[Haskell-cafe] Re: sizeFM type

Tomasz Zielonka t.zielonka at students.mimuw.edu.pl
Tue Apr 27 23:36:02 EDT 2004

On Tue, Apr 27, 2004 at 02:05:06PM -0400, S. Alexander Jacobson wrote:
> On Tue, 27 Apr 2004, Tomasz Zielonka wrote:
> > On Mon, Apr 26, 2004 at 08:43:44PM -0400, ajb at spamcop.net wrote:
> >
> > It seems I qualify as almost nobody, cause I've had a database with more
> > than 2^34 records on a single IA32 machine. However, I wouldn't ever
> > think of loading the entire database to RAM.
> Why not?  Its only 17gb.

About 17000000000 records, yes, but not 17GB. It was more like 300 GB
compressed data.

> If you are doing any substantive volume of transactions on that
> database* you may be better off storing the database in memory than
> buying the multiple CPUs and reliable RAID arrays you need to serve it
> off disk.

I was not a typical relational database, so it's hard to compare. I
managed on a 1 CPU P4 2.6 with 1GB mem and a RAID5 array. The database
engine was optimised for bulk processing, reducing disk seeks and memory

> A Dell PowerEdge 4600 w/ 24gb of memory and a
> write-caching controller w/ 6 36GB harddrives
> (RAID0 state,logging,status) costs around $34k.
> <http://www1.us.dell.com/content/products/compare.aspx/tower?c=us&cs=555&l=en&s=biz>
> Using e.g. a Prevayler style in memory logged data
> structures** you may be able to serve >100,000
> queries/second. (see e.g.
> <http://www.prevayler.org/wiki.jsp?topic=ScalabilityTestResults>)

We were able to achieve a throughput of 200,000 records per seconds (not
queries), which was about 3 times faster then necessary.

The data you provided is interesting. However, if I had such a machine,
my work could be much less challenging :)

> * OTOH, if this is just your personal mail archive
> then nevermind :- )


Best regards,

.signature: Too many levels of symbolic links

More information about the Haskell-Cafe mailing list