[Haskell-cafe] Hackage 2 maintainership

[Jeremy Shaw]

On Wed, Jun 20, 2012 at 11:06 AM, Ben Gamari <bgamari.foss at gmail.com> wrote:

> This list is definitely a start. One of the issues that was also
> realized is the size of the server's memory footprint. Unfortunately
> acid-state's requirement that all data either be in memory or have no
> ACID guarantees was found to be a bit of a limitation. If I recall
> correctly some folks were playing around with restructuring the data
> structures a bit to reduce memory usage. I really don't know what
> happened to these efforts.

I am one of those people, but as I (hopefully) mentioned at the time,
I had some other tasks I had to address first (namely, the release of
Happstack 7 and clckwrks). However,  starting this week I am finally
looking at some acid-state related issues. Part of this will be
building some tools to help analyze where all your RAM is going.

My current hypothesis is that for hackage 2, we should be able to
reduce RAM usage by 10 to 100 fold. The size of the data on disk is
only a few MB.. I currently can not think of any good reason why it
should take anywhere as much RAM as it currently does. So, it must be
doing it for some bad reason :) Though, I won't know until I do the
analysis. You'll probably see some blog articles as I work out the
process.

- jeremy