Updates of hackage-server and data loss

[Duncan Coutts]

On 8 February 2012 16:58, Erik Hesselink <hesselink at gmail.com> wrote:
> On Mon, Feb 6, 2012 at 19:07, Duncan Coutts
> <duncan.coutts at googlemail.com> wrote:
>> On 6 February 2012 15:22, Erik Hesselink <hesselink at gmail.com> wrote:
>>> Hi all,
>>>
>>> We run hackage-server internally at Silk. Every time we update to a
>>> new version, it is unable to read our old data. Is the acid-state
>>> being versioned correctly? Or is this expected behavior? It is very
>>> annoying for us. If this is expected, is there a way to work around
>>> it?
>>
>> Our approach to versioning is not via acid-state by by a database
>> backup/restore method. We export all canonical data to a tarball of
>> text and other standard format files. The reason we do this is because
>> we don't trust that the binary data format is sufficient for long term
>> preservation of all the data. Because we are using dump/restore, we
>> make no attempt to use the acid-state versioning scheme (it's
>> plausible we might do so in future as an extra, but probably not until
>> there are actual releases).
>>
>> The backu[/restore is done via the commands 'hackage-server backup'
>> and 'hackage-server restore'. It is also possible to generate backups
>> while the server is running by sending a USR1 signal (kill -USR1
>> $the_pid).
>>
>> Note also that testing of the backup/restore method is not quite
>> complete, but the testing framework for it is in place I think.
>
> I actually tried the backup at some point, and got the following output:
>
> hackage-server: Initializing happstack-state...
> hackage-server: Server data loaded into memory
> hackage-server: Preparing export tarball
> hackage-server: Saving export tarball
> hackage-server: Shutting down...
> hackage-server: /home/erik/state/: inappropriate type
>
> This was with the old code though, I haven't tried with the new code yet.

Ok, like I said it's not fully tested yet, so you may hit some
problems that route too, but that's the approach we intend to make
work.

> I just tried sending the signal to the running process, and got:
>
> hackage-server: Writing checkpoint...
> hackage-server: Done
>
> There's no backup to be found anywhere, though. Where should it create
> the backup?

Oh, sorry, I was getting myself confused. Yes, that just writes the
acid-state checkpoint, not the backup tarball. We can in principle
also do online backup snapshots but we don't have a UI for that yet.

Duncan