Hackage 2 and acid-state vs traditional databases

[Duncan Coutts]

On 6 September 2012 21:06, Duncan Coutts <duncan.coutts at googlemail.com> wrote:
> On 6 September 2012 19:49, Ian Lynagh <ian at well-typed.com> wrote:
>> * Only a single process can use the database at once. For example, if
>>   the admins want a tool that will make it easier for them to approve
>>   user requests, then that tool needs to be integrated into the Hackage
>>   server (or talk to it over HTTP), rather than being standalone.
>
>> * The database is relatively opaque. While in principle tools could be
>>   written for browsing, modifying or querying it, currently none exist
>>   (as far as I know).
>
> In both these points, I would argue that we should simply make all the
> data available via http REST interfaces. Instead of making the data
> available only locally to someone with direct access to the database,
> it should be there in machine readable form so that everyone can get
> it and experiment.
>
> In the approving user requests example, that just needs an HTTP
> PUT/POST. It doesn't need any web form. Totally scriptable with
> wget/curl etc.

I should probably say this more clearly...

The original design of the new server was that it should primarily be
a web database (REST and all that), with a user-facing website tacked
on.

That is, whatever extra fancy features it has, all of the data it
stores should be accessible and modifiable via http (i.e.
get/put/post) in at least one machine readable format. This alone
makes the system extensible, and extensible by third parties.

For the most part the current implementation does indeed make all the
data available via standard http methods. We can make that better by
adding more formats (esp json) and making sure we consistently make
everything available, and by making it discoverable that everything is
indeed available. The latter should be possible e.g. via an
automatically generated site map. We have URL templates for all the
resources that are available, and we can simply automatically collate
those and make them available on a generated page.

So if we've done it properly, there should be no need for direct
access to the database. And as I said before, doing it properly has
the great advantage that all the data is available to everyone to do
cool things we've not thought of yet.

Duncan