[Haskell-cafe] Amazon AWS storage best to use with Haskell?

[dokondr]

On Tue, Nov 1, 2011 at 10:53 AM, Neil Davies
<semanticphilosopher at gmail.com>wrote:

> Word of caution
>
> Understand the semantics (and cost profile) of the AWS services first -
> you can't just open a HTTP connection and dribble data out over several
> days and hope for things to work. It is not a system that has that sort of
> laziness at its heart.
>
> AWS doesn't supply a traditional remote file store semantics - is queuing,
> simple database and object store have all been designed for large scale
> systems being offered as a service to a (potentially hostile) large set of
> users - you can see that in the way that things are designed. There are all
> sorts of (sensible from their point of view) performance related limits and
> retries.
>
> The challenge in designing nice clean layers on top of AWS is how/when to
> hide the transient/load related failures.
>
>
>
As a straw-man approach I would go first to NData.Map backed by Data.Map
with addition of "flush" function  to write Data.Map to external key-value
store / NoSQL DB.
Another requirement for NData.Map is concurrent consistency, so different
clients could modify its state preserving "happen-before" relationship. For
this I would add to NData.Map a "reftresh" function, that updates local
copy from  external key-value store.

As for hSimpleDB package, it looks like it doesn't build on ghc7:
http://hackage.haskell.org/package/hSimpleDB


> The hSimpleDB package
>
> Interface to Amazon's SimpleDB service.
> PropertiesVersions0.1 <http://hackage.haskell.org/package/hSimpleDB-0.1>,
> 0.2 <http://hackage.haskell.org/package/hSimpleDB-0.2>, *0.3*Dependencies
> base <http://hackage.haskell.org/package/base-3.0.3.2> (≥3 & ≤4),
> bytestring <http://hackage.haskell.org/package/bytestring-0.9.2.0>, Crypto<http://hackage.haskell.org/package/Crypto-4.2.4>,
> dataenc <http://hackage.haskell.org/package/dataenc-0.14.0.2>, HTTP<http://hackage.haskell.org/package/HTTP-4000.1.2>,
> hxt <http://hackage.haskell.org/package/hxt-9.1.4>, network<http://hackage.haskell.org/package/network-2.3.0.7>,
> old-locale <http://hackage.haskell.org/package/old-locale-1.0.0.3>,
> old-time <http://hackage.haskell.org/package/old-time-1.0.0.7>,
> utf8-string <http://hackage.haskell.org/package/utf8-string-0.3.7>License
> BSD3AuthorDavid Himmelstrup 2009, Greg Heartsfield 2007MaintainerDavid
> Himmelstrup <lemmih at gmail.com>CategoryDatabase<http://hackage.haskell.org/packages/archive/pkg-list.html#cat:database>,
> Web <http://hackage.haskell.org/packages/archive/pkg-list.html#cat:web>,
> Network<http://hackage.haskell.org/packages/archive/pkg-list.html#cat:network>Upload
> dateThu Sep 17 17:09:26 UTC 2009Uploaded byDavidHimmelstrupBuilt onghc-6.10,
> ghc-6.12Build failureghc-7.0 (log<http://hackage.haskell.org/packages/archive/hSimpleDB/0.3/logs/failure/ghc-7.0>
> )
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/haskell-cafe/attachments/20111101/107cfe27/attachment.htm>