[Haskell-cafe] DB vs read/show for persisting large data

[Marc Weber]

Excerpts from Michael Snoyman's message of Wed Dec 14 14:34:30 +0100 2011:
> On Wed, Dec 14, 2011 at 3:31 PM, C K Kashyap <ckkashyap at gmail.com> wrote:
> Definite *don't* use read/show: if you make any updates to your data
> structures, all old files will be lost.

Well you can work around it:

  data MyDataV1 = {
    name :: String
  }
  deriving (Read,Show)

then you make an update:

    data MyDataV2 = {
      name :: String,
      age : Int
    }
    deriving (Read,Show)

then you can do 
  let (v1 :: MyDataV1) = tryReadDataToMaybe data
  let (v2 :: MyDataV2) = tryReadDataToMaybe data
  let real_data = upgrade v1 `or` v2


But you already see that you start writing boilerplate code.
It can be done for easy data structures .. But it soon will be a night
mare if you have complex data.

If you use a version control system you don't loose your data - it will
just be "hard to update".

For prototyping deriving binary or read/show instances are a nice way to
get started. serialization to JSON/XML can be implemented later when you
change your data format as well eventually.

So it depends on your task. If you want to use read/show etc you have to
think about file locking and such.

Have a look at the "derive" package (hackage) which can derive more
instances than just read/show (eg json).

You can still use a sqlite database use it as binary storage...
Depends on whether all your data fits into memory.

Marc Weber