[Haskell-cafe] Property checks and PostgreSQL?

[amindfv at mailbox.org]

On Sat, Mar 13, 2021 at 04:39:40PM +0500, Ignat Insarov wrote:
> Note: I posted this [elsewhere] — I hope it is permissible to also post
> here. There seems to be no single right venue for Haskell questions at this
> time.
> 
> [elsewhere]: https://discourse.haskell.org/t/property-checks-and-postgresql/2078
> 
> So, I have a three layer stack: PostgreSQL data base, Haskell back end,
> PureScript front end. And there is a fault somewhere in this stack: sometimes
> some items present in the data base do not make it to the front end. A small
> fortune in programmer hours has been spent searching for the fault but there is
> no progress. No one can figure out even roughly where the fault is. Our last
> hope is to property check the whole thing, from end to end, verifying that any
> things put into the data base can be retrieved by the front end.
> 
> We have in place a fixture that creates a new PostgreSQL cluster, runs the back
> end on it and makes some scripted HTTP requests. We use it to run unit tests,
> such like _«if I make this `PUT` request and then that `GET` request to the same
> end point, I should get back the same thing as I put in»_. In principle it would
> not be a problem to make a property check out of this property. Practically,
> tearing down the cluster, building a new pristine one and initializing the data
> base anew takes seconds, so this way even a trivial property check would take
> minutes to run.

Do you know where the majority of the time is spent? I.e. can you just drop the schema without re-creating the cluster? I'd guess it'd be fast but I don't know your setup.

> 
> Another thing I heard being done is to selectively drop and re-create exactly
> the tables affected by a run of a property check. This seems challenging to
> automate though. How can I know in a general way if a table has been touched?
> And then, how can I re-create exactly the affected tables given that the data
> base initialization code is an opaque blob?

At $WORK we do this: property tests on PSQL data dropping data between tests, but we know which tables are modified so we can just truncate the affected ones.

Tom