[Haskell-cafe] lazily updating dependencies, git submodules and cabal

[Dimitri DeFigueiredo]

Hello all,

I am facing a problem that I assume others have seen before, but am 
stumped for a solution. It's not about Haskell per se, but about how to 
organize my haskell code to play nicely with the build tools.

#The Setup

I am currently developing a bitcoin trading agent. I produce 2 
executables: the "Trader" and the "Tax" calculator. These 2 executables 
share a common dependency to talk to a SQLite3 database in their 
"Persistence" layer. (I actually have 3 levels of dependency, but we'll 
get to that later)

I spend most of my time working on the Trader and the Persistence layer. 
These are just different source code files: Trader.hs and 
Persistence.hs. The problem is that as I update the Persistence.hs file 
I break Tax.hs.

Because I am "lazy" and only really have to calculate taxes once I year. 
I do *not* want to have to continuously modify the Tax.hs file to keep 
up with the development in the Persistence layer. I just want it to use 
an old version, so that I only have to bring it up to speed later. In 
other words, I want to:

1. always be able to compile both executables,
2. delay updating an executable because of changes in a common 
dependency as much as possible.

I want to be lazy and delay updating my Tax.hs code to work with latest 
version of the Persistence layer a much as possible. At the same time, 
the Trader and Persistence layer are worked on simultaneously.

#My Current Solution

The immediate solution that comes to mind is to put these files in 3 
different packages. That way, I can use version 1.5 of the Persistence 
layer for the Tax calculator, but still develop version 2.0 at the same 
time as working on the Trader. I can use 'cabal sandbox add-source' to 
immediately get changes made to the Persistence layer as I develop the 
Trader. There is one minor problem with this approach: I have to 
manually track the version numbers. I need to put the proper version 
number in the "build-depends" section in the .cabal file. For example, 
for Tax.cabal I would have: "build-depends: persistence == 1.5".

There is a different approach that is completely orthogonal to using 
Cabal packages. I can put each of the 3 files (Trader.hs, Tax.hs and 
Persistence.hs) in 3 different repositories and use git submodules. The 
Persistence layer would be a submodule of both the Trader and the Tax 
repos. As each submodule just points to a specific commit in the 
Persistence repo, I don't have to manually track version numbers and the 
Trader and Tax calculator can point to different commits and use 
different versions of the Persistence layer.

The git solution is really convenient because when you clone a repo 
(with --recursive), it comes with the correct versions of the 
dependencies you need. This is what I did.

#The Problem

The problem I am facing is that I now have 3 (or more) levels and 
multiple dependencies. There are 2 changes to the original setup:
1. Persistence.hs now imports MarketModel.hs and that also changes 
pretty quickly as I develop the Trader.
2. The Trader now also depends on ProfitCalculator.hs that also imports 
MarketModel.hs

Using the 'cabal sandbox add-source' solution now requires that I 
"add-source" the MarketModel package too. Otherwise, any change I make 
to this package will not be tracked. In fact, I have to "add-source" two 
different versions of MarketModel (in general, there's an exponential 
blow up as the number of levels increases). The manual tracking of 
version numbers in the cabal file also becomes critical. If I forget to 
update a version number, the package will be reinstalled over an 
existing one that's being used. This quickly becomes a mess.

The git submodule solutions doesn't work either. Although GHC does the 
right thing and only complains about two modules with the same name if I 
use a symbol that is ambiguous. Cabal sets the paths for "import"ed 
modules in the "hs-source-dirs:" and that is global to the whole 
package. I can't say:

1. first, compile the Persistence.hs file by "import"ing files in the 
path /trader/persistence/marketmodel,
2. now, compile the ProfitCalculator.hs by "import"ing files in the path 
/trader/profitcalculator/marketmodel.

That path is the same for the whole project and I can't use different 
versions of MarketModel.hs

In short, it seems no solution works anymore. Is there a way to make 
this work? Is there another way for me to be lazy about updating my 
dependencies when building multiple targets? Any pointers on this would 
be much appreciated!


Thanks,


Dimitri