[Haskell-cafe] lazily updating dependencies, git submodules and cabal
defigueiredo at ucdavis.edu
Wed Sep 9 21:53:18 UTC 2015
I am facing a problem that I assume others have seen before, but am
stumped for a solution. It's not about Haskell per se, but about how to
organize my haskell code to play nicely with the build tools.
I am currently developing a bitcoin trading agent. I produce 2
executables: the "Trader" and the "Tax" calculator. These 2 executables
share a common dependency to talk to a SQLite3 database in their
"Persistence" layer. (I actually have 3 levels of dependency, but we'll
get to that later)
I spend most of my time working on the Trader and the Persistence layer.
These are just different source code files: Trader.hs and
Persistence.hs. The problem is that as I update the Persistence.hs file
I break Tax.hs.
Because I am "lazy" and only really have to calculate taxes once I year.
I do *not* want to have to continuously modify the Tax.hs file to keep
up with the development in the Persistence layer. I just want it to use
an old version, so that I only have to bring it up to speed later. In
other words, I want to:
1. always be able to compile both executables,
2. delay updating an executable because of changes in a common
dependency as much as possible.
I want to be lazy and delay updating my Tax.hs code to work with latest
version of the Persistence layer a much as possible. At the same time,
the Trader and Persistence layer are worked on simultaneously.
#My Current Solution
The immediate solution that comes to mind is to put these files in 3
different packages. That way, I can use version 1.5 of the Persistence
layer for the Tax calculator, but still develop version 2.0 at the same
time as working on the Trader. I can use 'cabal sandbox add-source' to
immediately get changes made to the Persistence layer as I develop the
Trader. There is one minor problem with this approach: I have to
manually track the version numbers. I need to put the proper version
number in the "build-depends" section in the .cabal file. For example,
for Tax.cabal I would have: "build-depends: persistence == 1.5".
There is a different approach that is completely orthogonal to using
Cabal packages. I can put each of the 3 files (Trader.hs, Tax.hs and
Persistence.hs) in 3 different repositories and use git submodules. The
Persistence layer would be a submodule of both the Trader and the Tax
repos. As each submodule just points to a specific commit in the
Persistence repo, I don't have to manually track version numbers and the
Trader and Tax calculator can point to different commits and use
different versions of the Persistence layer.
The git solution is really convenient because when you clone a repo
(with --recursive), it comes with the correct versions of the
dependencies you need. This is what I did.
The problem I am facing is that I now have 3 (or more) levels and
multiple dependencies. There are 2 changes to the original setup:
1. Persistence.hs now imports MarketModel.hs and that also changes
pretty quickly as I develop the Trader.
2. The Trader now also depends on ProfitCalculator.hs that also imports
Using the 'cabal sandbox add-source' solution now requires that I
"add-source" the MarketModel package too. Otherwise, any change I make
to this package will not be tracked. In fact, I have to "add-source" two
different versions of MarketModel (in general, there's an exponential
blow up as the number of levels increases). The manual tracking of
version numbers in the cabal file also becomes critical. If I forget to
update a version number, the package will be reinstalled over an
existing one that's being used. This quickly becomes a mess.
The git submodule solutions doesn't work either. Although GHC does the
right thing and only complains about two modules with the same name if I
use a symbol that is ambiguous. Cabal sets the paths for "import"ed
modules in the "hs-source-dirs:" and that is global to the whole
package. I can't say:
1. first, compile the Persistence.hs file by "import"ing files in the
2. now, compile the ProfitCalculator.hs by "import"ing files in the path
That path is the same for the whole project and I can't use different
versions of MarketModel.hs
In short, it seems no solution works anymore. Is there a way to make
this work? Is there another way for me to be lazy about updating my
dependencies when building multiple targets? Any pointers on this would
be much appreciated!
More information about the Haskell-Cafe