[Haskell-cafe] Audio search library; and debugging help!

[Richard Lewis]

Hi there,

For a little while I've been working on an audio search library. It
wraps over a C++ library written by some of my colleagues. It's
nearing the state where someone other than me might actually be able
to use it. So this message is partly an annoucement:

<https://github.com/TransformingMusicology/libaudioDB-haskell>

I'd be very interested to hear if anyone has a go at installing it and
running the "tests".

But this message is also a request for help, because I'm actually
stuck with a bug.

I've created a fork of the above repo here:

<https://github.com/ironchicken/libaudioDB-haskell>

which includes an extra commit which adds quite a few tracing
statements. So if anyone feels up to trying to help me with my bug,
then this would be the repo to use.

The bug goes something like this: One of the features I need to
implement is so-called "query rotation". The library implements audio
feature searching. For some kinds of multi-dimensional audio features,
it's useful to execute queries with them repeatedly moving the values
between the dimensions each time to get a kind of exhaustive
search. As a specific example (and for those who know a little music
theory), there's a feature known as chroma which gives a measure of
the signal energy in each of the 12 pitch classes for each window in
the audio. If you search with a vector of chroma features, you will
only find results that are at exactly the same pitch. But if you shift
the chroma search vector around 11 times, you can effectively search
for the query vector in every transposition (that's "transposition" as
in to transpose in music theory).

Anyway, I've tried to capture this in the library by allowing a call
to the Sound.Audio.Database.Query.query function to include a
QueryTransformer, which is essentially a function which takes a query
and returns a new query.

And then one instance of this is the function
S.A.D.Query.mkSequenceQueryWithRotation which uses another helper
function, S.A.D.Query.mkSequenceQueryMutateDatum. This function takes
a Ptr to a query, and scribbles over the memory to replace its
contents with an altered query vector: the result of some mutation
function.

Now, what originally alerted me to there being anything wrong with
this was that, when I execute a rotated query, I'm getting exactly the
same results as for a non-rotated query. As I started investigating,
the first thing I found was that the altered feature vectors were not
being altered in the way I expected. If I printed them and traced the
path of a single value, it wasn't moving step-wise between the bins,
but instead jumping around between them randomly. I eventually put in
enough putStrLn and trace calls to find that, I think, each time the
query is transformed, instead of doing just one transformation, it
does as many transformations as times it has iterated: i.e., first
iteration it does one transformation, second iteration it does two
transformations, and so on.

The recursive (sort of iterative?) query transformation is implemented
in S.A.D.Query.queryWithTransformPtr. But I'm now at a loss to say
whether there's anything actually wrong with this function.

So, I think this is the end of my message. I appreciate that this is a
very specific problem. I know it's normally helpful to try and
abstract a problem into something in terms of Foos and Bars, but I'm
not sure I can do this here. As a result, it would be a significant
undertaking for anyone to try and help out. So I would be even more
grateful than usual if anyone could! (And that's not really meant as a
grovelling plea, it's actually more of a warning: you probably
shouldn't try and help me.)

Thanks,
Richard
-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Richard Lewis
Computing, Goldsmiths' College
t: +44 (0)20 7078 5203
@: lewisrichard
http://www.transforming-musicology.org/
905C D796 12CD 4C6E CBFB  69DA EFCE DCDF 71D7 D455
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=