[Haskell-cafe] Requesting For Comments for a new simple-dsp library

[Tristan Cacqueray]

On Thu, Mar 21, 2024 at 19:04 Henning Thielemann wrote:
> On Thu, 21 Mar 2024, Tristan Cacqueray wrote:
>
>> I'd like to propose a new package named simple-dsp. I couldn't figure
>> out how to use the existing dsp library, so I made my own version
>> featuring a little GUI to try the code in realtime.
>>
>> I'd be happy to contribute this work directly to dsp.
>
> The 'dsp' package is pretty old, predating any Vector data type or refined 
> numeric type class hierarchies. I decided to leave it as it is, for 
> educational reasons, since it contains many literal translations of signal 
> processing algorithms from various text books. I have not used most of the 
> contained functions myself.
>
>> My goal is to be able to extract sensible discrete values from a sound 
>> input at 60 Hz for video production. I implemented low/band/high pass 
>> filters to compute 3 RMS volumes, but the values I'm getting are not 
>> sharp enough, e.g. the band pass picks the low end. Ideally I would 
>> extract the transient events of the percusive sound and the note pitch 
>> of the melodic elements.
>
> For a filter that computes lowpass, highpass and bandpass in one go, I use 
> the "universal filter", also known as "state variable filter":
>    https://hackage.haskell.org/package/synthesizer-core-0.8.3/docs/Synthesizer-Plain-Filter-Recursive-Universal.html
>
> You can see running examples here:
>    https://hackage.haskell.org/package/synthesizer-core-0.8.3/docs/src/Synthesizer.Plain.Filter.Recursive.Test.html#universalTest
>
> Frequencies are specified as ratio with respect to the sampling rate, e.g. 
> 440 Hz resonance frequency at 44100 Hz sampling rate translates to 
> 440/44100. For working on storable vectors you would instead use 
> Universal.causal and apply this to Data.StorableVector.Lazy. For very fast 
> processing I have the synthesizer-llvm package, which however depends on 
> the LLVM package in a compatible version. (LLVM-16 currently preferred.)
>

I see, thank you for the details, that sounds useful. The 'step'
function looks simple to use too. It's surprising that such a filter
seems to only need 2 values for its state.

>
> Unfortunately I do not understand your application well, thus I do not 
> know whether the state variable filter is the right choice for your 
> application.

The final application is named animation-fractal, it renders a video at
60 fps, using external inputs to drive a generative art function (shader).
For each frame, the process:
- collects inputs such as midi events or audio frames.
- applies a modulation function to produce discrete values.
- updates the shader input with the modulation outputs.

In this demo, I used 6 sound stems and extracted their peak RMS volume
(using 735 samples per frame, or 44100 / 60):
  https://www.youtube.com/watch?v=qZTqUQumC6s

My goal with simple-dsp, is to enable using a single sound file to drive
the animation, for example, by extracting the peak RMS volume of the
low/mid/high frequency to approximate the drum/melody/hats elements of
a sound and use that as the animation source. In a previous
implementation, I used an fft to extract such values, but it was
challenging to get a smooth result and that's why I'm looking into
filters. For what it's worth, my current implementation is done in the
'mkFilters' and 'updateFilters' of:
  https://gitlab.com/TristanCacqueray/animation-fractal/-/blob/main/src/AnimationFractal/Input/AudioFilter.hs

Note that it's ok if the signal gets distorted, as long as the value
somehow represent a meaningful element of the sound input.

Thanks again for getting back to me, I'll give synthesizer-core a try
soon.

-Tristan
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 515 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/haskell-cafe/attachments/20240321/efa7180b/attachment.sig>