[Haskell-cafe] Haskell audio libraries & audio formats
lemming at henning-thielemann.de
Mon Aug 25 02:52:15 EDT 2008
On Sun, 24 Aug 2008, Eric Kidd wrote:
> Greetings, Haskell folks!
> I'm working on a toy program for drum synthesis. This has lead me to
> explore the various sound-related libraries in Hackage. Along the way,
> I've learned several things:
> 1. There's a lot of Haskell sound libraries, but no agreement on how
> to represent buffers of audio data.
In my sound synthesis library
I use mainly two representations:
1. Chunky sequences based on StorableVector. This is for efficient storage
and functions that need to process data in a non-causal way. They are not
good for fusion.
2. A Stream like sequences without wrappers. In fact, they are no data
storages but generator functions equipped with a seed. I use them for
causal signal processes. I think that most functions needed for drum
synthesis are causal. By inlining them, many subsequent causal signal
processes are fused to one.
> 2. Some of the most useful sound libraries aren't listed in Hackage's
> "Sound" section, including HCodecs, SDL-mixer and hogg.
my library is also not at Hackage, it's too much experimental currently
> 3. The Haskell OpenAL bindings are great, but ALUT has been removed
> from MacOS 10.5. Of course, a pure-Haskell version of ALUT would
> be pretty easy to write, and it could be used as a fallback.
> 4. '[[a]]' is _not_ a good representation for multi-channel audio
> data. Manipulating even 10 seconds of audio is noticeably slow.
Using the signal representations I mentioned above, I can synthesize some
songs in realtime. For stereo signals I use Signal (a,a). However, a
StereoFrame data type with Storable instance might be even better.
> 5. HCodecs is probably the best library for reading WAVE files. It
> uses 'DiffUArray Int a' to represent audio data, and not '[[a]]'.
> It also supports MIDI and SoundFont, which is quite handy.
> 6. YampaSynth looks really cool, and it might be possible to turn it
> into a reusable MIDI synthesizer library for use with HCodecs.
> This would be extremely useful: it would provide a basis for all
> kinds of crazy MIDI-based programming in Haskell.
If you are after realtime synthesis you might also want to look into
SuperCollider. I posted some code to control SuperCollider via MIDI to
haskell-art. I can send it to you, if you like.
> What would the ideal Haskell sound API look like? Personally, I would
> love to see:
> a. Something like HCodecs's Data.Audio as the basic sound format.
> Data.Audio is an array-based API, and it supports converting
> between a wide range of common sample formats. I don't know how
> fast this is, but it already exists, and it's used by YampaSynth.
> b. OpenAL for sound playback, with a portable version of ALUT. This
> may require writing a pure-Haskell version of ALUT.
So far I used Sox' play command fed by a pipe with runInteractiveCommand.
> Other nice-to-have features might include:
> e. A standard MIDI format, based on either the HCodecs package or the
> midi package. (HCodecs is used by YampaSynth, and the midi package
> is used by alsa-midi.)
midi package is mainly used by Haskore
> f. A modular version of YampaSynth which can convert MIDI data
> structures into Data.Audio values.
> It looks like Haskell could be a really sweet audio programming
> environment with just a bit of integration work. What do folks think?
> Are there other libraries I should look at more closely? Other features
> that should be included in an ideal audio API?
What do you mean by 'API'? I think there should not be one library
handling all kind of music tasks. For interoperability common data
structures are nice. However, what signal representation concerns,
I think we need more than one representation, because different
applications have different needs.
> Haskell audio libraries
I have advertised my libraries for years in Haskell-Wiki and HCAR, but it
seems that people do not look there. :-(
> Reading and writing sound files:
> HCodecs: (Audible a) => DiffUArray Int a
> hsndfile: MArray with Float and Double
> HSoundFile: [[Double]]
> ALUT: OpenAL.AL.Buffer
> WAVE: [[Int32]]
> Playing sounds:
> OpenAL: Ptr UInt8, Ptr Int16, 1-N channels
> SDL-mixer: ForeignPtr ChunkStruct, ForeignPtr MusicStruct
Sox via pipe, using any data structure which can be written to a file
> Sound processing libraries:
> dsp: Array a, [a]
> jack: Operates via mapping functions
> mainMono :: (CFloat -> IO CFloat) -> IO ()
> mainStereo :: ((CFloat, CFloat) -> IO (CFloat, CFloat)) -> IO ()
synthesizer: StorableVector, State, Generic (older: list, fusable list)
> HCodecs: Reads and writes MIDI files
> midi: Reads and writes MIDI files
> alsa-midi: Uses midi library
> YampaSynth: Stand-alone program
Haskore uses MIDI and other back-ends like Haskell Synthesizer, CSound,
> Special-purpose APIs only (FFIs, etc.):
> hCsound: CsoundPtr
> hsc3: UGen
> No public sound-buffer API:
> hbeat: The relevant source files are missing!
> hogg: Very low-level API for Ogg internals only
> libmpd: No sound buffer API
> sonic-visualizer: No sound buffer API
> truelevel: Stand-alone program (uses WAVE)
> wavconvert: Stand-alone program
More information about the Haskell-Cafe