[Haskell-cafe] Haskell audio libraries & audio formats

[Eric Kidd]

Greetings, Haskell folks!

I'm working on a toy program for drum synthesis.  This has lead me to
explore the various sound-related libraries in Hackage.  Along the way,
I've learned several things:

  1. There's a lot of Haskell sound libraries, but no agreement on how
     to represent buffers of audio data.
  2. Some of the most useful sound libraries aren't listed in Hackage's
     "Sound" section, including HCodecs, SDL-mixer and hogg.
  3. The Haskell OpenAL bindings are great, but ALUT has been removed
     from MacOS 10.5.  Of course, a pure-Haskell version of ALUT would
     be pretty easy to write, and it could be used as a fallback.
  4. '[[a]]' is _not_ a good representation for multi-channel audio
     data.  Manipulating even 10 seconds of audio is noticeably slow.
  5. HCodecs is probably the best library for reading WAVE files.  It
     uses 'DiffUArray Int a' to represent audio data, and not '[[a]]'.
     It also supports MIDI and SoundFont, which is quite handy.
  6. YampaSynth looks really cool, and it might be possible to turn it
     into a reusable MIDI synthesizer library for use with HCodecs.
     This would be extremely useful: it would provide a basis for all
     kinds of crazy MIDI-based programming in Haskell.

At the end of this e-mail, you can find a list of Haskell sound
libraries and their supported audio formats.

What would the ideal Haskell sound API look like?  Personally, I would
love to see:

  a. Something like HCodecs's Data.Audio as the basic sound format.
     Data.Audio is an array-based API, and it supports converting
     between a wide range of common sample formats.  I don't know how
     fast this is, but it already exists, and it's used by YampaSynth.
  b. OpenAL for sound playback, with a portable version of ALUT.  This
     may require writing a pure-Haskell version of ALUT.
  c. Utility functions to convert Data.Audio to OpenAL.AL.Buffer.
  d. An easy-to-use API for audio processing.  It might be possible to
     start with something like Jack's 'mainMono' and 'mainStereo'
     functions, which map a function over an audio stream.

Other nice-to-have features might include:

  e. A standard MIDI format, based on either the HCodecs package or the
     midi package.  (HCodecs is used by YampaSynth, and the midi package
     is used by alsa-midi.)
  f. A modular version of YampaSynth which can convert MIDI data
     structures into Data.Audio values.

It looks like Haskell could be a really sweet audio programming
environment with just a bit of integration work.  What do folks think?
Are there other libraries I should look at more closely?  Other features
that should be included in an ideal audio API?

Thank you for your advice and feedback!

Cheers,
Eric


Haskell audio libraries

Reading and writing sound files:
  HCodecs: (Audible a) => DiffUArray Int a
  hsndfile: MArray with Float and Double
  HSoundFile: [[Double]]
  ALUT: OpenAL.AL.Buffer
  WAVE: [[Int32]]

Playing sounds:
  OpenAL: Ptr UInt8, Ptr Int16, 1-N channels
  SDL-mixer: ForeignPtr ChunkStruct, ForeignPtr MusicStruct

Sound processing libraries:
  dsp: Array a, [a]
  jack: Operates via mapping functions
    mainMono :: (CFloat -> IO CFloat) -> IO ()
    mainStereo :: ((CFloat, CFloat) -> IO (CFloat, CFloat)) -> IO ()

MIDI-based:
  HCodecs: Reads and writes MIDI files
  midi: Reads and writes MIDI files
  alsa-midi: Uses midi library
  YampaSynth: Stand-alone program

Special-purpose APIs only (FFIs, etc.):
  hCsound: CsoundPtr
  hsc3: UGen

No public sound-buffer API:
  hbeat: The relevant source files are missing!
  hogg: Very low-level API for Ogg internals only
  libmpd: No sound buffer API
  sonic-visualizer: No sound buffer API
  truelevel: Stand-alone program (uses WAVE)
  wavconvert: Stand-alone program