[Haskell-cafe] Neural Nets

[Marco Righele]

schrum2 at southwestern.edu wrote:

>Hello all. I was wondering if anyone knows of ways to implement neural nets in 
>Haskell, and if there are any sites or projects that you could direct me to. 
>I'm specifically interested in how to represent the neural net in Haskell, and 
>then how to represent the neural net as genes in a genetic algorithm.
>_______________________________________________
>  
>
Hi,
It mostly depends on which kind of nets you want to work on.
If feed-forward neural nets are enough for you, the best solution is 
probably to layer them and use vector to reperesent activation values 
and matrices to represent weights betwen neurons. The result will be at 
the same time elegant and efficient.
For example suppose we have some Vector and Matrix data types (there 
should be some implementations around) with the common operations defined
(What follows is not exactly Haskell, as I'm not going to write down 
everything, but it should give you the idea)
mmulv :: Matrix a -> Vector a ->Vector a
mmulm :: Matrix a -> Matrix a -> Matrix a
transpose :: Matrix a -> Matrix a
mmap :: Matrix a -> (a -> b) -> Matrix b
instance Functor Matrix where ...
etc.

For example let's suppose to have network with two layers, the input one 
with n neurons and the output one with m. All the weights can be 
represented by a mxn matrix. In mathematical term the output of the net 
will be o = net( W* net(i) ) where net is the activation function.
The following bit of code will evaluate the output of the net given the 
input value
eval :: Matrix Double -> Vector Double-> Vector Double
eval w i =
  let o = fmap net i
      i' = w `mmulv` o
  in
    fmap net i'

If we want to be a bit more general we can represent a network in the 
following way:
type ANN = ([Double -> Double], [Double->Double],[Matrix Double])

where the first element of the tuple is the list of the activation 
function and the second is the list of the their derivatives, and they 
both have one element more than the weights list (because we have one 
layer more than weights matrices)

evalAnn :: ANN -> Vector Double -> Vector Double
evalAnn (funs,dfuns,ws) = evalAnn' funs ws

evalAnn' [f] [] v = fmap f v
evalAnn' (f:fs) (w:ws) v = evalAnn' fs ws ( w `mmulv` (fmap f v))

You can get the training function in a similar way.
What you need is just a "train" function that takes the training 
examples and a partially trained network and returns an adjusted network
type Example = (Vector Double,Vector Double)
train :: [Example] -> ANN -> -> ANN

Iterating this function you can get the list of all the "states" of the 
network during the training and decide when to stop in any way you want 
(with validation on another set for example)

A final warning: I wrote all the code "on the fly" so expect it to be 
broken and far from working.
Hope it helps anyway.

Marco.