[Haskell-cafe] Hit a wall with the type system

[Henning Thielemann]

On Wed, 28 Nov 2007, Chris Smith wrote:

> > data AD a = AD a a deriving Eq
> >
> > instance Show a => Show (AD a) where
> >     show (AD x e) = show x ++ " + " ++ show e ++ " eps"
> >
> > instance Num a => Num (AD a) where
> >     (AD x e) + (AD y f)       = AD (x + y)         (e + f)
> >     (AD x e) - (AD y f)       = AD (x - y)         (e - f)
> >     (AD x e) * (AD y f)       = AD (x * y)         (e * y + x * f)
> >     negate (AD x e)           = AD (negate x)      (negate e)
> >     abs (AD 0 _)              = error "not differentiable: |0|"
> >     abs (AD x e)              = AD (abs x)         (e * signum x)
> >     signum (AD 0 e)           = error "not differentiable: signum(0)"
> >     signum (AD x e)           = AD (signum x)      0
> >     fromInteger i             = AD (fromInteger i) 0
> >
> > instance Fractional a => Fractional (AD a) where
> >     (AD x e) / (AD y f)       = AD (x / y) ((e * y - x * f) / (y * y))
> >     recip (AD x e)            = AD (1 / x) ((-e) / (x * x))
> >     fromRational x            = AD (fromRational x) 0
> >
> > instance Floating a => Floating (AD a) where
> >     pi                        = AD pi        0
> >     exp (AD x e)              = AD (exp x)   (e * exp x)
> >     sqrt (AD x e)             = AD (sqrt x)  (e / (2 * sqrt x))
> >     log (AD x e)              = AD (log x)   (e / x)
> >     (AD x e) ** (AD y f)      = AD (x ** y)  (e * y * (x ** (y-1)) +
> >                                               f * (x ** y) * log x)
> >     sin (AD x e)              = AD (sin x)   (e * cos x)
> >     cos (AD x e)              = AD (cos x)   (-e * sin x)
> >     asin (AD x e)             = AD (asin x)  (e / sqrt (1 - x ** 2))
> >     acos (AD x e)             = AD (acos x)  (-e / sqrt (1 - x ** 2))
> >     atan (AD x e)             = AD (atan x)  (e / (1 + x ** 2))
> >     sinh (AD x e)             = AD (sinh x)  (e * cosh x)
> >     cosh (AD x e)             = AD (cosh x)  (e * sinh x)
> >     asinh (AD x e)            = AD (asinh x) (e / sqrt (x^2 + 1))
> >     acosh (AD x e)            = AD (acosh x) (e / sqrt (x^2 - 1))
> >     atanh (AD x e)            = AD (atanh x) (e / (1 - x^2))
> >
> > diffNum        :: Num b        => (forall a. Num a        => a -> a) -> b -> b
> > diffFractional :: Fractional b => (forall a. Fractional a => a -> a) -> b -> b
> > diffFloating   :: Floating b   => (forall a. Floating a   => a -> a) -> b -> b
> >
> > diffNum f x        = let AD y dy = f (AD x 1) in dy
> > diffFractional f x = let AD y dy = f (AD x 1) in dy
> > diffFloating f x   = let AD y dy = f (AD x 1) in dy

Why do the functions have different number types after differentiation?
I thought, that just 'diff'

diff :: (AD a -> AD a) -> (a -> a)
diff f x   = let AD y dy = f (AD x 1) in dy

must work. What you do, looks like numbers with errors, but I suspect you
are right that 'automatic differentiation' is the term used for those
kinds of computations.