[Haskell-cafe] Loading a csv file with ~200 columns into Haskell Record

[dominic at steinitz.org]

I don’t know if this helps but here’s something I created in 5 minutes yesterday. Once you have the data you can convert to e.g. frames with all the help that types can give. I once used frames on a file with 10,000 columns and it worked (probably I shouldn’t have done this).

Also an option is in Chris Done’s blog: https://www.fpcomplete.com/blog/2016/09/data-haskell

PS My script below will fail if I have missing data but then I can use something better than read.

> {-# OPTIONS_GHC -Wall #-}
> 
> import qualified Data.Vector as V
> import qualified Data.ByteString.Lazy as BL
> import qualified Data.ByteString.Lazy.Char8 as BLC
> import Data.Csv
> import Data.List
> import Plots
> import Diagrams.Prelude
> import Diagrams.Backend.Rasterific
> 
> main :: IO ()
> main = do
>   evalsCsv <- BLC.readFile "/Users/dom/.local/share/simwork/data/Models/Lorenz.csv"
>   let records :: Either String (V.Vector (V.Vector BL.ByteString))
>       records = decode NoHeader $ BLC.unlines $ BLC.lines evalsCsv
>   case records of
>     Left err -> putStrLn err
>     Right evals -> do
>       let hdrs :: V.Vector String
>           hdrs = V.map BLC.unpack $ V.head evals
>       let zss :: [[Double]]
>           zss = map (map read) $ map (map BLC.unpack) $
>                 transpose $ V.toList $ V.map V.toList $ V.drop 1 evals
>       renderRasterific "diagrams/lorenz.png" (dims2D 500.0 500.0)
>                        (renderAxis $ jSaxis (zss!!0) (zss!!1))
>       renderRasterific "diagrams/lorenz1.png" (dims2D 500.0 500.0)
>                        (renderAxis $ jSaxis (zss!!0) (zss!!2))
>       renderRasterific "diagrams/lorenz2.png" (dims2D 500.0 500.0)
>                        (renderAxis $ jSaxis (zss!!0) (zss!!3))
>       renderRasterific "diagrams/lorenz3.png" (dims2D 500.0 500.0)
>                        (renderAxis $ jSaxis (zss!!1) (zss!!2))
>       renderRasterific "diagrams/lorenz4.png" (dims2D 500.0 500.0)
>                        (renderAxis $ jSaxis (zss!!1) (zss!!3))
>       renderRasterific "diagrams/lorenz5.png" (dims2D 500.0 500.0)
>                        (renderAxis $ jSaxis (zss!!2) (zss!!3))
> 
> jSaxis :: [Double] -> [Double] -> Axis B V2 Double
> jSaxis ts xs = r2Axis &~ do
>   linePlot' $ zip ts xs


Dominic Steinitz
dominic at steinitz.org
http://idontgetoutmuch.wordpress.com