<!DOCTYPE html><html><head><title></title><style type="text/css">p.MsoNormal,p.MsoNoSpacing{margin:0}</style></head><body><div>Hi Olaf,<br></div><div><br></div><div>I can't speak for SQL in general, but if you're targeting PostgreSQL note that the hasql library uses a binary wire format. You'll need to use parameterized queries to benefit from this, but it would let you avoid encoding digits to ASCII.</div><div><br></div><div>Ollie</div><div><br></div><div>On Tue, 9 Aug 2022, at 8:34 PM, Olaf Klinke wrote:<br></div><blockquote type="cite" id="qt" style=""><div>Dear Café, <br></div><div><br></div><div>I was discussing with a friend the fastest way to shovel data into a<br></div><div>SQL database after parsing the raw (textual) data with Haskell. I am<br></div><div>concerned about numeric conversion: In my experience parsing the file<br></div><div>structure is fast. The majority of the time is spent converting strings<br></div><div>of digits to numbers. I hoped that once Haskell has constructed a<br></div><div>Double, the backend driver can hand that binary value over to the<br></div><div>database engine. But Database.HDBC.Statement has a field <br></div><div><br></div><div> orginalQuery :: String<br></div><div><br></div><div>and HDBC has functions<br></div><div><br></div><div> prepare :: Connection -> String -> IO Statement<br></div><div> run :: Connection -> String -> [SqlValue] -> IO Integer<br></div><div><br></div><div>which makes me wonder whether the backend constructs a complete SQL<br></div><div>query statement in form of a string (including the SqlValues), and<br></div><div>hands that to the database driver to parse. Is that indeed so? <br></div><div>If yes, the numbers should never be parsed in the first place: The DB<br></div><div>will do that anyways. I might as well construct a new CSV file and let<br></div><div>the database do a bulk insert on that. The above is specific to the<br></div><div>HDBC backend. If there are other backends/frameworks that do the<br></div><div>marshalling more efficiently, please let me know. <br></div><div><br></div><div>Thanks<br></div><div>Olaf<br></div><div><br></div><div>_______________________________________________<br></div><div>Haskell-Cafe mailing list<br></div><div>To (un)subscribe, modify options or view archives go to:<br></div><div><a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe">http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe</a><br></div><div>Only members subscribed via the mailman list are allowed to post.<br></div></blockquote></body></html>