[Haskell-cafe] Binary Data Access via PIC…??

[Nick Rudnick]

On NL FP day, it struck me again when I saw an almost 1 MB *.hs file with
apparent sole purpose of getting a quantity of raw data incorporated to the
binary – applying some funny text encoding constructs. I remembered that,
to my best knowledge, with major downside that it's compile time, this
appears to be the best solution to me…

Another approach I did notice several times was, say, the use of super fast
parsing, to read in binary data at run time.

Did I miss something?

Or, more specifically – I am speaking about that kind of binary data which
is

(1) huge! – the 1 MB mentioned above rather being at the lower limit,
(2) completely independent from the version of the Haskell compiler,
(3) guaranteed (externally!) to match the structural requirements of the
application referred to,
(4) well managed in some way, concerning ABI issues, too (e.g. versioning,
metadata headers etc.),

and the question is in how far – as I believe other languages do, too – we
can exploit PIC (position independent code), to read in really large
quantities of binary data at run time or immediately before run time,
without the need for parsing at all.

E.g., a textual data representation Haskell file will generate an an object
file already, for which linking only should have a limited amount of
assumptions regarding its inner structure. Imagining I have a huge but
simple DB table, and a kind of converter which by some simplification of a
Haskell compiler generates an object file that equally matches these
(limited, as I believe) assumptions, and at the end can build a 'fake' the
linker accepts instead of one dummy file skeleton – couldn't that be a way
leading into the direction of directly getting in vast amounts of binary
data in one part?

In case there are stronger integrity needs, extra metadata like should be
usable for verification of the origin from a valid code generator.

Of course, while not completely necessary, true run time loading would be
even greater… while direct interfacing to foreign (albeit simple) memory
spaces deems much more intricate to me.

I regularly stumbled about such cases – so I do believe this to useful.

I would be happy to learn more about this – any thoughts…??

Cheers, and all the best, Nick
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/haskell-cafe/attachments/20190113/a60c7c39/attachment.html>