[Haskell-cafe] Storables and Ptrs

[Tyler Pirtle]

On Sun, Dec 5, 2010 at 9:46 PM, Antoine Latter <aslatter at gmail.com> wrote:
> On Sun, Dec 5, 2010 at 10:45 PM, Tyler Pirtle <teeler at gmail.com> wrote:
>> Hi cafe,
>>
>> I'm just getting into Foreign.Storable and friends and I'm confused
>> about the class storable. For GHC, there are instances of storable for
>> all kinds of basic types (bool, int, etc) - but I can't find the
>> actual declaration of those instances.
>>
>> I'm confused that it seems that all Storable instances operate on a
>> Ptr, yet none of these types allow access to an underlying Ptr. I
>> noticed that it's possible via Foreign.Marshal.Utils to call 'new' and
>> get a datatype wrapped by a Ptr, but this isn't memory managed - I'd
>> have to explicitly free it? Is that my only choice?
>
> The Storable class defines how to copy a particular Haskell type to or
> from a raw memory buffer - specifically represented by the Ptr type.
> It is most commonly used when interacting with non-Haskell (or
> 'Foreign') code, which is why a lot of the tools look like they
> require manual memory management (because foreign-owned resources must
> often be managed separately anyway).
>
> Not all of the means of creating a Ptr type require manual memory
> management - the 'alloca' family of Haskell functions allocate a
> buffer and then free it automatically when outside the scope of the
> passed-in callback (although 'continuation' or 'action' would be the
> more Haskell-y way to refer to the idea):
>
> alloca :: Storable a => (Ptr a -> IO b) -> IO b
>
> This can be used to call into C code expecting pointer input or output
> types to great effect:
>
> wrapperAroundForeignCode :: InputType -> IO OutputType
> wrapperAroundForeignCode in =
>  alloca $ \inPtr ->
>  alloca $ outPtr -> do
>    poke inPtr in
>    c_call inPtr outPtr
>    peek outPtr
>
> The functions 'peek' and 'poke' are from the Storable class, and I
> used the 'alloca' function to allocate temporary storage for the
> pointers I pass into C-land.
>
> Is there a particular problem you're trying to solve? We might be able
> to offer more specific advice. The Storable and Foreign operations may
> not even be the best solution to what you're trying to do.
>


Hey Antoine,

Thanks for the clarity, it's very helpful. There is in fact a particular
problem I'm trying to solve - persisting data structures. I'm a huge
fan of Data.Vector.Storable.MMap, and I'm interested in other things
like it - but i realize that the whole thing is built up/on/around
storables, and building vectors with storables (read == peek, write ==
poke, etc), because i'm trying to write the raw structures themselves
to disk (via mmap).

I am aware of Data.Binary, but I feel that this kind of serialization
for the application I'm building would be too cumbersome considering the
number of objects I'm dealing with (on the order of hundreds-of-millions
to billions), especially considering that the application I'm building
has some very nice pure-ish semantics (an append-only list). I'd
like the application to able to simply load a file and interact with
that memory - not have to load the file and then deserialize everything.

If you have any suggestions here, or if anyone has any general feelings
about the design or implementation of Data.Vector.Storable.MMap I'd be
very interested in hearing them. Or about any ideas involving persisting
native data structures in an append-only fashion, too. ;)

Thanks,


Tyler







> Take care,
> Antoine
>
>
>>
>> Is there a way that given just simply an Int I could obtain a Ptr from
>> it, and then invoke the storable functions on it? Or for that matter,
>> if I go and create some new data type, is there some generic
>> underlying thing (ghc-only or otherwise) that would let me have a Ptr
>> of it?
>>
>> Thanks,
>>
>>
>> Tyler
>>
>> _______________________________________________
>> Haskell-Cafe mailing list
>> Haskell-Cafe at haskell.org
>> http://www.haskell.org/mailman/listinfo/haskell-cafe
>>
>