major missing piece to arrays?

Simon Marlow simonmar at microsoft.com
Fri Jul 16 05:43:58 EDT 2004


On 15 July 2004 22:08, John Meacham wrote:

> On Thu, Jul 15, 2004 at 04:20:38PM +0200, Jérémy Bobbio wrote:
>> memcpy is available in Foreign.Marshal.Utils:
>> 
>>      copyBytes :: Ptr a -> Ptr a -> Int -> IO ()
>> 
>>      Copies the given number of bytes from the second area (source)
>>      into the first (destination);the copied areas may not overlap
>> 
>> Here is the result of a quick try to implement fast copy using it and
>> Data.Array.Storable:
> 
> Yeah,
> I know I can copy areas of memory allocated via the foreign library
> or C code around. What I am looking for is an efficient way to work
> with 
> the standard Arrays as provided by Data.Array.

The idea is that if you copy areas of an array using intermediate lists, GHC should do the appropriate deforestation to remove the lists.  Whether this actually happens in practice or not is another matter - but it would help greatly if we had examples that we can investigate where the deforestation isn't happening properly.

However, this still isn't going to be as fast as using memcpy.  A general array-copying operating using memcpy() is entirely possible - you're allowed to pass a ByteArray# to a foreign function as long as it is marked 'unsafe'.  Only a *pinned* ByteArray# can be passed to 'safe' FFI calls.

I've wondered in the past whether we should pin all IOUArrays, which would make StorableArray almost obsolete (except for making arrays from Ptrs returned from FFI calls).  However, having lots of small pinned IOUArrays could lead to bad memory performance, because each pinned object holds onto the page it resides in, wasting up to 4k of memory.

> Also, in my tests, arrays implemented via ByteArray# or Ptr a seem to
> be signifigantly faster than those implemented via ForeignPtr. Is this
> expected?

Yes, StorableArray suffers from this problem.  Specialising the array operations on StorableArray might help.

Cheers,
	Simon


More information about the Glasgow-haskell-users mailing list