Mixed boxed/unboxed arrays?

[J. Reinders]

Thanks for your suggestion. That sounds like a promising technique.

I have an implementation that mostly works here:
https://github.com/noughtmare/clutter
in the src/Counter.hs file.

The only problem is that I get segfaults or internal GHC errors if I run it on large files. I’ve adding some tracing and it seems to occur when I try to coerce back pointers from the hash table array to proper Haskell values in the ’toList’ function.

I can reproduce the problem by running ‘bash test.sh’.

Currently, I’m using the ‘ptrToAny' and ‘anyToPtr' functions to do the coercing, because that sounds like the safest option.

Do you know what’s going wrong or do you have a safer design for coercing the pointers?

I thought it might be because the compact region gets deallocated before all the pointers are extracted, but even if I add a ’touch c’ (where c contains the compact region) at the end it still gives the same errors.

> On 3 Aug 2022, at 05:30, Viktor Dukhovni <ietf-dane at dukhovni.org> wrote:
> 
> On Tue, Aug 02, 2022 at 05:32:58PM +0200, J. Reinders wrote:
> 
>>> Could you use `StablePtr` for the keys?
>> 
>> That might be an option, but I have no idea how performant stable
>> pointers are and manual management is obviously not ideal.
> 
> If your hash table keys qualify for being stored in a "compact region",
> you may not need per-key stable pointers, just (carefully) coercing the
> keys to pointers suffices to produce primitive "handles" that are stable
> for the lifetime of the "compact region".  The inverse (unsafe) coercion
> recovers the key.
> 
> This also has the advantage that a key count does not incur a high
> ongoing GC cost.  The keys are of course copied into the compact region.
> 
> With this you could store "pointer + count" in a primitive cell.  The
> hash table then holds a reference to the compact region and compacts
> keys on insert.
> 
>    https://hackage.haskell.org/package/compact-0.2.0.0/docs/Data-Compact.html
> 
> -- 
>    Viktor.
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs