Replacement for GMP

[Esa Ilari Vuokko]

Hi Peter,

Peter Tanski wrote:
> (This may sound naieve): the in { size, used, payload, sign } are all
> parts of the info-table for the payload and the RTS re-initialises the
> mathlib on each invocation, right?

I hope my answer helps, but if it gets you more confused,
maybe it's just because I'm confused...I'm certainly
not the best qualified to answer.

Infotables are static as far as I understand.
I was also a bit wrong, now that I read primops-code again.

 (Places of code that I read for this:
 http://darcs.haskell.org/ghc/rts/PrimOps.cmm
 search for #define GMP_TAKE_RET1 for an example of macro that
 is used to implement unary op.
 http://darcs.haskell.org/packages/base/GHC/Num.lhs
 search for data Integer)

The interesting ctor for Integer is:
| J# Int# ByteArray#

Here we see that J# has two params
 size (which also contains sign) of the data
 pointer to another object, garbage-collection managed bytearray

In GMP_TAKE1_RET1 we have
  MP_INT__mp_alloc(mp_tmp1)	= W_TO_INT(StgArrWords_words(d1));
  MP_INT__mp_size(mp_tmp1)	= (s1);
  MP_INT__mp_d(mp_tmp1)		= BYTE_ARR_CTS(d1);

Where s1 and d1 are the parameters from that constructor.
In GMP's case, size variable contains sign, and no sign variable
is needed.  Payload is the actual bytes in bytearray-object and
allocated size is the amount of bytes allocated for bytearray-object.

Because the memory allocation for GMP has been hijacked, we
can simply return the important bits.

  RET_NP(
      TO_W_(MP_INT__mp_size(mp_result1)),
      MP_INT__mp_d(mp_result1) - SIZEOF_StgArrWords);

RET_NP jumps back, or into, whatever needs the answer, we
return two things, int that has the size and sign from GMP
structure, and pointer to bytearray object - we need to adjust
the pointer location to point into bytearray-object, instead
of it's payload.  The return type is actually,
(# Int#, ByteArray# #), which is unboxed tuple
containing unboxed Int and pointer/reference to ByteArray-object.

> In the thread "returning to cost of Integer", John Meacham wrote:
> 
>> we could use the standard GMP that comes with the system since
>> ForeignPtr will take care of GCing Integers itself.
> 
> From current discussion: at present the allocation is done on the GC
> heap, when it could be done entirely by the mathlib.  The benefit to
> letting the mathlib handle memory would be that you could use the
> mathlib with another (non-Haskell) part of your program at the same time
> (see, e.g., (bug) Ticket #311).

There are other nicer things about that as well - untying Integer
(atleast mostly) from runtime/frontend and moving it more into
domain of libraries.

> (I am making an educated guess, here.) You probably chose to allocate
> GMP's memory on the GC heap because:

(I have no idea about original reasons.)

> (1) call-outs to another program are inherently impure since the
> type-system and execution order are not defined by the Haskell Runtime;
> and,

Another program?  I assume you meant outside pure haskell - "call-outs"
have side-effects.  We can get around by using unsafePerformIO,
which doesn't really differ that much from writing it in
C-- (and library in C), except we'd write haskell.

> (2) it was a stable way to ensure that the allocated memory would remain
> available to the thunk for lazy evaluation, i.e., so that the evaluation
> of the returned Bignum could be postponed indefinitely, correct?  Or
> could the evaluation itself be postponed until the value was called
> for--making operations on Integers and other Bignums lazy?

Uhm, naturally, haskell rts needs to control lifetime of the memory.
I am not sure what you're trying to say here, really.  Is the point
that we cannot free almost anything without permission from
garbage collector?  Because, yeah, we can't.

My guess (without having read the thread you mention) is that using
ForeignPtrs brings two cons: Possibly extra indirection and using
lots of Integers will mean (in many cases) a lots of finalisers that
get run, and running finalisers (and checking for them in the first
place) might be slow.

> In other words, it does not seem possible to simply hold a ForeignPtr to
> the returned value unless there were a way to release the memory when it
> was no longer needed.  If you wanted the mathlib to retain the value on
> behalf of GHC, you would have to modify the library itself.  In the end
> you have a specialized version of the library and a change to the
> procedure from:
> math_lib_init ; ...
> return out.payload ;
>     to:
> math_lib_init ;
> math_lib_evaluate ;
> math_lib_free ;

Only temporaries could be free'd here.  Anything else could be
needed again.

> An easier though less-efficient alternative would be to have GHC copy
> the value returned by the mathlib.  That would be stable and allow other
> systems to use the same mathlib concurrently (assuming the lib is
> thread-safe).

As I understand, you suggest here copying payload instead of merging
memory handling.  I don't think it's clearly, if ever, less efficient
than ForeignPtr-based approach.  But I'd guess it is *more* code
than current solution.

> The third alternative I suggested previously was to embed the Bignum
> processing in GHC itself.  I think it would be very difficult to
> maintain a solution that was both optimised and portable, at least in
> C--.  (I may be way-off here; I am simply going by a rudimentary
> knowledge of BLAST implementations.)

I don't think it differs much from doing the same in C.
It does seem shame to write bignum in C--, as we don't get many
elegance-style advantages from writing it in C-- instead of C.

> If I am correct about (2) above, the best conclusion I could draw from
> this is that the easiest solution would be to copy the memory on return
> from the mathlib.

Depends what your goals are.  If you don't care about GMP being rendered
unavailable to other non-haskell parts of the program, the current
solution is one of the easiest. The easiest solution must be either
writing bignum in haskell or using ForeignPtrs.

> directory about this.  One of the big ToDo's seems to be to correct the
> method of configuring this stuff using machdep.h or the equivalent on a
> local system, such as the sysctl-headers on Darwin.  For C-- this seems
> like it would be a bit more difficult than simply confirming whether (or
> how) the C implementation conforms to the current standard through the
> usual header system.

Neither C or C-- was meant to be used to detect what system can do.
It is simply a byproduct, which autotools takes to the extreme.
C does fit the bill better because there's so much framework built
for it, mainly because unixy OSes, and their kernels, are written in C.

As for what it has to do with topic at hand, I have no idea.
C-- is simply used as an intermediate language for the compiler, for
convience of calling conventions and such, some low-level operations
are written in it.

[snip lots more]
There's lots thoughts, but I found I had little to say about those,
sorry.

Best regards,
--Esa