[GHC] #9647: allocation of 10790760 bytes too large
GHC
ghc-devs at haskell.org
Mon Sep 29 14:06:24 UTC 2014
#9647: allocation of 10790760 bytes too large
-------------------------------------+-------------------------------------
Reporter: mirko | Owner:
Type: bug | Status: new
Priority: normal | Milestone:
Component: Compiler | Version: 7.8.3
Keywords: run time | Operating System: Windows
insufficient memory | Type of failure: Incorrect
Architecture: x86_64 (amd64) | warning at compile-time
Difficulty: Unknown | Test Case:
Blocked By: | Blocking:
Related Tickets: | Differential Revisions:
-------------------------------------+-------------------------------------
I got an error message and invitation to submit a bug
report. So here is the error message first, followed
by my description of what happened.
C:\Users\Owner\Documents\hs>ghc -o void void.hs -rtsopts
[1 of 1] Compiling Main ( void.hs, void.o )
Linking void.exe ...
C:\Users\Owner\Documents\hs>void +RTS -K400M -RTS > void.txt
void: internal error: allocation of 10790760 bytes too large
(GHC should have complained at compile-time)
(GHC version 7.8.3 for x86_64_unknown_mingw32)
Please report this as a GHC bug:
http://www.haskell.org/ghc/reportabug
C:\Users\Owner\Documents\hs>void +RTS -K40000M -RTS > void.txt
void: internal error: allocation of 10790760 bytes too large
(GHC should have complained at compile-time)
(GHC version 7.8.3 for x86_64_unknown_mingw32)
Please report this as a GHC bug:
http://www.haskell.org/ghc/reportabug
Description of what happened.
I gave my program really large numbers.
It contains the following line:
rr = (of40)*(r250^3041)
In the above of40 is (if I remember correctly) the product
of the first 40000 odd primes. I computed it with reduce
(a computer algebra system) which saved the result (as an
explicitly written out number in decimal notation), which
I copied and inserted in my hs file, initializing of40 with
its value. In the file it takes 2457 rows, 1024 cols, so
it appears to be about 2 515 968 (that is two million+)
decimal digits. As seen it is multiplied by r250^3041,
here r250 is a 250-digit number so r250^3041 is about
760 250 (seven hundred sixty thousand+) decimal digits,
so the product rr = (of40)*(r250^3041) seems to be more
than 3 000 000 (three million+) decimal digits.
I tried to compile it yesterday, I did get an error
message from the compiler: I do not remember exactly
what it was, I may try to reproduce it later and include
it, but it was something along the lines of insufficient
amount of memory. I run a Hewlett-Packard HPE 500y,
upgraded to 16 Gb RAM (the max it would take), Windows 7,
AMD Phenom(tm) II X6 1045T Processor 2.70 GHz (6 core).
So yesterday after I got the first error message I increased
the paging file size from automatic to 32768 Mb initial, and
65536 Mb max. Then tried to compile again, the compiler
gradually took more and more memory (I was watching the performance
using the Task Manager) so it gradually went up to using
all 16 Gb of RAM, and stayed there for I think at least 40 min,
could have been more than an hour (or even 2 hours, sorry
can't quite remember). At some point it seemed the computer
froze, the task manager stopped showing any activity, but I
did not attempt to immediately restart it, the mouse would not
respond anyway, and in a few minutes it fixed itself, it
seemed to continue compiling, after which it eventually
gave up and complained about insufficient amount
of memory again. So this morning I increased
the paging file to initial size 49152 Mb,
max size 196608 Mb (I don't know if such big numbers
make sense but I tried). I tried to compile again
starting around 9am, left my apt at around 4pm and
the compiler had not yet finished, and came back after
midnight, the compiler had finished. It had produced
a big exe file (I see now, around 9 pm, so it must
have kept compiling for around 12 hours):
09/27/2014 09:38 PM 20,602,960 void.exe
AVG (antivirus) claimed that this file was a threat and
refused to let me run it, so I had to disable AVG,
after which I got the error messages as already
enclosed at the beginning of this description.
The Paging file reports as: Recommended 24574 Mb,
Currently allocated: 163765 Mb.
Note I gave the options -rtsopts to the compiler,
I do not remember anymore what they mean, and whether
they are valid (or perhaps not used anymore), I got
them from a webpage perhaps a couple of years ago
and at that time they made sense, so I just kept using them.
I also gave the option +RTS -K400M -RTS when I tried
to execute the compiled program, and then
+RTS -K40000M -RTS, both versions generated the error
messages shown above. I tend to believe that giving
Haskell such large numbers inside the .hs file explains
all of the above problems, it might even be a bit
surprising it went so far before it finally gave up.
But I do not really know, so let me just report it.
So I changed the paging file back to automatic, and
I am compiling the same file (after renaming it) to
try to reproduce the error message that I got yesterday
(or rather the other day, now being after midnight).
It has been compiling for about 4 min so far, nothing
really visible, memory usage stays at 1.28 Gb (where
it was before I started compiling). I don't quite remember
what happened the first time when I tried to compile,
I think I got the error message (about insufficient
amount of memory) immediately, or almost immediately.
This does not happen now, I wonder if the computer
took into account that I changed the paging file
back to automatic, I may need to restart to make sure.
Well, ok, 13 min after it started compiling, the memory
usage started going up gradually. It goes from e.g.
13.9 to 14 Gb in about a second or two. So in about
2 min it went from 1.28 to 15.9 Gb, and now stays there
(I do not know how to interpret all that Task Manager shows me).
I may not be able to reproduce the first error message:
At that time I was running 5 copies of reduce, each
busy multiplying numbers for some hours. Right now
I am only running the GHC by itself, so no other load.
It is running for about 23 min now, the last 6 of which
the memory usage stays at 15.9 Gb. My computer may
have learned to increase the paging file on its own.
The computer refuses to give up this time, but I can't
wait, going to bed is long overdue, if it generates an
error message I'll copy it in the morning. Well, memory
usage decreased a bit to 15.1, this happened after it stayed
at 15.9 for about 15 min. It I remember correctly what had
happened the other day, it will increase again, and then give
an error message. It's at 15.3 Gb for about 3 min now. On the
other hand the processor is doing almost nothing, going between
0% and 1%. Memory 15.4, 15.8, processor went up to 16% to 17%
(which is = 1 out of 6 cores),
memory 15.9, this is about 40 min after it started compiling
and about 23 min after it went up to 15.9 Gb (15.8 presently).
I should perhaps be patient and wait for the compiler to give up
and give me an error message that I could copy, but I do not have
the patience to do that: Again the original error message was
something to the effect that the compiler refused to work
because of insufficient amount of memory. I kept trying to
compile, so apparently the compiler tried to be nice to me,
and eventually compiled my program in 9 hours, after which
when I tried to execute it, I was invited to write this report.
It keeps compiling right now, at 16% (1 of 6 cores) processor,
and 15.9 Gb memory usage, for about 47 min (30 min at 15.9).
P.S. Thanks for Richard Eisenberg and Herbert Valerio Riedel
for replying to my email. I initially had problems, trying
to register and submit a new bug report, the BotScout
stopped me, and it was too late at night to try to figure it,
so I just emailed Richard in my frustration. Now, BotScout
may not be the culprit: I did not read carefully what
it asked me to do, and it was something like
Evaluate `(!!6) ...'
My knowledge of Haskell is rather superficial (I use it since it
compiles and is relatively fast, and does not need an extra library
to work with large numbers), so I have to often consult the
tutorial for simple constructs. I have had some previous
experience with Common Lisp (or Scheme, a course taken 12
years ago) and in Lisp there is a command "evaluate", and
also single quotation marks are used within the language
occasionally. So, I just copied the entire line
Evaluate `(!!6) (whatever it was)'
and pasted it in the interpreter which gave me some
nonsense which I sent back to BotScout. BotScout then
gave me a second chance, and I do not quite remember
if I tried to be more careful, perhaps not, so BotScout
wouldn't even let me try anymore after that (even if
I started from scratch ... perhaps it remembered me).
After that happened I copied only the (!!6), etc, and
the my GHC interpreter did evaluate something meaningful
(a single number, no error message). I just didn't want
to spend the time to think that late at night, and
at first it seemed to me that the word Evaluate
might have been part of a Haskell expression.
I question if the above (perhaps too lengthy) report
is of any use, but again, I follow the instruction,
if in doubt, submit it. Thank you again!
--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9647>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler
More information about the ghc-tickets
mailing list