[polyml] Heap does not grow up to --maxheap

[Cezary Kaliszyk]

Dear all,

I have a PolyML program (below), for which maxheap is ignored:
The heap is kept at the minheap size and does not grow automatically.

For the three runs below the maxheap is set to the same amount (4GB):

If I run:
polyml --maxheap 4G --gcthreads 1 --minheap 4G < polybug.ml
Terminates in less than 1 sec.

If I run:
polyml --maxheap 4G --gcthreads 1 --minheap 500M < polybug.ml
It does not terminate (at least not in 15 minutes)

If I run:
polyml --maxheap 4G --gcthreads 1 --minheap 50M < polybug.ml
I get 'Run out of store' exception.

Does the heap not grow automatically up to maxheap? Is there a way
to make PolyML automatically adjust the heap size?

Tested both with 64bit polyml-5.5.0 (from Isabelle2013).
And with current repository version (r1702).

Regards,

Cezary

--
datatype tree = Node of tree * tree | Leaf of int;

fun go acc (Leaf _) = acc + 1
| go acc (Node (l, r)) = go (go (acc + 1) l) r;
fun size t = go 0 t;

fun build i j =
if i <= 0 then Leaf j
else Node (build (i-1) (2*j), build (i-2) (2*j+1));

val tim = ref (Timer.startRealTimer ());
fun tim_start () = tim := Timer.startRealTimer ();
fun tim_print () = print (Time.toString (Timer.checkRealTimer (!tim)) ^ "\n");

val t = build 0 0;
val al = 10000000;
val a = Array.array (al, t);

tim_start ();
fun irep i = if i = al then tim_print () else
let val _ = Array.update (a,i,build 0 i) in irep (i + 1) end;
irep 0;
irep 0;
irep 0;
irep 0;
irep 0;

[David Matthews]

Hi Cezary,
Thanks for reporting that and producing a small example. It's always
much easier to work with something like that. I've had a look at it and
I think there are actually several different bugs that are showing up
but it all seems to revolve around the very large array. The intention
is that the heap should grow and indeed it does with other examples.
I'll need to look into this in more detail and try and fix the bugs. In
the meantime you may be able to work around the problems by using a
different structure instead of the single array.

Regards,
David

Post by Cezary Kaliszyk
Dear all,
The heap is kept at the minheap size and does not grow automatically.
polyml --maxheap 4G --gcthreads 1 --minheap 4G < polybug.ml
Terminates in less than 1 sec.
polyml --maxheap 4G --gcthreads 1 --minheap 500M < polybug.ml
It does not terminate (at least not in 15 minutes)
polyml --maxheap 4G --gcthreads 1 --minheap 50M < polybug.ml
I get 'Run out of store' exception.
Does the heap not grow automatically up to maxheap? Is there a way
to make PolyML automatically adjust the heap size?
Tested both with 64bit polyml-5.5.0 (from Isabelle2013).
And with current repository version (r1702).
Regards,
Cezary

[David Matthews]

I've committed some changes to SVN trunk that should deal with the
problems you found. Typical ML programs allocate large numbers of small
cells and the code didn't deal properly with a 10 million element array.

I don't know how much this was real code and how much you cut down for
the example. It's generally better to avoid using too much mutable data
such as arrays or refs if that's possible.

Regards,
David