Use of psRealloc() expands the size of the memory footprint

[Paul Price]

This was originally noticed in the psTrace code (because it's called
repeatedly), which called psStringAppend. psStringAppend would allocate an
empty string, and then psRealloc to the proper size once it was known. I've
fixed this behaviour so that it only calls psRealloc if necessary, but that does
not eliminate the problem with psRealloc.

I will attach code to reproduce the problem. Note how the values returned by
sbrk(0) change --- they should stay the same if psRealloc() was working
properly. To prove this, exchange gcc_memory() for ps_memory() in the loop and
see that the numbers stay the same.

[Paul Price]

Program to reproduce the bug in psRealloc()

[Paul Price]

I expect that the source of this problem is described in bug 803.

* This bug has been marked as a duplicate of 803 *

[Paul Price]

Yes, the fix for bug 803 also results in constant values from sbrk(0) in the
attached program.

[Paul Price]

The fix from bug 803 worked, but was slow. I also don't believe that it is the
correct diagnosis.

The important point to note is that in the attached code, I'm allocating from
one bin, reallocating into a different bin, and then going over and over and
over. What this means is that I am moving any pointers that were stored in the
first bin into the second bin, so that when I want to get something from the
first bin, there's nothing there and it needs to malloc more memory. All of
this memory then piles up in the second bin, and my memory footprint grows and
grows.

Basically, the bottom line is that the use of the current recycling
mechanism assumes that the spectrum of memory sizes is flat in log space
(so that you get a steady state, as you pointed out). When this
assumption is violated (as above, by continually demanding a certain
size that is not returned as the same size), then all bets are off, and
the memory footprint will keep expanding.

This assumption can be enforced by setting a maximum limit to the number in any
recycle bin. I've rolled back the psAlloc + copy fix from bug 803, and
implemented this enforcement; code checked in to CVS head.

I also did some benchmarks and found that the use of the recycle bins really
isn't doing much for us in terms of time. That is to say, the system malloc (at
least, on my linux box) is even better at recycling than the PS memory system.
So after discussion with Josh and Gene, I have #ifdef-ed out the recycling code.

If you want to run with memory recycling, compile with "PS_MEM_USE_RECYCLE"

defined.

Using the fix applied above and running the attached code results in correct
(i.e., non-growing values) for psLib, both with and without recycling enabled.