STL Associative Containers - Problems with deallocating memory

STL Associative Containers - Problems with deallocating memory

Post by SmVuczc » Wed, 02 Jul 2008 01:35:01


Hi,

In our software we are using Associative Containers
to store a huge amount of items in some situations.

For example at one point we are using a STL set to store
a huge amount of strings.
Memory used by this STL set is often more than 200 MB.

When the set goes out of scope, all memory used by the set gets deallocated
automatically.

Using a Memory-Tool shows that all the deallocated memory
ends up in "Reserved Memory" and not in "Free Memory".

STL map shows the same effect.


Here is some sample code:

int main()
{
{

// At this point reserved memory is 3 MB

std::set<std::string> strSet;

for (int i=0; i<2000000; i++)
{
char tmp[32];
sprintf(tmp,"ABCDEFG1%010d",i);
std::string str (tmp);
strSet.insert(str);
}
}

// At this point reserved memory is 258 MB

return 0;
}


Such huge STL Containers only needed a few times in our software.
Mostly associative containers with moderate sizes are needed.

The problem is:
At this point there are several hundred MB of memory
reserved by the system for future use (with associative containers?).
The reserved memory is not available for other operations.

Is there a way to deallocate the memory
,used by a Associative Container,
so that memory is given completely back
to the "Free Memory" ?


Thanks for any help!
 
 
 

STL Associative Containers - Problems with deallocating memory

Post by Stephen Ho » Wed, 02 Jul 2008 03:39:52

> Is there a way to deallocate the memory

No. It has nothing to do with containers.
It is do with the heap.

If you malloc() 300Mb of memory then free() it, the memory is returned to
the heap manager, waiting to be recycled, and _NOT_ back to the OS. It is
like this so the act of allocating/deallocating in a loop does not
constantly go back and forwards to the OS (slow) but instead is recycled by
the heap (fast). That is also true for new/delete.
There is no way the heap can "know" when a cycle of memory
allocation/deallocation is over and unused memory can be returned to the OS.

Only if the compiler vendor has something like _heapshrink() as part of the
C library, called by the programmer at the right time, is unused memory
returned back to the OS.

Stephen Howe

 
 
 

STL Associative Containers - Problems with deallocating memory

Post by SmVuczc » Thu, 03 Jul 2008 01:36:03


OK, I understand.
Thanks for your answer.

Jens
 
 
 

STL Associative Containers - Problems with deallocating memory

Post by Triple-DE » Fri, 04 Jul 2008 15:51:12


(MS specific)
Try the _heapmin() function, it is supposed to release unused heap
memory to the OS.

DP
 
 
 

STL Associative Containers - Problems with deallocating memory

Post by SmVuczc » Sun, 06 Jul 2008 17:18:00


Thank you.
I will try this function.

Jens