HeapAlloc hangs CPU 100% - Please help

HeapAlloc hangs CPU 100% - Please help

Post by ruen » Fri, 08 Aug 2003 14:54:49


Hi,

We have new/delete operator overriden with HeapAlloc/HeapFree. (It's a long
story if you like to know why~ we didn't want to.)

The problem happens with recent update, that hangs up CPU 100% in HeapAlloc
sometimes.

We're sure that we're screwing up heap, so I made sample program that
deletes (HeapFree) twice a memory allocated by HeapAlloc. But, if that's
case, that should happen HeapFree, not in HeapAlloc. Anybody seen this
problem? It doesn't seem to be deleting-twice-problem, maybe memory overrun?

Thanks in advance,
 
 
 

HeapAlloc hangs CPU 100% - Please help

Post by Ivan Brugi » Fri, 08 Aug 2003 15:11:16

The Heap Manager keeps internally single-linked lists, double-linked lists
and implicit lists of blocks.
The 100% CPU can occur if you've made one of the double-linekd list
circular,
and you've fallen in the code path that searches the so called non-dedicated
list in the Alloc code path.

The standard way to attack a heap corruption is to
enable Full-PageHeap for your application,
and run your application under a de *** .

Download the systm de *** s from
http://www.yqcomputer.com/

Assuming they are installed in c:\de *** s, run gflags.exe to enable full
PageHeap for your application

c:\de *** s>gflags -i YourApp.exe +hpa

then run your usage scenario under de ***
(I would recomend using the cdb/ntsd/windbg de *** s,
since they provide de *** extensions to understand the heap structures).

PageHeap can expose a variety of problems:
for heap overrun, it will force an Access Violation on the spot.
for double delete, it will print a message in the de *** .
for reuse after delete, it will force an Access Violation on the spot.
for other problems it may not break on the spot,
but when the daly-free-queue is validated.

If you cannot make sense of the break, please
set up a system de *** with good symbols,
and report the output of the '~*kb' command to the newsgroup.


--
This posting is provided "AS IS" with no warranties, and confers no rights.
Use of any included script samples are subject to the terms specified at
http://www.yqcomputer.com/




long
HeapAlloc
overrun?

 
 
 

HeapAlloc hangs CPU 100% - Please help

Post by Ivan Brugi » Sat, 09 Aug 2003 04:58:47

could write 10 lines of code that causes that,
but they would not make you to get any closer to the solution of the
problem.
This looks like a classical corruption caused by a race condition.
BoundsChecker may introduce enough slow-down and
serialization that may not help you catch the problem.
Full PageHeap may have the same (serializaiton and slow-down) problem as
well,
in this case (nothing shows up with page-heap) I can show a couple of trick
to have the maximum speed and still some level of instrumentaiton.
BTW, which OS and Service Pack are you using ?
Please use cdb/ntsd instead of DevEnv.exe, since they are less invasive and
more efficient.

--
This posting is provided "AS IS" with no warranties, and confers no rights.
Use of any included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm


"ruen" < XXXX@XXXXX.COM > wrote in message
news:# XXXX@XXXXX.COM ...
debugger),
in
lists
full
structures).
that's


 
 
 

HeapAlloc hangs CPU 100% - Please help

Post by nikolobi » Mon, 11 Aug 2003 02:27:45

i,

Sorry for late. I had personal problem. Still I'm away from work, too.
We're using w2000 server and latest service pack. (Is there any
special reason you asked this?) Database is MSSQL, and we're using
JNI(Javascript communicating with C++), and we have multiple server
communicating via DCOM. I don't think any of this configuration has
problem since it has been stable for too long. A change we made
recently caused this problem.

Since I'm away, I can't answer what you wanted me to try now, but I
will try as soon as I get back. If the problem is solved by the time I
get back, then I'll let you know what it was, too.

Thanks a lot,

--ruen

"Ivan Brugiolo [MSFT]" < XXXX@XXXXX.COM > wrote in message news:< XXXX@XXXXX.COM >...
 
 
 

HeapAlloc hangs CPU 100% - Please help

Post by Ivan Brugi » Mon, 11 Aug 2003 05:33:14

was asking about your environment because PageHeap features have evolved
in time,
and the W2K ones do not have all the options we have in W2K3
and the ones we are working on moving forward.
Still, it can provided some value in your case.
One other thing you may want to check is a component that
links the non multithreaded static version of the C-Runtime.
May sound a stupid advice, but sometimes few compiler/linker
switches slip into a mainstream product.

--
This posting is provided "AS IS" with no warranties, and confers no rights.
Use of any included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm

"ruen" < XXXX@XXXXX.COM > wrote in message
news: XXXX@XXXXX.COM ...
news:< XXXX@XXXXX.COM >...
trick
and
rights.
through
with it
some
new
same
later.
circular
post
message
enable
spot.
specified at
(It's a
that
this
memory


 
 
 

HeapAlloc hangs CPU 100% - Please help

Post by alexp1111 » Sat, 30 Aug 2003 12:25:36

hanks for the information Ivan. Unfortunately no luck finding this
problem with Pageheap although it caught one other memory overwrite.
We are trying to get some ideas of places in our code that could be
causing this type of heap corruption. Maybe if you showed is this 10
line example you mentioned we could better understand the type of code
that could produce this 100% situation with the linked list corruption
you described. Also, the tricks to get maximum speed would help since
it was running slow once the server load went up a bit.

I'm curious what made you suspect this was caused by a race condition
?

We are running our server on Windows 2000 with the latest service
packs. Using Visual C++ 6.0 SP5, JNI and COM.


Thanks very much,
Alex


"Ivan Brugiolo [MSFT]" < XXXX@XXXXX.COM > wrote in message news:< XXXX@XXXXX.COM >...
 
 
 

HeapAlloc hangs CPU 100% - Please help

Post by Ivan Brugi » Sat, 30 Aug 2003 12:35:09

f PageHeap does make the problem go away
(after having exposed overrun, that's one of the best feattures of the
PageHeap code),
it's most likely a race condition.
PageHeap forces heap synchronization even when the original heap code did
not have it.
Could you please make sure none of you heap have the HEAP_NO_SYNCHRONIZE bit
set?
Most of the time it's cause by linking the static non-multithreaded version
of the C-Runtime.

--
This posting is provided "AS IS" with no warranties, and confers no rights.
Use of any included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm


"Alex" < XXXX@XXXXX.COM > wrote in message
news: XXXX@XXXXX.COM ...
news:< XXXX@XXXXX.COM >...
trick
and
rights.
through
with it
some
new
same
later.
circular
post
message
enable
spot.
specified at
(It's a
that
this
memory


 
 
 

HeapAlloc hangs CPU 100% - Please help

Post by Ivan Brugi » Sat, 30 Aug 2003 12:48:24

n the side,
if you set
HKLM\Software\Microsoft\Windows NT\CurrentVerstion\Image File Execution
Options\YourApp.exe
PageHeapFlags = REG_DWORD 0x1
you will have full pageheap without stack backtraces (this may save some CPU
cycle)
If you set PageHeapFlags to zero you will have less memory usage and fewer
checks.
I will go after the HEAP_NO_SERIALIZE first, anyway.

--
This posting is provided "AS IS" with no warranties, and confers no rights.
Use of any included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm


"Ivan Brugiolo [MSFT]" < XXXX@XXXXX.COM > wrote in message
news: XXXX@XXXXX.COM ...
bit
version
rights.
as
invasive
at
most
always
I'll
double-linked
list
no
in
if
seen


 
 
 

HeapAlloc hangs CPU 100% - Please help

Post by dvotypka_b » Thu, 04 Sep 2003 07:44:37

Hi Ivan,

Me, Alex and ruen all work together. We are undergoing serious time
pressure in getting these issues resolved. I tried e-mailing you but
it seems the address doesn't exist. I am wondering if you would be
interested in some consulting work? Or alternatively, if you had a
recommendation on someone qualified with low level kernel, system and
memory coding and debugging that may assist us in clearing up these
issues?

You can e-mail me at the above address and I will respond from my
proper e-mail address.

Thanks--
Dave
 
 
 

HeapAlloc hangs CPU 100% - Please help

Post by Ivan Brugi » Thu, 04 Sep 2003 08:17:43


One other way to attack the problem is to run the
!heap -a -h <heap_handle>
de *** extension command on all the heaps in the process,
and see if there is some complain / error message around.

A stack of the thread that is spinning would be a good starting place as
well.

--
This posting is provided "AS IS" with no warranties, and confers no rights.
Use of any included script samples are subject to the terms specified at
http://www.yqcomputer.com/