I have a multithreaded WPF application that is using > 600 threads after is has been running for more than 8 hours. All but approximately 10 of these threads have a stack trace that is very similar to this:
Stack trace 1:
ntkrnlpa.exe!NtInitialUserProcessBuffer+0x7b
ntkrnlpa.exe!MiAddWorkingSetPage+0x174
ntkrnlpa.exe!MiAddWsleHash+0x12a
ntkrnlpa.exe!PopSystemButtonHandler+0x141
ntkrnlpa.exe!KiInterruptTemplate+0x62
ntdll.dll!KiFastSystemCallRet
ntdll.dll!ZwWaitForMultipleObjects+0xc
KERNEL32.dll!WaitForMultipleObjectsEx+0x12c
mscorwks.dll!WaitForMultipleObjectsEx_SO_TOLERANT+0x6f
mscorwks.dll!Thread::DoAppropriateAptStateWait+0x3c
mscorwks.dll!Thread::DoAppropriateWaitWorker+0x13c
mscorwks.dll!Thread::DoAppropriateWait+0x40
mscorwks.dll!WaitHandleNative::CorWaitOneNative+0x156
mscorlib.ni.dll+0x1f68af
mscorlib.ni.dll+0x1caa17
WindowsBase.ni.dll+0x24ac34
WindowsBase.ni.dll+0x2aeb1e
WindowsBase.ni.dll+0x9445d
WindowsBase.ni.dll+0x9267f
mscorwks.dll!JITutil_IsInstanceOfAny+0x106
mscorlib.ni.dll+0x1e842f
mscorwks.dll!CallDescrWorker+0x33
mscorwks.dll!CallDescrWorkerWithHandler+0xa3
mscorwks.dll!MethodDesc::CallDescr+0x19c
mscorwks.dll!MethodDesc::CallTargetWorker+0x1f
mscorwks.dll!MethodDescCallSite::Call+0x1a
mscorwks.dll!ExecuteCodeWithGuaranteedCleanupHelper+0x9f
mscorwks.dll!ReflectionInvocation::ExecuteCodeWithGuaranteedCleanup+0x10f
mscorlib.ni.dll+0x235677
mscorlib.ni.dll+0x2202a5
mscorlib.ni.dll+0x1e839b
mscorwks.dll!CallDescrWorker+0x33
mscorwks.dll!CallDescrWorkerWithHandler+0xa3
mscorwks.dll!DispatchCallBody+0x1e
mscorwks.dll!DispatchCallDebuggerWrapper+0x3d
mscorwks.dll!DispatchCallNoEH+0x51
mscorwks.dll!AddTimerCallback_Worker+0x66
mscorwks.dll!Thread::DoADCallBack+0x32a
mscorwks.dll!Thread::ShouldChangeAbortToUnload+0xe3
mscorwks.dll!Thread::ShouldChangeAbortToUnload+0x30a
mscorwks.dll!Thread::ShouldChangeAbortToUnload+0x33e
mscorwks.dll!ManagedThreadBase::ThreadPool+0x13
mscorwks.dll!AddTimerCallbackEx+0x83
mscorwks.dll!AddTimerCallback+0x10
mscorwks.dll!ThreadpoolMgr::AsyncTimerCallbackCompletion+0x64
mscorwks.dll!UnManagedPerAppDomainTPCount::DispatchWorkItem+0x9a
mscorwks.dll!ThreadpoolMgr::ExecuteWorkRequest+0xaf
mscorwks.dll!ThreadpoolMgr::WorkerThreadStart+0x20b
mscorwks.dll!Thread::intermediateThreadProc+0x49
KERNEL32.dll!BaseThreadStart+0x37
Stack Trace 2:
ntkrnlpa.exe!NtInitialUserProcessBuffer+0x7b
ntkrnlpa.exe!MiAddWorkingSetPage+0x174
ntkrnlpa.exe!MiAddWsleHash+0x12a
ntkrnlpa.exe!PopSystemButtonHandler+0x141
ntkrnlpa.exe!KiInterruptTemplate+0x62
ntdll.dll!KiFastSystemCallRet
ntdll.dll!ZwWaitForMultipleObjects+0xc
KERNEL32.dll!WaitForMultipleObjectsEx+0x12c
mscorwks.dll!WaitForMultipleObjectsEx_SO_TOLERANT+0x6f
mscorwks.dll!Thread::DoAppropriateAptStateWait+0x3c
mscorwks.dll!Thread::DoAppropriateWaitWorker+0x13c
mscorwks.dll!Thread::DoAppropriateWait+0x40
mscorwks.dll!WaitHandleNative::CorWaitOneNative+0x156
mscorlib.ni.dll+0x1f68af
mscorlib.ni.dll+0x1caa17
WindowsBase.ni.dll+0x24ac34
WindowsBase.ni.dll+0x2aeb1e
WindowsBase.ni.dll+0x9445d
WindowsBase.ni.dll+0x9267f
mscorwks.dll!JITutil_IsInstanceOfAny+0x106
mscorlib.ni.dll+0x1e842f
mscorwks.dll!CallDescrWorker+0x33
mscorwks.dll!CallDescrWorkerWithHandler+0xa3
mscorwks.dll!MethodDesc::CallDescr+0x19c
mscorwks.dll!MethodDesc::CallTargetWorker+0x1f
mscorwks.dll!MethodDescCallSite::Call+0x1a
mscorwks.dll!ExecuteCodeWithGuaranteedCleanupHelper+0x9f
mscorwks.dll!ReflectionInvocation::ExecuteCodeWithGuaranteedCleanup+0x10f
mscorlib.ni.dll+0x235677
mscorlib.ni.dll+0x2202a5
mscorlib.ni.dll+0x1e839b
mscorwks.dll!CallDescrWorker+0x33
mscorwks.dll!CallDescrWorkerWithHandler+0xa3
mscorwks.dll!DispatchCallBody+0x1e
mscorwks.dll!DispatchCallDebuggerWrapper+0x3d
mscorwks.dll!DispatchCallNoEH+0x51
mscorwks.dll!AddTimerCallback_Worker+0x66
mscorwks.dll!Thread::DoADCallBack+0x32a
mscorwks.dll!Thread::ShouldChangeAbortToUnload+0xe3
mscorwks.dll!Thread::ShouldChangeAbortToUnload+0x30a
mscorwks.dll!Thread::ShouldChangeAbortToUnload+0x33e
mscorwks.dll!ManagedThreadBase::ThreadPool+0x13
mscorwks.dll!AddTimerCallbackEx+0x83
mscorwks.dll!AddTimerCallback+0x10
mscorwks.dll!ThreadpoolMgr::AsyncTimerCallbackCompletion+0x64
mscorwks.dll!UnManagedPerAppDomainTPCount::DispatchWorkItem+0x9a
mscorwks.dll!ThreadpoolMgr::ExecuteWorkRequest+0xaf
mscorwks.dll!ThreadpoolMgr::WorkerThreadStart+0x20b
mscorwks.dll!Thread::intermediateThreadProc+0x49
KERNEL32.dll!BaseThreadStart+0x37
Application uses System.Threading.Timer to periodically poll for data from several webservices using a WCF client proxy and at any giving time, could be making about 20 of these requests at the same time. Each call to a webservice instantiates a new proxy instance but the client is always closed when a response is received from the webservice.
Application also manipulates with bitmaps for a GIS and this is also done on a periodic interval. No where in code am I explicitly creating threads besides localized usage of the Timer class to poll for data periodically. The GIS does
use the BackgroundWorker but they do limit the thread count.
Anyone have an idea on what is spawning these new threads and why they are not being disposed?
TIA.
Yeah, looks like something you shouldn't ignore. They are the threadpool thread that the Timer class uses to make the callback. They are deadlocked, looks like they are waiting for a method call that's marshaled by COM to complete. There should be another thread in your program, one of the other 10 on which you created the GIS object. That thread is not pumping a message loop, a hard requirement for an STA thread that creates single apartment threaded COM components. Or it is stuck itself, not re-entering the message loop. Getting a managed stack trace ought to make it easier to see where the thread is stuck.
Trying to use threads on a COM object that explicitly doesn't support them (very few do) is pointless. Be sure to create the GIS object on your program's main UI thread. And use a DispatcherTimer. Creating your own STA thread that pumps a message loop can be a solution when the GIS component is taking too much of a hit on your user interface.
Problem was not GIS related but was due to System.Threading.Timer call backs queuing up as they were being called faster than they could be actioned. Problem was further compounded by the fact that timer call back was doing a Dispatcher.Invoke to the main user interface thread to refresh data grids. Dispatcher.Invoke was blocking when main user interface was busy, say when user was panning and zooming on map, and before timer call back got a chance to complete it was invoked again.
Fixed problem by temporarily stopping timer when it starts executing the call back and restarting it again after call back is complete. Also, instead of doing a Dispatcher.Invoke to get my grids updated, I changed it to a Dispatcher.BeginInvoke to avoid blocking.
Related
I am building a GUI (using C) which receives data to display from another application sending the data over a TCP socket. How do I do this using GTK (just a general overview of the approach I should take)? I have done a lot of searching and came across stuff about multithreading, GIOchannel etc. now I'm more confused than ever. There doesn't seem to be any conclusive articles or guides about how to actually achieve this.
There is basically one important rule:
You must call all gtk_* functions from the main thread.
If you update any widgets from another thread, you might get inconsistent results.
Of course, you don't want to wait for TCP data in that thread.
Therefore I would suggest you create a separate thread for doing the communication. In this thread you can wait for data and if you got anything that should affect what you show in your GUI, you can tell the main thread to do the required work.
A simple way to do this is to use g_idle_add() to enqueue a callback function. That callback function is then executed in context of main thread and can update your widgets.
The information what needs to be updated can be stored in some newly allocated memory that is passed to this callback where you have to free it afterwards.
I have an WPF application which contains many classes that support INotifyProprtyChange interface. The property on this classes are changed at high rate from different threads and the problem is that in some cases application UI thread hangs , along with the UI thread other threads that have code calling PropertyChange also hangs while trying update properties.
I did a deadlock check with Windbg but it could detect none.
Now i looked at the stack trace of the threads holding locks and what i always see is that at least one same or different thread has
00000000002d0ab8 000000007712908a [HelperMethodFrame_1OBJ: 00000000002d0ab8] System.Threading.SynchronizationContext.WaitHelper(IntPtr[], Boolean, Int32)
00000000002d13b0 000000007712908a [GCFrame: 00000000002d13b0]
00000000002d16c8 000000007712908a [HelperMethodFrame_1OBJ: 00000000002d16c8] System.Threading.ReaderWriterLock.AcquireReaderLockInternal(Int32)
00000000002d17f0 000007fef1bcf1a0 MS.Internal.ReaderWriterLockWrapper.get_ReadLock()
00000000002d1830 000007fef1ba8d44 System.ComponentModel.PropertyChangedEventManager.OnPropertyChanged(System.Object, System.ComponentModel.PropertyChangedEventArgs)
and another
000000001f64d138 000000007738186a [HelperMethodFrame_1OBJ: 000000001f64d138] System.Threading.ReaderWriterLock.AcquireReaderLockInternal(Int32)
000000001f64d260 000007fef1bcf1a0 MS.Internal.ReaderWriterLockWrapper.get_ReadLock()
000000001f64d2a0 000007fef1ba8d44 System.ComponentModel.PropertyChangedEventManager.OnPropertyChanged(System.Object, System.ComponentModel.PropertyChangedEventArgs)
So its looks like PropertyChangedEventManager wait for some thread to complete while other threads wait to acquire lock?
Now i am having troubles figuring out why this occurs as its sporadic and usually happens when there is many property change events. From what i see other threads in application that not calling property change continue to run normally.
As far as I know PropertyNotifyChanged is only supported in the UI thread.
Just guessing. In bigger environments one problem happends quiet easy: 2 properties update each other endlessly. The setter of prop A calls the setter of prop B which in turn calls the setter of A again. Usually of course not that obvious as described but could be worth to look at.
I checked which thread my Dispose(bool) methods get called on. When the app is running, it is always the UI thread that calls Dispose, say when clicking on the [x] to close a Form. But when I close the whole app, many Dispose methods get called on a (single) different thread. When I dump the stack trace, I see that they all get called from
System.ComponentModel.Component.Finalize().
Does that mean all my Dispose methods need to be made thread-safe? Or is WinForms guaranteeing that the UI thread won't touch these objects any more and does it also establish some kind of "happened-before" relationship between the UI thread and the one that's now finalizing?
Yes, the finalizer works on a separate thread. Usually this is no problem, because when an Object is finalized it is not reachable by any user thread (like the UI thread) anymore. So, you usually do not have to be thread-safe within your finalizer.
Why i should call Control.Invoke from non-ui thread? As i know any manipulations with control are the messages to control. So when i call, for example TextBox.Text = "text", it will produce message SendMessage(TextBox.Hanlde...). That message will be queued into UI thread message queue and dispatched by UI thread. Why i have to call invoke, even it will produce the same message?
There are two reasons MS developers made this restriction:
Some UI functions have access to thread-local storage (TLS). Calling these functions from another thread gives incorrect results on TLS operations.
Calling all UI-related functions from the same thread automatically serializes them, this is thread-safe and doesn't require synchronization.
From our point of view, we just need to follow these rules.
Because you cannot directly access UI controls from threads other than the thread they were created on. Control.Invoke will marshal your call onto the correct thread - allowing you to make a call from another thread onto the UI thread without needing to know yourself what the UI thread is or how to perform the marshalling.
Update: to answer your question, you don't have to use Control.Invoke - if you have code to marshal your call onto the correct thread and post a message to the message pump - then use that. This, however, is known as re-inventing the wheel. Unless you are doing something that changes the behaviour.
Is there a (or, do you have your own) preferred way to do background processing in slices on the UI thread in Windows Forms? Like OnIdle() in MFC?
In native Windows programming you could roll your own message loop to do this, but Application.Run() doesn't give us access to the message loop.
The Application.Idle event gives us no way to trigger it repeatedly.
I guess you could call native PostMessage() with P/Invoke (since there's no managed version) to post yourself a private "WM_IDLE" message, and override WndProc() to catch it. I don't know how this would get along with Application.Run().
So far I've used a short Timer for this, but I'm afraid I may be losing cycles sleeping, especially since the actual Timer resolution is coarser than the nominal 1 ms minimum.
The best option I've seen is to use a modified version of the Managed DirectX Render Loop designed by Tom Miller. By adding a call to Thread.Sleep() inside the render loop, you can pull your CPU usage down dramatically.
This does require a P/Invoke call to track that the application is still idle, but as long as it's idle, you can make a "timer" that fires continuously during the idle phases, and use that to do your processing.
That being said, on modern systems, you almost always have extra cores. I would suggest just doing the processing on a true background thread.
I thought of my own possible answer, inspired by Reed's talk of multithreading. I may have a way to retrigger Application.Idle:
Create a hidden form, let's call it formRetrigger.
In Application.Idle, launch my Retrigger() method on a thread pool thread.
Retrigger() calls formRetrigger.InvokeOnClick() (or any of the other "Control.Invoke" methods). I expect this to launch another message through Application's queue, causing Idle to get triggered again.