In Event Tracing for Windows, StartTrace accepts an EVENT_TRACE_PROPERTIES structure that allows for a FlushTimer which specifies how frequently unfull buffers should be flushed.
The thing is, FlushTimer is a ULONG representing seconds, but I want it to be very small so that it's nearly instantaneous (on the order of milliseconds).
I don't know how Process Monitor manages to get ETW events in real-time, but it does, so surely there must be a way to do it.
So the question is: How can I receive real-time events, you know, in real time?
ETW does not support real time notifications. Even the so-called EVENT_TRACE_REAL_TIME_MODE isn't really real-time as the documentation clearly says.
The premise of your question is wrong: Sysinternals Process Monitor does not use ETW to get its synchronous kind-of real-time process, thread, module, file and Registry events. You've got two options:
Use ETW - which is not what ProcMon does - and get events they way ETW provides them to you.
Do what ProcMon does - which is not consume ETW events - and get events synchronously, like ProcMon gets them.
Related
I have a piece of code which acts as a server and is switching different valves and other actuators. The commands for switching those actuators are being received via sockets from clients running on Android devices. Now Id like to add a module to my server which would allow me to schedule an calendar dependant switching of those actuators. I wonder how to do that.
Right now I am thinking of time.h of course, as the very basic. Then I could imagine to have a stucture which holds the programmable events (time, date and what to trigger) and a task which would endlessly compare the time/date to the actual time.. However I am not really sure if that is the right approach. Please hint me in the right direction.
What would the basic architecture look like?
Are there libraries in C I could youse?
Options include:
setitimer() to set next signal time, and using a signal handler.
having a loop, sleeping, polling and checking system time against time for the schedule.
I'm implementing some form of internal profiler. Is there a way to know when and for how long a thread is context switched out? I know windows has it w the event tracing api and I know perf logs how many context switches happens. Is there a way to do it on linux? Needing root privileges is not an issue since it will be an internal tool.
Sort of.
See http://man7.org/linux/man-pages/man2/getrusage.2.html about the getrusage() function.
Notice that the structure it returns has voluntary and involuntary context switch numbers. Also, you have user and system time. Other APIs return wall-clock time.
Any wall-clock time greater than your user and system time is time you weren't running.
Other than that, you could probably use the kernel ftrace ability. See https://www.kernel.org/doc/Documentation/trace/ftrace.txt
Read http://www.brendangregg.com/blog/2015-07-08/choosing-a-linux-tracer.html for even more options.
I've been looking at Windows's File System Filter Drivers. I started with this "FsFilter" example:
http://www.codeproject.com/Articles/43586/File-System-Filter-Driver-Tutorial
With effort, I managed to get it built and signed in versions that work on everything from 64-bit Win8 to 32-bit WinXP. (Well, as long as I run Bcdedit.exe -set TESTSIGNING ON to allow it to accept my test certificate, since I didn't pay Microsoft $250 to sign my .SYS file. :-/)
Now I want to modify FsFilter. I'd like write accesses to certain types of files to be trapped by the filter. I then want the user to receive a dialog box, in which they can either allow the access or deny it.
Perhaps obviously...the kernel-mode code cannot display the UI. It will have to signal some user mode process, which will (after an arbitrarily latent period of time) signal back the user's wish to the driver. I've looked a bit over
User-Mode Interactions: Guidelines for Kernel-Mode Drivers (here's Google's Cache as HTML, instead of .DOC)
I don't know what the best way to attack this is. The only example I've yet found to study is SysInternals FileMon. The driver it installs gathers data in a buffer, which is periodically requested by the .EXE according to a WM_TIMER loop:
// Have driver fill Stats buffer with information
if ( ! DeviceIoControl( SysHandle, IOCTL_FILEMON_GETSTATS,
NULL, 0, &Stats, sizeof Stats,
&StatsLen, NULL ) )
{
Abort( hWnd, _T("Couldn't access device driver"), GetLastError() );
return TRUE;
}
Should I use a similar technique? Perhaps the filter driver, upon receiving a request it wants to check, could place a record to track the request in a buffer that would contain two HEVENTs. It would then WaitForMultipleObjects on these two HEVENTs, which represent a signaled "YES" or "NO" from user mode on whether to allow access.
Periodically the monitor process (running in user mode) will poll the driver from another thread using a custom IOCTL. The filter driver would return the request information... as well as the two HEVENTs that request is waiting on. The monitor would wait for the user's feedback, and when available signal the appropriate event.
I could also invert this model. The user mode code could use a custom IOCTL to pass in data... such as HEVENTs which could be signaled by the driver, and just implement some kind of safe protocol. This would eliminate the need for polling.
Basically just looking for guidance on method, or a working example on the web! I'd also be interested to know what the mechanics would be on an asynchronous file access. I assume there's a way so a client making an async call that is being checked could keep running and only be held up when they waited on the request to finish...?
(Note: Along the way of getting the filters built and debugged, I learned there are some more modern techniques via "miniFilters"--which are part of something called the Filter Manager Model. But for the moment, I'm not that concerned as long as the legacy model is supported. It looks rather similar anyway.)
You (a.k.a. I) have pretty much enumerated the possibilities. Either poll the way FileMon does, or pass an event. Passing the event is probably a bit more error prone, and if you aren't a threading guru then there's probably more chance for error. But if you tend to make lots of mistakes then device drivers may not be for you...skydiving might be a poor choice too.
I'll offer taking a look at this project, but please note the disclaimers in the README. (It is only a test and investigation):
https://github.com/hostilefork/CloneLocker
And yes, to the extent that Microsoft and their driver model is to be something one worries about, miniFilters are the better choice these days.
I'm using PortAudio as a front-end to a speech synthesis (Text to Speech) engine, and I want to provide a synchronous speak function that waits until playback has completed.
It seems like all of the PortAudio functions that deal with this only wait until the underlying API has finished consuming the audio data, not until playback has finished.
Is this possible with PortAudio? If not, are there any good cross-platform alternatives to PortAudio (has to include a C interface) that might support this?
I am not sure if the streamFinished callback, as documented here:
http://portaudio.com/docs/v19-doxydocs/portaudio_8h.html#aa11e7b06b2cde8621551f5d527965838
is what you want. It may suffer from the same issue, but I think it would work.
Two other possibilities are:
Use lower latency settings.
Use the hardware timing. This information is available from calls like GetStreamTime(). For example:
get the current time
push x seconds of audio to the hardware
wait for the hardware clock to show the start time plus x seconds
You might also be interested in this document:
http://www.rossbencina.com/static/writings/portaudio_sync_acmc2003.pdf
I'm afraid I don't know of another API with better support for this sort of thing.
How can I determine if an ETW session is dropping events?
If it is dropping events, how can I configure the tracing session so that events are not dropped?
I've written a custom ETW provider to help with some debugging efforts. I'm currently capturing the trace data using logman.exe.
In viewing the results, it appears that some of the events are being dropped. Basically I'm seeking something like:
Event A
Event C
where their should be an intervening Event B, but one does not appear in the trace file. It should be impossible for that to happen, which leads me to believe that ETW is dropping events.
Of course, I'd like to verify that the problem I'm seeing is due to dropped events, and not caused by a bug in my code. I've tried Google, but wasn't able to come up with anything. Does any one know how I can check to see if events are being dropped?
It doesn't answer the question directly (how to detect drops), but it might explain drops:
EVENT_TRACE_NO_PER_PROCESSOR_BUFFERING
Writes events that were logged on different processors to a common
buffer. Using this mode can eliminate the issue of events appearing
out of order when events are being published on different processors
using system time. This mode can also eliminate the issue with
circular logs appearing to drop events on multiple processor
computers.
If you do not use this mode and you use system time, the events may
appear out of order on multiple processor computers. This is because
ETW buffers are associated with a processor instead of a thread. As a
result, if a thread is switched from one CPU to another, the buffer
associated with the latter CPU can be flushed to disk before the one
associated with the former CPU.
If you expect a high volume of events (for example, more than 1,000
events per second), you should not use this mode.
Note that the processor number is not included with the event. Not
available prior to Windows 7 and Windows Server 2008 R2.
I've been using logman to capture the results. It looks like tracelog will give me info about lost events, and I can tweek its buffer parameters to reduce the event loss.
If you use xperf to collect the logs, it generates a warning when events are lost. With xperf you can also play with the buffer size and can divide the logging to several loggers.