This is more of a conceptual/implementation question, rather than a specific language problem.
Does anybody have any insight into cursor movement recording?
It's very easy to get a cursor's current position, but how would you go about recording the path followed by the cursor?
(To the degree of detail where it could be plotted graphically without ambiguity as to the path taken)
I imagine you could record the cursor's current position repeatedly after a small duration, logging it all to make a list of chronologically visited coordinates,
but I'm not sure how frequent (or feasible) the recording should take place; every 10 ms?
I've not even encountered a method of sleeping for such short durations to the nescessary precision!
I'm concerned also about the performance of the sleeping and recording during intense CPU usage; when the user is using the mouse to interact with intensive software.
I'm not even entirely sure where the cursor is really moving.
If I sweep my cursor across the screen, has the computer (somewhere internally) acknowledged that I crossed all those pixels,
or has my mouse really told it "I was there, now I'm over here, now I'm there".
I do also seek a method of differentiating between fast and slow movement, but for now, I can just observe the plot spacing on a graph of the visited coordinates.
Does anybody have any insight into this?
Any potential pitfalls; are my concerns legitimate?
Am I going the wrong way about this?
(As is observable, I really need some guidance in the matter)
Thanks!
It's far easier to log the mouse movements within the same application - just log something on every WM_MOUSEMOVE message. You will get a message periodically updating the mouse pointer location. You will not get a WM_MOUSEMOVE message for every pixel the mouse crosses, but it will jump depending on how fast you move the mouse and how busy the system is.
Logging mouse movements in some other application is going to be slightly more involved. If you have written both the logger and the application being logged, then you can handle WM_MOUSEMOVE in the application being logged, and send a corresponding message to your logger application. Your choice of IPC; a simple SendMessage() might be sufficient.
Logging mouse movements across the whole system is a totally different problem. You may have to hook in at somewhere closer to the driver level.
I just thought of another approach - the CBT (Computer-Based Training) hooks are designed to provide exactly this sort of information across applications. I've never used these though, so you'll have to do more investigation.
Related
I'm trying to write my own software for security camera motion detection, but in the area of interest outside my house, there is a lot of vegetation motion that will obviously trigger recording if I use some of the more simple algorithms that rely just on the difference between images. Does anyone have any recommendations? I'm struggling to find motion detection information online. I'm guessing that I'll have to employ some edge detection, or maybe a filtering process.
Cheers,
Zan
Without having seen any of your recordings I would suspect that motion from the vegetation looks quite noisy and more random with only a few local edges as in contrast I would expect much stronger connected edges for people that move through the scenery. Also edges from objects moving on the floor will be mostly be oriented on specific directions for a longer period of time.
My first attempt would be
median filter on input image to reduce noise
difference image to previous (may be 2nd previous) image
some edge detector
build some edgelists based on the stronger
filter out weak/short edges
match edges from objects in last frame against the newly found
apply some tracking of positions and other features
classify object behaviour based on this features
consistent movement in one direction
consistently strong edges on the same object
object size
to trigger your recording
Alternatively you can jump on the recent Hype of Deep Neural Networks.
Look up online information and tools (and maybe embedded hardware) to train and run a CDNN.
Split your current videos into
videos there you do not want to be warned
videos there you do want to be warned
let the magic happen.
I have an application written in C using GTK (although the language is probably unimportant for this question).
This application has a fullscreengtk_window with a single gtk_drawing_area. For the drawing area, I have registered a tick callback via gtk_widget_add_tick_callback which just calls gtk_widget_queue_draw every tick. Inside the drawing area draw callback, I change the color of the entire window at regular intervals (e.g., from black to white at 1Hz).
Say that in this call to the draw callback I want to change the window from black to white. I would like to know the precise time (down to the nearest ms) that the change is actually drawn on the screen (ideally in the same units as CLOCK_MONOTONIC). I don't think this is the same thing as the GdkFrameClock available in the tick callback, which, as I understand it, is about the time of the frame, not the time when the frame is actually displayed on the screen.
If I just measure the CLOCK_MONOTONIC time in the drawing callback, and then use a photo-diode to measure when the actual change is via an attached A2D, the actual change is the display is understandably delayed by a number of refresh intervals (in my case, 3 screen refreshes).
Just as a summary: if I am in a GTK widget draw callback, is there any way to know the time when the display will actually be shown on the monitor in the units of CLOCK_MONOTONIC? Or alternatively, is there a way that I can block a separate thread until a specific redraw that I care about is actually displayed on the screen (a function I can write like wait_for_screen_flip())?
Update: Ideally, the same solution would work for any Linux compositor (X11 or Wayland), which is why I am hoping for a GTK/GDK solution, where the compositor is abstracted away.
Similarly to Uli's answer of the Present extension and PresentCompleteNotify for X11, Wayland has a similar protocol called wp_presentation_feedback:
https://cgit.freedesktop.org/wayland/wayland-protocols/tree/stable/presentation-time/presentation-time.xml
This protocol allows the Wayland compositor to inform clients when their content was actually displayed (turned to light). It is independent of the actual buffer mechanism used (EGL/SHM/etc). To use it, you call wp_presentation_get_feedback before wl_surface_commit; when the commit has completed, a presented event will be sent to the client from the new wp_presentation_feedback object, or discarded if it was never shown.
Presentation feedback is currently implemented in Weston; it is not yet implemented in Mutter, and I don't believe it's implemented in KWin either. GTK+ plans to support it when it becomes available in Mutter, but I don't have any great insight as to how it would be exposed through the GTK+ API.
That being said, if you can get access to the Wayland display, it's possible that you could use the interface directly yourself.
I just came across https://developer.gnome.org/gdk3/stable/gdk3-GdkFrameTimings.html#gdk-frame-timings-get-presentation-time which seems to do just like what you want and is part of Gdk. I do not know how to use it nor have I seen some example of it, but https://developer.gnome.org/gdk3/stable/gdk3-GdkFrameTimings.html#gdk3-GdkFrameTimings.description says
The information in GdkFrameTimings is useful for precise synchronization of video with the event or audio streams, and for measuring quality metrics for the application’s display, such as latency and jitter.
Take a look at https://cgit.freedesktop.org/xorg/proto/presentproto/tree/presentproto.txt. Specifically, you want PresentCompleteNotify events. Note that these can only tell you later when presentation actually happened, so (I think) you will not know ahead of time when this is (but you could perhaps guess based on recent notifies?).
Note that this is
a relatively new X11 extension, so might not actually be supported everywhere
depends on the driver used (and likely lots of other factors) for the quality of data
cannot be used from GTK since it requires a different way to display to the screen (you draw to a Pixmap and then use PresentPixmap to make it visible and ask for a notify)
Also note that this extension provides lots of other things. You can for example say "please display at time ". Just read the protocol specification from start to end. :-)
In real-time games, there is always a game loop that runs every few milliseconds, updates the game with new data and repaints the entire screen.
Is this something that is seen in other types of applications, other than games? A 'constant-update-loop'?
For example, imagine an application like MSPaint. The user can draw lines on the screen with the mouse. The line that is being drawn is displayed on the screen as it is being drawn.
Please imagine this line is actually made of a lot of smaller lines, each 2 pixels long. It would make sense to store each of these small lines in a List.
But as I said, the line that is being drawn (the large line, made out of lots of small lines) is displayed as it is being drawn. This means that a repaint of the screen would be necessary to display the new small line that was added the previous moment.
But - please correct me if I'm mistaken - it would be difficult to repaint only the specific part of the screen where the new small line was drawn. If so, a repaint of the entire screen would be necessary.
Thus it would make sense to use an 'update loop' to constantly repaint the entire screen, and constantly iterate over the list of lines and draw these lines over and over again - like in games.
Is this approach existent in non-game applications, and specifically in 'drawing' applications?
Thanks
Essentially you do have a loop in all applications, and games. How that is implemented depends on the system and desire of your application/game. I will loosely base my response toward the Windows system, if only because you mention MS Paint
MS Paint will likely NOT contain a List of lines like you are mentioning, instead it will edit the bitmap image directly each frame, coloring the required pixels immediately and then drawing it. In this situation drawing small portions of the image/application is as easy as telling "this part" of the image to draw itself "over there". So as you move the pencil tool around the pixels turn black and get drawn.
MS Paint and most applications will use a primary loop that WAITS for the next event, meaning, it will allow the operating system (Windows) to not process anything until it has messages/events to process (such as: Mouse Move, Button Press, Redraw, etc).
For a game, it needs to be a little differently. A game (typically) doesn't want the operating system to WAIT for the next message/event before processing continues. Instead here you Poll the operating system to check if there are messages to be handled, and if so handle them, if not continue with the game creating a single frame (perform an Update and Render/Draw the scene.)
MS Paint doesn't need to keep updating and drawing when the user is not interacting, and this is preferred for applications because constantly updating/drawing uses a lot of system resources and if every application did this, you wouldn't have 30 things running at the same time like you probably do now.
In my WPF app I have a control representing a pack of 20 cards (each about 150x80 px) that fan out in an arc, so they're all slightly overlapping in the centre of the arc. When the control is added there's an animation to fan them out.
After that, the fan/control can be moved around, and when the user hovers over a card it expands and then goes back to normal size when they move off it.
This all works fine, but has a noticeable effect on performance- everything is very jerky, presumably because when other things move all the overlapping stuff and transforms in the control are being constantly recalculated/redrawn.
Any suggestions for how I can improve performance while still keeping individual cards in the fan responsive?
To find the source of the slowdown you need to profile.
Try to find out whether or not WPF is switching back to software rendering or not.
After that try to run on a different computer with other (better) hardware/graphics card.
If it doesn't get any better there might be errors in your app.
I am working on a simple soccer simulation, I am using potential fields for collision avoidance more specifically following technique,
http://www.ibm.com/developerworks/java/library/j-antigrav/
only obstacles on the field are other players and they are constantly moving. Problem is it works if I assign really big push force to the characters since characters move at speed it takes some time to change direction but this has few drawbacks with such a high gravity I can never position an npc to grab the ball cause there is always some force pushing me around.
I though I could solve this by assigning pulling force to the ball but that actually made it worse. nps would go to the ball, ball starts pulling which makes npc push the ball it goes into a loop until npc crashes to a wall.
How I've implemented this is, I have a vector that would steer me towards my target then I add to that all gravitational forces acting on the npc and steer towards that direction.
Basically I am wondering what kind of improvements I can make? my current problem is not hitting other players precisely getting behind a ball without getting affected by other players.
I'm not sure that employing potential fields is the way to go here. If all players are directly between you and the ball, how do you get to it?
I'd be tempted to plot a straight line, then iteratively adjust the route for the position of other players, adjusting for their trajectories and — if you're really clever — anticipating changes in the same.
You could put some kind of limit as to the area of effect that another player's gravity well will have. Limit it to a small radius around that player so that when you're clear of other players, there are no forces acting on your character. You could also diminish the collision avoidance force when you're near the ball, reasoning that players care less about not hitting each other when it's time to kick the ball.