I am trying to make an application which will graphically repeat the mouse pointer, so I can ultimately make a mouse trail program, for Ubuntu 18.04 - and it seems, the way to do it is via X11/Xlib - although, these days I don't even know, as my machine says also wayland:
$ loginctl | while IFS= read line; do echo "$line"; if [[ $line == *"tty"* ]]; then sessnum=$(echo "$line" | awk '{print $1;}'); echo sessnum: $sessnum\; $(loginctl show-session $sessnum -p Type); fi; done
SESSION UID USER SEAT TTY
c1 121 gdm seat0 tty1
sessnum: c1; Type=wayland
2 1000 administrator seat0 tty2
sessnum: 2; Type=x11
2 sessions listed.
Regardless, I managed to put together an unholy assemblage of:
https://keithp.com/blogs/Cursor_tracking/ - which sets up the program for capturing raw mouse events, so the mouse pointer position can be extracted (and a redraw triggered) whenever the mouse pointer position changes
xosd.c (via https://github.com/AndreRenaud/XOSD) - I thought at first that On-Screen Display would have a special method to draw on top - but this sets up a topmost window, child of the root, where all drawing happens; and it also sets up event and timer thread
.... plus a ton of other code snippets (mostly from SO), which sort of does what I want (even if I don't really fully understand all of the layers and compositing that goes on in it). I posted this as a gist: xosd_track_cursor.c since it's 700+ lines (but can post it here if needed).
Here is how the application behaves (also see full-res imgur .mp4 video)
Basically, at start, the "OSD" topmost window is set up, and it's quite smaller than the desktop window - which helps us see the window border decorations around it (ultimately, I'd make this window the same size as the desktop).
At start, the desktop pixels at the location of this window have seemingly been copied as the window background.
Once the mouse pointer enters the OSD window, there is a draw of a circle, which becomes the mask for the OSD window (which again can be seen via the window border decorations) - and this circular window follows the mouse. Then, inside it, I draw a XFillRectangle to draw a lime rectangle, and then XPutImage to draw the pixels captured from the latest mouse pointer (the video doesn't show it, but also the copied cursor changed when the normal one does, say from left_ptr to bottom_side or xterm cursor bitmaps).
So far so good - but these are the problems, and questions:
All of the draws - both the lime rectangle and the mouse pointer copy - remain on the OSD window, and are not cleared upon redraw (which is quite obvious when the mouse pointer leaves the OSD window, so there is no masking). How can I erase these previous draws each time a new state is rendered?
When I click on window to change the focus, it is obvious (especially when the mouse pointer leaves the OSD window, so there is no masking) that the desktop "background" shown in the OSD window, shows the state when the program started. How can I capture the current state of the desktop background (that is, behind the OSD window), so I can use that for clearing the OSD window in the previous step?
(I thought I could hide the OSD window, then capture the desktop at the same location with XGetImage, maybe (?) - then show the window; but show always sends Expose event, which otherwise runs the expose function that does the redraw, and so I get a bunch of recursive calls hogging the application)
The mouse pointer copy is rendered with a black background - how can I make the drawing of mouse pointer copy transparent, where it is black now?
And, a sort of a bonus question (just curious here - obviously I'd rather not have the leftovers to begin with):
I first do XFillRectangle to draw a lime rectangle, then XPutImage to draw the pixels of the mouse pointer copy. I'd expect this to show the mouse cursor copy on top of the green pixels - and it is indeed so, while the OSD window is masked with the circle. But when the OSD window is shown in full, the leftovers make it seem as if the green pixels were drawn on top of the mouse cursor copy pixels. Why is this so?
Well, I think I got somewhere - the result is in the same gist, just different revision: gist: xosd_track_cursor.c (a31e9dff5); and it looks like this:
And so, to answer my questions:
How can I erase these previous draws each time a new state is rendered?
Cannot - not in the way the previous code was set up. It was set up as a override_redirect, meaning it would stay out of the management of any window manager. Furthermore, the default bit depth was 24, meaning that transparency was not supported, meaning that to grab the desktop "behind" (to use as "clear" background), we'd had to hide and then show the window, which used to cause recursion due to reaction to Expose events.
However, I saw in How to make an OpenGL rendering context with transparent background? that using an glXCreateContext might help - and it did. However, it turns out, it was not necessary - as soon as XMatchVisualInfo successfully returned a match for 32-bit depth for the OSD window (alpha transparency supported), then it was possible to define a "fully transparent" color, via XSetForeground, as 0x00000000 (as far as I see, that is 0xAARRGGBB format) - and use that to draw directly on the window with XFillRectangle -> that manages to clear the entire OSD window transparently.
The mouse pointer copy is rendered with a black background - how can I make the drawing of mouse pointer copy transparent, where it is black now?
Turns out, also this started working as soon as the window was created with XCreateWindow using settings from XMatchVisualInfo for 32-bit depth. By that, I mean that the result of XPutImage was such, that the transparent points in the cursor image were now "see-through"/transparent - whereas previously, the result of XPutImage showed black pixels at those locations.
But when the OSD window is shown in full, the leftovers make it seem as if the green pixels were drawn on top of the mouse cursor copy pixels. Why is this so?
Apparently, I didn't remember correctly what order the pixels were drawn in; when that demo capture was taken, indeed the mouse cursor pixels were copied first, and then the green pixels were copied on top. ( which now changes the question - how come the mouse cursor was visible in that capture, at all?! but now that the overall problem is solved, I'm not that curious :) )
Otherwise, few more notes on gist: xosd_track_cursor.c (a31e9dff5): since X11 has a client/server architecture, that means the user program can only queue requests to the server, and thus all of the drawing calls are asynchronous/non-blocking - and so, when we run, say, XFillRectangle and it exits, it does not mean that the drawing of pixels has been finished - just that the request has been sent to the queue, that ends up being sent to the server. Furthermore, in spite of commands like XFlush, XSync - there is never a guarantee that we can wait for a finished drawing operation; and there is no guarantee either that the server will honor any given request.
However, the less you try to do, the bigger the probability the X Server will honor the requests. So this version of the code actually makes a smallish window, 60x60 pixels, then sets it up so it is (centrally aligned) dragged by the motion of the mouse pointer. Then, the (main) mouse pointer is simply copied in this window at the same relative location.
Finally, there is a primitive attempt to do a mouse trail, by rendering two "ghost" copies of the mouse pointer, and have them be displaced by a history of mouse motion delta vector - the effect, as is visible on the gif, is not really amazing, but at least it's there as a "proof of concept", of sorts. Also, the window is setup at start as "click-through" using XShapeCombineRectangles - meaning the OSD window doesn't pick up/handle any mouse events (clicks) directly on it, instead everything is automatically passed to the windows below it, so the interaction remains the same, as if the program was not running at all.
(Note that to get the behavior of gist: xosd_track_cursor.c (a31e9dff5) shown in the gif, you should look up the defines DEBUGPRINT and MOUSE_TRAIL, and have them uncommented when you build)
I have an issue where I snap the window to the left and after I close the window I will save the location (Left, Top) along with size of the window. Next time the window is loaded I will try to apply the location and size.
I say try because the window might have been viewed on a screen that has larger resolution so it might not be visible at all on our new screen. What I do is - I see if the window(after I apply location and size) fits in the screen. If it doesn't I will show it on the center and if it does, well it's already where I want it.
My issue is when I snap window to the left. The actual Left property is not 0(zero) but -6.something. I suspect this is because the window has shadow around it so the location must be in minus so the actual form(the one with border) is touching left part of the screen. Because of my logic I will get this screen in the center.
Is there a bullet proof way of determining that the form is snapped to one of the sides?
Is there a bullet proof way of determining that the form is snapped to one of the sides?
No, I don't think so. There is at least no "IsSnapped" property or similar that you can use.
You will have to rely on the Left value and adjust it based on the difference in size between the different screens that are involved I am afraid.
I would happily provide a screenshot of this, however the problem is the captured image, is much larger than my actual desktop.
I am completely frustrated with this as I have tried using BitBlt with the desktop hdc AND the new "Graphics" commands.
My actual desktop resolution is 1920x1080 - 1080p .
BitBlt and "Graphics" both return that my resolution is 1536x864 # 96 DPI.
A form (WinForm), Maximized, borderless, and irrelevant of scaling mode the form is set to, also shows 1536x864 # 96 DPI.
Now the image that is captured, is like it is being done from 1920x1080, but clipping the region 1536x864 as the screenshot.
If I do PrintScreen directly using Prtscn button, I get the entire image, but still it is about 1.5-2x larger than what I actually see.
What I am looking for -- is a resolution for how I can take a picture of what is on my screen in the scale/dpi/whatever is going on here that it visually looks like. I have written a screen capture program, and using a few different examples for the RubberBand form (overlay form to select a region of the screen by drawing a box), and as you can imagine, this scaling crap is causing those box captures to be offset, and the contents are zoomed.
This is very annoying -- even to explain, however I am positive that most of you are familiar with the terms I use, and also know what to expect from taking a screenshot, so my explanation above should be pretty clear as to what my problem is.
Example/Consideration
Imagine, taking a picture of a window that is 300x300, and getting the top left 150x150 of that zoomed to 300x300 completely skipping the remainder of the window. Resulting image is still 300x300, but it's not what you selected.
Now imagine, you grab a picture of your screen by the only dimensions you can get programmatically, and then put the image into a picturebox. Even though both your screen and the picturebox claim to be the same dimensions and dpi, the image in the picturebox requires scrolling even if the picturebox is maximized to fullscreen on a borderless with no borders / etc. -- again, the picture is zoomed, but how is it still reporting that it's the same size as the form XD (comparing Graphics or BitBlt dimensions with the actual form. also tried comparing picturebox contents, and still same effect)
This, is EXACTLY what the effect is that is happening. When I try to capture a region or segment of the screen. I am not sure why windows api/crl is lying about this seemingly trivial stuff, however there must be a way to accurately obtain screenshots/capture regions without this faux zoom effect -- across all resolutions.
Thank you Hans Passant for pointing me in the right direction.
To add "true" dpi scaling support to a winforms application, you can make it so by adding the following block to your manifest :
Project > Add New Item > Visual C# Items > Application Manifest File
One the file has been added, open it up and look for a line like
</asmv1:assembly>
Whatever the "asmv" number is, (in the example above it is 1), use that to format the code:
<asmv1:application>
<asmv1:windowsSettings xmlns="http://schemas.microsoft.com/SMI/2005/WindowsSettings">
<dpiAware>true</dpiAware>
</asmv1:windowsSettings>
</asmv1:application>
Paste the above code (changing the asmv1 to whatever version the manifest is), just above the final closing line for the ""
Also, make sure your forms are set to AutoScale to dpi (and all sub-elements).
The form is set up to have no maximize button. When Win+D is pressed, all windows minimize including the form. When clicking on the application's taskbar entry, the form shows up but does not restore to the previous size. The form is reduced to just showing the title bar and the borders, w/o the client area.
I tried listening for WM_SIZE message's SIZE_RESTORED value, but couldn't seem to get the right size from RestoreBounds or GetWindowPlacement.
I'll try to do it the other way and look for SIZE_MINIMIZED and capture the rect, then restore from that rect on SIZE_RESTORED, when I come back.
In the meantime, any of you have an idea why the form does not restore correctly in the first place so I can apply a proper fix. Or suggest a better approach, if the proper fix is not possible.
You are doing something weird with your form - try creating an application from scratch, and observe the behavior on empty default form.
Then, go from there - add changes to the new form/project and see when the problem will start to occur.
I have an application that uses DwmExtendFrameIntoClientArea to draw a glass effect area at the top of my .NET Form. A strange side effect is that when you maximize and restore the window it grows taller. So if you keep maximizing/restoring it just keeps getting taller! Remove the call to DwmExtendFrameIntoClientArea to remove that nice effect and it stops going wrong.
Any ideas?