Capture window pixels regardless of its z-order - c

I'm actually trying to read a specific pixel on a window which is hidden by others. I want to use the GetPixel function from GDI library but it seems it only works with the global device context. I can't read pixel from a specific window and I don't understand why..
I found this article which uses the PrintWindow function to copy a specific window content to a temporary device context which can be read. But I can't reproduce it.
EDIT
Thank you all my problem is solved :)
This script give you the RGB color of the pointer on the choosen window, even though the window is hidden. Remind that this program must be launch with admin privileges to get the pixels of processes launched with admin privileges.
#define STRICT
#define WINVER 0x0501
#define _WIN32_WINNT 0x0501
// 0x0501 for PrintWindow function
// You must be at least running Windows XP
// See http://msdn.microsoft.com/en-us/library/6sehtctf.aspx
#include <stdio.h>
#include <string.h>
#include <windows.h>
#define WINDOW_LIST_LIMIT 32
#define WINDOW_NAME_LIMIT 1024
void FatalError(char* error)
{
printf("%s", error);
exit(-1);
}
HWND window_list[WINDOW_LIST_LIMIT];
unsigned int window_list_index = 0;
BOOL EnumWindowsProc(HWND window_handle, LPARAM param)
{
char window_title[WINDOW_NAME_LIMIT];
if(!IsWindowVisible(window_handle)) return TRUE;
RECT rectangle = {0};
GetWindowRect(window_handle, &rectangle);
if (IsRectEmpty(&rectangle)) return TRUE;
GetWindowText(window_handle, window_title, sizeof(window_title));
if(strlen(window_title) == 0) return TRUE;
if(!strcmp(window_title, "Program Manager")) return TRUE;
window_list[window_list_index] = window_handle;
window_list_index++;
printf("%u - %s\n", window_list_index, window_title);
if(window_list_index == WINDOW_LIST_LIMIT) return FALSE;
return TRUE;
}
int main(int argc, char** argv)
{
unsigned int i, input;
EnumWindows((WNDENUMPROC) EnumWindowsProc, (LPARAM) NULL);
printf("\nChoose a window: ");
scanf("%u", &input);
printf("\n");
if(input > window_list_index) FatalError("Bad choice..\n");
HDC window_dc = GetWindowDC(window_list[input - 1]), global_dc = GetDC(0), temp_dc;
if(!window_dc && !global_dc) FatalError("Fatal Error - Cannot get device context.\n");
POINT cursor, previous_cursor;
while(1)
{
temp_dc = CreateCompatibleDC(window_dc);
if(!temp_dc) FatalError("Fatal Error - Cannot create compatible device context.\n");
RECT window_rectangle;
GetWindowRect(window_list[input - 1], &window_rectangle);
HBITMAP bitmap = CreateCompatibleBitmap(window_dc,
window_rectangle.right - window_rectangle.left,
window_rectangle.bottom - window_rectangle.top);
if (bitmap)
{
SelectObject(temp_dc, bitmap);
PrintWindow(window_list[input - 1], temp_dc, 0);
DeleteObject(bitmap);
}
GetCursorPos(&cursor);
if(cursor.x != previous_cursor.x && cursor.y != previous_cursor.y)
{
COLORREF color = GetPixel(temp_dc, cursor.x - window_rectangle.left, cursor.y - window_rectangle.top);
int red = GetRValue(color);
int green = GetGValue(color);
int blue = GetBValue(color);
printf("\rRGB %02X%02X%02X", red, green, blue);
cursor = previous_cursor;
}
DeleteDC(temp_dc);
Sleep(50); // for lags
}
ReleaseDC(window_list[input - 1], window_dc);
return 0;
}
I've changed some things, now User32 isn't dynamically loaded.
It compiles with
gcc main.c -o main.exe -lGid32 -lUser32
Have a great day !

You are passing a process handle to GetDC. That's not right. Processes don't have device contexts, windows do. Remember a process can have many windows, or even none at all.
You need to get hold of the window handle, the HWND, for the window in question, and pass that to GetDC. I'd look to using FindWindow or EnumWindows to find your target top-level window.
Of course, there may be other problems with your code, but that's the one that jumps out at me.

HDC process_dc = GetDC(process_handle)
Well that's all kinds of wrong. GetDC accepts a window handle, not a process handle.
In order to find such errors, recompile with
#define STRICT
placed before your includes.

This is a bit of a confusing topic, so let's see if I can clarify a few things.
First things first: as both David and Ben have already answered, you're passing a process handle to the GetDC function, which is wrong. GetDC accepts a handle to a window (the HWND type), and it returns a device context (DC, the HDC type) corresponding to that window. You need to get that fixed before anything else will work.
Now, as the article you've read indicates, windows (assuming they've been correctly programmed) respond to the WM_PRINT or WM_PRINTCLIENT messages by rendering an image of themselves into the specified device context (HDC). This is a simple and effective way of capturing an "image" of a window, whether an overlapping window or the window of an individual control.
The rub comes in, as Hans mentioned in a comment, because handles to device contexts have process affinity, which means that the HDC you pass to the window in a separate process, into which it is supposed to render itself, will not be valid from that other process. Handles to device contexts cannot be passed across process boundaries. That's the primary reason that your code fails (or is going to fail, once you fix the handle type problems). The MSDN entry on GDI Objects makes this explicitly clear:
Handles to GDI objects are private to a process. That is, only the process that created the GDI object can use the object handle.
Fixing or getting around that is going to be a bit of an uphill battle. The only solution that I know of is to inject code into the other application's process that first creates a DC in memory, then sends the WM_PRINT or WM_PRINTCLIENT message to a window owned by that process to draw into that in-memory device context, and then transfers the resulted bitmap back to your own application. This is going to require that you implement some type of inter-process communication mechanism.
I've seen some anecdotal evidence that passing device context handles between processes via the WM_PRINT and WM_PRINTCLIENT messages "works", but it's unclear to me whether this is an artifact of the current implementation (and therefore subject to breaking in future versions of Windows), or if this is because Windows is actually handling the marshaling between processes. I haven't seen any documentation one way or the other. If this is a one-off project you're doing for fun or for a limited use, you might try it and get away with it. For other purposes, you probably want to investigate using IPC to really do this the right way.

Don't use GetDC for the DC to pass to PrintWindow. You need to create a compatible DC as you're doing (though you can pass it NULL to get a generic screen DC), then create a compatible bitmap the size of the window you're trying to capture and select it into the DC. Then pass that DC handle to PrintWindow.
Windows aren't required to respond properly to WM_PRINT or WM_PRINTCLIENT, so there may be some glitches even when you get this to work.

Related

How can I access a graphics card's output directly?

Do graphics cards typically write their output to some place in memory which I can access? Do I have to use the driver? If so, can I use OpenGL?
I want to know if it's possible to "capture" the output of a VM on Linux which has direct access to a GPU, and is running Windows. Ideally, I could access the output from memory directly, without touching the GPU, since this code would be able to run on the Linux host.
The other option is to maybe write a Windows driver which reads the output of the GPU and writes it to some place in memory. Then, on the Linux side, a program can read this memory. This seems somewhat impossible, since I'm not really sure how to get a process on the host to share memory with a process on the guest.
Is it possible to do option 1 and simply read the output from memory?
I do not code under Linux but in Windows (you are running it in emulator anyway) you can use WinAPI to directly access canvas of any window or even desktop from 3th party App. Some GPU overlays could be problematic to catch (especially DirectX based) but I had no problems with mine GL/GLSL for now.
If you got access to App source you can use glReadPixels for image extraction from GL directly (but that works only for current GL based rendering).
Using glReadPixels
As mentioned this must be implemented directly in the targeted app so you need to have it source code or inject your code in the right place/time. I use for screenshoting this code:
void OpenGLscreen::screenshot(Graphics::TBitmap *bmp)
{
if (bmp==NULL) return;
int *dat=new int[xs*ys],x,y,a,*p;
if (dat==NULL) return;
bmp->HandleType=bmDIB;
bmp->PixelFormat=pf32bit;
if ((bmp->Width!=xs)||(bmp->Height!=ys)) bmp->SetSize(xs,ys);
if ((bmp->Width==xs)&&(bmp->Height==ys))
{
glReadPixels(0,0,xs,ys,GL_BGRA,GL_UNSIGNED_BYTE,dat);
glFinish();
for (a=0,y=ys-1;y>=0;y--)
for (p=(int*)bmp->ScanLine[y],x=0;x<xs;x++,a++)
p[x]=dat[a];
}
delete[] dat;
}
where xs,ys is OpenGL window resolution, you can ignore the whole bmp stuff (it is VCL bitmap I use to store screenshot) and also can ignore the for it just copy the image from buffer to bitmap. So the important stuff is just this:
int *dat=new int[xs*ys]; // each pixel is 32bit int
glReadPixels(0,0,xs,ys,GL_BGRA,GL_UNSIGNED_BYTE,dat);
glFinish();
You need to execute this code after the rendering is done otherwise you will obtain unfinished or empty buffer. I use it after redraw/repaint events. As mentioned before this will obtain only the GL rendered stuff so if your App combines GDI+OpenGL it is better to use the next approach.
WinAPI approach
To obtain Canvas image of any window I wrote this class:
//---------------------------------------------------------------------------
//--- screen capture ver: 1.00 ----------------------------------------------
//---------------------------------------------------------------------------
class scrcap
{
public:
HWND hnd,hnda;
TCanvas *scr;
Graphics::TBitmap *bmp;
int x0,y0,xs,ys;
scrcap()
{
hnd=NULL;
hnda=NULL;
scr=new TCanvas();
bmp=new Graphics::TBitmap;
#ifdef _mmap_h
mmap_new('scrc',scr,sizeof(TCanvas() ));
mmap_new('scrc',bmp,sizeof(Graphics::TBitmap));
#endif
if (bmp)
{
bmp->HandleType=bmDIB;
bmp->PixelFormat=pf32bit;
}
x0=0; y0=0; xs=1; ys=1;
hnd=GetDesktopWindow();
}
~scrcap()
{
#ifdef _mmap_h
mmap_del('scrc',scr);
mmap_del('scrc',bmp);
#endif
if (scr) delete scr; scr=NULL;
if (bmp) delete bmp; bmp=NULL;
}
void init(HWND _hnd=NULL)
{
RECT r;
if (scr==NULL) return;
if (bmp==NULL) return;
bmp->SetSize(1,1);
if (!IsWindow(_hnd)) _hnd=hnd;
scr->Handle=GetDC(_hnd);
hnda=_hnd;
resize();
}
void resize()
{
if (!IsWindow(hnda)) return;
RECT r;
// GetWindowRect(hnda,&r);
GetClientRect(hnda,&r);
x0=r.left; xs=r.right-x0;
y0=r.top; ys=r.bottom-y0;
bmp->SetSize(xs,ys);
xs=bmp->Width;
ys=bmp->Height;
}
void capture()
{
if (scr==NULL) return;
if (bmp==NULL) return;
bmp->Canvas->CopyRect(Rect(0,0,xs,ys),scr,TRect(x0,y0,x0+xs,y0+ys));
}
};
//---------------------------------------------------------------------------
//---------------------------------------------------------------------------
//---------------------------------------------------------------------------
Again it uses VCL so rewrite bitmap bmp and canvas scr to your programing environment style. Also igneore the _mmap_h chunks of code they are just for debugging/tracing of memory pointers related to some nasty compiler bug I was facing at that time I wrote this.
The usage is simple:
// globals
scrcap cap;
// init
cap.init();
// on screenshot
cap.capture();
// here use cap.bmp
If you call cap.init() it will lock on the whole windows desktop. If you call cap.init(window_handle) it will lock on specific visual window/component instead. To obtain window handle from 3th app side see:
is ther a way an app can display a message without the use of messagebox API?
Sorry it is on SE/RE instead of here on SE/SO but My answer here covering this topic was deleted. I use this for video capture ... all of the animated GIFs in my answers where created by this code. Another example could be seen on the bottom of this answer of mine:
Image to ASCII art conversion
As you can see it works also for DirectX overlay with media player classic (even Windows PrintScreen function cant do it right). As I wrote I got no problems with this yet.
Beware visual stuff WinAPI calls MUST BE CALLED FROM APP MAIN THREAD (WNDPROC) otherwise serious problems could occur leading to random unrelated WinAPI calls exceptions anywhere in the App.

Xlib and Firefox behavior

I'm trying to create a small window manager (just for fun), but I'm having problems in handling windows created by Firefox (only with that application, other apps works fine)
The problem is, after I launch Firefox, and add my decoration, it seems to work fine, but if for example I try to click on the menu button, the (sub)window doesn't appear.
What seems to happen is that after the click, a ClientMessage event is fired with the following values:
Data: (null)
Data: _NET_WM_STATE_HIDDEN
Data: (null)
Data: (null)
Data: (null)
Now the problem is that I don't know how to show the window, which window.
I tried with:
XRaiseWindow
XMapWindow
I tried to get the transient window and show it
But without success. What I don't understand is that if this client message is generated by the menu subwindow or not.
How should I show a window that is in _NET_WM_STATE_HIDDEN?
Another strange problem is that after receiving the ClientMessage, I always receive 2 UnMapNotify Events.
I also have another question, if I want to show the "File, Edit" menù (in Firefox it appears, if I remember correctly, when you press the Alt button.
Maybe Firefox creates a tree of windows?
This is the loop where I handle the events:
while(1){
XNextEvent(display, &local_event);
switch(local_event.type){
case ConfigureNotify:
configure_notify_handler(local_event, display);
break;
case MotionNotify:
motion_handler(local_event, display);
break;
case CreateNotify:
cur_win = local_event.xcreatewindow.window;
char *window_name;
XFetchName(display, cur_win, &window_name);
printf("Window name: %s\n", window_name);
if(window_name!=NULL){
if(!strcmp(window_name, "Parent")){
printf("Adding borders\n");
XSetWindowBorderWidth(display, cur_win, BORDER_WIDTH);
}
XFree(window_name);
}
break;
case MapNotify:
map_notify_handler(local_event,display, infos);
break;
case UnmapNotify:
printf("UnMapNotify\n");
break;
case DestroyNotify:
printf("Destroy Event\n");
destroy_notify_handler(local_event,display);
break;
case ButtonPress:
printf("Event button pressed\n");
button_handler(local_event, display, infos);
break;
case KeyPress:
printf("Keyboard key pressed\n");
keyboard_handler(local_event, display);
break;
case ClientMessage:
printf("------------ClientMessage\n");
printf("\tMessage: %s\n", XGetAtomName(display,local_event.xclient.message_type));
printf("\tFormat: %d\n", local_event.xclient.format);
Atom *atoms = (Atom *)local_event.xclient.data.l;
int i =0;
for(i=0; i<=5; i++){
printf("\t\tData %d: %s\n", i, XGetAtomName(display, atoms[i]));
}
int nchild;
Window *child_windows;
Window parent_window;
Window root_window;
XQueryTree(display, local_event.xclient.window, &root_window, &parent_window, &child_windows, &nchild);
printf("\tNumber of childs: %d\n", nchild);
break;
}
Now in the clientmessage actually I'm just trying to see collect some information to understand what is happening. And what I can see from the code above, is that the window that raised the event contains one child (again: is that the menu? or not?)
The code for the MapNotify event, where I add the decoration is the following:
void map_notify_handler(XEvent local_event, Display* display, ScreenInfos infos){
printf("----------Map Notify\n");
XWindowAttributes win_attr;
char *child_name;
XGetWindowAttributes(display, local_event.xmap.window, &win_attr);
XFetchName(display, local_event.xmap.window, &child_name);
printf("\tAttributes: W: %d - H: %d - Name: %s - ID %lu\n", win_attr.width, win_attr.height, child_name, local_event.xmap.window);
Window trans = None;
XGetTransientForHint(display, local_event.xmap.window, &trans);
printf("\tIs transient: %ld\n", trans);
if(child_name!=NULL){
if(strcmp(child_name, "Parent") && local_event.xmap.override_redirect == False){
Window new_win = draw_window_with_name(display, RootWindow(display, infos.screen_num), "Parent", infos.screen_num,
win_attr.x, win_attr.y, win_attr.width, win_attr.height+DECORATION_HEIGHT, 0,
BlackPixel(display, infos.screen_num));
XMapWindow(display, new_win);
XReparentWindow(display,local_event.xmap.window, new_win,0, DECORATION_HEIGHT);
set_window_item(local_event.xmap.window, new_win);
XSelectInput(display, local_event.xmap.window, StructureNotifyMask);
printf("\tParent window id: %lu\n", new_win);
put_text(display, new_win, child_name, "9x15", 10, 10, BlackPixel(display,infos.screen_num), WhitePixel(display, infos.screen_num));
}
}
XFree(child_name);
}
Now can someone help me with these problems? Unfortunately I already googled many times, but without success.
To sum up, my issues are two:
1. How to show subwindows from Firefox
2. How to show the File, Edit menu.
UPDATE
I noticed something strange testing Firefox with xev to understand what events are fired in order to show an application. I saw that using Firefox in unity, and using Firefox in another window manger, the events fired are completely different. In Unity I have only:
ClientMessage
UnmapNotify
Instead using Firefox, for example with xfce4, the xevents generated are more:
VisiblityNotify (more than one)
Expose event (more than one)
But if I try to enable VisibilityChangeMask in my wm, I receive the following events:
ConfigureNotify
ClientMessage
MapNotify
2 UnMapNotify
UPDATE 2
I tried to read the XWMhints properties in the ClientMessage window (probably the menù window) and the values are:
For the flags 67 = InputHint, StateHint, WIndowGroupHint
For the initial state NormalState
UPDATE 3
I tried to look how another window manager works, and I was looking at the source code of calmwm. What is my understanding is that, when the ClientMessage event arrives, with a _NET_WM_STATE message, it updates these properties, and in the case of _NET_WM_STATE_HIDDEN it clears this property, and the result will be that the property will be deleted. So I tried to update my code to delete that property, but it's still not working. Anyway the relevant updated code in client_message_handler now looks like this:
Atom *atoms = (Atom *)local_event.xclient.data.l;
int i =0;
for(i=0; i<=5; i++){
printf("\t\tData %d: %s\n", i, XGetAtomName(display, atoms[i]));
if(i==1){
printf("\t Deleting Property: _NET_WM_STATE_HIDDEN \n");
XDeleteProperty(display, cur_window, atoms[i]);
}
}
It is only a test, and I'm sure that i=1 in my case is the _NET_WM_STATE_HIDDEN property.
Here a link to calmwm source code: https://github.com/chneukirchen/cwm/blob/linux/xevents.c
So I'm still stuck at that point.
UPDATE 4
Really I don't know if it helps, but I tried to read the window attributes in the MapNotify Event, and the window map_state is IsViewable (2).
UPDATE 5
I found a similar problem here in SO, using xlib with python: Xlib python: cannot map firefox menus
The solution suggests to use XSetInputFocus, i tried that on my XMapNotify handler:
XSetInputFocus(display, local_event.xmap.window, RevertToParent, CurrentTime);
But it still doesn't help, the firefox menu still doesn't appear!!
And i have the same problem with right-click.
UPDATE 6
Playing with xconfigurenotify event and unmap event i found that the:
Xconfigure request has 2 window fields: window and above, and when the
the xconfigurerequest.window value is the same of xunmap.window value.
And also that the xconfigurerequest.above is always changing, but xconfigurerequest.window is always the same in all events.
It seems that the xconfigurerequest.above is related to what menu i'm trying to open. For example:
if right-click on a page i get an id (always the same for every subsequent click)
if i right-clik on a tab, the above value is another one
and the same happen if i left-click the firefox main menu
Still don't know if that helps.
Really don't know
Anyone got any idea?
This question is ancient but for the benefit of anyone who stumbles across it looking for an answer to this, here's an edited (chopped to bits) sample of how I solved this based on the hints above:
while (event = xcb_poll_for_event(connection)) {
uint8_t actual_event = event->response_type & 127;
switch (actual_event) {
case XCB_MAP_NOTIFY: ;
xcb_map_notify_event_t *map_evt = (xcb_map_notify_event_t *)event;
if (map_evt->override_redirect) {
xcb_get_property_cookie_t cookie = xcb_icccm_get_wm_transient_for(connection, map_evt->window);
xcb_window_t transient_for = 0;
xcb_icccm_get_wm_transient_for_reply(connection, cookie, &transient_for, NULL);
if (transient_for) {
xcb_set_input_focus(connection, XCB_INPUT_FOCUS_POINTER_ROOT, transient_for, XCB_CURRENT_TIME);
}
xcb_flush(connection);
}
break;
case XCB_CLIENT_MESSAGE: ;
xcb_client_message_event_t *message_evt = (xcb_client_message_event_t *)event;
xcb_get_atom_name_cookie_t name_cookie = xcb_get_atom_name(connection, message_evt->type);
xcb_get_atom_name_reply_t *name_reply = xcb_get_atom_name_reply(connection, name_cookie, NULL);
int length = xcb_get_atom_name_name_length(name_reply);
char *atom_name = malloc(length + 1);
strncpy(atom_name, xcb_get_atom_name_name(name_reply), length);
atom_name[length] = '\0';
free(atom_name);
free(name_reply);
if (message_evt->type == ewmh->_NET_WM_STATE) {
xcb_atom_t atom = message_evt->data.data32[1];
unsigned int action = message_evt->data.data32[0];
xcb_get_atom_name_cookie_t name_cookie = xcb_get_atom_name(connection, atom);
xcb_get_atom_name_reply_t *name_reply = xcb_get_atom_name_reply(connection, name_cookie, NULL);
int length = xcb_get_atom_name_name_length(name_reply);
char *atom_name = malloc(length + 1);
strncpy(atom_name, xcb_get_atom_name_name(name_reply), length);
atom_name[length] = '\0';
if (action == XCB_EWMH_WM_STATE_REMOVE) {
if (atom == ewmh->_NET_WM_STATE_HIDDEN) {
xcb_delete_property(connection, message_evt->window, ewmh->_NET_WM_STATE_HIDDEN);
}
}
free(atom_name);
free(name_reply);
}
break;
}
}
By way of explanation, the important events to handle are MapNotify and ClientMessage because there's two main things that have to be taken care of, the window has to have its hidden state removed on request (the xcb_delete_property call) and the parent window of the transient has to gain input focus (the xcb_set_input_focus call; note that the window that the transient is a transient for gains focus, not the transient itself) or Firefox will immediately hide the transient again.
It also seems to be important for the transients to be stacked above their parent so a WM should respect the ConfigureRequest events.
PS Even if this is the accepted answer, the code of it is for xcb, if you need the code for xlib check my answer below, with the code adapted for xlib, it does cover only the MapNotify event
Use xtruss — an easy-to-use X protocol tracing program
Overview
Any programmer accustomed to writing programs on Linux or System V-type Unixes will have encountered the program variously known as strace or truss, which monitors another program and produces a detailed log of every system call the program makes – in other words, all the program's interactions with the OS kernel. This is often an invaluable debugging tool, and almost as good an educational one.
When it's a GUI program (or rather, the GUI-related behaviour of a program) that you want to understand or debug, though, the level of interaction with the OS kernel is rarely the most useful one. More helpfully, one would like to log all the program's interactions with the X server in the same way.
Programs already exist that will do this. I'm aware of Xmon and Xtrace. But they tend to require a lot of effort to set up: you have to run the program to establish a listening server, then manually arrange for the target program to contact that instead of the real server – including some fiddly work with xauth. Ideally, you'd like tracing a program's X operations to be just as easy as tracing its kernel system calls: you'd like to type a command as simple as strace program-name arguments, and have everything automatically handled for you.
Also, the output of those programs is less easy to read than I'd have liked – by which I largely mean it's less like strace than I'd like it to be. strace has the nice property of putting each system call and its return value on the same line of output, so that you can see at a glance what each response was a response to. X protocol monitors, however, tend to follow the structure of the X protocol faithfully, meaning that each request and response is printed with a sequence number, and you have to match the two up by eye.
So this page presents xtruss, my own contribution to the field of X protocol loggers. It has a command-line syntax similar to strace – in its default mode, you just prefix "xtruss" to the same command line you would have run anyway – and its output format is also more like strace, putting requests and responses on the same line of output where reasonably possible.
strace also supports the feature of attaching to an already-running process and tracing it from the middle of its run – handy when something goes wrong with a long-running process that you didn't know in advance you were going to need to trace. xtruss supports this same feature, by means of the X RECORD extension (provided your X server supports it, which modern X.Org ones do); so in that mode, you can identify a window with the mouse (similarly to standard programs like xwininfo and xkill), and xtruss will attach to the X client program that owns the window you specified, and begin tracing it.
Description
xtruss is a utility which logs everything that passes between the X server and one or more X client programs. In this it is similar to xmon(1), but intended to combine xmon's basic functionality with an interface much more similar to strace(1).
Like xmon, xtruss in its default mode works by setting up a proxy X server, waiting for connections to that, and forwarding them on to the real X server. However, unlike xmon, you don't have to deal with any of that by hand: there's no need to start the trace utility in one terminal and manually attach processes to it from another, unless you really want to (in which case the -P option will do that). The principal mode of use is just to type xtruss followed by the command line of your X program; xtruss will automatically take care of adjusting the new program's environment to point at its proxy server, and (also unlike xmon) it will also take care of X authorisation automatically.
As an alternative mode of use, you can also attach xtruss to an already-running X application, if you didn't realise you were going to want to trace it until it had already been started. This mode requires cooperation from the X server – specifically, it can't work unless the server supports the RECORD protocol extension – but since modern X.Org servers do provide that, it's often useful.
Ok, i'm going to answer my own question after only 4.5 years and half.
I'm going to revise Mr Lightning Bolt answer, and adapt it for XLIB, keeping focused on what he said about the Transient window. The answer probably will not be complete, but at least with that code snippet, now i'm able to open firefox menus.
I will accept his question, since he proposed the correct solution.
As lightning bolt pointed the key is the MapNotify Event,so the window manager should accept that kind of events, and when it is generated it should:
grab any transient window with XGetTransientWindowForHint
if any transient window is found, we need then to set input focus to it using XSetInputFocus.
The complete code, in your MapNotifyHandler, should looks like:
Window trans = None;
XGetTransientForHint(display, local_event.xmap.window, &trans);
if(trans != None){
XSetInputFocus(display, trans, RevertToParent, CurrentTime);
}

GTK Application doesn't refresh UI when using OpenCV

I have a simple GUI application I wrote in C for the RaspBerry PI while using GTK+2.0 to handle the actual UI rendering. The application so far is pretty simple, with just a few pushbuttons for testing simple functions I wrote. One button causes a thread to be woken up which prints text to the console, and goes back to sleep, while another button stops this operation early by locking a mutex, changing a status variable, then unlocking the mutex again. Fairly simple stuff so far. The point of using this threaded approach is so that I don't ever "lock up" the UI during a long function call, forcing the user to be blocked on the I/O operations completing before the UI is usable again.
If I call the following function in my thread's processing loop, I encounter a number of issues.
#include <opencv2/objdetect/objdetect.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <stdio.h>
#include <errno.h>
using namespace std;
using namespace cv;
#define PROJECT_NAME "CAMERA_MODULE" // Include before liblog
#include <log.h>
int cameraAcquireImage(char* pathToImage) {
if (!pathToImage) {
logError("Invalid input");
return (-EINVAL);
}
int iErr = 0;
CvCapture *capture = NULL;
IplImage *frame, *img;
//0=default, -1=any camera, 1..99=your camera
capture = cvCaptureFromCAM(CV_CAP_ANY);
if(!capture) {
logError("No camera interface detected");
iErr = (-EIO);
}
if (!iErr) {
if ((frame = cvQueryFrame(capture)) == NULL) {
logError("ERROR: frame is null...");
iErr = (-EIO);
}
}
if (!iErr) {
CvSize size = cvSize(100, 100);
if ((img = cvCreateImage(size, IPL_DEPTH_16S, 1)) != NULL) {
img = frame;
cvSaveImage(pathToImage, img);
}
}
if (capture) {
cvReleaseCapture(&capture);
}
return 0;
}
The function uses some simple OpenCV code to take a snapshot with a webcam connected to my Raspberry PI. It issues warnings of VIDIOC_QUERYMENU: Invalid argument to the console, but still manages to acquire the images and save them to a file for me. However, my GUI becomes sluggish, and sometimes hangs. If it doesn't outright hang, then the window goes blank, and I have to randomly click all over the UI area until I click on where a pushbutton would normally be located, and the UI finally re-renders again rather than showing a white empty layout.
How do I go about resolving this? Is this some quirk in OpenCv when using it as part of a Gtk+2.0 application? I had originally had my project setup as a GTK3.0 application, but it wouldn't run due to some check in GTK preventing multiple versions from being included in a single application, and it seems OpenCv is an extension of GTK+2.0.
Thank you.
there is something quite broken here:
CvSize size = cvSize(100, 100);
if ((img = cvCreateImage(size, IPL_DEPTH_16S, 1)) != NULL) {
img = frame;
cvSaveImage(pathToImage, img);
}
first, you create a useless 16-bit image (why even?), then you reassign(alias) that pointer to your original image, and then you don't cvReleaseImage it (memleak).
please, stop using opencv's deprecated c-api. please.
any noob will shoot into his foot using this (one of the main reasons to get rid of it)
also, you can only use ~30% of opencv's functionality this way (the opencv1.0 set)
again, please, stop using opencv's deprecated c-api. please.
Didn't you forget to free the img pointer ?
Also, I did in the past an app that stored uncompressed images on the disk, and things used to become sluggish. In fact, what was taking time was storing the images on the disk, as it was exceeding the max bandwidth of what the filesystem layer could handle.
So try to see is you can store compressed images instead (trading some CPU to save bandwidth), or store your images in RAM in a queue and save them afterwards (in a separate thread, or in an idle handler). Of course, if the video you capture is too long, you may end up with an Out Of Memory condition. I only had sequences of a few seconds to store, so that did the trick.

How to wait until window is mapped and viewable

What is the proper way to wait until an X11 window is mapped and viewable? Precisely, I want to wait until I can safely call XSetInputFocus() without running into any risks of the X server backfiring with the following error:
// X Error of failed request: BadMatch (invalid parameter attributes)
// Major opcode of failed request: 42 (X_SetInputFocus)
Currently this error happens quite often, especially on slow X servers or when trying to open a new window right after having changed the monitor resolution using libXrandr.
I already have a solution for this problem but it is pretty hacky because it polls the window attribute so I'd like to know whether or not there is a cleaner version.
Here is my current approach:
static Bool predicate(Display *display, XEvent *ev, XPointer arg)
{
return(ev->type == MapNotify);
}
static void waitmapnotify(struct osdisplayinfo *osd)
{
XEvent ev;
XWindowAttributes xwa;
XPeekIfEvent(osd->display, &ev, predicate, NULL);
do {
XGetWindowAttributes(osd->display, osd->window, &xwa);
usleep(1);
} while(xwa.map_state != IsViewable);
}
This code works fine but it is hacky so I'm putting it up for debate here - just in case there is a cleaner way of doing this.
Select SubstructureNotifyMask on the root window. You should get an event each time a top-level window is mapped, unmapped, moved, raised, resized etc. These are the events that potentially change visibility of top-level windows. This program prints a message whenever such an event happens:
#include <X11/Xlib.h>
#include <stdio.h>
int main ()
{
Display* d = XOpenDisplay(0);
int cnt = 0;
XEvent ev;
XSelectInput (d, RootWindow(d, DefaultScreen(d)), SubstructureNotifyMask);
while (1)
{
XNextEvent(d, &ev);
printf ("Got an event %d!\n", cnt++);
// <----- do your XGetWindowAttributes(...) check here
}
}
Note that you may not get events about your own windows getting mapped. This is because the WM is likely to reparent top-level windows to be children not of the root, but of intermediate decoration windows.
There are two ways to cope with the situation:
Check if your window parent, the parent of the parent, ... etc is the mapped window of the event.
Add XSelectInput (d, yourwindow, StructureNotifyMask); to the mix.
Note the first select has SubstructureNotifyMask and the second one StructureNotifyMask, a different mask.
As far as I know X11 lib does not expose any callback mechanics for the X11 Event handling. (you can easily build your own once you understand the event filtering model)
You might want to loop on the X11 event queue as I guest this should be more efficient being designed for that purpose. Moreover you can configure the events filter, so that you get only the events that are of interest for your specific window.
A useful (though dated) link might be:
Linux Journal X11 Tutorial Check page two for an example on installing filters and getting events from the X11 queue.

How do I get the selected text from the focused window using native Win32 API?

My app. will be running on the system try monitoring for a hotkey; when the user selects some text in any window and presses a hotkey, how do I obtain the selected text, when I get the WM_HOTKEY message?
To capture the text on to the clipboard, I tried sending Ctrl + C using keybd_event() and SendInput() to the active window (GetActiveWindow()) and forground window (GetForegroundWindow()); tried combinations amongst these; all in vain. Can I get the selected text of the focused window in Windows with plain Win32 system APIs?
TL;DR: Yes, there is a way to do this using plain win32 system APIs, but it's difficult to implement correctly.
WM_COPY and WM_GETTEXT may work, but not in all cases. They depend on the receiving window handling the request correctly - and in many cases it will not. Let me run through one possible way of doing this. It may not be as simple as you were hoping, but what is in the adventure filled world of win32 programming? Ready? Ok. Let's go.
First we need to get the HWND id of the target window. There are many ways of doing this. One such approach is the one you mentioned above: get the foreground window and then the window with focus, etc. However, there is one huge gotcha that many people forget. After you get the foreground window you must AttachThreadInput to get the window with focus. Otherwise GetFocus() will simply return NULL.
There is a much easier way. Simply (miss)use the GUITREADINFO functions. It's much safer, as it avoids all the hidden dangers associated with attaching your input thread with another program.
LPGUITHREADINFO lpgui = NULL;
HWND target_window = NULL;
if( GetGUIThreadInfo( NULL, lpgui ) )
target_window = lpgui->hwndFocus;
else
{
// You can get more information on why the function failed by calling
// the win32 function, GetLastError().
}
Sending the keystrokes to copy the text is a bit more involved...
We're going to use SendInput instead of keybd_event because it's faster, and, most importantly, cannot be messed up by concurrent user input, or other programs simulating keystrokes.
This does mean that the program will be required to run on Windows XP or later, though, so, sorry if your running 98!
// We're sending two keys CONTROL and 'V'. Since keydown and keyup are two
// seperate messages, we multiply that number by two.
int key_count = 4;
INPUT* input = new INPUT[key_count];
for( int i = 0; i < key_count; i++ )
{
input[i].dwFlags = 0;
input[i].type = INPUT_KEYBOARD;
}
input[0].wVK = VK_CONTROL;
input[0].wScan = MapVirtualKey( VK_CONTROL, MAPVK_VK_TO_VSC );
input[1].wVK = 0x56 // Virtual key code for 'v'
input[1].wScan = MapVirtualKey( 0x56, MAPVK_VK_TO_VSC );
input[2].dwFlags = KEYEVENTF_KEYUP;
input[2].wVK = input[0].wVK;
input[2].wScan = input[0].wScan;
input[3].dwFlags = KEYEVENTF_KEYUP;
input[3].wVK = input[1].wVK;
input[3].wScan = input[1].wScan;
if( !SendInput( key_count, (LPINPUT)input, sizeof(INPUT) ) )
{
// You can get more information on why this function failed by calling
// the win32 function, GetLastError().
}
There. That wasn't so bad, was it?
Now we just have to take a peek at what's in the clipboard. This isn't as simple as you would first think. The "clipboard" can actually hold multiple representations of the same thing. The application that is active when you copy to the clipboard has control over what exactly to place in the clipboard.
When you copy text from Microsoft Office, for example, it places RTF data into the clipboard, alongside a plain-text representation of the same text. That way you can paste it into wordpad and notepad. Wordpad would use the rich-text format, while notepad would use the plain-text format.
For this simple example, though, let's assume we're only interested in plaintext.
if( OpenClipboard(NULL) )
{
// Optionally you may want to change CF_TEXT below to CF_UNICODE.
// Play around with it, and check out all the standard formats at:
// http://msdn.microsoft.com/en-us/library/ms649013(VS.85).aspx
HGLOBAL hglb = GetClipboardData( CF_TEXT );
LPSTR lpstr = GlobalLock(hglb);
// Copy lpstr, then do whatever you want with the copy.
GlobalUnlock(hglb);
CloseClipboard();
}
else
{
// You know the drill by now. Check GetLastError() to find out what
// went wrong. :)
}
And there you have it! Just make sure you copy lpstr to some variable you want to use, don't use lpstr directly, since we have to cede control of the contents of the clipboard before we close it.
Win32 programming can be quite daunting at first, but after a while... it's still daunting.
Cheers!
Try adding a Sleep() after each SendInput(). Some apps just aren't that fast in catching keyboard input.
Try SendMessage(WM_COPY, etc. ).

Resources