Backstory:
I want to write a C program to automate clicks in a program running in OSx (in a desktop setting).
I first tried Using Quartz Event Services to simulate input events. But then I had this problem: Simulating mouse clicks on Mac OS X does not work for some applications, and the answers didn't help in my case.
CGEventRef click1_down = CGEventCreateMouseEvent(NULL, kCGEventLeftMouseDown, CGPointMake(posx, posy), kCGMouseButtonLeft);
CGEventSetIntegerValueField(click1_down, kCGMouseEventClickState, 0);
// This down click works about 5% of the time.
CGEventPost(kCGHIDEventTap, click1_down);
usleep(30000);
CGEventRef click1_up = CGEventCreateMouseEvent(NULL, kCGEventLeftMouseUp, CGPointMake(posx, posy), kCGMouseButtonLeft);
CGEventSetIntegerValueField(click1_up, kCGMouseEventClickState, 1);
CGEventPost(kCGHIDEventTap, click1_up);
// I've tried every combination of CGEventSetIntegerValueField, usleep and CFRelease, nothing seems to help
// The only thing helping is repeating the line: "CGEventPost(kCGHIDEventTap, click1_down);" 100s of times,
// then the down click works about 80% of the time, still not acceptable
I'm now turning to solution #3 suggested here: How can Mac OS X games receive low-level keyboard input events?
(this might also help How can I simulate the touch events by IOHIDEvent?)
I tried with Karabiner by sending a mouse click on key press:
<item>
<name>Right Mousebutton</name>
<identifier>rightMouseButton</identifier>
<autogen>__KeyToKey__ KeyCode::H, PointingButton::LEFT</autogen>
</item>
And this sends the click 100% of the time, but I want to send the click with by writing C code (to have greater control). Tough I'm not sure, Karabiner seems to use IOKit to send events, so I think this should work in my case, if I'm able to send mouse events with IOKit.
So my question is basically: how do I write a C program to simulate a mouse left click with IOKit ? The documentation is very sparse and I didn't manage to do it.
I tried getting inspiration from some projects:
https://github.com/tekezo/Karabiner
https://github.com/NoobsArePeople2/manymouse
Related
I have an unrepeatable bug of unknown origin in my single threaded window manager that occurs fairly infrequently (once every 2-3 weeks). Something happens that causes me to lose keyboard input. Mouse events are still handled properly so I know the event loop is still running, but the key press event is no longer triggered. Actually, the key is no longer grabbed. When I press XCB_MOD_MASK_4+2 to switch to desktop 2, the 2 will show up in the text editor or terminal that currently has the input focus, instead of being grabbed by the window manager. I thought maybe it was related to xcb_allow_events, so via IPC I can execute these three tests (from within the window manager, cmd is received from an external process):
if (strcmp(cmd,"test0")==0)
xcb_allow_events(wm.conn, XCB_ALLOW_ASYNC_KEYBOARD, XCB_CURRENT_TIME);
else if (strcmp(cmd,"test1")==0)
xcb_allow_events(wm.conn, XCB_ALLOW_SYNC_KEYBOARD, XCB_CURRENT_TIME);
else if (strcmp(cmd,"test2")==0)
keyboard();
void keyboard()
{
int i,m,k;
xcb_void_cookie_t cookie;
spawn("/usr/bin/xmodmap -e 'keycode 108 = Super_L'");
spawn("/usr/bin/xmodmap -e 'remove mod1 = Super_L'");
for (i=0; i<LENGTH(key_bindings); i++)
{
m = key_bindings[i].mod;
k = keysc(key_bindings[i].keysym);
info("grabbing key: %s (%d), mod: %d",key_bindings[i].keysym,k,m);
cookie = xcb_grab_key_checked(wm.conn, 0, wm.root, m, k, XCB_GRAB_MODE_ASYNC, XCB_GRAB_MODE_ASYNC);
if (xcb_request_check (wm.conn, cookie))
error("can't grab key");
}
}
None of these tests help. I know the keyboard function works properly because it works on window manager startup. Also I can see in the log file that the key grabs in the keyboard function are actually being executed (without error) when prompted via IPC. The current workaround is to send sigterm to the window manager process, and then restart the wm. At that point everything works fine again.
I'm looking for techniques that might be helpful in tracking down the source of this problem, or in correcting the problem once it occurs (another test). Unfortunately, since I have no clue of the source of this problem, or what triggers it, I cannot make a simple test case to demonstrate. BTW I check the log files when this happens, and I don't see any pattern leading up to the problem. Each function logs an entry on entrance and exit.
Update 2021-02-12: I thought a restart would be a good workaround until I found the root cause of this problem. My restart function contains only one line:
execvp(lwm_argv[0], lwm_argv);
where lwm_argv is the argv provided as an argument to main.
I was very surprised to see that this did not alleviate the problem. I have to completely kill the old process then launch an new one to alleviate the problem. So this problem is PID dependant??? Further, I'm fairly convinced that this problem is somehow related to the stdout/stderr output of other applications launched from within the window manager using execvp. I've stopped launching applications from within the window manager and the problem went away. Any ideas of how launching other applications (and their output) could be affecting the keygrabs within the window manager would be appreciated.
You could try using strace or perf trace on the X server to see what it is doing with the key events. It ought to read them from somewhere in /dev/input and send them as events to connected clients.
If it isn't sending you events, then you might need to dig into its internal state, perhaps by building a debug server and connecting to it with GDB, to see why it isn't sending those events.
But if it is sending events to your WM then they're getting lost somewhere in the library stack.
Is there a way to detect if the mouse has been moved ANYWHERE on the X Server or a keyboard event occured? I need to react on the user doing anything with the X11 input devices.
I only managed to detect events on my own window using GTK.
I am thankful for every information (it does not have to be full code, an entry point would be good enough!)
Yes, you can do this using the Xinput2 extension. A complete, but rather small, tool which does this for cursor events can be found here (unclutter-xfixes). As a disclaimer, I am the author of that tool.
Another good resource in tutorial form can be found here.
Using XInput2 has multiple benefits:
No need to constantly poll the position (resource efficient)
Does not interfere with / break applications like selecting mouse events on all windows would.
What you don't get easily using Xinput2 is the exact position (but you can query it when you need it), but my understanding is that you don't need it anyway.
Once you loaded the extension, which I won't show here, you can select all events like this:
XIEventMask masks[1];
unsigned char mask[(XI_LASTEVENT + 7)/8];
memset(mask, 0, sizeof(mask));
XISetMask(mask, XI_RawMotion);
XISetMask(mask, XI_RawButtonPress);
XISetMask(mask, XI_RawKeyPress);
masks[0].deviceid = XIAllMasterDevices;
masks[0].mask_len = sizeof(mask);
masks[0].mask = mask;
XISelectEvents(display, DefaultRootWindow(display), masks, 1);
XFlush(display);
In your event queue, you can now look for the corresponding events.
For modern X11 implementations, xinput --test-xi2 --root will display great detail about all X11 input events available on your root window. I use this in a shell script that needs to wait on any input event:
echo "DEBUG $(date) waiting on X event"
xinput --test-xi2 --root | head -n 15 >/dev/null
echo "DEBUG $(date) got X event"
I wrote a large complex C program around 20(!) years go. As far as I can recall it worked fine at the time in all respects - it was probably running on windows 95.
Now I need to use it again. Unfortunately the radio buttons in it do not appear to work properly any more (the ordinary push buttons are all behaving correctly). As I click on the radio buttons, I get some feedback that windows is acknowledging my click in as much as I see a dotted line appear around the button's text and the circle of the button goes grey for as long as my finger is on the button, but when I take my finger off I see that the selected button has not changed.
My suspicion is that I was perhaps getting away with some bad practice at the time which worked with windows 95 but no longer works on newer versions of windows, but I'm struggling work out what I did wrong. Any ideas?
EDIT: Its difficult to extract the relevant code because the message handling in this program was a tangled nightmare. Many buttons were created programatically at runtime and there were different message loops working when the program was in different modes of operation. The program was a customisable environment for running certain types of experiment. It even had its own built-in interpreted language! So I'm not expecting an answer like "you should have a comma instead of a semicolon at line 47", but perhaps something more like "I observed similar symptoms once in my program and it turned out to be ..... " .. or perhaps "the fact that the dotted rectangle is appearing means that process AAA has happened, but maybe step BBB has gone wrong".
EDIT: I've managed to extract some key code which my contain an error...
char *process_messages_one_at_a_time()
{
MSG msg;
int temp;
temp = PeekMessage(&msg,winh,0,0,PM_NOREMOVE);
if (temp)
{
GetMessage (&msg, NULL, 0, 0);
if (msg.message == WM_LBUTTONUP)
{
mouse_just_released_somewhere = TRUE;
}
TranslateMessage (&msg);
DispatchMessage (&msg);
}
if (button_command_waiting)
{
button_command_waiting = FALSE;
return (button_command_string);
}
else
{
return (NULL);
}
}
There are two simple things to check when using radio buttons. First is to make sure that each has the BS_AUTORADIOBUTTON property set. The second is to make sure that the first button in the tab order and the next control after the set of buttons (typically a group box) have the WS_GROUP property set, while the other buttons have it clear.
A few suggestions:
I'd try to use spy++ to monitor the messages in that dialog box, particularly to and from the radiobutton controls. I wonder if you'll see a BM_SETCHECK that your program is sending (ie, somewhere you're unchecking the button programatically).
Any chance your code ever checks the Windows version number? I've been burned a few times with an == where I should have used a >= to ensure version checking compatibility.
Do you sub-class any controls? I don't remember, but it seems to me there were a few ways sub-classing could go wrong (and the effects weren't immediately noticeable until newer versions of Windows rolled in).
Owner-drawing the control? It's really easy to for the owner-draw to not work with newer Windows GUI styles.
Working with old code like that, the memories come back to me in bits and pieces, rather than a flood, so it usually takes some time before it dawns on me what I was doing back then.
If you just want to get the program running to use it, might I suggest "compatibility mode".
http://www.howtogeek.com/howto/windows-vista/using-windows-vista-compatibility-mode/
However, if you have a larger, expected useful life of the software, you might want to consider rewriting it. Rewriting it is not anywhere near the complexity or work of the initial write because of a few factors:
Developing the requirements of a program is a substantial part of the required work in making a software package (the requirements are already done)
A lot of the code is already written and only parts may need to be slightly refactored in order to be updated
New library components may be more stable alternatives to parts of the existing codebase
You'll learn how to write current applications with current library facilities
You'll have an opportunity to comment or just generally refactor and cleanup the code (thus making it more maintainable for the anticipated, extended life)
The codebase will be more maintainable/compatible going forward for additional changes in both requirements and operating systems (both because it's updated and because you've had the opportunity to re-understand the entire codebase)
Hope that helps...
Almost every tutorial I find tells me to do this for my event loop:
XEvent event;
while (true)
{
XNextEvent(display, &event);
switch (event.type)
{
case Expose:
printf("Expose\n");
break;
default:
break;
}
}
However, clicking the X to close the program results in this message.
XIO: fatal IO error 11 (Resource temporarily unavailable) on X server ":0"
after 10 requests (10 known processed) with 0 events remaining.
It is indeed strange to me that the examples suggest using an infinite loop. That doesn't sound natural, and my other X11 programs don't do that. So I searched around. I found out how to capture the window close event.
Atom wmDeleteMessage = XInternAtom(mDisplay, "WM_DELETE_WINDOW", False);
XSetWMProtocols(display, window, &wmDeleteMessage, 1);
XEvent event;
bool running = true;
while (running)
{
XNextEvent(display, &event);
switch (event.type)
{
case Expose:
printf("Expose\n");
break;
case ClientMessage:
if (event.xclient.data.l[0] == wmDeleteMessage)
running = false;
break;
default:
break;
}
}
That works. It exits without errors. ... But I refuse to believe this is the normal way to do things. I mean, is this the only way to properly exit an X11 app? It seems like a lot of work just to capture the close event. How do I make a 'proper' event loop? Why is the close event so deeply buried? What am I missing?
The problem lays in the communication between X Server and the Window Manager.
When you call XCreateWindow or XCreateSimpleWindow, the X Server creates your window (not showing it until you explicitly map it on the screen by calling XMapWindow), and then the Window Manager is responsible for attaching all the decorations and buttons and system menu around your window.
You can call XDestroyWindow on your own to remove the window, and this usually means it just disappears from the screen, but your program is still running and the connection to the X Server is still open, so you can send it some more requests.
The problem begins when the user clicks that little X button attached to your window by the Window Manager, because it is not created by the X Server and it is not his business to decide what to do then. Now it's all in hands of Window Manager.
If the Window Manager simply called XDestroyWindow on your window, it would cause a problem if your application wanted to capture the closing event to do something before the window gets destroyed. So the convention has been established between the X Server and the Window Managers to handle this process.
The default behavior of most Window Managers is to destroy the window and close the connection with the X server, because this is what most users of Window Managers would expect: that when they close the window, the program will end (and the connection to the X Server will close with the closed window). And then, when you try to call XCloseDisplay(display), it will cause the IO error you've mentioned, because the connection to the server is already closed and the display structure is invalid.
Here's an excerpt from the Xlib documentation which explains this:
Clients that choose not to include WM_DELETE_WINDOW in the WM_PROTOCOLS property may be disconnected from the server if the user asks for one of the client's top-level windows to be deleted.
Yeah, it would be great if they didn't hide it so deep in their docs, though :-P
But when you already find it, fortunately it also hints for the solution.
If you want a different behavior (that is, to capture the closing event from the Window Manager), you need to use the WM_DESTROY_WINDOW protocol.
Another excerpt from the docs:
Clients, usually those with multiple top-level windows, whose server connection must survive the deletion of some of their top-level windows, should include the atom WM_DELETE_WINDOW in the WM_PROTOCOLS property on each such window. They will receive a ClientMessage event as described above whose data[0] field is WM_DELETE_WINDOW.
I had the same error and I wanted to know exactly what causes it and why. It took me some time to figure it out and find the proper explanation in the doc, so I put my explanation here to save the time of others uninformed.
There are no such things as "exit button" or "application" or "close event" in X11. This is by design.
Window decorations, exit buttons and many the other things we depend upon are not built into X11. They are implemented on top of the core X11 instead. The name of the particular set of conventions responsible for wmDeleteMessage is ICCCM, look it up.
Xlib only deals with the core X11 protocol. No built-in close event there.
There are toolkits that make dealing with ICCCM and all other things that are not built into X11 easier (GTK, wxWindows, Qt, ...) You probably want to use one of those.
I'd like to intercept the WM_DELETE_WINDOW message that is posted to a certain selection of windows that an application I'm writing (AllTray), so that I can act on it instead of the application receiving it. I'm currently looking at trying this at the GDK level via gdk_display_add_client_message_filter if possible, but I'd be happy with an Xlib solution if there is one as well; it seems to be possible, but I just don't seem to be understanding how I am to do it successfully.
Currently, I have two programs (written in C) that I am trying to use to get this figured out, the first one does nothing but create a window and register that it knows about WM_DELETE_WINDOW, and the second one attempts to catch that message, but seems to fail in doing so; it appears to do precisely nothing. Am I understanding the documentation wrong on this, or is there something additional that I need to be doing (or do I need to avoid using GDK entirely for this)?
The background is this: Prior to my re-write of AllTray, the way it would do things appears to be to try to intercept a mouse-click on the X button itself. For some window managers, this worked properly, for others it didn't work at all, and for others, the user had to configure it manually and instruct AllTray where the button for closing the window was. What I am looking for is a solution that doesn't involve a LD_LIBRARY_PRELOAD and will work for any window manager/application combination that conforms to the current standards and sends a WM_DELETE_WINDOW ClientMessage when the window is closed.
UPDATE: I'm still looking for an answer. The route that I am taking at the moment is to try to reparent the window and manage it myself, but I just cannot make it work. Upon reparenting, I don't seem to be able to get it back in any way. I may be missing something very fundamental, but I can't figure out how to actually make it appear it my own window again, to bring it back on the screen.
UPDATE 2: Alright, so I've hit another brick wall. The X server documentation says to set the StructureNotifyMask on the window's event mask to receive both MapNotify and ReparentNotify events. I'd be interested in receiving either. My current thinking was to create a window that served just as an event receiver, and then when I get events for interesting things, act on them by creating and reparenting. However, this simply doesn't seem to be working. The only events I actually receive are PropertyNotify events. So, this route doesn't seem to be doing very much good, either.
I don't know X11, but I googled using "Intercept WM_DELETE_WINDOW X11" as keywords. Found 17k - MarkMail and Mplayer-commits r154 - trunk/libvo. In both cases they are doing the same thing.
/* This is used to intercept window closing requests. */
static Atom wm_delete_window;
within static void x11_init(),
XMapWindow(display, win);
wm_delete_window = XInternAtom(display, "WM_DELETE_WINDOW", False);
XSetWMProtocols(display, win, &wm_delete_window, 1);
then, within static int x11_check_events(),
XEvent Event;
while (XPending(display)) {
XNextEvent(display, &Event);
if (Event.type == ClientMessage) {
if ((Atom)Event.xclient.data.l[0] == wm_delete_window) {
/* your code here */
}
}
}
See XInternAtom, XSetWMProtocols and XNextEvent.
After I wrote the above, I found Handling window close in an X11 app:
When a user clicks the close button
[x] on our X11 application we want it
to pop a a dialog asking “do you
really want to quit?”. This is a plain
X app. No fancy GTK or QT widgets
here. So how to catch the “window is
being closed” message?
The answer is to tell the Window
Manager we are interested in these
event by calling XSetWMProtocols and
registering a WM_DELETE_WINDOW message
with it. Then we’ll get a client
message from the Window Manager if
someone tries to close the window, and
it won’t close it, it’ll leave that us
up to us. Here’s an example….
// example.cpp
#include <X11/Xlib.h>
#include <X11/Xatom.h>
#include <iostream>
int main()
{
Display* display = XOpenDisplay(NULL);
Window window = XCreateSimpleWindow(display,
DefaultRootWindow(display),
0, 0,
500, 400,
0,
0, 0);
// register interest in the delete window message
Atom wmDeleteMessage = XInternAtom(display, "WM_DELETE_WINDOW", False);
XSetWMProtocols(display, window, &wmDeleteMessage, 1);
std::cout << "Starting up..." << std::endl;
XMapWindow(display, window);
while (true) {
XEvent event;
XNextEvent(display, &event);
if (event.type == ClientMessage &&
event.xclient.data.l[0] == wmDeleteMessage) {
std::cout << "Shutting down now!!!" << std::endl;
break;
}
}
XCloseDisplay(display);
return 0;
}
Unfortunately, the best answer to this question is a series of non-answers; there are technically ways to accomplish it, but they all have downfalls that make them extremely impractical:
Create an X11 proxy for an application, passing all X11 protocol messages back and forth between the application and the X server. The proxy would then filter out any interesting messages. The downside to this is that this is an awful lot of overhead for a single little tiny feature, and the X11 protocol is complex. There could also be unintended consequences, which makes this an even more unattractive option.
Launch as a standard application that acts as an intermediary between the window manager and “interesting” client applications. This breaks some things, such as XDnD. In effect, it is not unlike the first option, except that the proxy is at the Window level as opposed to the X11 protocol level.
Use the non-portable LD_PRELOAD library trick. This has several downsides:
It is non-portable across dynamic linkers: not all dynamic linkers support LD_PRELOAD, even among UNIX-like systems.
It is non-portable across operating systems: not all operating systems support featureful dynamic linkers.
It breaks network-transparency: the shared object/dynamic link library must reside on the host as the child process that is being executed.
Not all X11 applications use Xlib; it would be necessary to write one LD_PRELOAD module for each of the libraries that an application might use to talk with X11.
In addition to the last point, not all applications would be susceptible to LD_PRELOAD even if they ran under a linker that supported it, because they may not use a shared object or DLL in order to communicate with X; consider, for example, a Java application which uses an X11 protocol library written in Java itself.
On some UNIX-like operating systems, LD_PRELOAD libraries must be setuid/setgid if they are to be used with setuid/setgid programs. This is, of course, a potential security vulnerability.
I am quite sure that are more downsides that I cannot think of.
Implement an extension to the X Window system. Non-portable among X11 implementations, complex and convoluted as all get out, and absolutely out of the question.
Implement extensions or plug-ins to window managers. There are as many window managers as there are opinions on window managers, and therefore this is utterly infeasible.
Ultimately, I was able to finally accomplish my goal by using a completely separate mechanism; anyone who is interested, please see the Close-to-Tray support in AllTray 0.7.5.1dev and later, including the git master branch available on github.
Ok, to elaborate on my earlier suggestion, you might want to investigate XEmbed. At the least, that might give you some ideas to try.
Failing that, I'd have a look at how other similar software might be working (e.g. wmdock, or how GtkPlug/GtkSocket is implemented), though I believe in both those cases explicit support is required in the applications.
Hope that is more helpful.
You should read ICCCM that tells you how window manager communicates with client. Most of WM will create a frame window to contain your top-level window via reparenting. Thus, if your reparent may break the relationship known by WM and your client window.