Injecting Mouse Input in WPF Applications - wpf

I've been working on injecting input into a WPF application. What makes this project hard is that I need to be able to inject the input into the application even though it's running in the background (i.e. another application has the input focus). Using the SendInput() function is therefore out of the question.
So far, I've got keyboard input working but am having trouble injecting mouse input.
I used Spy++ to observe the window messages that get sent to the WPF window when I physically click the mouse button. I then simply craft these same mouse messages (such as WM_LBUTTONDOWN and WM_LBUTTONUP) manually and send them explicitly to the WPF window to emulate mouse input.
Unfortunately, this doesn't work as expected (not even when I, for testing purposes, have set the WPF window as the foreground window).
I've added a button to my test WPF window which when clicked displays a message box. Injecting the appropriate mouse messages when I've manually positioned the cursor over the button doesn't cause the button to be clicked, however (i.e. the clicked event isn't fired by the WPF framework).
If I add a handler for mouse clicks on the actual dialog (the client area), that handler does get called if I position the cursor over the dialog itself and inject the same window messages as before:
this.MouseLeftButtonDown += WndMouseDown;
public void WndMouseDown(object sender, EventArgs e)
{
...
}
Strangely enough, if I change the push mode of the button to Press (i.e. it's considered clicked on mouse down rather than the default mouse up), the button clicked event is now fired when I inject the same messages as before. (It's worth mentioning that the handler from the example above correctly fires for both mouse downs and ups, so it'd seem the WPF framework does process both messages successfully.)
It seems like there are some other criteria that need to be fulfilled in order for a mouse clicked event to be fired by the WPF framework. Does anybody know how mouse input is handled internally in WPF, or why it's not interpreting my mouse up and down messages as a click on the button?
(It's worth mentioning that this approach [sending window messages] works fine on ordinary Win32 windows, such as the Start->Run dialog. The difference here is that WPF only has one physical Win32 window and the rest is WPF specific, which means all window messages go to that top-level window rather than the actual button.)
I've been searching high and low for an answer to this and would appreciate any thoughts or ideas.

I'd highly suggest going the UIAutomation route. You create an AutomationElement by window handle. Crawl to the button and invoke it. I'd just like to know how you managed to get the keyboard input working. I am currently trying to resolve the converse issue. How to get a WPF window (I've managed to get a hWnd to it via Win32 calls), to respond to virtual keyboard messages. I've logged ++spy sessions on the window in question and replicated it's input without success.

Use UI Automation to do this - trying to manually simulate input via window messages is a textbook mistake, like trying to start a land war against Russia.

Your strategy is basically sound but in order to send a message to a window owned by another process you must first register the message.
Here is an article explaining the whole business. The sample code is unfortunately in VB but I'm sure that won't stop you.

Related

WPF - Simulate Mouse Move without moving the Mouse

I've a WPF window, which I transfer over websockets via https://wow.codeplex.com/ !
The problem now is, I need to simulate Mouse Events on that window. Clicks already work directly in WPF, but MouseMove does not. So I tried to use SendMessage API, but this does not work!
The problem is, I can not use APIs which really move the cursor, because multiple Clients could be connected!
I suspect some thing is overloading the mouse move events in the codeplex custom window and returning an unhandled exception. Put a debug pointer in the custom window mouse move and watch. You will get it resolved.

Focus indicator on a Microsoft droplist combo box not showing (sometimes)

I have a legacy application that uses Win32 (not MFC) to create graphical screens. One of the control types we use is a combo box with the droplist style (using the new Common Controls module). We have discovered a very strange thing about this control lately.
If I start the application with a mouse click, the focus indicator on the droplist does not show! This can be done using the Start->Run dialog (click OK after filling in the program to run). This can be done using a desktop shortcut (double-click). This can be done by clicking the Go button in Visual Studio 2010 to debug the program. Our user discovered this by using a small Delphi app that acts similar to Start->Run (fill in an entry field of a program to execute, and press a Launch button). If you click the Launch button, the focus indicator is missing.
If I start the application with a keypress, the focus indicator shows! Again, Start->Run dialog (but press Enter to activate the OK button). Desktop shortcut (single click to highlight the icon, then press Enter to activate it). F5 from within Visual Studio. Even the Delphi app - if you tab to the Launch button, and press Space or Enter, our application starts and shows the focus indicator on the drop list.
I have tried to create a small MFC application with just a drop list, and have started that application in all the ways described above. However, this small application always shows the focus indicator of the drop list, regardless of how I start it.
I have examined the styles of both drop lists (on my small application, as well as my actual application). They are slightly different, but changing the style of my small MFC app to match my desktop app doesn't change the behavior. Nor does changing my desktop app to match the style of the small MFC app change the behavior of my desktop app.
I modified my desktop application to require the user to press Enter after the main window is displayed, but before any graphical controls are shown. This causes the focus indicator to now show!
I have found that, when debugging my desktop app, if I set a breakpoint after the main window is shown, but before any controls are drawn, and then press F5 to continue, the focus indicator now shows. But if I click the Go button instead, the focus indicator does not show.
It seems like interacting with the keyboard somehow causes the focus indicator to show.
Note that a drop list combo box looks like a push button with a small down-arrow on the right hand side. The focus indicator is a dashed line near the inside border of the button.
I have struggled with this issue for two days now, with no indication that I am getting closer to an answer. So any insight into what may be causing this will be much appreciated.
The first comment to the question provided a partial answer. Changing the Control Panel setting for keyboard accelerators also causes the focus indicator to show, regardless of how the application is started. However, I believe this is only a partial answer.
When the CP setting is off, keyboard accelerators are supposed to show once you have pressed the Alt key. My understand, based on the blog post that Jonathon Potter linked to, is that focus indicators should show up once the keyboard is used. Entering text into an entry field probably doesn't count as "using the keyboard", but I would expect tabbing between controls to count as such.
Also, if the CP setting is off, the behavior of my application is different, depending on whether I start the application with a mouse click or a key press.
In summary, I find the behavior inconsistent. Perhaps it is correct. I find it hard to say.

Simulating multiple mouse events

I'm attempting to create a UI where I interact with normal WPF controls without the mouse. I want to support multiple cursors so using any regular input simulation (such as SendInput) doesn't work. I also tried interlacing SendInput messages to simulate to mouse cursors but that didn't work either (only got one mouse input). I also have the constraint that I do not want to use Windows MultiPoint.
I've tried sending events to the controls (testing it on Button) through mouse events when I detect my cursor position is above the controls using: MouseEnterEvent, MouseLeaveEvent, MouseMove, MouseDownEvent and MouseUpEvent. But except for the MouseDownEvent, none of it seems to work.
Here is an example of how I send the MouseEnterEvent:
System.Windows.Input.MouseEventArgs e =
new System.Windows.Input.MouseEventArgs(System.Windows.Input.Mouse.PrimaryDevice, DateTime.Now.Millisecond);
e.RoutedEvent = System.Windows.Input.Mouse.MouseEnterEvent;
elementUnderCursor.RaiseEvent(e);
Where elementUnderCursor is a UIElement.
I think that i wanted the same thing for my multimose kinect app be but if you have an emulated mouse driver you would just have to figure out how to tell the app that an event happened with that mouse driver instead of your usb mouse or ps/2 mouse driver.
For instance,
send the mousedown and up event to simulate mouse click for mouse #1,#2. Also, update mouse positon for mouse #1,#2.
My reasoning for the above is i wanted it to work in any application by running a similar program to the above program in the background as a service.
Update dsf works for simulating virtual mouse devices im working on a multiple mouse project with kinect so please look at the current progress at the site: http://kinectmultipoint.codeplex.com.
be warned multiple mouse drivers cant be built overnight.
also, download the windows ddk to simulate the mouse devices. the testgenerichid.wsf script has to be changed for your scenario but its possible.

WPF application gets focus event

My WPF application has windows that are running on separate dispatchers. I need to be able to tell those windows to activate when the main window has focus. What event should I listen too to know when the application has focus?
Application.Activated event is a good place. When you get this event, your application has been activated by the user (either mouse click or keyboard focus switched to it).
It's not clear from the question if you want to know about the application getting focus (in which case, Franci is right) or when the main window gets focus. For the main window (or any particular window), there is a GotFocus event inherited from UIElement http://msdn.microsoft.com/en-us/library/system.windows.uielement.gotfocus.aspx

Windows App Focus: Why does it require a click?

When I have 2 apps open and one has the focus but I want to execute a command in the other app, it requires a click to regain focus and another to execute the command. Is there some good reason why I couldn't take focus on MouseOver? I'm working with a WPF app if that is pertinent. TIA
EDIT: Oddly enough the MouseOvers work without focus.
I would not recommend doing this. This is not a standard way of working in Windows, so you will confuse your users. People are used to clicking into an application (or tabbing) to provide focus.
However, this is a configurable setting via the Accessability Tools in Windows. It can be enabled by choosing "Activate a window by hovering over it with the mouse" globally. Let your users specify this behavior if they want it.
The setting is configurable at a system-wide level. You should never ever override the user's current setting regarding this.
MS Windows Vista -- focus follows mouse (There's also a link on how to do it on XP.)
Edit: Normally, you can click a button on a form and both bring focus to the window and click it at the same time. The origins of the current setting "eating" the initial mouse click that brings focus to a window started as a fix to a bug in the Ribbon UI. The discussion is somewhere in this video: The Story of the Ribbon. Sorry I can't narrow it down more than that, but at least the video is a great insight and work watching - maybe you can send a message to Jensen Harris if you need a faster answer.
Edit 2: I just added a button to a WPF window, and I'm able to click it as long as I can see it - whether or not the window has focus.
You can take focus on MouseOver manually

Resources