I'm trying to create a more-or-less high-level UI test for a WPF control (but without the complexities of UI Automation – UIA works, but I'd rather avoid it if possible), and found RaiseEvent and InputManager.ProcessInput for synthesizing input events. What I couldn't figure out, though, is how mouse (and touch) events can be created with a particular position. MouseEventArgs has a GetPosition method that returns the position for the event, but how can I set it?
The reason why I avoided UI Automation here is two-fold: (1) I want the tests to run headless, if possible, thus injecting events instead of simulating mouse movement; (2) I need to test touch input as well, for which I didn't find a library to simulate them so far.
I tried inheriting from MouseDevice only to find that there are no virtual methods that are of use to me here, so that seems a dead end as well.
Related
In XAML-based apps, quite often I need to manipulate on-screen elements, resize or move stuff around. The situation arises usually for very custom UI controls.
The timing of stuff loading and data-binding etc. can give rise to irritating timing issues.
Traditionally, my colleagues and I have worked around these problems by 're-dispatching' work to the UI thread so it runs later, when dependent properties are in the 'final' states.
Is there a better way to do this? What strategies have you found that work?
The LayoutUpdated event can be very noisy, fine-grained, and deregistering requires forgoing a Lambda for a method and thus not being able to access enclosed variables from the outer logic - its a bit of a 'mare.
Edit
I'll give a tangible example. A custom control draws an outline around a face when doing facial recognition, so we're talking totally custom stuff here, nothing XAML does out the box.
The image needs to be scaled and sized and the paths and geometries scaled and sized so its all in alignment.
When programmatically changing the heights and widths of elements, the impact is not immediate, its only once the UI thread is relinquished back to the XAML framework does the rendering subsystems rearrange everything.
Thus, logic that depends upon the changed values needs to run after the framework has rearranged everything, and so this work needs scheduling to occur later on the UI thread, hence resorting to dispatching. It smells.
Many events and virtuals are called at times when requisite data is not yet available. Sometimes, work needs to be done upon data arrival (i.e. property change notification) which is does not typically trigger the XAML layout events. In this case, we resort to dispatcher hacks.
What about using the Loaded event?:
Loaded += LoadedHandler;
...
private void LoadedHandler(object sender, RoutedEventArgs e)
{
// Do something here when controls have loaded
}
I discovered that in WPF, this dependency property metadata argument exists to help with the problems I'm talking about.
FrameworkPropertyMetadataOptions.AffectsRender
The enum doesn't exist in Windows Store applications, presumably since its designers think the InvalidateArrange method is a suitable alternative.
Possibly, we should be a) invalidating the arrangement within our DP change handlers b) making these arrangements over multiple passes c) tolerating incorrect layout in-between passes.
Still, the using lambdas with the dispatcher is nice since variables used in calculations can be referenced from inside the closure.
I'm attempting to pan around the contents of a ScrollViewer in the same way you would pan around in a PDF document (scroll to zoom in/out, click + drag to pan) ScrollViewer has this functionality built in for Touch events (PanningMode), however this doesn't seem to translate to Click+Drag events. Is there a way to tell it to/emulate this functionality?
Panning is enabled internally by four virtual methods implemented by ScrollViewer:
OnManipulationCompleted
OnManipulationDelta
OnManipulationInertiaStarting
and OnManipulationStarting
So where are these virtual methods defined. Lets go up the hierarchy. We see that it's called on UIElement within the OnManipulationCompletedThunk (I'm sure there are concomitant methods for the rest as well).
Everything is still private at this point, we want something to tap into. Unfortunately this is the point at which both reflector and ILSpy failed me (well actually it didn't, the call site is in a different dll (PresentationCore) that I don't have loaded brb). Ok back. Once we look in PresentationCore, we have a vague idea that dependencyProperties are registered statically, so we find the .cctor. There are a couple lines of interest here.
ManipulationCompletedEvent = Manipulation.ManipulationCompletedEvent.AddOwner(typeof(UIElement));
and
EventManager.RegisterClassHandler(typeof(UIElement), ManipulationCompletedEvent, new EventHandler(UIElement.OnManipulationCompletedThunk));
We see that OnManipulationCompletedThunk is the registered callback for this class handler that listens for ManipulationCompletedEvent. Also, ManipulationCompletedEvent is not originally defined on UIElement, it is borrowed from the Manipulation static class via AddOwner.
Doing a search for the Manipulation class, I see that it's in the System.Windows.Input namespace within the same assembly. Is it public, yep. Cool! so at this point I know that if I fire the ManipulationCompletedEvent or any of it's buddies, it will eventually call into ScrollViewer. http://msdn.microsoft.com/en-us/library/system.windows.input.manipulation.aspx
In the documentation for this public static class, I see there are a bunch of interesting and possibly useful methods. The only one that isn't readily obvious is AddManipulator. What does this thing do? Clicks.. reads.. oh, "Each touch point is an IManipulator object. For example, if you use two fingers to resize an object, a TouchDevice, which implements IManipulator, is created for each finger." So TouchDevice is an IManipulator. Maybe that will give me some idea of how to create my own manipulator.
The Properties on TouchDevice give some clue as to features it supports. It's sort of like a MouseDevice (has the concept of capture, DirectlyOver etc.), but it doesn't support manipulation in the same way. Rather we want to do Manipulation in response to Mouse events. Lets look harder at TouchDevice to see how it really implements some of these features.
The methods TouchDevice is implementing are GetPosition and ManipultionEnded
GetPosition returns this.GetTouchPoint(relativeTo).Position;
relativeTo is a parameter
ManipulationEnded calls OnManipulationEnded forwarding a bool parameter named cancel. Not sure what cancel does yet. oh, turns out it's not used, weird but ok. This basically sets capture to null. Kindof at the end of the rabbit hole here so we'll have to back up and reevaluate.
All I really want to do is raise events manually on the UIElement and see if it works. The RaiseEvent method on UIElement should work for this. going to try brb. Err wait I missed something, all the events defined on the Manipulation class are marked as internal.
Clearly these events are meant only for internal consumption, and short of doing reflection we don't have an avenue there.
I think maybe using the Manipulation feature is overkill for what you're trying to do. There is probably a way to implement this with just drag event and a canvas.
Also, found this while googling and thought it might be of some relevance http://multitouchvista.codeplex.com/
I'm improving standart WPF TabControl. I want to add undocking functionality to it:
user drags the page just outside the TabControl and this page undocks in the window.
I want two events in this control - PageDragStart (raises when the page dragged outside) and PageDragEnd (raises when the page dropped outside)
I've got no problem with the first event.
But the second... OnDrop doesn't call, because the item dropped outside the tabcontol container. How can I know that it was dropped?
P.S. I want a universal control (so, undocking functionality shouldn't be connected and hardcoded with the window tabcontrol is placed or something like this)
Why use DoDragDrop at all? As I was reading your description, using Mouse.Capture by itself seemed the obvious solution:
Handle OnMouseLeftButtonDown on the tab and start capture
Handle OnMouseMove on the tab and update the cursor based on hit testing
Handle OnMouseLeftButtonUp on the tab, and stop the capture and make the appropriate change
The reasons you might ever consider DoDragDrop over simple mouse capture are:
Integration with Windows' OLE drag and drop so you can drag and drop between applications and technologies
Modal nature of DoDragDrop call (which actually seems to be more of a disadvantage to me)
Automated hit testing of targets
Standardized "drop operation" API to allow unrelated applications to handle copy vs move, etc.
You apparently don't need the OLE integration or multi-application support and you want to customize the hit testing, so it seems that DoDragDrop has no advantages over directly handling the mouse capture.
I solved the problem - in rather brutal and unsafe way. But for it's gonna work as the temporary solution.
Well, when I'm raising PageDragStart event, I call Mouse.Capture(this, CaptureMode.SubTree);
When the page is dropped somewhere - DoDragDrop throws different exceptions (COMException, NullReference (I couldn't find which object is null) and some others I don't remember).
I catch exception and call PageDragEnd event (if the property IsPageDraggingOut set to true).
As far as you can see this solution is really dirty and bad. But it works.
So, any other ideas (or some ideas how to work with Mouse.Capture properly)?
i would like to create a simple winforms or wpf application where i can drag and drop virtual "cards". this below is not exactly what i want to do, but it the closest thing that i found on the web to represent the user interface.
http://www.greenpeppersoftware.com/confluence/plugins/advanced/gallery-slideshow.action?imageNumber=1&pageId=24870977&decorator=popup&galleryTitle=Task+board+and+transitions
so basically i want to have columns in the GUI where i can drag and drag from one to the other.
My questions are:
would this be easier in winforms or wpf
where do i start?
In both winForms and WPF dragging and dropping can be done in a similar way by working with the events on the target DragOver and Drop.
However with WPF you have other options. You will also be able to make the application look better by having a thumbnail as you drag (this is possible in winforms but harder to achieve).
Have a look at this WPF sample it uses a helper class and think it does exactly what you need.
I agree with John in that WinForms and WPF are quite close to one another w.r.t. drag'n'drop. But WPF offers more of a "common base" for ItemsControl, allowing to implement more independent of the final UI elements used (ListBox, ListView, TreeView... can be easily switched). And obviously WPF allows much more fancy effects.
I would strongly recommend this blog post:
http://www.beacosta.com/blog/?p=53
both for some drag'n'drop basics and for a clean WPF drag'n'drop approach.
It shows a nice implementation of a rather generic helper for drag'n'drop from/to WPF ItemsControls, I really like that "Insertion Adorner". And I do like that the drag'n'drop code is nicely separated from the user control itself by using attached properties, which makes it much easier to use and maintain.
It would probably be slightly easier in WPF because of the Thumb control which provides easy to use built-in support for dragging. (If I remember correctly, in WinForms you would need to handle the mouse events yourself, whereas the WPF Thumb does this for you and translates them into drag start, delta and end events.)
However if you are much more familiar with one framework than the other than that would probably dwarf the difference that the Thumb control would make.
You should also have a look around for toolkits/frameworks that could handle this for you -- I think they exist for both WinForms and WPF (not sure though).
A good way for darg and drop are explained as
Detect a drag as a combinatination of MouseMove and MouseLeftButtonDown
Find the data you want to drag and create a DataObject that contains the format, the data and the allowed effects.
Initiate the dragging by calling DoDragDrop()
Set the AllowDrop property to True on the elements you want to allow dropping.
Register a handler to the DragEnter event to detect a dragging over the drop location. Check the format and the data by calling GetDataPresent() on the event args. If the data can be dropped, set the Effect property on the event args to display the appropriate mouse cursor.
When the user releases the mouse button the DragDrop event is called. Get the data by calling the GetData() method on the Data object provided in the event args.
You can find the complete article here
Background: I have a little video playing app with a UI inspired by the venerable Sasami2k, just updated to use VMR9 (i.e. Direct3D9 with DirectShow) and be less unstable. Currently, it's a C++ app using raw Win32, through necessity: none of the various toolkits are worth a damn. WPF, in particular, was not possible, due to its airspace restrictions.
OK, so, now that D3DImage exists it might be viable to mix and match D3D/VMR9/DirectShow and WPF. Given past frustrations with Win32's inextensibility, this seems like a good thing.
But y'know, I'm falling at the first hurdle here.
With Win32 I have created (very easily) a borderless window that's resizable, resizes proportionately, snaps to the screen edges, and takes up the whole screen (including taskbar area) when maximized. It's a video app, so these are all pretty desirable properties.
OK, so, how to do the same with WPF?
In Win32, I use:
WM_GETMINMAXINFO to control the maximize behaviour
WM_NCHITTEST to control the resize borders
WM_MOVING to control the snap-to-screen-edges
WM_SIZING to control the resize aspect ratio
However, looking at WPF it seems that the various events arrive too late, unless I'm misunderstanding the documentation?
For example, I don't know when I'm mid-move, as LocationChanged says it fires only once the window has moved (which is too late).
Similarly, it appears that StateChanged only fires once the window has been restored/maximized (when I need the information prior to the maximize, to tell the system the correct maximize size).
And I seem to be completely overlooking where the system tells me about resizes. Likewise the hit testing.
So, uh, am I missing something here, or do I have no choice but to drop back to hooking the wndproc of this thing anyway? Can I do what I want without hooking the WndProc?
If I have to use the WndProc I might as well stick with my existing codebase; I want to have simpler, cleaner UI code, and moving away from the WndProc is fundamental to this.
If I do have to hook the WndProc, I have to wonder--why? Win32 has got the sizing/sized, moving/moved, poschanging/poschanged window messages, and they're all useful. Why wouldn't WPF replicate the same set of events? It seems like an unnecessary gap in functionality.
Plus, it means that WPF is tied to a specific USER32-dependent implementation. This means that MS can't (in Windows 7 or 8, say) invert the display layer to make WPF "native" and emulate HWNDs and WndProcs for legacy apps--even though this is precisely what MS should be doing.
OK, to answer my own question, I was missing Adorners (never came back in any of the searches I did, so it doesn't seem that they're as widely known as they perhaps should be).
They seem rather more complex than the WndProc overrides, unfortunately, but I think it should be possible to manhandle them into doing what I want.
And I seem to be completely overlooking where the system tells me about resizes. Likewise the hit testing.
For the resizing you're indeed missing the SizeChanged event.
AFAIK there is sadly no OnSizeChanging, OnLocationChanging and OnStateChanging event on a Window in .NET
I saw that one, but as far as I can tell it only fires after the size has changed, whereas I need the event to fire during the resize. Unless I'm misreading the docs and it
actually fires continuously?
It does not fire continuously but you can probably use the ResizeBegin and ResizeEnd events and be able to do that.
Aren't they WinForms events?
Hmm, you're right.
In code you can set the WindowStyle property to "None" and WindowsState to "Maximized"
Im not sure what the Xaml would look like.
Can you perhaps override the ArrangeOverride and/or MeasureOverride to make up for those missing resize events? Measure is the first pass, and occurs when a layout needs to adjust for a new size, so it's kind of like a size changing event.