I'm rendering a WPF grid with multiple elements (buttons, textbox, ...) to a bitmap which is then used as a texture for a 3D surface in a Direct3D scene. For user interaction I create a 3D ray from the 2D mouse cursor position into the 3D scene finding the intersection point with the gui surface. So I know where the user has clicked on the WPF grid, but from there I'm stuck:
How can I simulate mouse events on the WPF elements while they are not actually shown in an open window but rendered off-screen?
Recently, I was looking into UIAutomation and RaiseEvent but these are used to send events to individual elements, not the whole visual tree. Traversing the tree manually and looking for elements at the cursor position would be an option but I haven't found a way to do this accurately. VisualTreeHelper.HitTest is a good start but instead of finding TextBox it finds TextBoxView and instead of ListBox it finds a Border.
EDIT: returning HitTestResultBehavior.Continue in the result callback of HitTest lets me walk through all elements at a given point. I can now send mouse events to all these elements but the values of the MouseEventArgs object are those of the real mouse. So I have to create a custom MouseDevice which apparently is impossible.
PointHitTestParameters p = new PointHitTestParameters(new Point(
((Vector2)hit).X * sourceElement.ActualWidth,
(1 - ((Vector2)hit).Y) * sourceElement.ActualHeight));
VisualTreeHelper.HitTest(sourceElement,
new HitTestFilterCallback(delegate(DependencyObject o)
{
return HitTestFilterBehavior.Continue;
}),
new HitTestResultCallback(delegate(HitTestResult r)
{
UIElement el = r.VisualHit as UIElement;
if (el != null)
{
MouseButtonEventArgs e = new MouseButtonEventArgs(Mouse.PrimaryDevice, 0, MouseButton.Left);
if (leftMouseDown) e.RoutedEvent = Mouse.MouseDownEvent;
else e.RoutedEvent = Mouse.MouseUpEvent;
el.RaiseEvent(e);
}
return HitTestResultBehavior.Continue;
}), p);
You might be able to send windows messages (like WM_MOUSEMOVE) to the WPF window's HWND, via the PostMessage(..) win32 method. I believe these messages would be read by WPF and executed as if it came from a user.
If you are feeling really brave, you can try my hack to get access to the WPF IDirect3DDevice9Ex. From there you can copy the backbuffer in the swapshchain to your d3d scene. If you are using 9Ex (Vista D3D9), you can take advantage of the shared resources (share surfaces/textures/etc between devices and processes) feature to help with performance.
You may still have to do some trickery with windows messages for interactivity.
Related
For a Winforms textbox, when you click and drag over the text, it highlights it. Is there a way to determine which direction the user dragged over?
There is no way to get this info using the Windows TextBox selection API. For example, the EM_GETSEL message defines the starting and ending character positions of the selection, but in a predefined (sorted) order.
However you could get this info by handling the control's MouseMove event. For example:
textBox1.MouseMove += new MouseEventHandler(textBox1_MouseMove);
void textBox1_MouseMove(object sender, MouseEventArgs e)
{
Control tbCtrl = sender as Control;
// the mouse coordinate values here are relative to the coordinates of the control that raised the event
int mouseX = e.X;
...
}
By applying some logic to mouseX you could potentially discover the average direction the cursor is moving. It would work best if the user is making a linear motion. You could also handle the textbox's drag-and-drop event for similar mouse information if you only wanted the event raised while the user was dragging the mouse.
I try to create a window, that can be moves or resized via multi-touch gestures. I tried it this way. I captures the TouchDown-Event of the window and saved all active TouchDevices in a List, to know which TouchDevices are active. I catch updated and deactivated event of the TouchDevices to know when they are moved and when they are deactivated. I save the Left and Top Property of the Window and the position where the TouchDevice started and everytime the Updated event is called I move the Window to the new Position relative to the new position of the TouchDevice. This works if I move the finger. But if I don't move the finger (or just very little) the window suddenly began shaking (moving chaotically) and then soon disappears to a position outside of the screen.
I think the problem here is, that the function "GetTouchPoint" of the TouchDevice only give relative coordinates related to the window (even if I set the parameter null instead if the window reference). And because the Window moves the relative position of the TouchDevice (that doesn't move) changes too. So I did a research but wasn't able to find a way to determine the screen coordinates of the touch device.
So I hope anyone can help me how to get the absolute coordinates of the TouchDevice. Or can help me find another way to "DragMove" the window with touch.(I tried DragMove, but that only works for mouse clicks, not TouchDowns) Also I like to resize the window when two Touch Devices are active and therefor I also need absolute coordinates because otherwise same effect happens.
I ran into this issue because my taskbar is on the right edge of the screen, effectively pushing the maximized window to the right. This issue also arises when the application's window is not maximized, and floating somewhere on the screen.
Here is an extension method that fixes the coordinates based on the application window's position.
public static Point FixCoordinates(this Point point)
{
var left = Application.Current.MainWindow.Left;
var top = Application.Current.MainWindow.Top;
return new Point(point.X + left, point.Y + top);
}
You may want to pass in a window that hosts your touched control as a parameter. In my case it is application's main window.
Also, since you tagged the question with "multi-touch", here is a method which averages multiple touch coordinates:
public static Point GetAverage(this IEnumerable<Point> points)
{
var averageX = points.Average(p => p.X);
var averageY = points.Average(p => p.Y);
return new Point(averageX, averageY);
}
And I use it in the code like this:
private void TouchAdornment_TouchDown(object sender, TouchEventArgs e)
{
var touchPosition = (sender as UIElement).TouchesOver.Select(t => t.GetTouchPoint(null).Position).GetAverage().FixCoordinates();
}
I have stack panel with custom controls in it. I attach standard MouseDragElementBehavior to each item.
When I drag to the right the item moves underneath the other items.
What would be a viable solution to create better user experience - to show better visual cue - how the item moves and where is it going to be dropped.
After a bit of tinkering I realised that nothing can be dragged within stack panel to the right not being coverd by other elements .. unless you drag the very right item..
What I did to resolve it:
Created a visual cue (half transparent shape of a generic item to represnt it during the drag operation)
Made the cue invisible (width=0) and keep it always as the very last element of the stack panel children
Subscribed the stack panel to mouse left button up, down, move
Emulated drag/drop with code
Once the drag initated I turn the cue to visible and set its translate transform to the current mouse coordinates
Adjust translate transform on every mouse move event
On drop I hide the cue again and rearrange items in a way I want.
To stress again - whatever way you do - you have to manipulate with the last element in StackPanel.Children collection....
If the MouseDragElementBehavior doesn't work the way you need it to, you could always descend from the class and customize it to your needs. For example:
public class DragBehvaior : MouseDragElementBehavior
{
public DragBehvaior()
{
this.DragBegun += new MouseEventHandler(DragBehvaior_DragBegun);
}
void DragBehvaior_DragBegun(object sender, MouseEventArgs e)
{
var element = this.AssociatedObject as UIElement;
// Calculate the correct ZIndex here.
Canvas.SetZIndex(element, 100);
}
}
We use Windows Forms and custom user controls, and I would like to be able to rotate the panel hosting the userControl in a particular form. I have seen similar functionnalities with WPF, but I can't use it for the moment. Is it possible to achieve the rotation of a panel and its children using possibly built-in .NET methods or GDI+?
I have seen some pretty cool visual effect with menus that are displayed in game development, so I was wondering if it would be possible to create similar effects using Windows Forms.
Rotating a panel and its children in Windows Forms is not something directly supported, and I think it will end up being a buggy headache that could easily suck up lots of time. It's especially painful to think about when you could do this in WPF with zero lines of C# code and only a tiny bit of XAML.
You can use rotations in GDI+ by calling the RotateTransform method on a Graphics object.
However, rotating an entire control is not so simple, and will depend heavily on how the control is implemented.
If it's a composite UserControl that has other controls inside of it, you're out of luck.
If it's a sinlge control that paints itself, try inheriting the control, overriding the OnPaint method, and calling RotateTransform on the Graphics object. However, you will probably have trouble with it. In particular, you will probably need to override all of the mouse events and call the base control's events with rotated coordinates.
You can get halfway there by calling the DrawToBitmap method on your panel, then rotating the bitmap and displaying it e.g. in a PictureBox:
var bitmap = new Bitmap(panel.Width, panel.Height);
panel.DrawToBitmap(bitmap, new Rectangle(Point.Empty, panel.Size));
bitmap.RotateFlip(RotateFlipType.Rotate270FlipNone);
var pictureBox = new PictureBox();
pictureBox.Location = panel.Location;
pictureBox.SizeMode = PictureBoxSizeMode.AutoSize;
pictureBox.Image = bitmap;
Controls.Remove(panel);
Controls.Add(pictureBox);
Rotation angles other than 90-degree increments are also possible, if you draw the bitmap into another bitmap using GDI:
var bitmap2 = new Bitmap(bmp.Width + 75, bmp.Height + 100);
var graphics = Graphics.FromImage(bmp2);
graphics.TranslateTransform(bitmap2.Width / 2, bitmap2.Height / 2);
graphics.RotateTransform(-15f);
graphics.TranslateTransform(-bitmap.Width / 2, -bitmap.Height / 2);
graphics.DrawImageUnscaled(bitmap, Point.Empty);
graphics.Dispose();
The problem of course is that you're only displaying an image of your panel, and not the panel itself, so it's no longer possible to interact with the controls inside.
That could probably be done as well, but you would have to mess with window messages, which gets quite a bit more complicated. Depending on your needs you might also be able to get away with handling click and key events on the PictureBox, manipulating the controls in the panel, and then updating the image.
I would like to create a visual effect when an item in a listbox is double clicked. So far I have drag and drop functionality where the item is visually attached to the mouse and can be moved to a drop target. From that functionality I am able to animate the item using the same logic of getting the item container, however I am not able to leave the items control. Is there any way to remove the item from the ListBox and visually animate it to another location? Basically the main list box is a hand of cards. When a card is double-clicked I want it to visually move from the hand listbox to a discard listbox. As of now, the logic for moving the item from one collection to another is no problem, however I would really like an animated visual representation of this event. Any ideas or references on how to do something like this would be appreciated.
Thanks,
Brandon
Further Details on what I have attempted:
There are some concepts that I do not yet have a strong grasp of yet, which has led me to run face first into this wall. I have a method that I pass in(some may be unnecessary) the ListBox as an ItemsControl, a FrameworkElement that is the listbox item, and the data object associated with the ListBox item. What I attempted to do was FindVisualChild of the ListBoxItem that is a canvas. I can do that. In my mind I was wanting to somehow clone the canvas either as a canvas or as a bitmap, add it to the children of the child of the page in the same location, remove the ListBoxItem from the ListBox, and animate the clone to the discard pile. When the animation completes the clone will be removed or hidden, and as that object is added to the discard pile collection it will effectively take the replace the clone.
My problem with this, is that I feel like there really is a simpler way of doing this using the adorner layer or something. I also, don't know how I would position the clone at the exact same position in the element further up the visual tree. I am going to keep working on it and researching other approaches, and I will just hope that someone will share some insight on this.
Here's some code I worked up to draw a visual to a bitmap. You may be able to adapt this to your needs, and draw the bitmap by adorning the UIElement that represents a common ancestor of the two list views. Note the use of FrameworkElement.TransformToAncestor to get the coordinates of a nested element in terms of an ancestor element.
public static BitmapSource CreateBitmapFromElement(FrameworkElement element, Double dpiX, Double dpiY)
{
Size elementSize = new Size(element.ActualWidth, element.ActualHeight);
Visual root = GetAdornerDecoratorAncestor(element);
Rect elementBounds = element.TransformToAncestor(root).TransformBounds(new Rect(elementSize));
RenderTargetBitmap rtb = new RenderTargetBitmap((Int32)(elementBounds.Size.Width * dpiX / 96.0),
(Int32)(elementBounds.Size.Height * dpiY / 96.0),
dpiX,
dpiY,
PixelFormats.Pbgra32);
DrawingVisual dv = new DrawingVisual();
using (DrawingContext dc = dv.RenderOpen())
{
VisualBrush vb = new VisualBrush(root);
vb.ViewboxUnits = BrushMappingMode.Absolute;
vb.Stretch = Stretch.None;
vb.Viewbox = elementBounds;
dc.DrawRectangle(vb, null, new Rect(new Point(), elementBounds.Size));
}
rtb.Render(dv);
return rtb;
}
public static Visual GetAdornerDecoratorAncestor(DependencyObject obj)
{
while(obj != null && !(obj is AdornerDecorator))
{
obj = VisualTreeHelper.GetParent(obj);
}
return obj as AdornerDecorator;
}
OK, you could try taking a Visual element and setting it's background to a visualbrush of your ListItem and animate that to the other list box. You can wait for the storyboard completed event to actually do the switch. If this were me, I would animate from one box only to the edge of the other. If the switch happens fast enough it should look pretty seamless to the user. Finding the exact position of where the item is supposed to go into the list box would be pretty complex based on any sorting/filtering rules you have.
If the two listboxes are always in the same position, you could try animating the double-clicked item to a predetermined spot, let's say half-way between the old list and the new list. Then execute the code to move the item to the new list, but use a style that immediately starts an animation on that item starting it from that predetermined location and animating to its location in the new list. You'd probably have to tweak the initial offset of the new item at runtime based on where it's inserted in the list.