WPF 3d ImageBrush material help - wpf

Question

Im trying to apply a material to my GeometryModel3D at runtime like so:
var model3D = ShardModelVisual.Content as GeometryModel3D;
var materialGroup = model3D.Material as MaterialGroup;
BitmapImage image;
ResourceLoader.TryLoadImage("pack://application:,,,/AnzSurface;component/path file/img.png", out image, ".png");
var iceBrush = new ImageBrush(image);
var grp = new TransformGroup();
grp.Children.Add(new ScaleTransform(0.25, 0.65, 0.5, 0.5));
grp.Children.Add(new TranslateTransform(0.0, 0.0));
iceBrush.Transform = grp;
var iceMat = new DiffuseMaterial(iceBrush);
materialGroup.Children.Add(iceMat);
Which all works fine, and the material gets added.
What I dont understand is how I can map the users click on the screen to the offsets that need to be applied to the TranslateTransform.
I.e. at the moment, x: -0.25 moves the material backwards along the X axis, but I have NO IDEA how to get that type of a coordinate from the users mouse click...
when I do:
e.MouseDevice.GetPosition(ShardsViewPort3D);
that gives me normal X/Y corrds of the mouse click...
Thanks for any help you can give!

Answer 1

It sounds like you want to slide the material around on your geometry when you click on it. Here's how:
Use hit testing to translate your X/Y coordinates from the mouse click into a RayMeshGeometry3DHitTestResult as described in my earlier answer. This will give you the MeshGeometry3D that was hit, the vertices of the triangle that was hit, and the relative position on that triangle.
Look up each vertex index (VertexIndex1, VertexIndex2, VertexIndex2) in the MeshGeometry3D.TextureCoordinates to get the texture coordinates. This will give you three (u,v) pairs as Point objects. Multiply each the (u,v) pairs by the corresponding weight from the hit test result (VertexWeight1, VertexWeight2, VertexWeight3) and add the pairs together, ie:
uMouse = u1 * VertexWeight1 + u2 * VertexWeight2 + u3 * VertexWeight3
vMouse = v1 * VertexWeight1 + v2 * VertexWeight2 + v3 * VertexWeight3
Now you have a point (uMouse, vMouse) that indicates where on your material your mouse was clicked.
If you want a particular point on your texture to move to exactly where the mouse was clicked, just subtract the (uMouse, vMouse) where the mouse was clicked from the (u,v) coordinate of the location in the material you want to appear under the mouse, and set this as your TranslateTransform. If you want to handle dragging, store the computed (uMouse,vMouse) where the drag started and the transform as of the drag start, then as dragging progresses compute the new transform as:
translate = (uMouse,vMouse) - (uMouseDragStart, vMouseDragStart) + origTranslate
In code you'll write this as Point additions. I spelled it out as (u,v) in this explanation because I thought it was easier to understand if I did so. In actuality the code to compute (uMouse, vMouse) will look more like this:
var uv1 = hit.MeshHit.TextureCoordinates[hit.VertexIndex1];
var uv2 = hit.MeshHit.TextureCoordinates[hit.VertexIndex2];
var uv3 = hit.MeshHit.TextureCoordinates[hit.VertexIndex3];
var uvMouse = new Vector(
uv1.X * hit.VertexWeight1 + uv2.X * hit.VertexWeight2 + uv3.X * hit.VertexWeight3)
uv1.Y * hit.VertexWeight1 + uv2.Y * hit.VertexWeight2 + uv3.Y * hit.VertexWeight3);
and the code to update the transform during a drag will look something like this:
...
var translate = translateAtDragStart + uvMouse - uvMouseAtDragStart;
... = new TranslateTransform(translate.X, translate.Y);
You'll have to adapt this to the exact situation.
Note that your HitTest callback may be called multiple times, starting at the closest mesh and moving back. It may even be called with 2D hits, for example if a 2D object is in front of your Viewport3D. So you'll want to check each hit to see if it is really what you want, for example during dragging you want to keep checking the position on the mesh being dragged even if it is no longer foremost. Return HitTestResultBehavior.Stop from you callback once you have acted on the mesh being dragged.