Converting an MLMultiArray (output shape) to SCNGeometry - scenekit

has anyone had luck efficiently converting an MLMultiArray to custom geometry? I've been able to convert MLMultiArray -> UIImage, though unsure how to efficiently create a custom geometry.
Big picture - I'm trying to mask a Scenekit Scene by utilizing an SCNNode with custom geometry. Essentially, I'd like to create a portal effect as seen in many of the online demos/tutorials. Though, rather than using an SCNBox as the geometry for the masking node, I'd like to use output data from my CoreML model as the "masking" shape.
If anyone can share code samples or high-level approaches, I'd greatly appreciate it! Thank you
Example app using custom geometry for portal mask
https://itunes.apple.com/us/app/dark-subject-one/id1312987602
Image of custom geometry mask used in portal

Related

Decals and React

I’ve been stuck for days, and couldn’t figure it out.
I’m trying to put a small graphic (as in Team Badge, and player’s number at the back) onto a basketball jersey. I can’t use Texture as anything I specified will cover the whole jersey.
From this https://threejs.org/examples/#webgl_decals I believe decal is the way to go. What I don’t understand is that I need to shove a Mesh into the Constructor of DecalGeometry. What kind of mesh should I use? Not the mesh of the basketball jersey?
Is this the right step?
get the upload of PNG from browser
create DecalGeometry using an arbitrary Geometry (may be a square?)
specify some material (I want it to be transparent)
create a new mesh using the above DecalGeometry and material (I can’t figure out what values to use for the decal projector…)
load the PNG into a Texture
load the basketball jersey’s Mesh
merge the basketball jersey’s Mesh with the new Mesh containing the DecalGeometry
Also, how do I control the size of the graphic inside the DecalGeometry? As the graphic has color, I’m not sure if the basketball jersey’s color will affect the graphic and make the graphic look broken?
Thanks a lot!

X3D, Shape Creation, Curve, Manipulation

I have created a box in X3D but i want to curve the top right and left corners of the box.
Any ideas on how? At the moment i have the code for the box only.
I cannot find much on the net about simple shape manipulation.
Thanks
You can't do it manually, directly in X3D.
You have to use a software like Blender or 3DS Max to create non-primitive shapes (more complex shapes than cubes, spheres, cylinders and so on.) There you have all kinds of functions that would change your object and then you can export it in several formats, including X3D.

LABVIEW matrix to graylevel picture

I'm using camera mightex bte-b050-u
and using the example the company gave using labview I created a 2D array and want to transform it into a grayscale photo. I have no idea how to do it since I don't know labview and using the code in matlab it has many errors.
I'd appreciate if someone could please let me know how I can take this 2D array and represent it as a grayscale photo. (beam profiler if you wish)
P.s
I have no idea why they ignored 28 "words" so I just tried going with that logic and transform it to my camera according to the pixels I have.
I would encourage you to read about intensity graph.
And use the linked article which would explain on how to change the color scale on the graph.
Changing the color on an Intensity Graph
Here is an example that converts a 2-D array into an image: Convert Array to Image.
In short, use Flatten Pixmap.vi, Draw Flattened Pixmap.vi, and Picture to Pixmap.vi from the NI-IMAQ palette, which is part of the Vision Application Software SDK.

How to render a VisualElement to a Vector Based Image?

I am working on a Chart Control,
I need to implement the Chart Snapshot feature for capturing a vector based image in high quality of Curves and Texts, in any requested size.
Can anyone suggest me any solution or a pointer to resolve this problem.
Any kind of help in deeply appreciated.
If you have to save your image in a vector format you can using tracing. Potrace is an open source bitmap-to-vector tracer library (but considers that bitmap tracing is imperfect). Also considers that wpf is linked closely with XAML, a vector graphics markup languages, so you may want convert raster graphics to XAML (though Charles Petzold you can't embed a bitmap in a XAML file)
If you need to draw a vector image you can use the Shape element that provides a base class for shape elements, such as Ellipse, Polygon, and Rectangle; and add the shape as Panel child. If you are dealing with thousands of shape I suggest to use the DrawingVisual class, a visual object that can be used to render vector graphics on the screen, and its RenderOpen method.
To zoom you have to work with transformations, in particular ScaleTrasnform and apply the transformation to your panel or to your shapes.
Hope this help.
You can take a look at XamlToy but I have not already try it.
http://xamltoys.codeplex.com/

Attaching 2D objects to 3D objects in WPF

I have a Viewport3D object containing a scene of hierarchical Model3D objects (grouped in Model3DGroups).
I need to "attach" 2D objects to 3D elements in the scene.
I read some articles on 3D projection but I couldn't fully understand how to apply my WPF scene data to those solutions.
What is the proper way to do that? I also tried Petzold.Media3D library but I didn't understand how to use it...
Thanks
I think you are looking for the Viewport2DVisual3D class. It allows you to render any Visual2D into the 3D space.
Searching the web for more info on this class will give you examples and videos explaining how it works.
Anyway, if you are doing any WPF 3D you also need to know about the Helix Toolkit - an awesome (free) WPF 3D toolkit.
It has two classes added recently that might also be relevant to what you are doing:
BillboardVisual3D (quad that always faces camera) and
TextBillboardVisual3D (text that always faces camera).
There are useful if you want to make some 2D image that must always face towards the 3D camera.

Resources