i'm trying to create a 3D animation using WPF. To modelize a form i see that we can use PerspectiveCamera or matrixCamera. What's the difference between the both?
Perspective camera makes objects farther away look smaller, like what we see in real life or through a camera zoom lens. You can achieve the same thing using a matrix camera, but you can also do a lot more. The perspective camera and orthogonal camera are both special cases of the more general matrix camera. With a matrix camera, you could create a perspective in one dimension different from the other, like a panorama view. It requires more work than the other camera types to do the same thing, but has more possibilities.
Related
I would like to place a hairstyle after facial landmark detection. I'm able to render 2D images properly. I would like to render 3D model. I thought of using SceneKit to render 3D model. I would like to know how Instagram, snapchat and other face filter apps are rendering 3D models. I observe SceneKit coordinate system is different from UIKit coordinate system. I have googled but couldn't find the conversion of coordinate system. Could anyone help. Thanks.
Look for worldUp and simdWorldUp instance properties to understand how ARKit constructs a scene coordinate system based on real-world device motion (also, you can inspect ARConfiguration.WorldAlignment enum).
Please, look at this SO post: Understand coordinate spaces in ARKit for complete info.
And remember, ARAnchor is your best friend when placing 3D object. Click here
for further details.
I am working on a AR based solution in which I am rendering some 3D models using SceneKit and ARKit. I have also integrated CoreML to identify objects and render corresponding 3D objects in scene.
But right now I am just rendering it in the center of screen as soon I detect the object(Only for the list of objects that I have). Is it possible to get the position of the real world object so that I can show some overlay above the object?
That is if I have a water bottled scanned, I should able to get the position of the water bottle. It could be anywhere in the water bottle but shouldn't go outside of it. Is this possible using SceneKit?
All parts of what you ask are theoretically possible, but a) for several parts, there’s no integrated API to do things for you, and b) you’re probably signing yourself up for a more difficult problem than you think.
What you presumably have with your Core ML integration is an image classifier, as that’s what most of the easy to find ML models do. Image classification answers one question: “what is this a picture of?”
What you’re looking for involves at least two additional questions:
“Given that this image has been classified as containing (some specific object), where in the 2D image is that object?”
“Given the position of a detected object in the 2D video image, where is it in the 3D space tracked by ARKit?”
Question 1 is pretty reasonable. There are models that do both classification and detection (location/bounds within an image) in the ML community. Probably the best known one is YOLO — here’s a blog post about using it with Core ML.
Question 2 is the “research team and five years” part. You’ll notice in the YOLO papers that it gives you only coarse bounding boxes for detected objects — that is, it’s working in 2D image space, not doing 3D scene reconstruction.
To really know the shape, or even the 3D bounding box of an object means integrating object detection with scene reconstruction. For example, if an object has some height in the 2D image, are you looking at a 3D object that’s tall with a small footprint, or one that’s long and low, receding into the distance? Such integration would require taking apart the inner workings of ARKit, which nobody outside Apple can do, or recreating an ARKit-alike from scratch.
There might be some assumptions you can make to get very rough estimates of 3D shape from a 2D bounding box, though. For example, if you do AR hit tests on the lower corners of a box and find that they’re on a horizontal plane, you can guess that the 2D height of the box is proportional to the 3D height of the object, and that its footprint on the plane is proportional to the box’s width. You’d have to do some research and testing to see if assumptions like that hold up, especially in whatever use cases your app covers.
What is the correct way to zoom in and out of a scene in SceneKit?
So when I enable the standard camera control in a scene and pinch in and out the scene gets bigger and smaller. What is that pinch really doing?
Is it changing the scale of the whole scene? Is it moving the camera closer?
I want to implement the same effect but programmatically.
What should I do to obtain the same effect?
When you pinch it's the field of view (xFox and yFov properties) of your camera that's changed. Changing the field of view is not the best way to zoom because it can dramatically change the perspective.
Moving the camera closer to your object is a good solution.
Also note that the "free camera" behavior is suitable for 3D viewers (such as Preview.app) but will rapidly become frustrating in any other app. At this point you might want to implement your own camera controller.
At any given point the camera has a position in space, it has a rotation for each of its own axis compared to each of the world axis, to have a zoom in and zoom out, you have to move the camera in the +z/-z axis direction.
Along the Cameras own Z/-Z axis.
For those on OSX, I used this in my SCNView subclass:
override func scrollWheel(theEvent: NSEvent) {
let cam = pointOfView!.camera
cam!.xFov = cam!.xFov - Double(theEvent.deltaY)
cam!.yFov = cam!.yFov - Double(theEvent.deltaY)
}
There are two (minor) problems that could be addressed with a little extra code. One is that the values can go negative, at which point the image is flipped inside-out. The other is that mouse acceleration can cause the zoom level to go too fast if you really spin the wheel. Limits on both of these would be a good idea, but in my app the behaviour was fine as it is above.
Can someone tell me the difference between Point Sprites and Billboards in OpenGL? I read a lot about both of them and I'm getting confused more and more about when to use which of them and whether there is actually a difference?
Wikipedia only knows about Sprites (Billboard redirects there):
In computer graphics, a sprite is a two-dimensional bitmap that is integrated into a larger scene, most often in a 2D video game. Originally, the term sprite referred to fixed-sized objects composited together, by hardware, with a background.3 Use of the term has since become more general.
One source states:
Sprite
A sprite is the traditional term given to a 2D image displayed in a game
Billboard
... you need to re-orient each particle so that it's facing the viewer.
This technique of re-orienting the sprites is called billboarding.
Another source:
Billboarding is a popular technique used in 3D graphics programming.
Billboarding allows an object (usually a quad) to always face a given
camera. Here are some common uses of billboarding:
– particles – halo surrounding an object – trees rendering
For the particular case of particles, the billboarding is a GPU
built-in feature when point-sprites are used (a single point is
transformed to a billboarded quad).
Yet another states that both face the camera, but billboards only rotate about their vertical axis (think trees).
Some references specifically for OpenGL:
https://learnopengl.com/In-Practice/2D-Game/Rendering-Sprites
https://learnopengl.com/In-Practice/2D-Game/Particles (billboarding)
https://www.opengl-tutorial.org/intermediate-tutorials/billboards-particles/billboards/
Live Examples by Three.js/WebGL (though I can't tell the difference):
https://threejs.org/examples/?q=billboard#webgl_points_billboards
https://threejs.org/examples/?q=sprite#webgl_points_sprites
I'm trying to write an CAD-like application in WPF(.NET 4.0) that needs to be able to display a lot of 2D points/lines. It will be used to display CAD-plans of entire cities with zoom, pan, rotate and point snapping on mouseover.
Right now I purely use WPF. I read the objects from the CAD file draw them into a StreamGeometry, use it as stroke of a new Path and add it to a Canvas, with several transforms.
My problem is that this solution doesn't scale well enough. It works fine with small CAD-files, but when I want to display like half a city(with houses and land boundaries) it is very very delayed.
I also tried to convert my CAD-file to an image, but
- a resolution a 32000x32000 is sometimes not enough
- when zooming out the lines are too thin.
In the end I need to be able to place this on a Canvas(2D/3D) as background.
What are my best options here?
Thanks,
Niklas
wpf is not good for a large 3d models. im afraid it is too slow. Your best bet is direct 3d or openGL
However, even with the speed of direct3d,openGL you will still need to work out how to cull as many polygons/vertices as possible before the rendering of the scene if you are trying to show an entire city.
there is a large amount of information on this (generally under game development)
there are a few techniques including frustrum culling, near and far plane culling.
also, since you probably have a static scene you may be able to use binary spacial partitioning.
As I understand the subject is 2D CAD system within WPF.
Great! I use it...
OpenGL and DirectX are in infinite loop OnDraw always. The CPU works all the time.
WPF/Silverlight 2D is smart model.
Yes, total amount of elements (for example, primitives inherited from Shape) must be not so much. But how many?
I tested own app (Silverlight). WPF will be a bit faster I hope...
Here my 2D CAD results. Performance is still great. Each beam consists of multiple primitives.
Use a VirtualCanvas like this one from Chris Lovett.