In a 3D scene we often need to apply labels (little textelements or icons) next to 3D object that is moving around (rotation, translation) in the scene. These labels should always face the camera but still move with the object. This technique I believe is called billboard.
An additional cool feature would be if the label would stay always at the same size - no matter how far away the associated object is. So the label seems to live in 2D screenspace and not in the 3D scenegraph.
Does anyone figures out a clever way how to do this in WPF?
For billboarding you need to make sure that the face normal is pointing towards the camera. The algorithm is that the dot product between the face normal and the view direction should be -1 (minus one).
I have some old C code that does this, but it's probably not particularly useful.
For keeping the object the same size you'd need to work out the screen size and then apply a transform to keep it the constant size you desired.
However, if you want the object to appear as though it's in 2D space, why not draw it in a 2D overlay? This will solve both the billboarding and scaling problem at the same time. You work out the screen location of your label and then use the 2D drawing functions.
Related
I am working on a AR based solution in which I am rendering some 3D models using SceneKit and ARKit. I have also integrated CoreML to identify objects and render corresponding 3D objects in scene.
But right now I am just rendering it in the center of screen as soon I detect the object(Only for the list of objects that I have). Is it possible to get the position of the real world object so that I can show some overlay above the object?
That is if I have a water bottled scanned, I should able to get the position of the water bottle. It could be anywhere in the water bottle but shouldn't go outside of it. Is this possible using SceneKit?
All parts of what you ask are theoretically possible, but a) for several parts, there’s no integrated API to do things for you, and b) you’re probably signing yourself up for a more difficult problem than you think.
What you presumably have with your Core ML integration is an image classifier, as that’s what most of the easy to find ML models do. Image classification answers one question: “what is this a picture of?”
What you’re looking for involves at least two additional questions:
“Given that this image has been classified as containing (some specific object), where in the 2D image is that object?”
“Given the position of a detected object in the 2D video image, where is it in the 3D space tracked by ARKit?”
Question 1 is pretty reasonable. There are models that do both classification and detection (location/bounds within an image) in the ML community. Probably the best known one is YOLO — here’s a blog post about using it with Core ML.
Question 2 is the “research team and five years” part. You’ll notice in the YOLO papers that it gives you only coarse bounding boxes for detected objects — that is, it’s working in 2D image space, not doing 3D scene reconstruction.
To really know the shape, or even the 3D bounding box of an object means integrating object detection with scene reconstruction. For example, if an object has some height in the 2D image, are you looking at a 3D object that’s tall with a small footprint, or one that’s long and low, receding into the distance? Such integration would require taking apart the inner workings of ARKit, which nobody outside Apple can do, or recreating an ARKit-alike from scratch.
There might be some assumptions you can make to get very rough estimates of 3D shape from a 2D bounding box, though. For example, if you do AR hit tests on the lower corners of a box and find that they’re on a horizontal plane, you can guess that the 2D height of the box is proportional to the 3D height of the object, and that its footprint on the plane is proportional to the box’s width. You’d have to do some research and testing to see if assumptions like that hold up, especially in whatever use cases your app covers.
I am trying to write a PCL document, which has several drawing objects (lines, rectangles, texts...)
I found that if I draw the rectangles before anything else, they appear in the right position and size. However, if I draw them among the rest of objects, they are drawn smaller and in the wrong place.
The PCL seems to be OK (although this is yet to be proven), but it has made me think that perhaps graphic objects must be drawn in a particular order (I am using HPGL/2, by the way).
Does someone know if this is so? I have not been able to find anything in the PCL Manual nor in the internet (which leads me to believe that there is not such drawing order).
Perhaps you have written position or scale commands that unintentionally affect your rectangles.
I'm trying to write an CAD-like application in WPF(.NET 4.0) that needs to be able to display a lot of 2D points/lines. It will be used to display CAD-plans of entire cities with zoom, pan, rotate and point snapping on mouseover.
Right now I purely use WPF. I read the objects from the CAD file draw them into a StreamGeometry, use it as stroke of a new Path and add it to a Canvas, with several transforms.
My problem is that this solution doesn't scale well enough. It works fine with small CAD-files, but when I want to display like half a city(with houses and land boundaries) it is very very delayed.
I also tried to convert my CAD-file to an image, but
- a resolution a 32000x32000 is sometimes not enough
- when zooming out the lines are too thin.
In the end I need to be able to place this on a Canvas(2D/3D) as background.
What are my best options here?
Thanks,
Niklas
wpf is not good for a large 3d models. im afraid it is too slow. Your best bet is direct 3d or openGL
However, even with the speed of direct3d,openGL you will still need to work out how to cull as many polygons/vertices as possible before the rendering of the scene if you are trying to show an entire city.
there is a large amount of information on this (generally under game development)
there are a few techniques including frustrum culling, near and far plane culling.
also, since you probably have a static scene you may be able to use binary spacial partitioning.
As I understand the subject is 2D CAD system within WPF.
Great! I use it...
OpenGL and DirectX are in infinite loop OnDraw always. The CPU works all the time.
WPF/Silverlight 2D is smart model.
Yes, total amount of elements (for example, primitives inherited from Shape) must be not so much. But how many?
I tested own app (Silverlight). WPF will be a bit faster I hope...
Here my 2D CAD results. Performance is still great. Each beam consists of multiple primitives.
Use a VirtualCanvas like this one from Chris Lovett.
I want to draw a tilemap in a (ANSI C, C99 cannot be used due to windows compatibility) game that uses GL for accelerated graphics, although the game is a top-down 2D perspective using textured quads.
The popular opinion for handling a timemap seems to use a GL vertex buffer object, which I am about to write. However, I realized I want some tiles to go a little beyond vertical bounds, faking a slanted aerial view. That will make whatever is directly above the block to be partially covered by the tile.
If I use a VBO here, I will need to draw the entire tilemap in one sitting. Meaning that any object I draw afterwards will be directly on top of the tilemap.
What would be the sanest approach to this problem? Should I draw the tilemap first, then the entities (players/enemies) and then the excess vertical space so they cover the entities, and finally the effects that display over both? (such as shots, explosions, etcetera). But this would give me the issue of shots not being covered by terrain, and if I change the order, terrain covering large explosions awkwardly.
Alternatively I can sort all visual objects and draw them in a top-down fashion, but that would mean I need to change textures often, as sorting by texture wouldn't help too much in this specific case.
As well, I want to be able to modify the colors of every individual vertex in the grid in a dynamic way, so that entities can cast colors into the map. From what I am understanding, the way to achieve this would be with a vertex shader. Is this correct?
EDIT: A last thing. If I draw a VBO like that tilemap that is larger than the screen,by translating, does GL automatically cull out-of-view faces or do I need to reform the VBO every time I move the "camera"?
A VBO is just a piece of abstract memory reserved in the graphics memory. You can place data in any layout and arrangement as you like. You can use a single VBO to store several independent meshes. gl{Vertex,Normal,TexCoord,Color,Attrib}Pointer functions are used to set the offset into memory, that means either process address space or offset into the bound VBO.
Furthermore once can easily draw only subsets of the bound data with either glDrawArrays and glDrawElements by choosing approriate first element or indices in the index buffer.
So, no, you don't have to draw entire VBOs.
I actually answered my own question. I needed to separate the map in two: blocks that have empty space directly on top, and then the rest. Effects will be drawn in two passes, "regular" and "top" "layer"
I feel pretty bad about having an useless question lying around though, so if some admin needs to purge it, please go ahead.
Hey, I'm working on a map editor for my game, and I'm trying to convert the mouse position to a position in the game world, the view is set up using gluPerspective
A good place to start would be the function gluUnProject, which takes mouse coordinates and calculates object space coordinates. Take a look at http://nehe.gamedev.net/data/articles/article.asp?article=13 for a basic tutorial.
UPDATE:
You must enable depth buffering for the code in that article to work. The Z value for mouse coordinates is determined based on the value in the depth buffer at that point.
In your initialization code, make sure you do the following:
glEnable(GL_DEPTH);
A point on the screen represents an entire line (an infinite set of points) in 3D space.
Most people with questions similar to yours are really trying to select an object by clicking on it. If that's what you're after, OpenGL offers a selection mode that's generally more effective than trying to convert the screen coordinate into real-world coordinates.
Using selection mode is (usually) pretty simple: you start with gluPickMatrix, which you use to specify a small box around the click point. You then draw your scene in selection mode. When you're done, instead of actually drawing anything, it gives you back records of what would have been drawn in the box you specified. If memory serves, those are arranged in Z order, so the first one in the list is what would have displayed front-most (i.e., the one you usually want).