Can anyone provide any details, code snippets, examples, etc. of how to go about building something as cool as this "Rock Wall" that Obscura Digital built?
http://www.obscuradigital.com/work/detail/rock-wall/
Let's just pretend we have access to whatever technology is required. Where do I start?
http://www.microsoft.com/surface/en/us/default.aspx
http://weblogs.asp.net/scottgu/archive/2007/05/30/microsoft-surface-and-wpf.aspx
http://multitouchvista.codeplex.com/
My understanding of the question is, what kind of libraries are available for multitouch (specifically in .Net)?
If this is the case, see here and here. Neither actually requires a Microsoft Surface table, but it does require multi-touch compatible hardware and Windows 7.
http://www.perceptivepixel.com/ founded by Jeff Han
Have a look here:
TouchKit: the open source, multi-touch screen developer's kit
http://www.gizmag.com/touchkit-the-open-source-multi-touch-screen-developers-kit/9852/
You need a large piece of plexiglass, an infrared webcam, a projector, several infrared LED's, and some sort of translucent material like butcher paper.
The infrared LED's are placed along the outer edge of the glass, and illuminate the glass with infrared light. The projector takes the computer image and projects it onto the butcher paper, which is mounted behind the glass. When you touch the glass with your finger, it produces an image in the infrared camera that can be processed by the computer.
The whole thing can be built for less than $1000. TouchKit offers everything but the projector in a pre-assembled form for $1580, including shipping.
Related
This is one of those messy real world problems. I have a device that provides two video stream of 1920 x 1080 # 30 FPS. I have the documentation and the SDK for the video device which is unmanaged, naturally. It support GPUDirect. We have tested a SDK which is a SharpDX/SlimDX/DirectShow.Net type wrapper basically around DirectShow for our purposes. Licensing legal issues may prevent the use of this SDK. Being new to this world I am trying to piece together an alternative solution.
The question is the sample I have appears that they take the frame buffer in a surface and apply that to the input pin of a filter. After that is a mystery. I see the video in a a WPF window, can overlay text etc. is DirectShow the best or most direct way to display streaming video or will I find that using DirectX and shaders using say SharpDX easier?
I understand the question is broad in nature but I believe there are many others who are faced with this myriad of tools but do not have enough experience in this graphics world to make the best choice and look to those who have gone before to nudge us in the right direction.
Opinion based questions are basically considered off-topic, so I am skipping that part and besides that,
DirectShow and DirectX are working together only at the point of presentation of video. Video Mixing Renderer 9 and Enhanced Video Renderer components are utilizing Direct3D to present video. That is, you are supposed to choose the most appropriate for you: you either work with DirectX directly, or you are using DirectShow's "gate" to Direct3D.
Then DirectShow itself, you typically use it when you have a source interfaced as DirectShow filter, or you need to apply other DirectShow filters for processing, or you would like to synchronize video and audio, or you otherwise have a good reason to use capabilities of DirectShow in terms of using available components. If you have a video feed comping from non-DirectShow sources, then you need to deliver that into non-DirectShow destination then you might prefer to not use DirectShow at all. DirectShow and Media Foundation are primary APIs for video streaming and those you mentioned are merely wrappers.
Quick Question
How to show webcam rapidly on Adobe AIR application with Stage3D?
Detailed question
About
My goal is to create prototype of AR (Augmentation Reality) mobile application. I have chosen Adobe Flash AIR for good 3D graphics support on mobile and AIR apps easy to porting to many mobile platforms (iOS, Android, Blackberry Playbook).
Purpose
I want to show up complex 3D model (so i need to use Stage3D). And underneath a video from Front Camera. As usual AR application.
Here is examples
(source: augmentedplanet.com)
Problem
Stage3D not transparent at all so i can't use StageVideo for the rapid showing of content of Camera, because StageVideo doesn't seen under Stage3D.
So
And only decision i have found - it's to create flat surface with dynamic texture updating.
Here is example of integration of webcamvideo with Starling Framework (Stage3D). But with many ordinary mobile devices we get such a big texture updating (almost as size of screen resolution), that any app will fall down to low fps or even crashes down. What i have done on my Galaxy Note for example. With 320x200 texture size it's has fairly good performance but look ugly at AR app.
So is any brilliant solution for create AR on AIR? Is anybody got same challenge?
This use case is unfortunately not well supported in AIR. Your best bet is really the manual upload. It might help to add votes to feature requests on Adobe forums for transparent Stage3D.
Now for why this feature was low priority: If you are doing AR you are probably already doing CPU work on the video. That means you already read back the camera data for processing either on the CPU or as a Stage3D texture. That's the expensive part, not the uploading a texture back to Stage3D.
In order for this to be useful there would need to be a lot of complicated code paths working together flawlessly. On all supported devices:
Read back low resolution camera for CPU or GPU AR image processing
Show passthrough high resolution camera image
Overlay 3D with blending
This is unfortunately very hard. On many mobile chipsets video/camera, CPU, and 3D are very separate units so it is hard to share data between them without stalling or copying. It can be done very well IF you target specific hardware. I know this does not solve your problem but I hope it explains why this use case does not work well in AIR yet. I think you have those options:
Go with AIR and readback/upload. It will be very slow on some HW, but it will work reliably.
Go native. It will be a huge win in the best case, but you need a lot of custom code and testing for every single target.
Go native on a single very narrow platform. Many very cool AR demos do this. Look at SDKs for AR from GPU vendors. Most of them have one.
Make the best out of a bad situation: Stick with low res and uploads but add some interesting filters on the video. Once you paid for the upload doing Stage3D stuff with you texture is very cheap.
I hope this helps a bit. While developing Stage3D this exact use case came up every now and then. I still think it is really cool! Maybe this post explained why it never made the top of the list yet.
Question:
I look for a program that can do the job for the following questions:
On a website the user can see a animated 'movie' of a house exterior
He can rotate the house left-right, up-down
When he clicks on, lets say a door, a price and some text will show up
.
About the framework the program uses:
Please no unity3d or another animation framework that is not widely used. Maybe Flash, Shockwave or Silverlight?
.
The questioned program:
It may be a program that a most company use in this scenario. But at best a program that can be used by semi-profs like I think i am ;-)
Thanks in advance!
You could use Flash for this.
There are several open source 3D libraries such as:
http://away3d.com/
http://blog.papervision3d.org/
http://www.flashsandy.org/
http://alternativaplatform.com/en/
...
Furthermore, you will be able to leverage GPU acceleration natively in the Flash Player 11.0 release.
Cheers
I would like to develop on a mobile device, probably with Windows Mobile platform, which device would you guys recommend for me to use? But one requirement on the device is that I need that device to be able to capture signature (human writing on the device).
Many Thanks
Sounds like any PocketPC (or "Professional") device would do. Personally I currently have an HTC Touch Pro, which would be suitable for what you describe. For testing purposes, it might also be a good idea to also test on a device with smaller resolutions (I have an HTC Elfin for example that would do well here, but it shows its age..). Or on the emulator provided with the SDKs.
However some of the newer ones have capacitive touch screens and no stylus. (For example HTC HD2.) So while HD2 is otherwise an excellent phone, capturing a human writing on it probably wouldn't work as well as older devices with resistive screens and a stylus.
Any Windows Mobile 6.0/6.1 based phone with resistive touch screen. A good example is HTC Touch Pro2.
Also, some S60 5th edition phones, such as N97, may be a good choice.
It depends on the languages you intend to use to write the application.
If it's Java, I'd go with an android system.
If you're a c# coder I'd say any windows device would be ok.. You could likely get one on EBay for dirt cheep and for the most part you can be sure your code will run on most of them (I know this is a generality), especially if you keep the code simple.
If you're a c++ developer then the same as above, any pocket PC/winmobile will do.
If you own a mac, and you've written apps in objective c then IPhone is a good choice.
In part you have to base decisions like this on what you know going in.
If you're a blank slate.... well then you've got to figure out what direction you want to take your knowledge. I've personally picked Java and Android because I've recently been convinced it's the road to the future.
Are there any controls that anyone is aware of that I can use to stream firewire video into a WPF app. I do not need camera control or capture just the video. I need WPF hosting because I'll be adding WPF content on top.
I was hoping that with the addition of having direct X surfaces in WPF something like this might appear.
Ideally looking for something relatively high level (not a direct show guy at all).
Thanks,
Brian
There are a couple really good video rendering packages for WPF. This guy Jeremiah Morrill has a blog where he discusses his numerous render projects. There's the WPF Win32 render project, and a number of low-level techniques he documents for how to access accelerated playback, Media Foundation .NET, DVD controls, etc... I believe his blog is titled "Jer's One Stop Shop".
Reading over his blog in general is a good idea if you are in to video/WPF. Last I checked, "MediaKit", one of his more comprehensive projects, enables easy use of DirectShow (simple xaml and your off and running, so don't worry) and other well known native interfaces. It's very robust and actively maintained, if not that specific project, check into some of the recent API's he's contributing on, some various Win7 media support also.
The only reason I'd bring up this other project, Augmented Reality, is that you remarked about adding content "on top". You should definitely check out wpfAugRel if your doing a lot of video production. Where to get an add on for it eludes me, but I'm sure you can find it off that site, but it allows for you to script in python some fairly slick real-time video production.
-- edit --
Right, look at this google code page, it has some video's (picture's worth a thousand words right?), but regardless, it allow's you to mix in 3D content into live-action, through the use of "marker" prop's, essentially bit's of paper with some easily machine recognizable feature's, that facilitate their underlying engine to inject computer rendered output into a real world scene, highly dynamic, so you can toss these marker's around and the 3D content move's fluidly... anyhow good luck.
Check out this article by UberDemo. It captures video into a WMV file with Windows Media Encoder and WPF. There is a paragraph about how to do the preview in a WPF application.