I've seen ARkit and I've seen the demo for animoji on the keynote, but I'm wondering if there is a way to create your own animoji (that will work within messages) within Xcode.
You can use ARKit to provide facial movement data to animate your own 3D models. In conjunction with an iMessage app, you should be able to export videos of animated characters similar to animojis.
Take a look at ARBlendShapeLocation (documentation) which provides high-level facial feature detection. You could track these features and use them to animate your models.
I'd also recommend watching the recent Apple developer video called "Face Tracking with ARKit" (link) which gives a good overview of the API's available.
When you're ready to jump right in, start with this face-tracking sample code (link) from Apple. (thanks #rickster)
Note that these features are only available on the new iPhone X.
Related
This is one of those messy real world problems. I have a device that provides two video stream of 1920 x 1080 # 30 FPS. I have the documentation and the SDK for the video device which is unmanaged, naturally. It support GPUDirect. We have tested a SDK which is a SharpDX/SlimDX/DirectShow.Net type wrapper basically around DirectShow for our purposes. Licensing legal issues may prevent the use of this SDK. Being new to this world I am trying to piece together an alternative solution.
The question is the sample I have appears that they take the frame buffer in a surface and apply that to the input pin of a filter. After that is a mystery. I see the video in a a WPF window, can overlay text etc. is DirectShow the best or most direct way to display streaming video or will I find that using DirectX and shaders using say SharpDX easier?
I understand the question is broad in nature but I believe there are many others who are faced with this myriad of tools but do not have enough experience in this graphics world to make the best choice and look to those who have gone before to nudge us in the right direction.
Opinion based questions are basically considered off-topic, so I am skipping that part and besides that,
DirectShow and DirectX are working together only at the point of presentation of video. Video Mixing Renderer 9 and Enhanced Video Renderer components are utilizing Direct3D to present video. That is, you are supposed to choose the most appropriate for you: you either work with DirectX directly, or you are using DirectShow's "gate" to Direct3D.
Then DirectShow itself, you typically use it when you have a source interfaced as DirectShow filter, or you need to apply other DirectShow filters for processing, or you would like to synchronize video and audio, or you otherwise have a good reason to use capabilities of DirectShow in terms of using available components. If you have a video feed comping from non-DirectShow sources, then you need to deliver that into non-DirectShow destination then you might prefer to not use DirectShow at all. DirectShow and Media Foundation are primary APIs for video streaming and those you mentioned are merely wrappers.
I am developing a webapp which the most challenging aspect is the 3D model section. It also contains other things such as drag and drop and a sliding bar with arrows each side which the user can go through to select different items.
I have been looking into WebGL but it seems IE doesn't support it without using a plugin. This isn't ideal so I was wondering what other options I have.
Flash? Silverlight? Anything else?
Silverlight is WAY too rare to be used for any meaningful development, and ajax is simply not stunning enuogh. That leaves us with Flash, which in my opinion, is the best of the three options. Flash Player 11 introduces the new Stage3D API which can create pretty stunning 3D graphics. There are also many AS3 libraries for 3D rendering. I prefer Away3D
Just found this, which looks promising.
http://iewebgl.com/
Edit: Disregard, seems the user has to download the executable for it to work.
at the moment flash is most popular plugin for 3D based web apps. which is cross browser, but no mobile browsers support :(
Many technologies and every one have his disatvantages
I want to create a new application. It will basically be a Deep Zoom application that users can draw annotations on (that will save to a DB so other users can see those annotations.) At first it will just simply run in a browser. However, the app would be useful if it could be used by enthusiasts in the field, so ability to run on smartphones or other handheld devices would be massively beneficial. 3G/4G signal is likely to be practically non existent in those places, so having the ability to download all the images and info for an "area" would be good.
I can't decide on which technology to use. Silverlight Deep Zoom apps look really nice in browsers, but I have heard that it is not a widely supported technology that MS might be ditching anyway and the only smartphones that would be capable of running Silverlight would be Windows phones = a very small share of the smartphone market. Flash will probably never run on iPhones/Apple products in general. So should I use HTML5? HTML5 all seems a little confusing to me at the moment, would it even be possible to make a HTML5 Deep Zoom application that users could annotate?
Any thoughts and advice would be really handy, thanks for reading.
I wrote a Deep Zoom app that supported annotation for a proof of concept a couple of years ago.
I used Django for this, however it is not approach I would recommend. If i was doing the same job again I would use CanvasZoom, which is based on HTML5. Canvas Zoom can be embedded into a webpage through javascript. There is a guide on how to do this here:
a link
Unfortunately you need to run Microsoft DeepZoom composer on the original image first in order too generate the deep zoom data that CanvasZoom will use. If you want your app to run in a browser it is likely that you will have to go for the following approach.
User selects image.
Image gets uploaded to server
Server creates deep zoom information
Use a PHP based approach so you have a canvaszoom page for the image.
The annotations will probably complicate matters, I did this with javascript when I attempted it. The trick is to work out when the image has been zoomed in (with canvas zoom there are preset zoom levels) and redraw the annotation regions. I found this approach non-trivial but not overly complicated.
Canvas Zoom is MIT licensed, so you can do what you like with it.
Good luck with your project.
How can I create an animated tile on WP7 similar to the XBox Live Tile? What are the options for Animating tiles in general?
After a good bit of research I leaning towards it is not possible. You can use push notification to update the tile which does add some flexibility and opportunity but achieving the same effect as the XBox live tile is looking unlikely. With that said I do understand the motivation for the limitation if animations were available this feature would be abused and would ultimately lead to slow devices.
In a recent Channel 9 video (about 11 minutes in) the softie presenters say this requires Microsoft "special sauce", e.g. this feature is not available to non Microsoft developers. I am guessing (only a guess!) that they will eventually open this feature. The link is here: http://channel9.msdn.com/Shows/This+Week+On+Channel+9/TWC9-mvcConf-WP7-Tools-update-AsmSpy-JavaScript-OO
Are there any controls that anyone is aware of that I can use to stream firewire video into a WPF app. I do not need camera control or capture just the video. I need WPF hosting because I'll be adding WPF content on top.
I was hoping that with the addition of having direct X surfaces in WPF something like this might appear.
Ideally looking for something relatively high level (not a direct show guy at all).
Thanks,
Brian
There are a couple really good video rendering packages for WPF. This guy Jeremiah Morrill has a blog where he discusses his numerous render projects. There's the WPF Win32 render project, and a number of low-level techniques he documents for how to access accelerated playback, Media Foundation .NET, DVD controls, etc... I believe his blog is titled "Jer's One Stop Shop".
Reading over his blog in general is a good idea if you are in to video/WPF. Last I checked, "MediaKit", one of his more comprehensive projects, enables easy use of DirectShow (simple xaml and your off and running, so don't worry) and other well known native interfaces. It's very robust and actively maintained, if not that specific project, check into some of the recent API's he's contributing on, some various Win7 media support also.
The only reason I'd bring up this other project, Augmented Reality, is that you remarked about adding content "on top". You should definitely check out wpfAugRel if your doing a lot of video production. Where to get an add on for it eludes me, but I'm sure you can find it off that site, but it allows for you to script in python some fairly slick real-time video production.
-- edit --
Right, look at this google code page, it has some video's (picture's worth a thousand words right?), but regardless, it allow's you to mix in 3D content into live-action, through the use of "marker" prop's, essentially bit's of paper with some easily machine recognizable feature's, that facilitate their underlying engine to inject computer rendered output into a real world scene, highly dynamic, so you can toss these marker's around and the 3D content move's fluidly... anyhow good luck.
Check out this article by UberDemo. It captures video into a WMV file with Windows Media Encoder and WPF. There is a paragraph about how to do the preview in a WPF application.