I'm writing a Video application utilizing D3DImage. Frames are from memory and rendered as textures in native code with DirectX9, finally exposed by D3DImage to the WPF GUI. I have some Overlays on top, created with WPF's painting framework (Text, shapes etc.). Up to this point, it works like a charm.
Now, I would like encode the composited image from my underlying native C++ code. Video is 640x480 BGR, 25 FPS and has to be rendered and encoded in parallel, also on older Hardware with Windows versions down to XP/SP3.
Problem is, I cannot find any documentation describing the composition process between WPF and D3DImage. They 'blend' in some sense, but what is the meaning of this? And is it possible to get a handle to the WPF's part of the drawing or even the composited image in my native C++ code?
p.s: I'm also open to managed solutions, but didn't found much performant up to now.
There is global static method called "CompositionTarget.Rendering". Add an event to that and every time WPF renders that method will be called before WPF presents(the FPS can vary though). So just updated your renderTarget accordingly.
There might be a better way, but i'm not aware of it.
NOTE:: Also for D3DImage on WindowsXP you use a D3D9 device with a lockable renderTarget while on Vista/7 you use a D3D9Ex device with a non-lockable renderTarget. Just a note.
Related
I have a considerable challenging subject i'm trying to figure out.
Within WPF/WinForms I can create a WebBrowser component which has significant limitations, these would be drastically resolved/reduced if i can bounce the directx surface from the web browser control to a DirectX surface i have setup.
A few things to note:
The WebBrowser component hosts IWebBrowser2 OLE/ActiveX component in a "floating" window above existing content as a child window of the WPF window.
I know and can get (without hacks) the HWND of the floating window; and the sub-classed HWND's of the children which have the actual Internet Explorer component running.
I can confirm that the window is rendering with directX, but i do not have any handles to anything outside of a HWND. I do not know the surface its rendering to, the device, or anything else.
What I've found as potential solutions:
BitBlt the child window to a WPF surface to solve the problem; this is an option of last resort as it would require a timer to capture, and update the bitmap. This seems wasteful and doesn't seem to be all that great.
The option to "Redirect" a HWND's directx surface has been noted as "trivial" by Microsoft Blog writers, but they never actually explain how. So there may be a non-directx non-gdi method thats more high-level that might work if anyone knows of one.
Use a swap chain to bounce the directx surface from the source WebBrowser HWND to the new target. This is my optimal choice; but its difficult to even begin since I do not have anything other than the HWND of the target. (I'm not limited to WPF, other tech such as OLE/COM/MFC/ATL/WinForms solutions are great!)
Is there a way to access the directx device from a OLE/COM object?... Kind of a hail marry, using reflection/debuggers i can't seem to find any reference to it. But this is also somewhat hack-y as i'm digging deep into the internals of the implementation.
Is there any pointers, hints or direction anyone can provide on how i might best accomplish this with the minimal "hackery" and maximum performance?
To begin with, I really don't think you can force WebBrowser to render directly onto your custom DirectDraw surface. However, you might be able to provide a DD surface's HDC to draw onto.
If you want to play with this, WebBrowser ActiveX control implements Windowless ActiveX Controls interfaces. In theory, you could implement a windowless ActiveX host and use IViewObject::Draw to draw onto the DD surface's HDC. I cannot predict what the performance of this would be, but I doubt it would even closely approach the native DirectDraw performance of Trident rendering engine.
I also posted a somewhat related code which uses OleDraw (which indirectly calls IViewObject::Draw).
In our WPF application, we have a need to display about 64 real-time level meters for an audio application. The tests we've thrown at WPF, even when rendering basic primitives as efficiently as we can still show it to be nowhere near where our application needs to be, often times bogging down the main thread so much to the point that it's non-responsive to input.
As such, we have to go with something more optimized for graphics performance such as DirectX (via SlimDX or SharpDX) or OpenGL/ES (via Atlas which converts it to DirectX calls.)
My question is if it's possible to create multiple, small DirectX-based areas, each representing an individual meter, or for that matter, is that even the right approach? I was under the understanding that you have to run it as at a minimum, the entire window, not a portion thereof.
The issues I see with the latter are airspace issues wherein you can't have WPF content in front of DirectX content in the same window, and we really don't want to have to redo all of our controls in DirectX since for the other non-meter 95% of our UI WPF is great!
I have read that you can render DirectX to a brush, then use that inside WPF, or using the WriteableBitmap class which gives you direct access to the buffers WPF then uses in its Render thread, both of which don't seem to suffer from the Airspace issues, but that seems we'd be right back at the same place with WPF being the bottleneck since it still has to do the rendering.
We are of course going to dedicate a few weeks to sample applications testing all of the above, but I'm wondering if I'm even headed in the right direction, and/or if there are any caveats we can avoid by talking to people with experience doing something like this to avoid common pitfalls, etc. As such, any comments will be appreciated.
I'm hoping we can perhaps even start a wiki somewhere to discuss this topic as it seems to be a popular one, albeit spread all over the place making it hard for new entrants to get the information they seek.
With wpf / d3d interop, You should always try to create the smallest number of interop calls. So you should prefer rendering all 64 level meters in a single render target (also it allows you to batch your primitive rendering and draw everything in the smallest number of gpu calls).
you should try to use the D3DImage API that allows you to share your own D3D texture with the wpf renderer.
If WPF can't really handle these 64 moving bars, you could go with a single D3DImage and use Direct3D9 for rendering all bars at once directly to it. For your specific scenario, you shouldn't have any performance problem.
I have a WPF application that is intended for overlaying a HUD in a live stream. The original idea was to create a plugin for xsplit (a popular application for presenting live streams) to display the content of the WPF application. The problem with this approach is that rendering a bitmap to the COM interface of xsplit is far to damaging in CPU performance to release the application (As I believe there are issues in xsplit's COM interface as well as using RenderTargetBitmap taxing the CPU).
I've been looking at directly rendering the overlay into the game (The target DirectX application) because it provides a number of benefits. Chiefly it circumvents the performance problems in xsplit, but also opens the application up to a variety of streaming and capture applications.
I'm not a very experienced with DirectX but I think this is the outline of the solution
Initialize the WPF application and capture WPF's Direct3d device (via this method)
Find and hook the target DirectX application's EndScene call (using EasyHook+Slimdx)
Render contents of the WPF Device's surface ontop of hooked DirectX application
The main question I have is how to accomplish step 3 using SlimDX. I'd hope a solution could somehow reuse the surface and not rely on copying as the goal is to not impact the performance of the hooked application. I'd also like to be able to limit the region and support transparency. I am also wondering if using WPF's Direct3d device in the hooked DirectX application's device might cause any instabilities.
Any insight would be appreciated, thank you.
I'm trying to do the same. What I've found so far is that you can render your wpfvisualtree to a bitmap and afterwards write is bitmap to the d3d device captured in point 2.
void render(Direct3D.Device device)
{
wpfRenderTargetBitmap.Render(WpfVisualTree);
wpfRenderTargetBitmap.CopyPixels(devicePtr);
}
I didnt test this yet but I think I'm on the right track with this. The only problem I now have is that I loose all interactivity from my window. Button clicks and so on will no longer be captured...
Any help on that would be nice.
One of the projects I work on has some pre-existing reports that are printed via MFC's printing support and rendered more or less directly to a printer DC via GDI. We've started doing some new (unrelated) reports via WPF/XAML since we're transitioning new UI to WPF anyway and it's so much better to work with for layout.
The other shoe has finally dropped, and I've got the need to add some new functionality to an existing printed report, and the new functionality practically begs to be implemented with WPF. Our existing WPF reports are implemented via XAML pages sent to an XpsDocument (in-memory, not on disk) via XpsDocumentWriter. I would like to be able to continue to use this strategy, and take the approach of writing WPF/XAML reports that happen to have some pages rendered via GDI.
My first naive attempt was to embed an HwndHost in the UIElement that gets rendered in the XpsDocumentWriter, but that doesn't seem to work. No surprise but it was worth a try.
The next obvious solution, IMO, would be to render the GDI graphics to an appropriate sized and scaled bitmap, and render that bitmap to a page in the XpsDocument. That would work, but page-sized bitmaps (especially in-memory ones) seem like a recipe for high memory usage and poor performance on slower computers.
Ideally I'd like to render the GDI content to a metafile or some other vector format that could then be translated to XPS. But this has to be an automatic process that works every time since it's just a document printing feature. OTOH it's an application for in-house users so we can put up with some performance degredation
WPF development is not my main task, so I'd describe myself as a novice without much detailed knowledge of the underlying details. I just wanted to make sure I'm not missing something obvious before I revert to using a bitmap as the transfer medium, although I haven't turned up any other decent options in my search so far.
Anything I should be looking into?
One way of doing this would be to create a WriteableBitmap in WPF and blit the GDI drawn image directly to it so it can be rendered in your XPS document. An initial step could be to do a straight blit from your GDI DC (get a pointer to GDI DC, pointer to WriteableBitmap and use Platform Invoke to call memcpy). Later work could involve converting the MFC GDI drawing to vanilla WPF (using a library such as WriteableBitmapEx which has gdi like drawing methods).
Although the first approach above would involve two bitmaps, its the best way I can currently think of without a huge re-write. The second method may or may not be possible out of the box, since WriteableBitmaps's drawing support is not as extended as GDI. A final method I just thought of would be to use GDI via Platform Invoke and draw directly on the WriteableBitmap surface. This would allow a port without a massive re-write and would give you the performance you need, while keeping the code familiar.
I'm looking for an Image control for WPF which can rapidly change images. The built in WPF one is quite slow for the image sizes im using (scaled). I only need about ~3 FPS. I have considered dropping to WinForms and even D3D but I'm not sure thats the best way.
Can anyone suggest something?
WPF's Image control uses the native "Windows Imaging" and Direct3D subsystems of Windows to do all its dirty work, so if used with the right paremeters it will be pretty much as fast as anything you will find.
I suspect the problem is that your settings are causing Windows Imaging load the image at full resolution, then having Direct3D scale it. The solution to this is to do the scaling as you load the image by setting DecodePixelHeight and DecodePixelWidth on the BitmapImage you are using as an ImageSource.
Another technique that many graphics apps use to speed things up is to preload the images in the background. For example, the Windows picture viewer automatically starts loading the next image as soon as the current image is shown.
If you are preloading images, consider doing it in a separate thread. Also make sure you use BitmapCacheOption.OnLoad when you create the BitmapImage or the preloading won't actually occur (the default is OnDemand).