I have a native codebase that creates an image every 50 ms. I want to use WPF to render this image into a WPF GUI view.
The image is an hbitmap constructed from a data structure of type boost::numeric::ublas::matrix. i.e. I start with a matrix of float values, then I create a HBITMAP, and then draw this HBITMAP using the device context of MFC. But now, I want to replace my MFC GUI with a WPF GUI, because it looks nicer. I plan to use C++/CLI to achieve the interop.
My question is as follows : How should I setup the transfer of my image across the interop boundary?
Should I transfer it over while it is still a structure of floats, or shall I transfer after I created the bitmap? I have heard marshaling is a big drain on efficiency, and so must be handled very carefully. Is there a type that is common to managed and native world, that can be used for this?
I am a complete newbie to interoping, any other related resources you share will be greatly appreciated.
Or, is there a way to completely avoid the transfer? - by drawing from native code into a WPF GUI view
My application is very performance sensitive, any alternative you suggest that achieves good performance will be greatly appreciated.
I've been through the same options and found that - at least for .NET 4.5 - the InteropBitmap is the best way to go, as long as your image stays the same size (e.g. for something like video streaming).
In brief, you create a memory mapped file (either in using C++/CLI or C# with p/invoke), and use that as the source of pixel data. The InteropBitmap is created over that same memory mapped file, and you can use it as an ImageSource in WPF.
In my application, a background thread updates the MMF and invalidates the InteropBitmap and the WPF front end just binds to the image source as normal.
Related
I have a WPF application that is intended for overlaying a HUD in a live stream. The original idea was to create a plugin for xsplit (a popular application for presenting live streams) to display the content of the WPF application. The problem with this approach is that rendering a bitmap to the COM interface of xsplit is far to damaging in CPU performance to release the application (As I believe there are issues in xsplit's COM interface as well as using RenderTargetBitmap taxing the CPU).
I've been looking at directly rendering the overlay into the game (The target DirectX application) because it provides a number of benefits. Chiefly it circumvents the performance problems in xsplit, but also opens the application up to a variety of streaming and capture applications.
I'm not a very experienced with DirectX but I think this is the outline of the solution
Initialize the WPF application and capture WPF's Direct3d device (via this method)
Find and hook the target DirectX application's EndScene call (using EasyHook+Slimdx)
Render contents of the WPF Device's surface ontop of hooked DirectX application
The main question I have is how to accomplish step 3 using SlimDX. I'd hope a solution could somehow reuse the surface and not rely on copying as the goal is to not impact the performance of the hooked application. I'd also like to be able to limit the region and support transparency. I am also wondering if using WPF's Direct3d device in the hooked DirectX application's device might cause any instabilities.
Any insight would be appreciated, thank you.
I'm trying to do the same. What I've found so far is that you can render your wpfvisualtree to a bitmap and afterwards write is bitmap to the d3d device captured in point 2.
void render(Direct3D.Device device)
{
wpfRenderTargetBitmap.Render(WpfVisualTree);
wpfRenderTargetBitmap.CopyPixels(devicePtr);
}
I didnt test this yet but I think I'm on the right track with this. The only problem I now have is that I loose all interactivity from my window. Button clicks and so on will no longer be captured...
Any help on that would be nice.
One of the projects I work on has some pre-existing reports that are printed via MFC's printing support and rendered more or less directly to a printer DC via GDI. We've started doing some new (unrelated) reports via WPF/XAML since we're transitioning new UI to WPF anyway and it's so much better to work with for layout.
The other shoe has finally dropped, and I've got the need to add some new functionality to an existing printed report, and the new functionality practically begs to be implemented with WPF. Our existing WPF reports are implemented via XAML pages sent to an XpsDocument (in-memory, not on disk) via XpsDocumentWriter. I would like to be able to continue to use this strategy, and take the approach of writing WPF/XAML reports that happen to have some pages rendered via GDI.
My first naive attempt was to embed an HwndHost in the UIElement that gets rendered in the XpsDocumentWriter, but that doesn't seem to work. No surprise but it was worth a try.
The next obvious solution, IMO, would be to render the GDI graphics to an appropriate sized and scaled bitmap, and render that bitmap to a page in the XpsDocument. That would work, but page-sized bitmaps (especially in-memory ones) seem like a recipe for high memory usage and poor performance on slower computers.
Ideally I'd like to render the GDI content to a metafile or some other vector format that could then be translated to XPS. But this has to be an automatic process that works every time since it's just a document printing feature. OTOH it's an application for in-house users so we can put up with some performance degredation
WPF development is not my main task, so I'd describe myself as a novice without much detailed knowledge of the underlying details. I just wanted to make sure I'm not missing something obvious before I revert to using a bitmap as the transfer medium, although I haven't turned up any other decent options in my search so far.
Anything I should be looking into?
One way of doing this would be to create a WriteableBitmap in WPF and blit the GDI drawn image directly to it so it can be rendered in your XPS document. An initial step could be to do a straight blit from your GDI DC (get a pointer to GDI DC, pointer to WriteableBitmap and use Platform Invoke to call memcpy). Later work could involve converting the MFC GDI drawing to vanilla WPF (using a library such as WriteableBitmapEx which has gdi like drawing methods).
Although the first approach above would involve two bitmaps, its the best way I can currently think of without a huge re-write. The second method may or may not be possible out of the box, since WriteableBitmaps's drawing support is not as extended as GDI. A final method I just thought of would be to use GDI via Platform Invoke and draw directly on the WriteableBitmap surface. This would allow a port without a massive re-write and would give you the performance you need, while keeping the code familiar.
I'm writing a Video application utilizing D3DImage. Frames are from memory and rendered as textures in native code with DirectX9, finally exposed by D3DImage to the WPF GUI. I have some Overlays on top, created with WPF's painting framework (Text, shapes etc.). Up to this point, it works like a charm.
Now, I would like encode the composited image from my underlying native C++ code. Video is 640x480 BGR, 25 FPS and has to be rendered and encoded in parallel, also on older Hardware with Windows versions down to XP/SP3.
Problem is, I cannot find any documentation describing the composition process between WPF and D3DImage. They 'blend' in some sense, but what is the meaning of this? And is it possible to get a handle to the WPF's part of the drawing or even the composited image in my native C++ code?
p.s: I'm also open to managed solutions, but didn't found much performant up to now.
There is global static method called "CompositionTarget.Rendering". Add an event to that and every time WPF renders that method will be called before WPF presents(the FPS can vary though). So just updated your renderTarget accordingly.
There might be a better way, but i'm not aware of it.
NOTE:: Also for D3DImage on WindowsXP you use a D3D9 device with a lockable renderTarget while on Vista/7 you use a D3D9Ex device with a non-lockable renderTarget. Just a note.
I'm trying to cross compile some libraries for WPF and Silverlight.
Currently one of the libraries depends on System.Drawing.Bitmap which is not available in Silverlight.
It's a class that represents a device specific image format,
and holds a reference to a Bitmap instance in order to display this image
in a WPF application.
What could I use instead of the Bitmap class in order to hold a reference
to an Image which I can display in both silverlight and wpf ? (cannot be file based, must be in memory).
System.Windows.Media.Imaging for WPF and Silverlight. This one shows how to work with it in memory: Silverlight 4.0: How to convert byte[] to image?
The closest you will get is the WriteableBitmap class but these classes a quite different between frameworks. If you are looking to create a common library to use in both your WPF and Silverlight applications you will probably need to create an abstract that your applications code to and then write two different implementations of the abstract.
If both chunks of code return something tha,t at least lexically, is a "WriteableBitmap" then you might be able to integrate that into common code.
Worth looking at for the Silverlight side of the implementation would be WriteableBitmapEx.
I'm looking for an Image control for WPF which can rapidly change images. The built in WPF one is quite slow for the image sizes im using (scaled). I only need about ~3 FPS. I have considered dropping to WinForms and even D3D but I'm not sure thats the best way.
Can anyone suggest something?
WPF's Image control uses the native "Windows Imaging" and Direct3D subsystems of Windows to do all its dirty work, so if used with the right paremeters it will be pretty much as fast as anything you will find.
I suspect the problem is that your settings are causing Windows Imaging load the image at full resolution, then having Direct3D scale it. The solution to this is to do the scaling as you load the image by setting DecodePixelHeight and DecodePixelWidth on the BitmapImage you are using as an ImageSource.
Another technique that many graphics apps use to speed things up is to preload the images in the background. For example, the Windows picture viewer automatically starts loading the next image as soon as the current image is shown.
If you are preloading images, consider doing it in a separate thread. Also make sure you use BitmapCacheOption.OnLoad when you create the BitmapImage or the preloading won't actually occur (the default is OnDemand).