So here's my setup:
Camera images coming in at 1920x1080 # 25 FPS
Writing image data to WriteableBitmap on UI thread (simple copy, no processing)
Two Image controls in two different windows on two different monitors has their Source property set to the WriteableBitmap
Some generic UI stuff goes over the camera images
This works great, uses about 4% CPU on an old laptop (8 logical processors). The video is as smooth as can be. However, my UI has some animations (stuff moving around). When the camera display is running, those animations gets choppy.
Right now, the camera image is in Gray8 format, so it will be converted (I guess when calling WritePixels?). I also tried forcing one of the animations to 25 FPS too, no change.
Where should I start to resolve this? Is the issue that I'm locking the bitmap for too long, or is there something else going on? From what I can see locking the bitmap will cause the render thread to block, so moving that to another thread seems pointless. And it does feel like that somewhat defeats the purpose of WriteableBitmap.
This is always going to be tricky because you're capturing at 25FPS whilst WPF tries to update at 60. It's difficult to offer any meaninful advice without seeing a testable project but I'd probably start by doing the updates in a CompositionTarget.Rendering handler.
Related
I have a WPF application showing a large image (1024x1024) inside a ZoomableCanvas. On top of the image, I have some 400 ellipses. I'm handling touch events by setting IsManipulationEnabled="True" and handling the ManipulationDelta event. This works perfectly when zooming slowly. But as soon as I start zooming quickly, there is a sudden frame-rate drop resulting in a very jumpy zoom. I can hear the CPU fan speeding up when this occurs. Here are some screenshots from the WPF Performance Suite at the time when the frame-rate drops:
Perforator
Visual Profiler
Software renderer kicks in?
I'm not sure how to interpret these values, but I would guess that the graphics driver is overwhelmed by the amount of graphics to render, causing the CPU to take over some of the job. When zooming using touch, the scale changes by small fractions. Maye this has something to do with it?
So far, I have tried a number of optimization tricks, but none seem to work. The only thing that seems to work is to lower the number of ellipses to around 100. That gives acceptable performance.
Certainly this is a common problem for zoomable applications. What can I do to avoid this sudden frame-rate drop?
UPDATE
I discovered that e.DeltaManipulation.Scale.X is set to 3.0.. in the ManipulationDelta event. Usually, it is around 1.01... Why this sudden jump?
UPDATE 2
This problem is definitely linked to multi-touch. As soon as I use more than one finger, there is a huge performance hit. My guess is that the touch events flood the message queue. See this and this report at Microsoft Connect. It seems the sequence Touch event -> Update bound value -> Render yields this performance problem. Obviously, this is a common problem and a solution is nowhere to be found.
WPF gurus out there, can you please show how to write a high performance multi-touch WPF application!
Well I think you've just reached the limits of WPF. The problem with WPF is that it tesselates (on CPU) vertex grafics each time it is rendered. Probably to reduce video memory usage. So you can imagine what happens when you place 600 ellipses.
If the ellipses are not resized then you could try to use BitmapCache option. In this way ellipses will be randered just once in the begining and then will be stored as textures. This will increase memory usage but should be ok I think.
If your ellipses are resized then previous technic won't work as each ellips will be rerendered when resized and and it will be even slower as this will rewrite textures (HW IRTs in perforator).
Another possibility is to design special control that will use RenderTargetBitmap to render ellipses to bitmaps and then will render it through Image control. In this way you can control when to render ellipses you could even render them in parralel threads (don't forget about STA). For example you can update ellipse bitmaps only when user interaction ends.
You can read this article about WPF rendering. I don't agree with the author who compares WPF with iOS and Android (both use mainly bitmaps compared to WPF). But it gives a good explanation about how WPF performs rendering.
I'm using VB.net 2010 and WPF 4. I need to have a smooth transition between two videos played on the mediaelement. I absolutely cannot use anything that requires me to use a winhost in the WPF window, as that will make my project impossible (since the video is full screen, and the controls are over the video)
Basically, I need for the video to play through, and then smoothly go to another video specified in code behind. I cannot splice the two videos together - they must be separate.
How do I have the videos transition smoothly, with no "blink"?
I'm guessing without testing here. You're probably going to need some CPU cores and a good video card.
If you have the memory, use two MediaElements.
Queue up both videos, one on each element.
Set the opacity of the second one to completely transparent. They're UIElements so this should work...
Use timers of some kind keyed from the start of playback on the first one so that you get an event a couple of seconds before playback ends.
With that event delegate, start the video in the second MediaElement, animate the first one's opacity to zero while simultaneously animating the second one to fully opaque.
If you need to do it again, set up the timer again and make sure your delegate animates things the other way.
I've written a simple game-like app in WPF. The number of objects drawn is well within WPF capabilities - something like a few hundred ellipses and lines with simple fills. I have a DispatcherTimer to adjust the positions of the objects from time to time (1/60th of a second).
The code to compute the new positions can be quite intensive when there are lots of objects, and can fully load a processor. Whenever this occurs, WPF starts skipping frames, presumably trying to compensate for the "slowness" of my application.
What I would much rather happen is for all the frames to be drawn anyway, only slower. The dropped frames do not add any speed - because visual updates were pretty quick anyway.
Can I somehow force WPF to have my changes to the visuals be reflected on the screen regardless of whether WPF thinks it's a good idea?
Unfortunately I don't think there's anything you can do about this, although I will happily be corrected! WPF is designed to be an application creation framework, not a games library, so it priortises application performance and "usability" over framerate. This actually works very well when producing applications as it allows you to use quite rich animations and effects while maintaining perceived performance on lower end systems.
The only thing I think you might be able to try is push your movement code's Dispatcher priority down slightly to below Render (Loaded is the next one down) using something like:
this.Dispatcher.BeginInvoke(DispatcherPriority.Loaded, MoveMyStuff);
I don't have any kind of test harness to verify if that will help though.
This issue was fixed by using a Canvas with an OnRender override instead of creating and moving UIElements. This does mean that everything needs to be drawn by hand in OnRender, but it can now run at any FPS consistently, without skipping any frames.
Im trying to create a PNG sequencer class that will allow me to change an ImageBrush's ImageSource property via an animation.
The issue is that I have around 150 PNG files to load, and it really really affects performance when I have a few animations on the screen.
I have read a little about RenderTargetBitmap and also WriteableBitmap but Im not sure how to get a big performance boost, because I really do need it.
Im getting down to 6fps in some cases, which is obviously not acceptable.
In my Sequencer class, I just update a CurrentFrame DP that changes the ImageSource property of the ImageBrush.
Any ideas on how to increase the performance here?
Step 1 is loading all your images in ahead of time (preferably on a background thread). You should have your BitmapImage objects initialized with the CacheOption = BitmapCacheOption.OnLoad. You may already be doing this or it might not be the problem (the images cache by default).
However the rendering thread also needs to do some work when you change the image source. If you're not displaying at the source image size, that might be a problem, as by default the Image control uses the high quality Fant scaling algorithm. In that case you could get a performance increase by calling RenderOptions.SetBitmapScalingMode(uiImage, BitmapScalingMode.LowQuality); on your Image. Low quality scaling is orders of magnitude faster. However even after that there's still a bit of work involved. If you want to get the fastest animation possible, you can create an Image control for every frame, then overlap them all on the same place and change which one appears on top. You'll still take the hit on the render thread loading all the images in, but the actual animation should be quite snappy.
I've created a WPF control (inheriting from FrameworkElement) that displays a tiled graphic that can be panned. Each tile is 256x256 pixels at 24bpp. I've overridden OnRender. There, I load any new tiles (as BitmapFrame), then draw all visible tiles using drawingContext.DrawImage.
Now, whenever there are more than a handful new tiles per render cycle, the framerate drops from 60fps to zero for about a second. This is not caused by loading the images (which takes in the order of milliseconds), nor by DrawImage (which takes no time at all, as it merely fills some intermediate render data structure).
My guess is that the render thread itself chokes whenever it gets a large number (~20) of new BitmapSource instances (that is, ones it had not already cached). Either it spends a lot of time converting them to some internal DirectX-compatible format or it might be a caching issue. It cannot be running out of video RAM; Perforator shows peaks at below 60MB, I have 256MB. Also, Perforator says all render targets are hardware-accelerated, so that can't be it, either.
Any insights would be appreciated!
Thanks in advance
Daniel
#RandomEngy:
BitmapScalingMode.LowQuality reduced the problem a little, but did not get rid of it. I am already loading tiles at the intended resolution. And it can't be the graphics driver, which is up-to-date (Nvidia).
I'm a little surprised to learn that scaling takes that much time. The way I understood it, a bitmap (regardless of its size) is just loaded as a Direct3D texture and then hardware-scaled. As a matter of fact, once the bitmap has been rendered for the first time, I can change its rotation and scale without any further freezes.
It's not just with a large number of images. Just one large image is enough to hold up rendering until it has been loaded in, and that can be quite noticable when your image dimensions start getting up in the thousands.
I do agree with you that it's probably the render thread: I did a test and the UI thread was still happily dispatching messages while this render delay was taking place from trying to display a fully pre-cached BitmapImage.
It must be doing some sort of conversion or preparation on the image, like you were speculating. I've tried to mitigate this in my app by "rendering" but hiding the image, then revealing it when I need to show it. However this is less than ideal because the rendering freezes happen anyway.
(Edit)
Some followup: After a discussion on the MS WPF alias I found what was causing the delays. On my Server 2008 machine it was a combination of old video drivers that don't support the new WDDM driver model and a delay for resizing the image.
If the source image size is different from the display size, that will delay the render thread before the image shows up. By default an image is set to the highest quality, but you can change the scaling options for rendering by calling RenderOptions.SetBitmapScalingMode(uiImage, BitmapScalingMode.LowQuality); . Once I did that, the mysterious freeze before displaying an image went away. An alternative, if you don't like the quality drop in scaling, is to load the BitmapImage with DecodePixelWidth/Height equal to the size it will be displayed at. Then if you load the BitmapImage on a background thread, you should have no delay in displaying it.
Also try these;
/* ivis is declared in XAML <Image x:Name="iVis" UseLayoutRounding="True" SnapsToDevicePixels="True" /> */
iVis.Stretch = Stretch.None;
RenderOptions.SetBitmapScalingMode(iVis, BitmapScalingMode.NearestNeighbor);
RenderOptions.SetEdgeMode(iVis, EdgeMode.Aliased);
VisualBitmapScalingMode = BitmapScalingMode.NearestNeighbor;
iVis.Source = **** your bitmap source ****
I was having some trouble with performance when using a huge amount of "A" channel color's, waiting until after the image had rendered to scale it worked much better for me.
Also, as you said your using a tiled graphic?
You would usually use a TileBrush to simply set as the Brush on your FrameworkElement. If you are animating them or adding new ones dynamically, you could generate your brushes then apply them to your object as you go manually too, be sure to Freeze them if you can. Also, VisualBitmapScalingMode is a property of any Visual.