I'm trying to understand animation loops in WebGl, along with React.
Basically, does the browser repaint specific elements without repainting the whole screen? Does a a reflow always refresh the whole screen? What exactly is the difference between what happens when you manually refresh the window with a refresh button, verses what reflow or repaint? I know this technology has been around awhile, but I never really dove into it much before.
I'm just trying to to understand how a webGL code animates on a canvas element and how React changes elements on a screen, without the whole screen reloading. I've read about requestAnimationFrame, and about how React bundles and diffs virtual dom changes, reducing the number of requests for rerender, but my question is more about the entire window reloading, verses rerendering individual components only. Im building a site with gatsby/react and babylonjs, and just trying to wrap my head around the underlying concepts. Thanks.
What the browser does is somewhat undefined. You give it HTML elements, it somehow draws them on the screen. How it optimizes that process is undefined, all that's defined is what the results are kind of supposed to be like. I say "kind of" because even a simple element like <p>Hello world</p> will be rendered differently on different browsers and differently on the same browser on different OSes or in the same browser on the same OS but with different OS settings etc..
In general the browser builds a tree of nodes (the elements and their contents) called the DOM. It then walks that tree and builds whatever it needs to apply the CSS and then render those elements. Most browsers would try to cache data at various points in that process so that if something changes on the page they don't have to compute everything from scratch. Example might include they generate a glyph (The pixel for a letter), store those pixels somewhere, next time they need to draw the letter at the same size they can just use the pixels they already generated instead having to rasterize that letter from the font definition.
does the browser repaint specific elements without repainting the whole screen?
That's up to the browser.
Does a a reflow always refresh the whole screen?
That's up to the browser. If the browser has way to figure out it only has to compute a partial reflow of some branch of the tree then it might not have to refresh the whole screen.
What exactly is the difference between what happens when you manually refresh the window with a refresh button, verses what reflow or repaint?
Refreshing a window is like killing a program and re-running it from scratch. All the data has to be reloaded either from the network or from the cache, the text gets parsed into elements, etc..
Reflow is computing where all the elements belong, where words or elements wrap, what size they are.
Repaint is drawing the elements. You can repaint without reflow.
how a webGL code animates on a canvas element
A canvas is just a rectangle of pixels, similar to an <img>. The difference is you can get one of several APIs to affect those pixels ("2d", "webgl", "webgl2", "webgpu", ...)
When you change those pixels the browser knows that <canvas> element needs to be re-drawn. How it re-draws it is up to the browser but at does have to at least follow the rules of the spec so for example a <canvas> like pretty much all elements, has CSS applied (it could have border, a background image/color/pattern, rounded corners, etc....). Elements are composited on top of each other so you might have elements in front of the canvas, you might have elements behind the canvas.
Like I said above, what the browser does to draw the elements is undefined but you can certainly imagine that if it can figure out the only thing that changed is the canvas's content, and there are no elements in front of the canvas, and there is nothing behind the canvas, and the canvas is opaque, then it could, potentially, just re-draw the canvas area only.
That situation is rare though. For example, most three.js examples have text at the top positioned over the canvas (the title of the example). Many also have an FPS meter. Some have a drop down UI. All of that is drawn over the canvas so at a minimum, the new contents of the canvas have to be drawn into the window and then those other elements have to rendered on top of that.
Again, how that, happens is up to the browser. It could use software rendering to draw those elements pixel by pixel, or it's possible it has stored the contents of those elements in textures and draws them as quads on top using the GPU.
how React changes elements on a screen, without the whole screen reloading
I'm not sure what you mean by reloading. React keeps its own "virtual DOM". It then tries to apply the changes in the virtual DOM to the actual browser DOM. If there are no changes needed to some elements those elements will not be affected.
From the POV of the browser, nothing is different. All the browser sees is the DOM. If you make changes to the DOM (using React or anything else), then, once your current event exits, the browser will schedule a task to walk the DOM and re-draw the page (using optimizations to re-compute/re-draw less is up to the browser).
Related
In WPF, is there a way to detect that the actual render size (measured in screen units) changed?
I have elements which contain a rendered bitmap. If these elements are placed inside a Viewbox (or some other control that deals with RenderTransforms), I want to render the bitmap in the actual size on screen, so that no interpolation is done.
The main idea is that I want to place some complex parts of the UI in bitmaps as these would otherwise (when drawed in retained mode) reduce the render framerate and UI responsivity, making the application a pain to use. As a side effect, I would like to draw the lines inside these controls with constant thickness, even if scaled.
One way would be to check the size on screen with every render pass (or in some given time interval), and if it changed redraw the bitmap. However, I would like to know if there is maybe a built-in way to achieve this.
I know very little about programming, but I'm excited about the prospect of Polymer making these advanced objects as simple as working in an HTML doc. Anyway, there's some basic stuff I can't figure out, like how to get a section to scroll to fit nested content, like a bunch of cards.
Here's an example
In a full, widescreen monitor, all 8 cards on the "100 years" tab are displayed fine. But as soon as you resize the window to something much narrower, like a smartphone screen, you realize you can't scroll, and thus can't access all of the content.
How do I enable scrolling for this?
The core-animated-pages element is overflow: hidden, and all the visible elements are children of that element.
If you set the core-animated-pages element to overflow: auto, then it will be scrollable.
I have 2 possible ways to display my FlowDocument:
FlowDocumentScrollViewer
Upside:
- This just presents me the data, with no attention to pages, so the user simply scrolls through everything. On printing I can add a header and a footer and the pages are decided there.
- When I resize my window, the content stays correctly at 100% zoom, as I want it to be.
Downside:
- With a lot of data it just crashes, it seems to render all controls at once, or something.
FlowDocumentPageViewer
Upside:
- With a lot of data it's still fast.
Downside:
- It decides pages for me, which are irrelevant.
- When I resize my window, the content zooms out to fit the window. Which makes the content unreadable very quickly. Possible fix to this is surrounding the control with a ScrollViewer, which works. But when you scroll down to view bottom page content, at the end it goes to the next page, and if you then scroll up too far it goes to previous page, very annoying.
What I eventually want is the FlowDocumentScrollViewer, but then with fast loading time.
Anyone with any ideas/tips on this matter? Much appreciated!
Use a FlowDocumentReader then the user can can go scroll or page at run time. This will not solve stability problems. I display some documents with 200,000 characters and it is stable for me. It load via Dispatcher so may want to look there.
I'm currently playing with touch gesture support and I wired up an element that can be zoomed in, rotated, etc through manipulationdelta. The elements container is a border however whenever i gesture the element it moves across the screen and I want to keep it inside the border where they can only use that area of the screen to manipulate the element. Any ideas as to how to accomplish this? Thanks!
Seeing how nobody has replied, I'll just say I think I've come up with a workaround by hosting a window inside a grid... that way the manipulation is always bound to the child window.
I've created a WPF control (inheriting from FrameworkElement) that displays a tiled graphic that can be panned. Each tile is 256x256 pixels at 24bpp. I've overridden OnRender. There, I load any new tiles (as BitmapFrame), then draw all visible tiles using drawingContext.DrawImage.
Now, whenever there are more than a handful new tiles per render cycle, the framerate drops from 60fps to zero for about a second. This is not caused by loading the images (which takes in the order of milliseconds), nor by DrawImage (which takes no time at all, as it merely fills some intermediate render data structure).
My guess is that the render thread itself chokes whenever it gets a large number (~20) of new BitmapSource instances (that is, ones it had not already cached). Either it spends a lot of time converting them to some internal DirectX-compatible format or it might be a caching issue. It cannot be running out of video RAM; Perforator shows peaks at below 60MB, I have 256MB. Also, Perforator says all render targets are hardware-accelerated, so that can't be it, either.
Any insights would be appreciated!
Thanks in advance
Daniel
#RandomEngy:
BitmapScalingMode.LowQuality reduced the problem a little, but did not get rid of it. I am already loading tiles at the intended resolution. And it can't be the graphics driver, which is up-to-date (Nvidia).
I'm a little surprised to learn that scaling takes that much time. The way I understood it, a bitmap (regardless of its size) is just loaded as a Direct3D texture and then hardware-scaled. As a matter of fact, once the bitmap has been rendered for the first time, I can change its rotation and scale without any further freezes.
It's not just with a large number of images. Just one large image is enough to hold up rendering until it has been loaded in, and that can be quite noticable when your image dimensions start getting up in the thousands.
I do agree with you that it's probably the render thread: I did a test and the UI thread was still happily dispatching messages while this render delay was taking place from trying to display a fully pre-cached BitmapImage.
It must be doing some sort of conversion or preparation on the image, like you were speculating. I've tried to mitigate this in my app by "rendering" but hiding the image, then revealing it when I need to show it. However this is less than ideal because the rendering freezes happen anyway.
(Edit)
Some followup: After a discussion on the MS WPF alias I found what was causing the delays. On my Server 2008 machine it was a combination of old video drivers that don't support the new WDDM driver model and a delay for resizing the image.
If the source image size is different from the display size, that will delay the render thread before the image shows up. By default an image is set to the highest quality, but you can change the scaling options for rendering by calling RenderOptions.SetBitmapScalingMode(uiImage, BitmapScalingMode.LowQuality); . Once I did that, the mysterious freeze before displaying an image went away. An alternative, if you don't like the quality drop in scaling, is to load the BitmapImage with DecodePixelWidth/Height equal to the size it will be displayed at. Then if you load the BitmapImage on a background thread, you should have no delay in displaying it.
Also try these;
/* ivis is declared in XAML <Image x:Name="iVis" UseLayoutRounding="True" SnapsToDevicePixels="True" /> */
iVis.Stretch = Stretch.None;
RenderOptions.SetBitmapScalingMode(iVis, BitmapScalingMode.NearestNeighbor);
RenderOptions.SetEdgeMode(iVis, EdgeMode.Aliased);
VisualBitmapScalingMode = BitmapScalingMode.NearestNeighbor;
iVis.Source = **** your bitmap source ****
I was having some trouble with performance when using a huge amount of "A" channel color's, waiting until after the image had rendered to scale it worked much better for me.
Also, as you said your using a tiled graphic?
You would usually use a TileBrush to simply set as the Brush on your FrameworkElement. If you are animating them or adding new ones dynamically, you could generate your brushes then apply them to your object as you go manually too, be sure to Freeze them if you can. Also, VisualBitmapScalingMode is a property of any Visual.