I'm kind of writing a little game engine -- purely to understand how these work from the inside. I currently don't want to mess with OpenGL or DirectX, so I stick to drawing on a control with GDI+ and all that WinForms stuff.
Obviously, I need to handle input. More specifically, keyboard events. This, however, poses a problem:
protected override void OnKeyDown(KeyEventArgs e)
{
Trace.WriteLine(string.Format("KD {0:hh:MM:ss.fff} {1}",
DateTime.Now, e.KeyCode));
}
This code (even with the shortest repeat delay set in the Keyboard applet in the Control Panel) yeilds the following:
KD 10:02:18.318 Right
KD 10:02:18.570 Right
KD 10:02:18.598 Right
KD 10:02:18.639 Right
KD 10:02:18.667 Right
KD 10:02:18.701 Right
As you can see, there's a 0.25 sec. delay between first two events. This results in sluggish movements of objects on screen, obviously: it first moves slightly to the right, then pauses for a noticeable moment, and then continues on.
How do I resolve that issue? Can this be done in pure WinForms or should I go DirectInput (or whatever is the kosher way nowadays?) route?
Using Windows messages isn't the best way to go for input interaction.
I know nothing about WinForms, but I assume on key events use messages.
I've used DirectInput (v7) many years ago, and it was really fast.
Perhaps GetKeyboardState or GetAsyncKeyState in a game loop are good alternatives.
Related
I am trying to smoothly scroll some images across a window using DirectX11 via SharpDX in a WPF application.
A bit of background:
The images are signal returns, varying by time, that have been loaded from a file and loaded into D3D as a texture array, and the image itself is rendered as a series of textured quads (a series of adjoining rectangles in a very long horizontal line).
The viewport is set up so that the left and right edges represent time offsets, and the signal is displayed for the time period that falls between these times.
The DirectX implementation is very simple; the quad vertices are generated once, and a very small per-frame buffer contains a simple 2D world-view transform that updates scale and translation according to the current visible range/zoom etc.
As far as I can tell, the D3D implementation is not part of the problem I am having - it really is a very simple set up, and seems to be rendering extremely quickly (as it should), and although I do have some fancier stuff (streaming of textures from disk as required), I have disabled all of these and am running with very simple (small) textures whilst I try and resolve the issue I am having..
The problem:
Scrolling of the image is not smooth when the visible time range is animated. The image "jitters" much of the time, and frankly looks awful (when it is not jittering, it looks great).
The setup:
DirectX is rendered to a D3DImage, with a bit of work going on behind the scenes to make DX11 work - this code is taken from https://sharpdxwpf.codeplex.com/
There are multiple D3DImages (up to a total of 4), arranged in a grid (I have been testing with two, both of which contain signals for the same time period and are animated together).
These D3DImages are drawn by a DrawingVisual (which is hosted by a custom FrameworkElement) where they are used as the source of an ImageBrush. The frameworkelement object triggers a render as required, and the drawing visual handles the D3D render call and draws a rectangle to fill the control using the D3DImage brush.
The time range value is animated using a WPF DoubleAnimation. The visuals that are currently displayed are bound to this value via INotifyPropertyChanged and trigger a render (via InvalidateVisual) on each change.
DrawingVisual render code, triggered by change of "Position" value (start of visible time range):
// update scene per-frame buffer first, with world-view transform
using (DrawingContext dc = RenderOpen()
{
_scene.Renderer.Render(_draw_args);
_image.Invalidate();
dc.DrawRectangle(_brush, null, new Rect(viewer.RenderSize));
}
What I have tried:
I have tried a number of things (and searched a lot) to try and determine whether the issue is due to jittery render request timing, or if the issue is further into the render process.
Hooking into CompositionTarget.Rendering
Firstly driving the update of the position value via CompositionTarget.Rendering, and leaving the rest of the process as-is (i.e. elements react to the change of this value):
(needless to say, this is very much "test" code):
Stopwatch rsw;
long last_time = 0;
void play()
{
rsw = new Stopwatch();
last_time = 0;
rsw.Start();
CompositionTarget.Rendering += rendering;
}
void rendering(object sender, EventArgs e)
{
double update_distance = rsw.ElapsedMilliseconds - last_time;
Position += 10 * update_distance;
last_time = rsw.ElapsedMilliseconds;
}
Results - worse than the simple double animation.
Using a DispatcherTimer. Similar to above, using a timer in conjunction with a stopwatch to judge the elapsed time a little better. Worse results again, with the interesting side note that CPU usage dropped from about 7% (half a core) to 0.7%.
The original attempt, along with variants 1 & 2, all try to update a single value that then triggers the render of interested parties. Out of interest, I logged the time differences between render requests as they reached the Drawing visual - although the DisptacherTimer actually gave the most consistent results there (both WPF animation and CompositionTarget dropped the odd frame), the DisptacherTimer also gave the most jittery animation.
Updating the image, but not invalidating the visual. Updating the source of an ImageBrush updates the displayed image, without having to re-render the DrawingVisual. This method produced very jittery results.
CompositionTarget.Rendering in the framework element itself. This produced the best results of the lot, but still not perfect. Jitter would happen, then dissipate, only to return again. In this approach, the framework element that holds the DV hooks up to CompositionTarget.Rendering, and the visual queries the current position, which is being animated independently. I could almost live with this approach.
Attempting the wait until the D3D scene is rendered, before invalidating the image (no discernible improvement):
_scene.Renderer.Render(_draw_args);
_scene.Renderer.Device.ImmediateContext.End(q);
while (!(_scene.Renderer.Device.ImmediateContext.IsDataAvailable(q)))
Thread.Yield();
_image.Invalidate();
Observations:
I really don't think this is a performance issue as such. My dev machine has a good graphics card, 8 i7 cores etc, and this is a simple rendering operation.
This really seems like some sort of synchronisation issue between D3D and the WPF rendering, but I have no idea how to begin looking into this.
If I have two images animating in parallel, the jitter is much more pronounced on the first of the two (usually).
If I actively resize the window whilst animating, animation is perfectly smooth (despite the fact that a lot of extra work is being done, as the D3D context is being resized constantly).
EDIT
I've taken things back as far as possible to try and isolate the problem, and the issue seems to be fundamental to the way in which D3DImage is updated & rendered by WPF.
I have modified the WPFHost example in the SharpDX Samples solution so that the simple trigangle that is displayed is animated across the screen. This example is hosted in a DX10ImageSource that is rendered by a DPFCanvas on CompositionTarget.Rendering.
This example couldn't be more simple, and is about as "close to the metal" as you can get whilst rendering a D3DImage in WPF. A single triangle, translated across the screen by a value calculated from the time difference between renders. The stutter remains, coming and going, as if some sort of synchronisation issue. It baffles me, but essentially makes SharpDX unusable within WPF for any sort of smooth animations, which is extremely disappointing.
If anyone is interested in reproducing this problem, the SharpDX samples are available here: https://github.com/sharpdx/SharpDX-Samples
I made the following simple changes to the WPFHost example:
void IScene.Render()
{
...
EffectMatrixVariable wv = this.SimpleEffect.GetVariableBySemantic("WorldView").AsMatrix();
wv.SetMatrix(this.WorldViewMatrix);
...
}
void IScene.Update(TimeSpan sceneTime)
{
float x = (float)sceneTime.Milliseconds * 0.001f - 0.5f;
WorldViewMatrix = Matrix.Translation(x, 0, 0);
}
and in shader Simple.fx:
float4x4 WorldViewTransform : WorldView;
PS_IN VS( VS_IN input )
{
PS_IN output = (PS_IN)0;
output.pos = mul(input.pos, WorldViewTransform);
output.col = input.col * Overlay;
return output;
}
D3DImage is fundamentally broken.
If you don't need to overlay XAML elements on top of D3D or you do not need to resize too frequently the D3D surface, prefer to host a HWND/WinForm into your WPF apps and render to it directly using regular rendering loop. If you want a bit more details about the reasons, you can check this issue.
I have a WPF application showing a large image (1024x1024) inside a ZoomableCanvas. On top of the image, I have some 400 ellipses. I'm handling touch events by setting IsManipulationEnabled="True" and handling the ManipulationDelta event. This works perfectly when zooming slowly. But as soon as I start zooming quickly, there is a sudden frame-rate drop resulting in a very jumpy zoom. I can hear the CPU fan speeding up when this occurs. Here are some screenshots from the WPF Performance Suite at the time when the frame-rate drops:
Perforator
Visual Profiler
Software renderer kicks in?
I'm not sure how to interpret these values, but I would guess that the graphics driver is overwhelmed by the amount of graphics to render, causing the CPU to take over some of the job. When zooming using touch, the scale changes by small fractions. Maye this has something to do with it?
So far, I have tried a number of optimization tricks, but none seem to work. The only thing that seems to work is to lower the number of ellipses to around 100. That gives acceptable performance.
Certainly this is a common problem for zoomable applications. What can I do to avoid this sudden frame-rate drop?
UPDATE
I discovered that e.DeltaManipulation.Scale.X is set to 3.0.. in the ManipulationDelta event. Usually, it is around 1.01... Why this sudden jump?
UPDATE 2
This problem is definitely linked to multi-touch. As soon as I use more than one finger, there is a huge performance hit. My guess is that the touch events flood the message queue. See this and this report at Microsoft Connect. It seems the sequence Touch event -> Update bound value -> Render yields this performance problem. Obviously, this is a common problem and a solution is nowhere to be found.
WPF gurus out there, can you please show how to write a high performance multi-touch WPF application!
Well I think you've just reached the limits of WPF. The problem with WPF is that it tesselates (on CPU) vertex grafics each time it is rendered. Probably to reduce video memory usage. So you can imagine what happens when you place 600 ellipses.
If the ellipses are not resized then you could try to use BitmapCache option. In this way ellipses will be randered just once in the begining and then will be stored as textures. This will increase memory usage but should be ok I think.
If your ellipses are resized then previous technic won't work as each ellips will be rerendered when resized and and it will be even slower as this will rewrite textures (HW IRTs in perforator).
Another possibility is to design special control that will use RenderTargetBitmap to render ellipses to bitmaps and then will render it through Image control. In this way you can control when to render ellipses you could even render them in parralel threads (don't forget about STA). For example you can update ellipse bitmaps only when user interaction ends.
You can read this article about WPF rendering. I don't agree with the author who compares WPF with iOS and Android (both use mainly bitmaps compared to WPF). But it gives a good explanation about how WPF performs rendering.
This is more of a conceptual/implementation question, rather than a specific language problem.
Does anybody have any insight into cursor movement recording?
It's very easy to get a cursor's current position, but how would you go about recording the path followed by the cursor?
(To the degree of detail where it could be plotted graphically without ambiguity as to the path taken)
I imagine you could record the cursor's current position repeatedly after a small duration, logging it all to make a list of chronologically visited coordinates,
but I'm not sure how frequent (or feasible) the recording should take place; every 10 ms?
I've not even encountered a method of sleeping for such short durations to the nescessary precision!
I'm concerned also about the performance of the sleeping and recording during intense CPU usage; when the user is using the mouse to interact with intensive software.
I'm not even entirely sure where the cursor is really moving.
If I sweep my cursor across the screen, has the computer (somewhere internally) acknowledged that I crossed all those pixels,
or has my mouse really told it "I was there, now I'm over here, now I'm there".
I do also seek a method of differentiating between fast and slow movement, but for now, I can just observe the plot spacing on a graph of the visited coordinates.
Does anybody have any insight into this?
Any potential pitfalls; are my concerns legitimate?
Am I going the wrong way about this?
(As is observable, I really need some guidance in the matter)
Thanks!
It's far easier to log the mouse movements within the same application - just log something on every WM_MOUSEMOVE message. You will get a message periodically updating the mouse pointer location. You will not get a WM_MOUSEMOVE message for every pixel the mouse crosses, but it will jump depending on how fast you move the mouse and how busy the system is.
Logging mouse movements in some other application is going to be slightly more involved. If you have written both the logger and the application being logged, then you can handle WM_MOUSEMOVE in the application being logged, and send a corresponding message to your logger application. Your choice of IPC; a simple SendMessage() might be sufficient.
Logging mouse movements across the whole system is a totally different problem. You may have to hook in at somewhere closer to the driver level.
I just thought of another approach - the CBT (Computer-Based Training) hooks are designed to provide exactly this sort of information across applications. I've never used these though, so you'll have to do more investigation.
I've written a simple game-like app in WPF. The number of objects drawn is well within WPF capabilities - something like a few hundred ellipses and lines with simple fills. I have a DispatcherTimer to adjust the positions of the objects from time to time (1/60th of a second).
The code to compute the new positions can be quite intensive when there are lots of objects, and can fully load a processor. Whenever this occurs, WPF starts skipping frames, presumably trying to compensate for the "slowness" of my application.
What I would much rather happen is for all the frames to be drawn anyway, only slower. The dropped frames do not add any speed - because visual updates were pretty quick anyway.
Can I somehow force WPF to have my changes to the visuals be reflected on the screen regardless of whether WPF thinks it's a good idea?
Unfortunately I don't think there's anything you can do about this, although I will happily be corrected! WPF is designed to be an application creation framework, not a games library, so it priortises application performance and "usability" over framerate. This actually works very well when producing applications as it allows you to use quite rich animations and effects while maintaining perceived performance on lower end systems.
The only thing I think you might be able to try is push your movement code's Dispatcher priority down slightly to below Render (Loaded is the next one down) using something like:
this.Dispatcher.BeginInvoke(DispatcherPriority.Loaded, MoveMyStuff);
I don't have any kind of test harness to verify if that will help though.
This issue was fixed by using a Canvas with an OnRender override instead of creating and moving UIElements. This does mean that everything needs to be drawn by hand in OnRender, but it can now run at any FPS consistently, without skipping any frames.
I made a ragdoll similar to the one in this demo. This rag doll will be used for a turn based rpg game where the physics will be used for animations such as character taking damage, dying, falling down, etc.
What I am pondering at the moment is as how should go about this, should I stick the rag doll by the head to the background (leaving the body dangling) and basically throw around its body parts around as to simulate punching etc (as shown in Fig 1), or stiffen the joints and statically rotate and move the body parts for the actions taken(as shown in Fig 2), and when it comes to the character dying(or a similar action) just loosen the joints and let the rag doll fall down. Or is there a better way to go about doing this?
I am new to farseer physics and don't even know if what I mentioned is even possible or overwhelmingly hard to do.
Illustration http://img3.imageshack.us/img3/8681/charactermovementrg5.jpg
Please note that the red line in the figures represents the character's arm
Not sure that ragdolls are the way to go here, if you want animations. But if you do want to use them, I'd lock the feet to the floor and have some rotation springs in the joints so that when no forces are applied, the body stands upright. Then if it gets a hit, it'll kind of bend over, but should rebound to it's stand-up state afterwards (you may have to help it along the way back, e.g. apply some forces/torques until it's back where you want it).
For animations, such as the character punching, you could perhaps apply a spring joint (I think that's the name). Connect it to the fist and the destination, and the arm should automatically move there. You could do the same with a kick, just release the lock on that foot. However, I think it might be hard to get it to look right. On the other hand, it would look unique to other games, even though it might look kinda funny.
If you're skilled, you might wanna create an animation editor and save an animation as a sequence of forces and torques that need to be applied to limbs in order to get them to where and when they should be.
I think a better approach is to have an animated sprite played, rather than going through joint manipulation .
Maybe you can use some RotateTransform implementations to articulate arms and legs.
For sure animated sprites are the best, and painless, way of doing this.