Context
I've been programming mainly as a hobby for some time now, mostly in C# and Java. I made many application (Windows Forms or Java Forms) that required animated content. In Java I would use Graphics.drawX() and redraw in function of time. When the animations were happening frequently the resolution would diminish or the application would slow down. I never gave it thought until I played a video game on the same computer that had so much trouble rendering a simple Java app. How can my computer instantly render a complex moving environment but rush a displaying a home-made 2048 game? I figured it must be because either I am misusing the draw functions, either because those functions are not optized for real-time render.
Question :
How can I directly access the display without having to go through preprogrammed functions?
I realize this maybe hard in higher level languages so let's say in C on a Windows OS. (But I would appreciate any answer relating to any language and/or OS)
I know it's a really vague question but I can't seem to find the right words to Google it appropriatly. Thank you very much for your help!
You can't (or maybe I should say should never) try to access the graphics driver directly on Windows. You used to have write directly to video memory to do graphics prior to Windows as DOS did not support graphics or display management and the stability of those programs were always a bit dicey. On Windows, it owns the screen and you have to work through it to access it.
The very concept of a Windows-based OS is that the OS owns the display and gives application access to a virtual display so that the OS can hide it or move it around. In most cases this does not cause a speed problem; but, in certain cases like gaming you need more speed; so, DirectX allows you tor transfer some of those task to the graphics card to get you the speed you need.
For more info on DirectX, check out Microsoft's Graphics and Gaming Resources
Related
Way way back in the day Itried to learn C from a game programming book. If I recall correctly, one of the first things your game "engine" would do would be to switch display modes to render. This involved a bit of asm to switch to a 640x480 display mode (mode 13 maybe?) so you could draw directly to the screen. Something like that.
My question is, what is the modern equivalent of this? I'm interested in writing a command line program that does something similar; drops into some kind of raster mode for me to draw to, but, I do not assume that my program would be running under some kind of window manager like kde, unity,Aqua etc.
Would this be something that OpenGL could provide (or does OpenGL assume a window manager too). My proposed program isn't a game, but would ideally start with a basic clear screen that I can draw primitives (2d lines, circles rects etc)
Cheers!
Modern operating systems don't give programmers as convenient access to low-level graphics routines as they used to. Partially, this is due to the advent of the GPU, which makes utilizing the graphics hardware a much more significant challenge than if you only had a CPU. The other reason is as window managers have gotten more and more complex, the graphical sandbox each operating system gives a programmer is more constrained.
That being said, OpenGL is definitely worth looking at. Its cross-platform, versatile, and automatically utilizes any hardware available (including the graphics card). OpenGL itself doesn't directly provide access to a windowing context, but you can easily create that with the OpenGL utility library (GLUT). OpenGL is however very low-level, you'll have to deal with frame buffers and flushing and bit masks and all sorts of low-level nonsense that can make OpenGL development a nightmare if you haven't done it before.
If I were starting a project, I would probably want a more robust graphics environment that provides drawing functions and windowing out of the box. Both SDL and SFML provide low-level graphics APIs that will be a little more friendly to start with. They are both implemented on top of OpenGL so you can use any of the OpenGL features when you want to, but you don't have to worry about some of the more tedious details.
As a side note, C might not be the best language to get started with graphics programming these days. If you want a really simple graphics environment that is becoming more relevant every day, you might want to check out what the web has to provide. JavaScript and the HTML5Canvas provide a very simple interface for drawing primitives, images, etc.
I have a WPF application that acquires images from a camera, processes these images, and displays them. The processing part has become burdensome for the CPU, so I've looked at moving this processing to the GPU and running custom CUDA kernels against them. The basic process is as follows:
1) acquire image from camera
2) load image onto GPU
3) call CUDA kernel to process image
4) display processed image
A WPF-to-CUDA-to-Display Control strategy is what I'm trying to figure out.
It seems natural that once the image is loaded onto the GPU that it would not have to be unloaded in order to be displayed. I've read that this can be done with OpenGL, but do I really need to learn OpenGL and include it in my project in order to do a fast display of a CUDA-processed image?
I understand (I think) the issues of calling CUDA kernels from C#. My plan is to either build an unmanaged library around my CUDA calls, which I later wrap for C# -- OR -- try to decide on which one of the managed wrappers (managedCUDA, Cudafy, etc.) to try. I worry about using one of the prebuilt wrappers because they all appear to be lightly supported...but maybe I have the wrong impression.
Anyway, I'm feeling a bit overwhelmed after days of researching the possible options. Any advice would be greatly appreciated.
The process of taking a result of CUDA computation and using it directly on the device for a graphics activity is called "interop". There is OpenGL "interop" and there is DirectX "interop". There are plenty of CUDA sample codes demonstrating how to interact with computed images.
To go directly from computed data on the device, to display, without a trip to the host, you will need to use one of these 2 APIs (OpenGL or DirectX).
You mentioned two of the managed interfaces I've heard of, so it seems like you're aware of the options there.
If the processing time is significant compared to (much larger than) the time taken to transfer the image from host to device, you might consider starting out by just transferring the image from host to device, processing it, and then transferring it back, where you can then use the same plumbing you have been using to display it. You can then decide if the additional effort for interop is worth it.
If you can profile your code to figure out how long the image processing takes on the host, and then prototype something on the device to find out how much faster it is, that will be instructive.
You may find that the processing time is so long you can even benefit from the double-copy arrangement. Or you may find the processing time is so short on the host (compared to just the cost to transfer to the device) that the CUDA acceleration would not be useful.
WPF has a control named D3DImage to directly show DirectX content on screen and in the managedCuda samples package you can find a version of the original fluids sample from Cuda Toolkit using it (together with SlimDX). You don’t have to use managedCuda to realize Cuda in C#, but you can take it to see how things can be realized: managedCuda samples
I am sorry if it appears this question has been done to death. I've done plenty of research however, and it seems there is no well known solution to what seems a simple problem: take a screenshot in windows.
There's a catch of course - the screenshot is to be manipulated in some way (gpu side, with shaders/etc), so there is no option of a slow copy to system memory. Instead the copy must somehow stay in graphics memory. GetFrontBuffer and the like a limited in this sense (they don't work full stop, I've checked).
I am aware of several closed questions on the stack exchange network ("Don't bother, not possible") and 2 with open bounties that amount to solving this problem.
Windows 7 introduces some changes to the graphics system, so now there is a compositing window manager,etc. Apparently GDI is also now 'hardware accelerated' so I was hoping this would have exposed a simple path for a possible solution:
gdi desktop window device context (in gpu memory) -> some direct2d or direct3d surface
In my particular case, just getting the DC in gpu memory is sufficient, but I am looking for a general solution.
So, how does one screnshot gpu side in Windows?
OK, so I recently purchased an Acer T232HL touch screen display to hook up to my Macbook Pro as a secondary monitor. To give you an idea, here's my setup.
OS X doesn't support this monitor in any way, so as you can see in the screenshot I'm actually running Windows 8 through VMware, which proxies the USB connection to Windows perfectly where the touch events are supported. But obviously this isn't ideal.
There's at least one 3rd party driver for OS X that looked sort of promising, but it doesn't seem to support multitouch from this device, it's expensive, and generally was a pain to get working to the small degree it was. There's also mt4j but best I could tell after running their examples, it doesn't support this device at all.
So here's my question: what exactly am I in for if I wanted to write a driver for this thing? I'm mostly a web developer with years of experience with Ruby, Objective-C (and a little C), Javascript, etc. I have never ventured into any kind of hardware programming, so from the surface this feels like an interesting while intimidating challenge.
I know at some level I need to read data from USB. I know this will probably mean trying to reverse engineer whatever protocol they're using for the touch events (is it possible this will be entirely custom?). However I haven't got a clue where to start - would this be a kernel extension? In C, I presume? Would love a high level overview of the moving parts involved here.
Ultimately I want to use the touch screen to drive a specialized web interface (running in Chrome), so ideally I could proxy the touch events straight to Chrome without the OS actually moving the mouse cursor to the touch location (so have the UI behave just as it would on an iPad), but regardless of whether this is technically possible, I'd love to start with just getting something working.
You're going to want to start with Apple's I/O Kit documentation. You can hope that the touchscreen isn't completely custom, though there must be some part that's not standard USB HID, or it would work already. If there are any linux (or other open source) drivers available, you'll have the advantage that somebody already did some of the reverse engineering for you. As an alternative to the I/O Kit, you might also want to look into libusb, which might make your brain hurt less when getting started. If you end up needing to write a kext, that might not help you anymore, though.
As to some of your specific questions:
would this be a kernel extension?
Maybe, maybe not. I'm not really up on the Mac OS X driver situation, but I did write some totally user-space USB code for OS X many years ago. Maybe you'll be as lucky.
In C, I presume?
Probably. I/O Kit itself is written in a subset of C++, so you can probably use that too, if you prefer.
is it possible this will be entirely custom?
Unfortunately, yes, it's possible.
Good luck!
My .Net Winforms application creates three OpenGL rendering contexts in my main window, and then allows the user to popup other windows where each window has two more rendering contexts (using a splitter). At around the 26th rendering context, things start to go REALLY slow. Instead of taking a few milliseconds to render a frame, the new rendering context takes between 5 and 10 seconds. It still works, just REALLY SLOW! And OpenGL does NOT return any errors (glGetError).
The other windows work fine. Just the new rendering contexts after a certain number slow down. If I close those windows, everything is fine -- until I reopen enough windows to pass the limit. Each rendering context has its own thread, and each one uses a simple shader. The slow down appears to happen when I upload a texture. But the size of the texture has no effect on how many contexts I can create, nor does the size of the OpenGL window.
I'm running on nVidia cards and see this on different GPU's with different amounts of memory and different driver versions. What's the deal? Is there some limit to how many rendering contexts an application can create?
Does anyone else have an application with LOTS of rendering contexts going at the same time?
As Nathan Kidd correctly said, the limit is implementation-specific, and all you can do is to run some tests on common hardware.
I was bored at today's department meeting, so i tried to piece together a bit of code which creates OpenGL contexts and tries some rendering. I tried rendering with and without textures, with and without forward-compatible OpenGL context.
It turned out that the limit is pretty high for GeForce cards (maybe even no limit). For desktop Quadro, there was limit of 128 contexts that were able to repaint correctly, the program was able to create 128 more contexts with no errors, but the windows contained rubbish.
It was even more interesting on ATi Radeon 6950, there the redrawing stopped at window #105, and creating rendering context #200 failed.
If you want to try for yourself, the program can be found here: Max OpenGL Contexts test (there is full source code + win32 binaries).
That's the result. One piece of advice - avoid using multiple contexts where possible. Multiple contexts can be understood in application running at mulitple monitors, but applications on a single monitor should resort to a single context. Context switching is slow. And that's not all. Applications where OpenGL windows are overlapped by another windows require hardware clipping regions. There is one hardware clipping region on GeForce, eight or more on Quadro (CAD applications often use windows and menus that overlap OpenGL window, in contrast with games). In case more regions are needed, rendering falls back to software - so again - having lots of OpenGL windows (contexts) is not a very good idea.
The best bet is that there is no real answer to this question. It probably depends on some internal limitation of the driver, hardware of even the OS. Something you might want to try to check is the number of available texture units using glGet(GL_MAX_TEXTURE_UNITS) but that may or may not be indicative.
A common solution to avoid this is to create multiple viewports within a single context rather than multiple contexts in a single window. It shouldn't be too hard to unite the two contextes that share a window to a single context with two viewports and some kind of UI widget to serve as the splitter. Multiple windows are a different story and you may want to consider completely re-thinking your UI design if there is an actual need for 26 separate OpenGL windows.
It's hard for me right now to think of a real UI use case that would actually require 26 different OpenGL windows operating simultaneously. maybe another option is to create a pool of say 5-10 contexts and reuse them only in the windows (tabs?) that are currently visible to the user. I didn't try it but it should be possible to create a context inside a plain window that contain nothing else and then move that window from parent window to parent window to whichever top-level window it is needed in.
EDIT -
Well, actually it's not that hard to think of one. The latest Chrome (9.x.x), supporting WebGL may want to open many tabs each with a WebGL context... I wonder if they handle this in any way. Just tried it and ran out of memory after 13 tabs... That would actually be a good check for you as well to see if its something you're doing wrong or if chrome and firefox (4.0.x-beta) have the same problem
Given the diverse nature of OpenGL drivers, your best bet is probably to check the behavior of the major drivers (AMD / Intel / NVIDIA / MS Software Render) and on first startup run a test. E.g. if you can see that NVIDIA always slows down like you saw, then just run a quick loop till you see where the limit is on that machine (or rather, card). It's not much fun, but I think it's pretty hard to reliably push the limits otherwise.
In other words "best bet" is just like previously answered, you can't know beforehand.
If you go through that much trouble to set OpenGL up in a over-the-top multi-threaded fashion, you could as well benefit from it and consider switching to Vulkan. See, by design, the OpenGL architecture funnels all the hard earned context/thread separated drawing operations into one single driver thread that then redistributes all these calls acrosss virtual hardware threads that map onto each context. The driver is in essence a huge bottleneck because it is not itself threaded, despite any glewmx sitting aroung. It is simply not designed to handle this well.
That said, I am curious if you used an older version of Glew, or if you do all the extension handling in some other way, since latest glew libs no longer support mx. One more reason to switch.