Working with C and Win32, I would like to know how to implement a secondary OpenGL thread for loading resources(textures and VBOs).
From what I found, this should be done with wglShareLists(), but I am unsure about how to set up the secondary thread:
Do I need a new device context or only a new rendering context?
What wgl functions do I need to call?
You don't need a new context because you can reuse the same device context of the first one. Btw, you can specify another device context, but depending on your platform you should take care about it (on Windows, device contextes must have the same pixel format), otherwise you could fail to share objects between two contextes
Create both context in the main thread, the second one sharing with the first one. Then, make the first one current on the main thread, while making the other one current on the secondary thread.
Note that you can share with any render context: all sharing contextes "see" the same object by its name, indeed they share an object name space. Two different object name spaces (i.e. two non-sharing contextes) can have defined the same object (i.e. texture object name is 1), but the same name actually points to different object depending on the current context.
Objects created by the secondary thread are visible concurrently and consistently. However, not all objects can be shared across contextes. Keep in mind that sometimes it happens that the driver supports unexpected objects, and other times it happens that driver doesn't support correctly an expected object.
OpenGL is becoming an object oriented language. You can see a certain pattern for creating objects:
Generate name (GenTextures, GenBuffers)
Define object (BindTexture, BindBuffer)
Object existence (IsTexture, IsShader, IsFramebuffer)
Delete name (and object)
(Note that an object created with Gen routines exists only when they are bound)
Object classes could be
Display lists
Texture objects
Buffer objects
Shader objects and program objects
Renderbuffer objects
Framebuffer objects
Query objects
Sync objects
Transform feedback objects
I would suggest to use a "runtime" test, like the following:
private bool TestSharingObject(RenderContext rContextShare)
{
uint texture = 0;
// rContextShader is a context sharing with this RenderCOntext
this.MakeCurrent(true);
if (Caps.TextureObject.Supported == true) {
// Generate texture name
Gl.GenTextures(1, out texture);
// Ensure existing texture object
Gl.BindTexture(Gl.TEXTURE_2D, texture);
Gl.BindTexture(Gl.TEXTURE_2D, 0);
// Self test
if (Gl.IsTexture(texture) == false)
throw new NotSupportedException();
}
// Make current sharing context
rContextShare.MakeCurrent(true);
return ((texture != 0) && (Gl.IsTexture(texture) == true));
}
Another suggestion would be to run on secondary thread operations that are CPU intensive, and not directly affecting drawing system windows buffers. A good example would be a shader compilation, since the compilation runs on CPU side; keep also in mind that the driver could async your operations, and that OpenGL implementations may pipeline different operations..
Load resources however you need to in your secondary thread and then pass ownership of them back to your primary for the GL calls.
Sharing "lists" between GL contexts should work. Doing all your GL calls on your primary thread does work :)
It works!, only if wglShareLists is done by the main thread for all the worker thread.
Using message map, pass the reference of rendering context and make the main thread create the rendering context, use wglShareLists by main thread alone. Then call wglMakeCurrent on the worker thread with the rendering context created by main thread. It is imperative that wglShareLists is called before any gl operation is done on the rendering context. This is due to the requirement on parameter hglrc2(the OpenGL rendering context to share display lists with hglrc1), that it should not contain any existing display lists when wglShareLists is called.
Related
I want to load some textures and meshes in a separate thread while the main program is showing a loading screen because it takes a few seconds to load all resources. I'm using OpenGL and GLFW. I tried to accomplish this with the following code:
void *status;
if(pthread_create(&loader, NULL, &loader_func, NULL))
{
fprintf(stderr, "Error creating loader thread\n");
return 1;
}
while(_flags & FLAG_LOADING)
{
vec3 color = { 0.1, 0.3, 1.0 };
if(glfwWindowShouldClose(window))
{
resource_destroy();
glfwTerminate();
return 0;
}
GL_CHECK(glClearColor(0.1, 0.1, 0.1, 1.0));
GL_CHECK(glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT));
font_renderer_activate();
render_string(&_font_menu, "Loading...", _width / 2, _height / 2, 64,
color, ALIGN_V_CENTER | ALIGN_H_CENTER);
glfwSwapBuffers(window);
glfwPollEvents();
}
if(pthread_join(loader, &status))
{
fprintf(stderr, "Error joining loader and main thread\n");
return 1;
}
if(*(int *)status)
{
fprintf(stderr, "Error loading resources\n");
return 1;
}
loader_func() is not rendering to the screen, and only uses OpenGL functions for creating VAOs, VBOs etc. and loading data into them.
The problem is that after the loading text shows up on screen and loading has finished, nothing shows up on screen (EDIT: except the textual HUD) and I'm getting a lot of debug error messages in my log (I'm wrapping all OpenGL calls in a macro that checks for errors with glGetError):
main.c:588
glBindTexture(GL_TEXTURE_CUBE_MAP, cubemap_texture);
GL_Error 0x502: GL_INVALID_OPERATION
main.c:589
glDrawArrays(GL_TRIANGLES, OFFSET_SKYBOX, VERTICES_SKYBOX);
GL_Error 0x502: GL_INVALID_OPERATION
main.c:629
glDrawArrays(GL_TRIANGLES, OFFSET_SELECTOR, VERTICES_SELECTOR);
GL_Error 0x502: GL_INVALID_OPERATION
When I call loader_func directly, there are no errors and the main render loop works correctly.
I read that to use OpenGL functions in another thread it is required to call glfwMakeContextCurrent but that wouldn't work in my case, because then the loading screen wouldn't be rendered. My only idea was to utilize a second library like SDL to create a window while loading, then destroy it and create a new window with GLFW for use with OpenGL. Is that what I want to achieve possible with just OpenGL?
The easiest way to handle this is to have the main thread create and manage all of the OpenGL objects, while the loading thread does the File IO (easily the slowest part of the loading). Once the loading thread is finished with loading a particular asset, it can deliver the loaded data to the main thread via <insert your favorite thread-safe mechanism here>, which can do the final OpenGL uploading part.
After all, it's not like rendering a loading screen is a huge performance drain or something, so the cost of uploading on the main thread will be minimal. This also permits you to do that loading bar thing, since your main thread will frequently be getting the results of the loading process, so it knows at any time how much of the loading is finished.
If you absolutely must have two threads both making OpenGL calls for some reason, then you should also have two OpenGL contexts, each being current in a different thread, and the two contexts sharing objects with each other. GLFW is perfectly happy to provide this if you ask it nicely. Create your main window as normal, then set the GLFW_VISIBLE hint to GLFW_FALSE, and create a second window (with an arbitrary resolution). You should pass the main window as the last parameter to glfwCreateWindow, so that the two contexts can share objects. You can then set each window current in different contexts and you're fine.
One word of caution. Contexts that share objects between them only share certain objects. Objects which reference other objects cannot be shared (also query objects are unsharable for some reason). VAOs reference buffer objects, so they can't be shared. So there's no point in trying to create them on the off-screen context.
Say I wanted to store objects statically inside their own class. Like this:
public class Foo
{
private static int instance_id = 0;
public static List<Foo> instances = new List<Foo>();
public Foo()
{
instances[instance_id++] = this;
}
}
Why?
I don't need to create unique array structures outside the class (one will do).
I want to map each object to a unique id according to their time of birth.
I will only have one thread with the class in use. Foo will only exist as one set in the program.
I did searching, but could find no mention of this data structure. Is this bad practice? If so, why? Thank you.
{please note, this question is not specific to any language}
There are a couple of potential problems I can see with this setup.
First, since you only have a single array of objects, if you need to update the code so that you have lots of different groups of objects in different contexts, you'll need to do a significant rewrite so that each object ends up getting associated with a different context. Depending on your setup this may not be a problem, but I suspect that in the long term this decision may come back to haunt you.
Second, this approach assumes that you never need to dispose of any objects. Imagine that you want to update your code so that you do a number of different simulations and aggregate the results. If you do this, then you'll end up having your giant array storing pointers to objects you're not using. This means that you'll (1) have a memory leak and (2) have to update all your looping code to skip over objects you no longer care about.
Third, this approach makes it the responsibility of the class, rather than the client, to keep track of all the instances. In some sense, if the purpose of what you're doing is to make it easier for clients to have access to a global list of all the objects that exist, you may want to consider just putting a different list somewhere else that's globally accessible so that the objects themselves aren't the ones responsible for keeping track of themselves.
I would recommend using one of a number of alternate approaches:
Just have the client do this. If the client needs to keep track of all the instances, just have them always create the array they need and populate it. That way, if multiple clients need different arrays, they can do so. You also avoid the memory leak issues if you do this properly.
Have each object take, as part of its constructor, a context in which to be constructed. For example, if all of these objects are nodes in a quadtree, have them take a pointer to the quadtree in which they'll live as a constructor parameter, then have the quadtree object store the list of the nodes in it. After all, it seems like it's really the quadtree's responsibility to keep track of everything.
Keep doing what you're doing, but using something with weak references. For example, you might consider using some variation on a WeakHashMap so that you do store everything, but if the objects are no longer needed, you at least don't have a memory leak.
C functions that are passed to Lua to so that Lua can call native functions are static functions and thus not related to an object instance.
In my application, I have multiple sessions. Each session runs on its own thread, has its own data and its own scripts, and a script of the session N must be able to access the data of that session N.
The problem is that when you register a native C function to be callable from Lua, I can't pass the instance of the session object to make it available in the static Lua function.
One possibility could to be store the instance of the session into a static variable that can be called from the static C function (called from Lua), but this looks dirty and would require synchronization that could cause some scripts to hang.
A second possibility might be to create an Lua object that represents the session and call member methods on it so that, in my C function, I would have a way to access the Lua object representing the session ("this") and retreive the actual native session instance represented by this object. But I have no idea how to do that.
Is there a way to create Lua objects representing native object instances in Lua so that native Lua functions would have access to that native object instance ?
When you register a C function with Lua, you can store additional values as so-called "upvalues". The C function can then retrieve those values when it's called:
// ...
// push session instance pointer
lua_pushlightuserdata( L, (void*)get_current_session() );
// create a closure with one upvalue
lua_pushcclosure( L, my_static_C_function, 1 );
// ... (the resulting Lua function is at the stack top now)
In your static C function you access the instance data like this:
static int my_static_C_function( lua_State* L ) {
session_t* instance = (session_t*)lua_touserdata( L, lua_upvalueindex( 1 ) );
instance->do_something();
return 0;
}
Since each session has its own thread, why don't you link them together?
Whenever your function is called, get the session from the current thread.
You might want to take a look at the manuals about userdata.
Since "Each session runs on its own thread" you could simply save session data as static thread-local.
Better would be to transport session data by userdata, with a metatable bound for associated functions, and don't rely on static stuff.
As last alternative, let's combine: save session data as userdata/pointer Lua-state-local.
In all cases you have to care about class instances destruction because userdata are freed in C-style, same as thread data.
So I was using an array to hold objects, then iterating over them to call a particular method. As the objects in this array could be enumerated/mutated on different threads I was using an NSRecursiveLock.
I then realised that as this method is always called on every object in this array, I could just use an NSNotification to trigger it, and do away with the array and lock.
I am just double checking that this is a good idea? NSNotification will be faster than a lock, right?
Thanks
First, it seems that the locking should be done on the objects, not on the array. You are not mutating the array, but the objects. So you need one mutex per object. It will give you a finer granularity and allow concurrent updates to different objects to proceed in parallel (it wouldn't with a global lock).
Second, a recursive lock is complete overkill. If you wish to have mutual exclusion on the objects, each object should have a standard mutex. If what you are doing inside the critical section is cpu bound and really short, you might consider using a spinlock (it won't even trap in the OS. Only use for short CPU bound critical sections though). Recursive mutex are meant to be used when a thread k can (because of its logic) acquire another lock on an object it has already locked himself (hence the recursive name).
Third, NSNotifications (https://developer.apple.com/library/mac/documentation/Cocoa/Reference/Foundation/Classes/NSNotification_Class/) will allocate memory, drop the notification in a notification Center, do locking to actually implement that (the adding to the center) and finally dispatch the notifications in the center and deallocate the notification. So it is heavier than a plain simple locking. It "hides" the synchronization APIs inside the center, but it does not eliminate them, and you pay the price of memory allocations/deallocations.
If you wish to modify the array (add/remove), then you should also synchronize on these operations, but that would be unfortunate as access to independent entries of the array would now collide.
Hope that helps.
I have two loops:
One loop gets data from a device and processes it. Scales received variables, calculates extra data.
Second loop visualizes the data and stores it.
There are lots of different variables that need to passed between those two loops - about 50 variables. I need the second loop to have access only to the newest values of the data. It needs to be able to read those variables any time they are needed to be visualized.
What is the best way to share such vector between two loops?
There are various ways of sharing data.
The fastest and simplest is a local variable, however that is rather uncontrolled, and you need to make sure to write them at one place (plus you need an indicator).
One of the most advanced options is creating a class for your data, and use an instance (if you create a by-ref class, otherwise it won't matter), and create a public 'GET' method.
In between you have sevaral other options:
queues
semaphores
property nodes
global variables
shared variables
notifiers
events
TCP-IP
In short there is no best way, it all depends on your skills and application.
As long as you're considering loops within the SAME application, there ARE good and bad ideas, though:
queues (OK, has most features)
notifiers (OK)
events (OK)
FGVs (OK, but keep an eye on massively parallel access hindering exec)
semaphores (that's not data comms)
property nodes (very inefficient, prone to race cond.)
global variables (prone to race cond.)
shared variables (badly implemented by NI, prone to race cond.)
TCP-IP (slow, awkward, affected by firewall config)
The quick and dirty way to do this is to write each value to an indicator in the producer loop - these indicators can be hidden offscreen, or in a page of a tab control, if you don't want to see them - and read a local variable of each one in the consumer loop. However if you have 50 different values it may become hard to maintain this code if you need to change or extend it.
As Ton says there are many different options but my suggestion would be:
Create a cluster control, with named elements, containing all your data
Save this cluster as a typedef
Create a notifier using this cluster as the data type
Bundle the data into the cluster (by name) and write this to the notifier in the producer loop
Read the cluster from the notifier in the consumer loop, unbundle it by name and do what you want with each element.
Using a cluster means you can easily pass it to different subVIs to process different elements if you like, and saving as a typedef means you can add, rename or alter the elements and your code will update to match. In your consumer loop you can use the timeout setting of the notifier read to control the loop timing, if you want. You can also use the notifier to tell the loops when to exit, by force-destroying it and trapping the error.
Two ways:
Use a display loop with SEQ (Single Element Queue)
Use a event structure with User Event. (Do not put two event structures in same loop!! Use another)
Use an enum with case structure and variant to cast the data to expected type.
(Notifier isn't reliable to stream data, because is a lossy scheme. Leave this only to trigger small actions)
If all of your variables can be bundled together in a single cluster to send at once, then you should use a single element queue. If your requirements change later such that the transmission cannot be lossy, then it's a matter of changing the input to the Obtain Queue VI (with a notifier you'd have to swap out all of the VIs). Setting up individual indicators and local variables would be pretty darn tedious. Also, not good style.
If the loops are inside of the same VI then:
The simplest solution would be local variables.
Little bit better to use shared variables.
Better is to use functional global variables (FGVs)
The best solution would be using SEQ (Single Element Queue).
Anyway for better understanding please go trough this paper.