Ansi C dynamic include - c

I was assigned to edit part of Ansi C application but my knowledge of pure C is just basics. Anyway current situation is I have map1_data1.h, map1_data2.h, map2_data1.h, map2_data2.h and variables in those files are always connected to the map name = map1_structure in map1_data1.h and so on.
In app there is #include for each file and in code then something like
if (game->map == 1){
mapStructure = map1_structure
} else {
mapStructure = map2_structure
}
I have to extend this to be able to load the map dynamicly so something like
void loadMap(int mapId){
mapStructure = map*mapId*_structure // just short for what i want to achieve
}
My first idea to do so was removing map name connection in variables name in map1_data.h and have just structure variable in there. That requires only one header file at time to be loaded and thats where I'm stucked. Havent found any clues to do so on google.
I would like to have it as variable as possible so something like #include "map*mapId*_data1.h" but should be ok to have 1 switch in one place in whole app to decide on what map to be loaded.
One more thing, the app keeps running for more than 1 game = it will load various maps in one run.

Judging from the comments, you have a single type, call it Map, which is a structure type containing a collection of different data types, including 3D arrays and points and so on. You need to have some maps built into the program; later on, you will need to load new maps at runtime.
You have two main options for the runtime loading the maps:
Map in shared object (shared library, dynamically loaded library, aka DLL).
Map in data file.
Of these two, you will choose the data file over the shared object because it is, ultimately, simpler and more flexible.
Shared Object
With option 1, only someone who can compile a shared library can create the new maps. You'd have a 'library' consisting of one or more data objects, which can be looked up by name. On most Unix-like systems, you'd end up using dlopen() to load the library, and then dlsym() to find the symbol name in that library (specifying the name via a string). If it is present in the library, dlsym() will return you a pointer.
In outline:
typedef void *SO_Handle;
const char *path_to_library = "/usr/local/lib/your_game/libmap32.so";
const char *symbol_name = "map32_structure";
SO_Handle lib = dlopen(path_to_library, RTLD_NOW);
if (lib == 0)
...bail out...
map_structure = dlsym(lib, symbol_name);
if (map_structure == 0)
...bail out...
You have to have some way of generating the library name based on where the software is installed and where extensions are downloaded. You also have to have some way of knowing the name of the symbol to look for. The simplest system is to use a single fixed name (map_structure), but you are not constrained to do that.
After this, you have your general map_structure read for use. You can invent endless variations on the theme.
Data file
This is the more likely way you'll do it. You arrange to serialize the map structure into a disk file that can be read by your program. This will contain a convenient representation of the data. You should consider the TLV (type-length-value) encoding scheme, so that you can tell by looking at the type what sort of data follows, and the length tells you how many of them, and the value is the data. You can do this with binary data or with text data. It is easier to debug text data because you can look at and see what's going on. The chances are that the difference in performance between binary and text is small enough (swamped by the I/O time) that using text is the correct way to go.
With a text description of the map, you'd have information to identify the file as being a map file for your game (perhaps with a map format version number). Then you'd have sections describing each of the main elements in the Map structure. You'd allocate the Map (malloc() et al), and then load the data from the file into the structure.

Related

What is variable attribute SEC means?

Currently, I'm tracing a bpf program and found something I can't understand.
There're several declaration like:
struct bpf_map_def SEC("maps") map_parsing_context = {
...
};
struct {
...
} map_keys SEC(".maps");
My question are:
What did this syntax called?
Is this same with __attribute__((section("name")))?
What is different between map and .map? Are they just user-defined sections?
This is for declaring the structure of the object (in this case, the keys for the map) for BTF.
SEC() is the same as __attribute__((section("name"), used)) so what it does is putting the defined object into the given ELF section.
Now, eBPF maps were (and still can) be defined in the maps ELF section for libbpf to be able to parse the relevant metadata and find or create these maps before the program can use them, hence: SEC("maps").
The other call to SEC() is to put additional metadata into the .maps ELF section: this information is a declaration of the structure of the map_keys, it tells what fields the key is made of. Libbpf will use this information to generate the BTF information related to the structure of the map. This information is passed to the kernel, and can be used for several purposes. One use case is to see the structure of the map when examining it, for example with bpftool map dump, and to easily dump each key (and value) fields individually instead of printing a raw single hexadecimal blob for the whole structure the key (or value) is made of. The kernel may also use it for other advanced eBPF features: for example, using spinlocks on fields in map entries requires the BTF information to be present, so that the kernel can understand what the struct fields are and which one should be locked.
[Edit] Using the maps section is the “legacy” way to declare maps, still valid when declaring without BTF. When BTF information is added, everything (map metadata and structure layout) should go into the .maps section, although redundancy with metadata in maps is supported. Usually, you don't use maps when declaring into .maps, simply because this is useless.
Some pointers related to BTF: blog post, kernel documentation. See also OP's link in the comments.
So to answer your questions directly:
The syntax is used to declare the map metadata (size, type, etc.) (maps) as well as the structure of the map keys (.maps), in this case.
The SEC() macro itself is the same as the one you provide.
maps and .maps are different ELF sections. They are defined by the user, but they are used as a convention by libbpf to parse data from the ELF object and to extract metadata for the map, and for metadata and the structure of map entries, respectively.

How to create Datasets Like MNIST in Pytorch?

I have looked Pytorch source code of MNIST dataset but it seems to read numpy array directly from binaries.
How can I just create train_data and train_labels like it? I have already prepared images and txt with labels.
I have learned how to read image and label and write get_item and len, what really confused me is how to make train_data and train_labels, which is torch.Tensor. I tried to arrange them into python lists and convert to torch.Tensor but failed:
for index in range(0,len(self.files)):
fn, label = self.files[index]
img = self.loader(fn)
if self.transform is not None:
img = self.transform(img)
train_data.append(img)
self.train_data = torch.tensor(train_data)
ValueError: only one element tensors can be converted to Python scalars
There are two ways to go. First, the manual. Torchvision.datasets states the following:
datasets are subclasses of torch.utils.data.Dataset i.e, they have __getitem__ and __len__ methods implemented. Hence, they can all be passed to a torch.utils.data.DataLoader which can load multiple samples parallelly using torch.multiprocessing workers.
So you can just implement your own class which scans for all the images and labels, keeps a list of their paths (so that you don't have to keep them in RAM) and has the __getitem__ method which given index i reads the i-th file, its label and returns them. This minimal interface is enough to work with the parallel dataloader in torch.utils.data.
Secondly, if your data directory can be rearranged into either structure, you can use DatasetFolder and ImageFolder pre-built loaders. This will save you some coding and automatically provide support for data augumentation routines from torchvision.transforms.

How do I get access to GUI elements in a IUP dialog loaded from a LED file?

I’m in love with IUP! However I cannot figure out how to get programmatic access (in C) to GUI elements in a dialog loaded by IupLoad() from a LED file.
One extremely laborious way would be to edit the LED file so as to manually give handle names to each single GUI element, then manually define corresponding variables for each element in C, then manually load handles into each variable by using IupGetHandle().
One comfortable way to do it would be to convert the LED file to a C header file using the built-in Layout Dialog tool. The resulting code makes each element available to the application in a simple array called Ihandle* containers[]. But this way deprives us of the benefits of LED files, such as the ability to edit GUI of a binary application by the user and keeping the C code small.
Is there no good way to do it?
Do I overrate the benefits of a third way, if it existed?
I cannot find any IupLoad() example in the directory with C examples.
My own example below explicitly defines one handle name for the top element (dialog) only. It features a very simple dialog where defining each element manually wouldn’t be a hard work at all. But this is only a test example for Stack Overflow and my question is relevant to complex dialogs.
C file:
#include <stdlib.h>
#include <iup.h>
int main(int argc, char **argv)
{
IupSetGlobal("UTF8MODE", "YES");
// IupSetGlobal("UTF8MODE_FILE", "YES");
IupOpen(&argc, &argv);
if(IupLoad("dropdown.led")) IupMessage("Error", "Failed to load LED.");
else {
Ihandle *dropdown = IupGetHandle("dropdown");
IupShow(dropdown);
IupMainLoop();
}
IupClose();
return EXIT_SUCCESS;
}
Corresponding dropdown.led file:
dropdown = DIALOG[TITLE=dropdown.led](
HBOX[CMARGIN=10x10,CGAP=10](
LIST[VALUE=3, 1=я, 2=ты, 3=оно, 4=мы, 5=вы, 6=они, DROPDOWN=YES](do_nothing),
LIST[VALUE=3, 1=ik, 2=je, 3=hij, 4=we, DROPDOWN=YES](do_nothing)
)
)
Which brings us to another question: how can I make Russian characters visible? But this issue is owed a separate thread which I will accordingly create.
All questions that pertain to this particular example:
How do I get access to GUI elements in a IUP dialog loaded from a LED file? (current)
How can I make Russian letters visible in a IUP dialog loaded from a LED file?
A gap in IUP dropdown lists
The way os to use IupGetHandle to get access to some element then use IupGetChild*, GetBrother, GetParent functions to get the element you want.
Another option is to use the NAME attribute. You set it on the element you want then use IupGetDialogChild to retrieve the element given the NAME value.

How to get the address of a variable, if I have the name of the variable in string during runtime?

I'm expecting something like below.
int genVar001, genVar002, genVar003;
int adrgenVar, status;
char strvariable[] = "genVar001";
status = somefunction(&strvariable, adrgenVar);
Where 'adrgenVar' will have the address of genVar001.! If the variable is not available the status should return error.
Why it is required: for manipulation of the values in the runtime (via keyboard or files).
Lookup table will not help me because it would increase the job; maintain the table if anything new is added.
Edit:
I'm working with PLCs, Here is the library which will allow me to do so.
Function: PV_xgetadr.
C doesn't have any sort of introspection capabilities in the standard, there's no way to do what you're aiming for in a portable manner. And the non-portable manners (like using the executable's debug info or symbol table) would be a terrible idea.
But that doesn't mean there's no solution. Judging by your variable names, what you should be doing is:
int genVar[3];
Then you can dynamically index into that array at runtime. If that's not flexible enough, use a linked list or a hash map - there are quite a few implementations of those for C available (none in the standard though).

Serialize Data Structures in C

I'd like a C library that can serialize my data structures to disk, and then load them again later. It should accept arbitrarily nested structures, possibly with circular references.
I presume that this tool would need a configuration file describing my data structures. The library is allowed to use code generation, although I'm fairly sure it's possible to do this without it.
Note I'm not interested in data portability. I'd like to use it as a cache, so I can rely on the environment not changing.
Thanks.
Results
Someone suggested Tpl which is an awesome library, but I believe that it does not do arbitrary object graphs, such as a tree of Nodes that each contain two other Nodes.
Another candidate is Eet, which is a project of the Enlightenment window manager. Looks interesting but, again, seems not to have the ability to serialize nested structures.
Check out tpl. From the overview:
Tpl is a library for serializing C
data. The data is stored in its
natural binary form. The API is small
and tries to stay "out of the way".
Compared to using XML, tpl is faster
and easier to use in C programs. Tpl
can serialize many C data types,
including structures.
I know you're asking for a library. If you can't find one (::boggle::, you'd think this was a solved problem!), here is an outline for a solution:
You should be able to write a code generator[1] to serialize trees/graphs without (run-time) pre-processing fairly simply.
You'll need to parse the node structure (typedef handling?), and write the included data values in a straight ahead fashion, but treat the pointers with some care.
For pointer to other objects (i.e. char *name;) which you know are singly referenced, you can serialize the target data directly.
For objects that might be multiply refernced and for other nodes of your tree you'll have to represent the pointer structure. Each object gets assigned a serialization number, which is what is written out in-place of the pointer. Maintain a translation structure between current memory position and serialization number. On encountering a pointer, see if it is already assigned a number, if not, give it one and queue that object up for serialization.
Reading back also requires a node-#/memory-location translation step, and might be easier to do in two passes: regenerate the nodes with the node numbers in the pointer slots (bad pointer, be warned) to find out where each node gets put, then walk the structure again fixing the pointers.
I don't know anything about tpl, but you might be able to piggy-back on it.
The on-disk/network format should probably be framed with some type information. You'll need a name-mangling scheme.
[1] ROOT uses this mechanism to provide very flexible serialization support in C++.
Late addition: It occurs to me that this is not always as easy as I implied above. Consider the following (contrived and badly designed) declaration:
enum {
mask_none = 0x00,
mask_something = 0x01,
mask_another = 0x02,
/* ... */
mask_all = 0xff
};
typedef struct mask_map {
int mask_val;
char *mask_name;
} mask_map_t;
mask_map_t mask_list[] = {
{mask_something, "mask_something"},
{mask_another, "mask_another"},
/* ... */
};
struct saved_setup {
char* name;
/* various configuration data */
char* mask_name;
/* ... */
};
and assume that we initalize out struct saved_setup items so that mask_name points at mask_list[foo].mask_name.
When we go to serialize the data, what do we do with struct saved_setup.mask_name?
You will need to take care in designing your data structures and/or bring some case-specific intelligence to the serialization process.
This is my solution. It uses my own implementation of malloc, free and mmap, munmap system calls. Follow the given example codes. Ref: http://amscata.blogspot.com/2013/02/serialize-your-memory.html
In my approach I create a char array as my own RAM space. Then there are functions for allocate the memory and free them. After creating the data structure, by using mmap, I write the char array to a file.
Whenever you want to load it back to the memory there is a function which used munmap to put the data structure again to the char array. Since it has virtual addresses for your pointers, you can re use your data structure. That means, you can create data structure, save it, load it, again edit it, and save it again.
You can take a look on eet. A library of the enlightenment project to store C data types (including nested structures). Although nearly all libs of the enlightenment project are in pre-alpha state, eet is already released. I'm not sure, however, if it can handle circular references. Probably not.
http://s11n.net/c11n/
HTH
you should checkout gwlib. the serializer/deserializer is extensive. and there are extensive tests available to look at. http://gwlib.com/
I'm assuming you are talking about storing a graph structure, if not then disregard...
If your storing a graph, I personally think the best idea would be implementing a function that converts your graph into an adjacency matrix. You can then make a function that converts an adjacency matrix to your graph data structure.
This has three benefits (that may or may not matter in your application):
adjacency matrix are a very natural way to create and store a graph
You can create an adjacency matrix and import them into your applications
You can store and read your data in a meaningful way.
I used this method during a CS project and is definitely how I would do it again.
You can read more about adjacency matrix here: http://en.wikipedia.org/wiki/Modified_adjacency_matrix
Another option is Avro C, an implementation of Apache Avro in C.
Here is an example using the Binn library (my creation):
binn *obj;
// create a new object
obj = binn_object();
// add values to it
binn_object_set_int32(obj, "id", 123);
binn_object_set_str(obj, "name", "Samsung Galaxy Charger");
binn_object_set_double(obj, "price", 12.50);
binn_object_set_blob(obj, "picture", picptr, piclen);
// send over the network
send(sock, binn_ptr(obj), binn_size(obj));
// release the buffer
binn_free(obj);
If you don't want to use strings as keys you can use a binn_map which uses integers as keys.
There is also support for lists, and all these structures can be nested:
binn *list;
// create a new list
list = binn_list();
// add values to it
binn_list_add_int32(list, 123);
binn_list_add_double(list, 2.50);
// add the list to the object
binn_object_set_list(obj, "items", list);
// or add the object to the list
binn_list_add_object(list, obj);
In theory YAML should do what you want http://code.google.com/p/yaml-cpp/
Please let me know if it works for you.

Resources