Context
I'm currently working on a firmware for a STM32F411CEU6, using STM32CubeIDE, I'm going to be programming several UC's, everyone of them is going to have an ID (a 32 bit unsigned number), this number is static and it will never be change in his lifespan, we are a small team but maybe we will have to program a few hundred of these devices, so changing the value associated whit that ID in the code manually will be kinda exhausting, and time consuming, so, my question is:
¿Is there a way to compile different versions of firmware so it generate several .bin files, each one whit the only difference that this single constant change?
¿Is there a way to automate this process?
What have I thought
I have thought on defining this constant (and other constants if I have to) on a header file, then use something like Python to make different versions of the code, but then I would have to open every project or workspace and still have to compile and produce every .binfile manually, ¿Is there a way to produce the .bin file from python (using the STM32CubeIDE), or something like that?
Additional information
Working on a STM32F411CEU6
Using STM32CubeIDE
I have basic knowledge in python C++
Medium-advance knowledge in C
Thanks in advance!
Any help would be very much appreciated
Here are a few ideas.
The STM32F411 chip is pre-programmed (by STMicro at the factory) with a 96-bit unique device ID. Perhaps you can use the device's unique ID for your purposes rather than creating and assigning your own ID value. See Section 24.1 of the reference manual. This seems much safer than trying to create and manage a different bin file for each ID value.
If you really want your own custom ID value, then program the ID value separately from the firmware bin file so that you don't need to create/manage different bin files for each unit. Write the program so that the ID value is at a known fixed address in ROM. Use the linker scatter file to reserve that address for the ID value. Program the ROM of each unit in two steps, the bin file and the ID value.
If you really want to incorporate the ID value into the bin file then you can use a tool such as srec_cat.exe to concatenate bin (also hex or srec) files. It's very versatile and you should study the man page. One example of how you could use this tool is this: In the source code for your program, declare your unique ID value a constant pointer to a constant value located at a fixed address in ROM beyond the end of the ROM consumed by the bin file. Build the bin file like normal. Then run srec_cat.exe to concatenate the unique ID value to the bin file with the appropriate offset. You could write a script to do this repeatedly for each unique ID value. Perhaps this script runs as a post-build action from the IDE. This solution could work but it seems like a maintenance nightmare to ensure the right bin file gets programmed onto the right device.
If using a hex file is an option, you could avoiding the need for re-compilation like so:
Reserve some flash space outside of your program (optionally configure the linker script to make sure no data is placed in that section).
Use a python script to generate intel hex data with the required ID placed in the reserved location.
Simply concatenate the two hex files and program as usual. I tested this with STM32 ST-LINK Utility / STM32CubeProgrammer.
To generate the hex data, you can use the intelhex package. For example:
import struct
from intelhex import IntelHex
from io import StringIO
ID_FLASH_ADDRESS = 0x8020000
hex_data = StringIO()
ih = IntelHex()
ih.puts(ID_FLASH_ADDRESS, struct.pack('<I', chip_id))
# Output data to variable
ih.write_hex_file(hex_data)
# Get the data
hex_data.getvalue().encode('utf-8')
Notes:
See the struct documentation for the meaning of '<I'.
I output the data to a variable, but you could also write directly to a file. See intelhex documentation.
Related
I need to embed a binary file within an executable generated with gcc on Linux, to be executed in the host (not in a separated device).
In addition, I want to be able to change that binary content externally by using obcjcopy --update-section.
I could do that with __attribute__(("section")), but the problem is that the mentioned binary file might have different sizes at different moments, so I want to allocate a section of a fixed maximum size. Thus, I can update slightly bigger/smaller binaries in the future.
Apart from the above, I would like to give a default value to that particular section at build time (a predefined binary file that is available at build time).
This can be done with a linker script. However, as far as I understand, I would need to modify the OS default linker script, what I want to avoid.
The only thing that comes to my mind is to create an array on that section with a fixed size, using the first bytes for allocating the default binary file and padding the rest with 0xFF's for instance.
Is there a better way to do this?
As ikegami has mentioned, it's enough to specify the maximum size of the array and then initialise the values you need.
I am trying to generate an incrementing value at load time to be used to "serialize" a PCB with a unique code value. Not an expert in ld or preprocessor commands, so looking for some help.
The value will be used in a unique ID for each board that the code is loaded on and will also be used as a counter for boards in the field.
I have no preconceived idea of how I might accomplish this, so any workable answer to get me started, including a pre-preprocessor macro is fine. In my olden days, I recollect adding a couple lines to the linker file that would accomplish this, but I have been unable to resurrect that information anywhere (including my brain's memory cells).
The simpler the answer, the better.
My solution to the problem was remarkably simple.
The binary contained
const char *serial = "XY-00000";
I then wrote a short program that boiled down to:
char uniqueserial [8];
/* Generate serial - this was an SQL call to the manufacturing DB */
char *array;
/* Read binary into array */
memcpy(memmem(array, "XY-00000",8), uniqueserial,8);
/* Write array to temp bin file for flashing */
Depends on the serial template string being unique in the binary. Use strings command to check. I disable crc protected object files due to taste. I like my embedded binaries being exact memory dumps.
The linker is not the right place for two reasons:
the executable can be loaded with the same id in several devices, making your approach void.
You should have to link the executable for each device you are programming, which poses an spent of cpu resources.
The best place is to patch the executable at loading time with the serial number.
Select a data patern as token to initialize your variable with the device id (a pattern difficult to happen elsewhere in your program binary) and initialize your serial number variable to that data pattern (better if you do it statically initializing an array variable or something similar)
Make a program to be executed on each download to device that search for the pattern in the executable file, before loading the binary program into the device and writes the correct value to be programmed into the device (beware that you are patching a binary, so you cannot think on variable lenght strings or the like, that can trash all the work made by the linker)
Once patched the binary executable, you can download it to the device.
Another solution is to reserve a fixed area in your linker script for all this kind of information. Then, put all your device information variables there. Then get the exact positions in rom for the individual variables and include the proper data in the loaded image. In this case, the linker is your friend, reserving some fixed segment in your device's rom allocated for storing the device's individual data (you can put there mac addresses, serial numbers, default configuration, etc.)
Please forgive me if there is a glaringly obvious answer to this question; I haven't found it because I'm not entire sure what I'm looking for. It may well be this duplicates a question I haven't found; sorry.
I have a C executable that uses text, audio, video, icons and a variety of different file types. These files are stored locally; the folder structure is large and deep and would need to be installed alongside the application for it to operate correctly (not that I anticipate it being distributed I'm looking to package my own work for convenience).
In my own opinion it would be more convenient if the file library was stored in a single file that remained accessible to the application for example alongside /usr/bin/APPLICATION or in the most appropriate location; accessed by the executable when required.
I searched for questions similar and found suggestions that indicated two possible options Resource Files which appear to be native to Windows and Including files at compile. The first question leads to an answer similar to the second and doesn't answer the question relating to the existence of resource files for linux executables. It (like the second) looks at including the datafile in the compilation process. This is not so useful as if I only want to update my resources I'm forced to recompile the entire application (the media is dynamically added).
QUESTION: Is there a way to store a variety of file types in one single file accessible to an executable in linux, and if so how would you implement this?
My thoughts on this initially were to create a .zip or .gz file which might also offer compression as an added bonus but I have no idea how (or if it is even possible) to access data within such a file on the fly. I'm equally uncertain if there is a specific file type or library that offers a more suitable solution. Also I know virtually nothing about .dat files could these be used in this context on a linux system?
I do not understand why you would use a single file at all. Considering the added complexity (and increased chance of bugs creeping in) of file extraction and the associated overheads, I do not see how it would be "more convenient".
I have a C executable that uses text, audio, video, icons and a variety of different file types.
So do many other Linux applications. The normal approach, when using package management, is to put the architecture independent data (icons, audio, video, and so on) for application /usr/bin/YOURAPP in /usr/share/YOURAPP/, and architecture dependent data (like helper binaries) in /usr/lib/YOURAPP. It is extremely common for the latter two to be full directory trees, sometimes quite deep and wide.
For locally compiled stuff, it is common to put these in /usr/local/bin/YOURAPP, /usr/local/share/YOURAPP/, and /usr/local/share/YOURAPP/ instead, just to avoid confusing the package manager. (If you check ./configure scripts or read Makefiles, this is the chief purpose of the PREFIX variable they support.)
It is also common for the /usr/bin/YOURAPP to be a simple shell script, setting environment variables, or checking for user-specific overrides (from $HOME/.YOURAPP/), ending up with exec /usr/lib/YOURAPP/YOURAPP.bin [parameters...], which replaces the shell with the actual binary executable without leaving the shell in memory.
As an example, /usr/share/octave/ on my machine contains a total of 138 directories (in a hierarchy of up to 7 directories deep) and 1463 files; about ten megabytes of "stuff" all told. LibreOffice, Eagle, Fritzing, and KiCAD take hundreds of megabytes there each, so Octave is not an extreme example in any way either.
You have several alternatives (TODO: add more ;)):
You can read some archiver file format specifications, writting code to read/write to those archivers, and waste your time doing so.
You can invent a dirty, simple file format, for example ("dsa" stands for "Dirty and Simple Archiver"):
#include <stdint.h>
// Located at the beginning of the file
struct DSAHeader {
char magic[3]; // Shall be (char[]) { 'D', 'S', 'A' }
unsigned char endianness; // The rest of the file is translated according to this field. 0 means little-endian, 1 means big-endian.
unsigned char checksum[16]; // MD5 sum of the whole file. (when calculating checksums, this field is psuedo-filled with zeros).
uint32_t fileCount;
uint32_t stringTableOffset; // A table containing the files' names.
};
// A dsaHeader.fileCount-sized array of DSAInodeHeader follows the DSAHeader.
struct DSANodeHeader {
unsigned char type; // 0 means directory, 1 means regular file.
uint32_t parentOffset; // Pointer to the parent directory, or zero if the node is in the root.
uint32_t offset; // The node's type-dependent header starts here.
uint32_t nodeSize; // In bytes for files, and in number of entries for directories.
uint32_t dataOffset; // The file's data starts at this offset for files, and a pointer to the first DSADirectoryEntryHeader for directories.
uint32_t filenameOffset; // Relative to the string table.
};
typedef uint32_t DSADirectoryEntryHeader; // Offset to the entry's DSANodeHeader
The "string table" is a contiguous sequence of null-terminated character strings.
This format is greatly simple (and portable ;)). And, as a bonus, if you want (de)compression, you can use something like Zip, BZ2, or XZ to (de)compress your file (those programs/formats are archiver-agnostic, i.e, not dependent on tar, as commonly believed).
As last last (or first?) resort, you may use an existent library/API for manipulating archivers and compressed file formats.
Edit: Added support for directories :).
I have a C executable that uses text, audio, video, icons and a variety of different file types. These files are stored locally; the folder structure is large and deep and would need to be installed alongside the application for it to operate correctly.
Considering the added complexity of associated differrent file types alongwith folder structure large and deep and required installed with application. Adding a single resources file would be difficult or would say near to immpossible to trace changes in case if you want to change resources dynamically. Certainly, adding resources to executable file is not an option as it will be increase the size of executable file and needed frequent re-complation in case of update of resources.
After giving consideration on all aspects of your project it seems to me the solution would be using INI file. INI would be stored at definate location and other resources location should be prived in INI File. As with INI you can store the locations of resources, hash keys and sizes easily and would easy check the changes or update the resources.
Since you are using already compressed versions of File type and thus General Zipping algos would not work as the rate would be very low. Thus recommend to use 7z algos for compression. From various algo I would suggest to opt of xz zipping algo as it is currently used by many opensource project to compress the binaries and decrease the size.
Foreach file compression its crc32 or hash value should also included in INI file to check the validity of data transfered.
Lets say you have:
top-level-folder/
|
- your-linux-executable
- icon-files-folder/
- image-files-folder/
- other-folders/
- other-files
Do this (inside top-level-folder)
tar zcvf my-package.tgz top-level-folder
To expand, do this:
tar zxvf my-package.tgz
I have a general question about finished applications. When I go into the files of a windows computer application, some files make sense as to why they are there, such as the executable, various media files, .dll files, etc. However, what I don't understand is how there's potentially thousands of different files, located in hundreds of different directories (counting hierarchy) with anywhere between dozens and hundreds of different filetypes. Some of the filetypes don't even seem like actual files, the extension could be something completely obscure. How does the application know how to work with that? Are all of those files hand-written and compiled or are many of them supplied automatically upon generating a desktop application (which would vary based on the application, of course)? I've never actually compiled an application in any language, as I've been studying JavaScript as a starting point, and I recognize that JavaScript is not intended for creating standalone applications, it's used to implement inside HTML. This is why I have so many questions about the generation of the application itself.
To provide an example, a few of the file extensions I see contained in the Audacity application folder which I don't recognize are as follows: .lsp .raw .mo .ny .exp
Even that is a very short list compared to the amount of filetypes/extensions I usually encounter which I have no knowledge of. So, all in all, my main question is why there's such a crazy amount of files, folders, and filetypes/extensions being used by an application. Hopefully someone can help me understand.
Extra question, for those who might care to answer it:
What does it mean when you open a file in an application like Notepad++ (or a .plist editor) and it's just a bunch of unreadable characters? I'm assuming that means it's a compiled file, but I could use some clarification. This happens when I try to open an .exe, a .dll, etc. I understand why I can't edit things like that in a text editor of course, yet why all the strange symbols and characters? Why wouldn't it just throw an error upon trying to open it? Are all the strange characters just a way of attempting to interpret already compiled code?
Bear with me, I'm pretty new to programming and I'm trying to get a better understanding of the process behind actually generating a GUI-based desktop application. As I said before, my current knowledge doesn't extend to the point of actually compiling an application.
Thank you for any help, I really appreciate it.
Focusing on your extra question: you have to learn what a binary file and a text file is, but in short: Imagine you have a simple calculator program that stores the result in a file. Lets say the result you want to store is the number 64. You have to options to do it: saving it as text (characteres 6 and 4) or as a binary data.
If you store it as a text, you need two bytes: one for the code of the character 6 and other for the character 4. You can open that file with the notepad and you'll see that two characteres '64'.
If you store it as a binary value, you only need one byte, but if you open it with the notepad, you'll see the character whose code is 64: 'A'
Most of such "strange" files are resources needed by parts of the application. A complex application is constructed very modular, and each component may need to load different additional resources, often depending on conditions decided at runtime.
For example, on startup if a Qt-based application reads it should use German translation, it may load trans/de_DE.qm from a directory also containing other language files. Or a game may load level by level from different files depending on how far you've come.
Your second question is quite simple. Most resource files are read by an application function as stream of bytes. If e.g. such stream contains '005a' as 4 bytes, you'll see strange symbols in notepade.exe since that editor interprets such bytes as ASCII code, which means it prints the symbols it finds at place 0, 0, 5, and a in the ASCII table. But the application actually reads it in as 4 x 8 bits = 32bit value, which may mean a 32bit integer value of a variable in my simple example. So the variable value is set to 0x5a wich is decimal 90.
For an embedded system with GUI, we are using our custom translation system. The strings are stored in the code ROM of a microcontroller.
Currently we have up to 10 languages and about 400 translated strings (varies depending on the product variant).
Our current system is rather cumbersome and I believe that gettext would be a better solution.
As far as I understand gettext, it requires the use of bindtextdomain to set the directory containing the compiled translations files (*.mo).
Is there a way to instead read the translation from memory? That is I would like to include the compiled *.mo files in the binary, and set up gettext to use these. Alternatively, the translation data would be stored in a data EEPROM without a filesystem.
Or can you recommend a different translation system for use in a microcontroller system (16 or 32 bit, 256 to 512 kbyte ROM) with a C interface?
Edit: I should add that being able to maintain translations apart from the microcontroller firmware would be a reason to switch to gettext. The appropriate translation data would be loaded by the user with a configuration software that we already supply with our systems.
This is what I would do: I would include the binary ".mo" in constant variables in the code. You can write a simple converter from binary to a char array and have that ".mo" file compiled inside your program. You would have an array of ".mo" file data, each with a different language.
I would modify libintl sourcecode to access one of those arrays. Check the file loadmsgcat.c, the function _nl_load_domain. See there how it tries to use mmap() (so it's prepared to have everything in memory). Just add some code there to decide which element of your ".mo" array to use based on the language requested.
I haven't tried this, but this is what I would try given your situation. It doesn't look too hard.