I'm collecting program counter samples from a ARM Cortex M3. A long list like this:
0x8005b2a
0x8001324
0x8005b34
0x8001318
The pc is sampled periodically. I now want to have a static flat profile from the running program. Like (g)prof is doing with support of the linux kernel.
Is there a way to convert these PC samples in a (g)prof readable format or are there other tools that give me a profile based on these pc samples and an *.elf / *.lst file ?
Hans Müller, as I understand, gprof format is not so easy to generate (https://sourceware.org/binutils/docs/gprof/File-Format.html - "new file format is defined in header file gmon_out.h. It consists of a header containing the magic cookie and a version number, ... Histogram records consist of a header that is followed by an array of bins...")
I can recommend generation of
gpreftools (pprof tool - https://gperftools.googlecode.com/git/doc/cpuprofile-fileformat.html) or
callgrind.out formats (callgrind_annotate and kcachegrind GUI tool - http://valgrind.org/docs/manual/cl-format.html but you should resolve symbol names and kcachegrind requires name compression to be turned on)
Easier way (for flat) is to use some awk/perl/python scripting and addr2line tool from binutils (you need addr2line with support of target architecture). This tool will give you function names from address (if you correctly map PC samples to virtual addresses of elf binary), and your script should sum samples for every function, then sort. It is harder to handle callgraph in small scripts.
gpreftools's pprof is just script which can run addr2line for you (you still need right variant of addr2line). It is capable of summing samples for functions and sorting, and even may call objdump to get annotated disassembly.
Both formats may be used for flat profiles, but they support callgraphs to some level. pprof allows you to save full backtrace in every event; while callgrind.out format stores only pairs caller-callee (cfn), and kcachegrind may incorrectly guess pathes to hot code.
Related
Context
I'm currently working on a firmware for a STM32F411CEU6, using STM32CubeIDE, I'm going to be programming several UC's, everyone of them is going to have an ID (a 32 bit unsigned number), this number is static and it will never be change in his lifespan, we are a small team but maybe we will have to program a few hundred of these devices, so changing the value associated whit that ID in the code manually will be kinda exhausting, and time consuming, so, my question is:
¿Is there a way to compile different versions of firmware so it generate several .bin files, each one whit the only difference that this single constant change?
¿Is there a way to automate this process?
What have I thought
I have thought on defining this constant (and other constants if I have to) on a header file, then use something like Python to make different versions of the code, but then I would have to open every project or workspace and still have to compile and produce every .binfile manually, ¿Is there a way to produce the .bin file from python (using the STM32CubeIDE), or something like that?
Additional information
Working on a STM32F411CEU6
Using STM32CubeIDE
I have basic knowledge in python C++
Medium-advance knowledge in C
Thanks in advance!
Any help would be very much appreciated
Here are a few ideas.
The STM32F411 chip is pre-programmed (by STMicro at the factory) with a 96-bit unique device ID. Perhaps you can use the device's unique ID for your purposes rather than creating and assigning your own ID value. See Section 24.1 of the reference manual. This seems much safer than trying to create and manage a different bin file for each ID value.
If you really want your own custom ID value, then program the ID value separately from the firmware bin file so that you don't need to create/manage different bin files for each unit. Write the program so that the ID value is at a known fixed address in ROM. Use the linker scatter file to reserve that address for the ID value. Program the ROM of each unit in two steps, the bin file and the ID value.
If you really want to incorporate the ID value into the bin file then you can use a tool such as srec_cat.exe to concatenate bin (also hex or srec) files. It's very versatile and you should study the man page. One example of how you could use this tool is this: In the source code for your program, declare your unique ID value a constant pointer to a constant value located at a fixed address in ROM beyond the end of the ROM consumed by the bin file. Build the bin file like normal. Then run srec_cat.exe to concatenate the unique ID value to the bin file with the appropriate offset. You could write a script to do this repeatedly for each unique ID value. Perhaps this script runs as a post-build action from the IDE. This solution could work but it seems like a maintenance nightmare to ensure the right bin file gets programmed onto the right device.
If using a hex file is an option, you could avoiding the need for re-compilation like so:
Reserve some flash space outside of your program (optionally configure the linker script to make sure no data is placed in that section).
Use a python script to generate intel hex data with the required ID placed in the reserved location.
Simply concatenate the two hex files and program as usual. I tested this with STM32 ST-LINK Utility / STM32CubeProgrammer.
To generate the hex data, you can use the intelhex package. For example:
import struct
from intelhex import IntelHex
from io import StringIO
ID_FLASH_ADDRESS = 0x8020000
hex_data = StringIO()
ih = IntelHex()
ih.puts(ID_FLASH_ADDRESS, struct.pack('<I', chip_id))
# Output data to variable
ih.write_hex_file(hex_data)
# Get the data
hex_data.getvalue().encode('utf-8')
Notes:
See the struct documentation for the meaning of '<I'.
I output the data to a variable, but you could also write directly to a file. See intelhex documentation.
I am trying to run some MC9S12DP256 example files, but I want to see the code to understand it. Are there any ways to convert a .s19 or .abs file to a C code?
An ".s19" or an ".abs" file contains mainly the machine code of the application. The source code of it is not included, independent of the language used to write it. Even if it were written in assembly language, all symbolic informations and comments are excluded.
However, you can try to de-compile the machine code. This is not a trivial or quick task, you need to know the target really well. I did this with software for other processors, it is feasible for code up to some KB.
These are the steps I recommend:
Get a disassembler and an assembler for the target processor, optimally from the vendor.
Let it disassemble the machine code into assembly source code. You might need to convert the ".s19" file into a binary file, one possible tool for this is "srecord".
Assemble the resulting source code again into ".s19" or ".abs", and make sure that it generates the same contents as your original.
Insert labels for the reset and interrupt entry points. Start at the reset entry point with your analysis.
Read the source code, think about what it does.
You will quickly "dive" into subroutines that execute small functions, like reading ADC or sending data. Place a label and replace the numerical value at the call sites with the label.
Expect sections of (constant) data mixed with executable code.
Repeat often from point 3. If you have a difference, undo your last step and redo it in another way until you produce the same contents.
If you want C source, it is commonly much more difficult. You need a lot of experience how C is compiled into machine code. Be aware that variables or even functions are commonly placed in another sequence than they are declared. If you want to go that route, you usually also have to use the exact version of the compiler used to generate the original machine code.
Be aware that the original might be produced with any other language.
I am trying to generate an incrementing value at load time to be used to "serialize" a PCB with a unique code value. Not an expert in ld or preprocessor commands, so looking for some help.
The value will be used in a unique ID for each board that the code is loaded on and will also be used as a counter for boards in the field.
I have no preconceived idea of how I might accomplish this, so any workable answer to get me started, including a pre-preprocessor macro is fine. In my olden days, I recollect adding a couple lines to the linker file that would accomplish this, but I have been unable to resurrect that information anywhere (including my brain's memory cells).
The simpler the answer, the better.
My solution to the problem was remarkably simple.
The binary contained
const char *serial = "XY-00000";
I then wrote a short program that boiled down to:
char uniqueserial [8];
/* Generate serial - this was an SQL call to the manufacturing DB */
char *array;
/* Read binary into array */
memcpy(memmem(array, "XY-00000",8), uniqueserial,8);
/* Write array to temp bin file for flashing */
Depends on the serial template string being unique in the binary. Use strings command to check. I disable crc protected object files due to taste. I like my embedded binaries being exact memory dumps.
The linker is not the right place for two reasons:
the executable can be loaded with the same id in several devices, making your approach void.
You should have to link the executable for each device you are programming, which poses an spent of cpu resources.
The best place is to patch the executable at loading time with the serial number.
Select a data patern as token to initialize your variable with the device id (a pattern difficult to happen elsewhere in your program binary) and initialize your serial number variable to that data pattern (better if you do it statically initializing an array variable or something similar)
Make a program to be executed on each download to device that search for the pattern in the executable file, before loading the binary program into the device and writes the correct value to be programmed into the device (beware that you are patching a binary, so you cannot think on variable lenght strings or the like, that can trash all the work made by the linker)
Once patched the binary executable, you can download it to the device.
Another solution is to reserve a fixed area in your linker script for all this kind of information. Then, put all your device information variables there. Then get the exact positions in rom for the individual variables and include the proper data in the loaded image. In this case, the linker is your friend, reserving some fixed segment in your device's rom allocated for storing the device's individual data (you can put there mac addresses, serial numbers, default configuration, etc.)
I'm not sure if this is specific to the processor I'm using, so for what it's worth I'm using a Cortex M0+. I was wondering: if I generate a hex file through gcc using -fPIC, I produce...Position Independent Code. However, the intel hex file format that I get out of objcopy always has address information on each line's header. If I'm trying to write a bootloader, do I just ignore that information, skip the bytes relating to it, and load the actual code into memory wherever I want, or do I have to keep track of it somehow?
The intel-HEX format was specially designed to programm PROMs, EPROMS or processors with an internal EPROM and is normally used with programmers for theses devices. The addresses at the beginning of the records have not much to do with the program code directly. They indicate at which address of the PROM the data will be written. Remember also that the PROM can be mapped anywhere into the address space of the processor, thus the final address can change anyway.
As long as you don't want to program a PROM you must remove anything except the data from the records. (Don't forget the checksum at the end ;-)
As I understand the intel-HEX format the records must not be contiguous, there may be holes in between.
Some remarks:
The -f PIC parameter is not responsible for the intel-HEX format. I think that somewhere in your command lines you'll find -O ihex. If you want to have a file that could be executed, objcopy provides better suited output formats.
As long as you don't write earlier stages of the boot process by yourself, you don't load your bootloader - it will be loaded for you. The address at which this will happen is normally fixed and not changeable. So there is no need for position independent code, but it doesn't hurt either.
I am doing an ongoing project to write a simplified OS for hobby/learning purposes. I can generate hex files, and now I want to write a script on the chip to accept them via serial, load them into RAM, then execute them. For simplicity I'm writing in assembly so that all of the startup code is up to me. Where do I start here? I know that the hex file format is well documented, but is it as simple as reading the headers for each line, aligning the addresses, then putting the data into RAM and jumping to the address? It sounds like I need a lot more than that, but this is a problem that most people don't even try to solve. Any help would be great.
way too vague, there are many different file formats and at least two really popular ones that use text with the data in hex. So not really helping us here.
writing a script on chip means you have an operating system running on your microcontroller? what operating system is it and what does the command line look like, etc.
assembly is not required to completely control everything (basically baremetal) can use asm to bootstrap C and then the rest in C, not a problem.
Are you wanting to download to ram and run or wanting to download and then burn to flash to reset into in some way?
Basically what you are making is a bootloader. And yes we write bootloaders all the time, one for each platform, sometimes borrowing code from a prior platform sometimes not.
First off on your development computer, windows, mac, linux, whatever, write a program (in C or Pascal ideally, something you can easily port to the microcontroller) that reads the whole file into an array, then write some code that basically accepts one byte at a time like you would if you were receiving it serially. Parse through that file format whatever format you choose (initially, then perhaps change formats if you decide you no longer like it) take real programs that you have built which the disassembler or other tools should have other output options to show you what bytes or words should be landing at what addresses. Parse this data, printf out the address/byte or address/word items you find, and then compare that to what the toolchain showed. carve the parsing tool out and replace the printf with write to memory at that address. and then yes you jump to the entry point if you can figure that out and/or you as the designer decide all programs must have a specific entry point.
Intel hex and motorola s-record are good choices (-O ihex or -O srec, my current favorite is --srec-forceS3 -O srec), both are documented at wikipedia. and at least the gnu tools will produce both, you can use a dumb terminal program (minicom) to dump the file into your microcontroller and hopefully parse and write to ram as it comes in. If you cant handle that flow you might think of a raw binary (-O binary) and implement an xmodem receiver in your bootloader.