Suppose there is a C program, which stores its version in a global char* in main.c. Can the buildsystem (gnu make) somehow extract the value of this variable on build time, so that the executable built could have the exact version name as it appears in the program?
What I'd like to achieve is that given the source:
char g_version[] = "Superprogram 1.32 build 1142";
the buildsystem would generate an executable named Superprogram 1.32 build 1142.exe
The shell function allows you to call external applications such as sed that can be used to parse the source for the details required.
Define your version variable from a Macro:
char g_version[] = VERSION;
then make your makefile put a -D argument on the command line when compiling
gcc hack.c -DVERSION=\"Superprogram\ 1.99\"
And of course you should in your example use sed/grep/awk etc to generate your version string.
You can use any combination of unix text tools (grep, sed, awk, perl, tail, ...) in your Makefile in order to extract that information from source file.
Usually the version is defined as a composition of several #define values (like in the arv library for example).
So let's go for a simple and working example:
// myversion.h
#define __MY_MAJOR__ 1
#define __MY_MINOR__ 8
Then in your Makefile:
# Makefile
source_file := myversion.h
MAJOR_Identifier := __MY_MAJOR__
MINOR_Identifier := __MY_MINOR__
MAJOR := `cat $(source_file) | tr -s ' ' | grep "\#define $(MAJOR_Identifier)" | cut -d" " -f 3`
MINOR := `cat $(source_file) | tr -s ' ' | grep '\#define $(MINOR_Identifier)' | cut -d" " -f 3`
all:
#echo "From the Makefile we extract: MAJOR=$(MAJOR) MINOR=$(MINOR)"
Explanation
Here I have used several tools so it is more robust:
tr -s ' ': to remove extra space between elements,
grep: to select the unique line matching our purpose,
cut -d" " -f 3: to extract the third element of the selected line which is the target value!
Note that the define values can be anything (not only numeric).
Beware to use := (not =) see: https://stackoverflow.com/a/10081105/4716013
Related
I'm writing a Terminal Match-Anything Pattern Rule, i.e. %::, that, as expected, will run only if no other target is matched. In its recipe I want to iterate over makefile's explicit targets and check if the found pattern ($*) is the beginning of any other target
By now I'm successfully getting all desired targets in a space-separated string and storing it in a variable TARGETS, however I couldn't turn it in an array to be able to iterate over each word in the string.
For instance
%::
$(eval TARGETS ::= $(shell grep -Ph "^[^\t].*::.*##" ./Makefile | cut -d : -f 1 | sort))
echo $(TARGETS)
gives me just what I was expecting:
build clean compile deploy execute init run serve
The Question
How could I iterate over each of $(TARGET) string words inside a GNU Make 4.2.1 loop?
I found a bunch of BASH solutions, but none of them worked in my tests:
Reading a delimited string into an array in Bash
How to split one string into multiple strings separated by at least >one space in bash shell?
It's generally a really bad idea to use eval and shell inside a recipe. A recipe is already a shell script so you should just use shell scripting.
It's not really clear exactly what you want to do. If you want to do this in a recipe, you can use a shell loop:
%::
TARGETS=$$(grep -Ph "^[^\t].*::.*##" ./Makefile | cut -d : -f 1 | sort); \
for t in $$TARGETS; do \
echo $$t; \
done
If you want to do it outside of a recipe you can use the GNU make foreach function.
I have the source code for a huge (exaggerated) c program with multiple files. I was hoping if there was any way to find all the functions (both standard and builtin) used (both declared and called) in the program. I know I can compile it and track the function and system calls using tools like ltrace and strace, by redirecting the output to a file first and then using grep to select the function calls. Or I can use regex on shell on all the files, but I don't know regex (yet). So does any one know a tool that could help me to quickly find all the functions called and declared in a c program?
Check if this helps, comments in code:
#!/bin/bash
# For each .c in the current path (recursive)
for f in $(find . -name '*.c'); do
# Extract the functions and his location with aux-info
# Send the results to output.info
# Send errors to /dev/null
gcc -aux-info output.info $f 2>/dev/null
# Extract function names from output.info excluding std headers and comments
grep -Ev "/usr|compiled from" output.info
done
# Remove output.info
rm output.info
Since it seems that you are trying to extract your own code following a pattern: type function(params), you can avoid using gcc -aux-info, try the following:
#!/bin/bash
# For each .c in the current path (recursive)
for f in $(find . -name '*.c'); do
# Extract the lines starting with a letter or underscore
# and ending with ')' or '{' or '}' ignoring trailing spaces
grep --with-filename --line-number --color '^[a-z|A-Z|_].*[)|{|} ]$' $f
done
Another way combining readelf and awk, notice that in this case you need to provide the name of the program/binary instead of the source files:
# Column 3 != 0 and Column 4 = "FUNC" and Column 8 not starting with _
readelf -sW your_program | awk '$3 != 0 && $4 == "FUNC" && $8 !~ /^ ?_/ {print $8}'
Because of function pointers, no, there's not. Consider this program:
#include <dlfcn.h>
int main(int argc, char **argv) {
void *handle = dlopen(argv[1], RTLD_LAZY);
void (*f)(void) = dlsym(handle, argv[2]);
f();
dlclose(handle);
}
It should be obvious that you can't possibly hope to make a list of all the functions that it can call.
#David, gave me a good answer, but after learning for loops in bash I came up with a method that is a little cleaner as far as messages are concerned. And the best thing about this method is that it scans the file statically which means even if some functions are assigned to a pointer as #joseph pointed you will be able to see a list of all the function calls made. The out put of this program isn't as clean as it could be as I don't know much regex, so if any one knows a better way don't forget to clean it up,
#!/bin/bash
#go through each .c file in the directory
for f in $(find . -maxdepth 1 -name "*.c");do
#print the name of the file
echo "$f"
#print the content of the file
#and then choose the lines that contain function calls
cat -n $f|grep -v "if\|else\|for\|while\|return"| grep --color=always "_*(\|[[:alpha:]]*("
done
Edit: I think this has been answered successfully, but I can't check 'til later. I've reformatted it as suggested though.
The question: I have a series of files, each with a name of the form XXXXNAME, where XXXX is some number. I want to move them all to separate folders called XXXX and have them called NAME. I can do this manually, but I was hoping that by naming them XXXXNAME there'd be some way I could tell Terminal (I think that's the right name, but not really sure) to move them there. Something like
mv *NAME */NAME
but where it takes whatever * was in the first case and regurgitates it to the path.
This is on some form of Linux, with a bash shell.
In the real life case, the files are 0000GNUmakefile, with sequential numbering. I'm having to make lots of similar-but-slightly-altered versions of a program to compile and run on a cluster as part of my research. It would probably have been quicker to write a program to edit all the files and put in the right place in the first place, but I didn't.
This is probably extremely simple, and I should be able to find an answer myself, if I knew the right words. Thing is, I have no formal training in programming, so I don't know what to call things to search for them. So hopefully this will result in me getting an answer, and maybe knowing how to find out the answer for similar things myself next time. With the basic programming I've picked up, I'm sure I could write a program to do this for me, but I'm hoping there's a simple way to do it just using functionality already in Terminal. I probably shouldn't be allowed to play with these things.
Thanks for any help! I can actually program in C and Python a fair amount, but that's through trial and error largely, and I still don't know what I can do and can't do in Terminal.
SO many ways to achieve this.
I find that the old standbys sed and awk are often the most powerful.
ls | sed -rne 's:^([0-9]{4})(NAME)$:mv -iv & \1/\2:p'
If you're satisfied that the commands look right, pipe the command line through a shell:
ls | sed -rne 's:^([0-9]{4})(NAME)$:mv -iv & \1/\2:p' | sh
I put NAME in brackets and used \2 so that if it varies more than your example indicates, you can come up with a regular expression to handle your filenames better.
To do the same thing in gawk (GNU awk, the variant found in most GNU/Linux distros):
ls | gawk '/^[0-9]{4}NAME$/ {printf("mv -iv %s %s/%s\n", $1, substr($0,0,4), substr($0,5))}'
As with the first sample, this produces commands which, if they make sense to you, can be piped through a shell by appending | sh to the end of the line.
Note that with all these mv commands, I've added the -i and -v options. This is for your protection. Read the man page for mv (by typing man mv in your Linux terminal) to see if you should be comfortable leaving them out.
Also, I'm assuming with these lines that all your directories already exist. You didn't mention if they do. If they don't, here's a one-liner to create the directories.
ls | sed -rne 's:^([0-9]{4})(NAME)$:mkdir -p \1:p' | sort -u
As with the others, append | sh to run the commands.
I should mention that it is generally recommended to use constructs like for (in Tim's answer) or find instead of parsing the output of ls. That said, when your filename format is as simple as /[0-9]{4}word/, I find the quick sed one-liner to be the way to go.
Lastly, if by NAME you actually mean "any string of characters" rather than the literal string "NAME", then in all my examples above, replace NAME with .*.
The following script will do this for you. Copy the script into a file on the remote machine (we'll call it sortfiles.sh).
#!/bin/bash
# Get all files in current directory having names XXXXsomename, where X is an integer
files=$(find . -name '[0-9][0-9][0-9][0-9]*')
# Build a list of the XXXX patterns found in the list of files
dirs=
for name in ${files}; do
dirs="${dirs} $(echo ${name} | cut -c 3-6)"
done
# Remove redundant entries from the list of XXXX patterns
dirs=$(echo ${dirs} | uniq)
# Create any XXXX directories that are not already present
for name in ${dirs}; do
if [[ ! -d ${name} ]]; then
mkdir ${name}
fi
done
# Move each of the XXXXsomename files to the appropriate directory
for name in ${files}; do
mv ${name} $(echo ${name} | cut -c 3-6)
done
# Return from script with normal status
exit 0
From the command line, do chmod +x sortfiles.sh
Execute the script with ./sortfiles.sh
Just open the Terminal application, cd into the directory that contains the files you want moved/renamed, and copy and paste these commands into the command line.
for file in [0-9][0-9][0-9][0-9]*; do
dirName="${file%%*([^0-9])}"
mkdir -p "$dirName"
mv "$file" "$dirName/${file##*([0-9])}"
done
This assumes all the files that you want to rename and move are in the same directory. The file globbing also assumes that there are at least four digits at the start of the filename. If there are more than four numbers, it will still be caught, but not if there are less than four. If there are less than four, take off the appropriate number of [0-9]s from the first line.
It does not handle the case where "NAME" (i.e. the name of the new file you want) starts with a number.
See this site for more information about string manipulation in bash.
I need to get all the file extension types in a folder. For instance, if the directory's ls gives the following:
a.t
b.t.pg
c.bin
d.bin
e.old
f.txt
g.txt
I should get this by running the script
.t
.t.pg
.bin
.old
.txt
I have a bash shell.
Thanks a lot!
See the BashFAQ entry on ParsingLS for a description of why many of these answers are evil.
The following approach avoids this pitfall (and, by the way, completely ignores files with no extension):
shopt -s nullglob
for f in *.*; do
printf '%s\n' ".${f#*.}"
done | sort -u
Among the advantages:
Correctness: ls behaves inconsistently and can result in inappropriate results. See the link at the top.
Efficiency: Minimizes the number of subprocess invoked (only one, sort -u, and that could be removed also if we wanted to use Bash 4's associative arrays to store results)
Things that still could be improved:
Correctness: this will correctly discard newlines in filenames before the first . (which some other answers won't) -- but filenames with newlines after the first . will be treated as separate entries by sort. This could be fixed by using nulls as the delimiter, or by the aforementioned bash 4 associative-array storage approach.
try this:
ls -1 | sed 's/^[^.]*\(\..*\)$/\1/' | sort -u
ls lists files in your folder, one file per line
sed magic extracts extensions
sort -u sorts extensions and removes duplicates
sed magic reads as:
s/ / /: substitutes whatever is between first and second / by whatever is between second and third /
^: match beginning of line
[^.]: match any character that is not a dot
*: match it as many times as possible
\( and \): remember whatever is matched between these two parentheses
\.: match a dot
.: match any character
*: match it as many times as possible
$: match end of line
\1: this is what has been matched between parentheses
People are really over-complicating this - particularly the regex:
ls | grep -o "\..*" | uniq
ls - get all the files
grep -o "\..*" - -o only show the match; "\..*" match at the first "." & everything after it
uniq - don't print duplicates but keep the same order
you can also sort if you like, but sorting doesn't match the example
This is what happens when you run it:
> ls -1
a.t
a.t.pg
c.bin
d.bin
e.old
f.txt
g.txt
> ls | grep -o "\..*" | uniq
.t
.t.pg
.bin
.old
.txt
i am trying to set a variable's value to use during the compilation.
I try to do that in a separate makefile target
svnversion:
SVN_REV=$(shell svnversion -cn | sed -e 's/.*://' -e 's/\([0-9]*\).*/\1/' | grep '[0-9]')
$(info svn_rev = $(SVN_REV))
I have read that this is the way to set the value of a variable.
Yet when i run 'make' I see :
SVN_REV=613
svn_rev =
so the variable seems to be empty. Afterwards I expect that this variable will be present while the compilation takes place (in other targets). Is this the case? or should I add an 'export' command in the svnversion target? and how to I address the SVN_REV variable? $(SVN_REV) or $$(SVN_REV).
thank you
You are actually assigning the value SVN_REV in the subshell that is concluded at the end of the line,
what you probably want is:
svnversion:: SVN_REV=$(shell svnversion -cn | sed -e 's/.*://' -e 's/\([0-9]*\).*/\1/' | grep '[0-9]')
svnversion:
$(info svn_rev = $(SVN_REV))
This sets the variable when the target is set.
If this isn't what was intended, say you want to do some processing with the variable, then you need to make each line a continuation of the previous one using the horrible ; \ at the end of line semantics. If you are then referencing shell variables (like the one evaluated in your first line), then you need to use the $$ syntax before the variable name
e.g.
svnversion:
SVN_REV=$(shell svnversion -cn | sed -e 's/.*://' -e 's/\([0-9]*\).*/\1/' | grep '[0-9]'); \
echo svn_rev = $$SVN_REV
but because it's in a shell, you can't use the variable in the $(info command, as that takes place outside of the evaluation of the target.
You can't use variables between different lines in rules or even between different rules. Each line is being executed in it's own shell, so there's no way of passing informations this way. (Petesh stated that right now as I just see)
if you need to store intermediate informations, use files like this:
foo:
uname -m > current_arch
...
bar:
gcc -m $$(cat current_arch) ...
...
You may also set a macro if the command being executed is not too time-consuming and does not depend on when it is called during the build process:ant
ARCH = $$(uname -m)
bar:
gcc -m $(ARCH) ...
But this is not a variable being set but a macro substitution. The actual command passed to the shell when calling make bar would be:
gcc -m $(uname -n)
and then the uname command would be executed.