I am running an embedded C project based on a tricore-gcc compiler.
I made a compile_commands.json file that works well.
However, there are some arguments that clang-tidy cannot recognize.
For example I get the following
error: unknown argument: '-fstrict-volatile-bitfields' [clang-diagnostic-error]
when I run clang-tidy.
This is not a surprise of course, since -fstrict-volatile-bitfields is an option for gcc but not for clang-tidy.
However, I would like to suppress output messages due to unknown compiler options.
I tried to append --extra-arg=-Wno-unknown-warning-option as suggested here but the problem persists.
I had the same problem when using clang-tidy for a project compiled with GCC. The two compilers have different set of recognized flags. My compile_commands.json, which was generated by intercept-build, contained a number of options used by GCC that clang-tidy did not understand.
I solved this by a somewhat hackish solution of redacting the compile database before initiating the static analysis pass.
My script contains a number of Sed lines similar to this:
sed -i -E 's#-fconserve-stack##g' compile_commands.json
sed -i -E 's#-maccumulate-outgoing-args##g' compile_commands.json
sed -i -E 's#-mindirect-branch-register##g' compile_commands.json
sed -i -E 's#-mindirect-branch=thunk-extern##g' compile_commands.json
sed -i -E 's#-mpreferred-stack-boundary=3##g' compile_commands.json
sed -i -E 's#-mfunction-return=thunk-extern##g' compile_commands.json
This way, the unsupported flags are made invisible to clang-tidy, which allows to work without errors.
Using the command line I get the desired output
$ ./program $(< file.txt)
./program 1 2 3 4 5
But with a makefile
all: program file.txt
./program $(< file.txt)
#rm -f program
program: program.c
gcc program.c -o program
I get the output
gcc program.c -o program
./program
Because in a makefile, the $(...) syntax is used for variable interpolation. So your makefile tries to expand the value of a makefile variable / environment variable named < file.txt. And if unset, it expands to an empty string.
Proof:
all:
echo $(< file.txt)
and file.txt containing
now it works
Then execute
% env '< file.txt=Hello world' make
echo Hello world
Hello world
i.e. by setting an environment variable named < file.txt to value Hello world , the greeting was printed. The fix is to escape the $ character by doubling it:
all:
echo $$(< file.txt)
and then
% make
echo $(< file.txt)
now it works
Q.E.D.
Finally, while the $() interpolation syntax in POSIX shells, the $(< file.txt) is not but you can replace it with $(cat file.txt) so it works with minimally POSIX-conforming shells. Of course in a makefile you again need to double the dollar, therefore getting the maximally compatible
$$(cat file.txt)
Alternatively you can use the similar makefile facility which is $(shell ), i.e.
$(shell cat file.txt)
would work too... (now with one $). Finally you can read files with $(file ) GNU makefile function too, i.e.
all:
echo $(file <file.txt)
would work alike but wouldn't call shell at all.
The substitution you are trying to use is a Bash feature, but make out of the box runs the regular Bourne shell sh, where this syntax is not available (even when sh is a symlink to Bash, as is still common on some Linux distributions).
Requiring the contents of the file to be specified on the command line looks like a design flaw, anyway; it's probably much better if your C program simply reads and processes standard input (or perhaps accepts a list of file names, and falls back to stdin if none are specified, like many Unix file processing utilities).
If this is just for a test case to run the program with parameters from a file, check out xargs.
xargs ./program <file.txt
If you insist on using Bash-only syntax, add
SHELL=/bin/bash
(or whatever full path is correct on your system); but understand that this limits the portability of your Makefile.
Still, you'll need to double any literal dollar sign which should be passed through and exposed to the shell.
I'm looking for a simple bash script which, when given the name of a system header, will return its full path from which it would be read in a #include <header> statement. I already have an analogous thing for looking up the library archive used by linker.
ld -verbose -lz -L/some/other/dir | grep succeeded | sed -e 's/^\s*attempt to open //' -e 's/ succeeded\s*$//'
For example, this will return the path of libz archive (/lib/x86_64-linux-gnu/libz.so on my system).
For the requested script I know that I could take a list of include directories used by gcc and search them for the file myself, but I'm looking for a more accurate simulation of what's happening inside the preprocessor (unless it's that simple).
Pipe the input to preprocessor and then process the output. Gcc preprocessor output inserts # lines with information and flags that you can parse.
$ f=stdlib.h
$ echo "#include <$f>" | gcc -xc -E - | sed '\~# [0-9]* "\([^"]*/'"$f"'\)" 1 .*~!d; s//\1/'
/usr/include/stdlib.h
It can output multiple files, because gcc has #include_next and can improperly detect in some complicated cases where multiple filenames are included with the same name, like in f=limits.h. So you could also filter exactly second line, knowing that the first line is always going to be stdc-predef.h:
$ f=limits.h; echo "#include <$f>" | gcc -xc -E - | sed '/# [0-9]* "\([^"]*\)" 1 .*/!d;s//\1/' | sed '2!d'
/usr/lib/gcc/x86_64-pc-linux-gnu/10.1.0/include-fixed/limits.h
But really search the include paths yourself, it's not that hard:
$ f=limits.h; echo | gcc -E -Wp,-v - 2>&1 | sed '\~^ /~!d; s/ //' | while IFS= read -r path; do if [[ -e "$path/$f" ]]; then echo "$path/$f"; break; fi; done
/usr/lib/gcc/x86_64-pc-linux-gnu/10.1.0/include-fixed/limits.h
You can use the preprocessor to do the work:
user#host:~$ echo "#include <stdio.h>" > testx.c && gcc -M testx.c | grep 'stdio.h'
testx.o: testx.c /usr/include/stdc-predef.h /usr/include/stdio.h \
You can add a bit bash-fu to cut the part you are interested in
I know "gcc -E main.c" option gives all the values pre-processed output. Is there a way to just expand the user defined macros?
For example
#define MACRO(z) z+z
c = a + (MACRO(z))
When I use this hypothetical gcc option, I should see:
c = a + (z+z)
I do not want any other system-defined MACRO's to be expanded. Is there an option in GCC?
Effectively what you want to do is to have the preprocessor ignore errors opening include files. I think there is a compiler with such a feature, but GCC isn't it. (Or maybe I'm confusing it with the GCC feature to ignore missing include files when generating dependency files.)
However what you can do instead is to create a shadow directory of empty files matching all the names and sub-directories for the system headers. You can then use the -nostdinc (to ignore the system include directories) and -I option (to point at your new shadow directory).
The shadow directory can be created with something like:
mkdir $HOME/tmp/emptystdinc
cd $HOME/tmp/emptystdinc
find /usr/include -type d -print | sed 's|/usr/include|.|' | xargs mkdir -p
find /usr/include -type f -print | sed 's|/usr/include|.|' | xargs touch
Now see what your post-processed code looks like:
cc -E -nostdinc -I$HOME/tmp/emptystdinc test.c
Good luck! This works fine for me!
Quoting from cpp man page:
-undef
Do not predefine any system-specific or GCC-specific macros. The standard predefined macros remain defined.
This means that macros such as __FILE__ or __LINE__ will be expanded, but this may be sufficient for your needs. You can also undefine these standard predefined macros using -U. The full list can be found here.
How do I find out the files in the current directory which do not contain the word foo (using grep)?
If your grep has the -L (or --files-without-match) option:
$ grep -L "foo" *
You can do it with grep alone (without find).
grep -riL "foo" .
This is the explanation of the parameters used on grep
-L, --files-without-match
each file processed.
-R, -r, --recursive
Recursively search subdirectories listed.
-i, --ignore-case
Perform case insensitive matching.
If you use l (lowercased) you will get the opposite (files with matches)
-l, --files-with-matches
Only the names of files containing selected lines are written
Take a look at ack. It does the .svn exclusion for you automatically, gives you Perl regular expressions, and is a simple download of a single Perl program.
The equivalent of what you're looking for should be, in ack:
ack -L foo
The following command gives me all the files that do not contain the pattern foo:
find . -not -ipath '.*svn*' -exec grep -H -E -o -c "foo" {} \; | grep 0
The following command excludes the need for the find to filter out the svn folders by using a second grep.
grep -rL "foo" ./* | grep -v "\.svn"
If you are using git, this searches all of the tracked files:
git grep -L "foo"
and you can search in a subset of tracked files if you have ** subdirectory globbing turned on (shopt -s globstar in .bashrc, see this):
git grep -L "foo" -- **/*.cpp
You will actually need:
find . -not -ipath '.*svn*' -exec grep -H -E -o -c "foo" {} \; | grep :0\$
I had good luck with
grep -H -E -o -c "foo" */*/*.ext | grep ext:0
My attempts with grep -v just gave me all the lines without "foo".
Problem
I need to refactor a large project which uses .phtml files to write out HTML using inline PHP code. I want to use Mustache templates instead. I want to find any .phtml giles which do not contain the string new Mustache as these still need to be rewritten.
Solution
find . -iname '*.phtml' -exec grep -H -E -o -c 'new Mustache' {} \; | grep :0$ | sed 's/..$//'
Explanation
Before the pipes:
Find
find . Find files recursively, starting in this directory
-iname '*.phtml' Filename must contain .phtml (the i makes it case-insensitive)
-exec 'grep -H -E -o -c 'new Mustache' {}' Run the grep command on each of the matched paths
Grep
-H Always print filename headers with output lines.
-E Interpret pattern as an extended regular expression (i.e. force grep
to behave as egrep).
-o Prints only the matching part of the lines.
-c Only a count of selected lines is written to standard output.
This will give me a list of all file paths ending in .phtml, with a count of the number of times the string new Mustache occurs in each of them.
$> find . -iname '*.phtml$' -exec 'grep -H -E -o -c 'new Mustache' {}'\;
./app/MyApp/Customer/View/Account/quickcodemanagestore.phtml:0
./app/MyApp/Customer/View/Account/studio.phtml:0
./app/MyApp/Customer/View/Account/orders.phtml:1
./app/MyApp/Customer/View/Account/banking.phtml:1
./app/MyApp/Customer/View/Account/applycomplete.phtml:1
./app/MyApp/Customer/View/Account/catalogue.phtml:1
./app/MyApp/Customer/View/Account/classadd.phtml:0
./app/MyApp/Customer/View/Account/orders-trade.phtml:0
The first pipe grep :0$ filters this list to only include lines ending in :0:
$> find . -iname '*.phtml' -exec grep -H -E -o -c 'new Mustache' {} \; | grep :0$
./app/MyApp/Customer/View/Account/quickcodemanagestore.phtml:0
./app/MyApp/Customer/View/Account/studio.phtml:0
./app/MyApp/Customer/View/Account/classadd.phtml:0
./app/MyApp/Customer/View/Account/orders-trade.phtml:0
The second pipe sed 's/..$//' strips off the final two characters of each line, leaving just the file paths.
$> find . -iname '*.phtml' -exec grep -H -E -o -c 'new Mustache' {} \; | grep :0$ | sed 's/..$//'
./app/MyApp/Customer/View/Account/quickcodemanagestore.phtml
./app/MyApp/Customer/View/Account/studio.phtml
./app/MyApp/Customer/View/Account/classadd.phtml
./app/MyApp/Customer/View/Account/orders-trade.phtml
When you use find, you have two basic options: filter results out after find has completed searching or use some built in option that will prevent find from considering those files and dirs matching some given pattern.
If you use the former approach on a high number of files and dirs. You will be using a lot of CPU and RAM just to pass the result on to a second process which will in turn filter out results by using a lot of resources as well.
If you use the -not keyword which is a find argument, you will be preventing any path matching the string on the -name or -regex argument behind from being considered, which will be much more efficient.
find . -not -regex ".*/foo/.*" -regex ".*"
Then, any path that is not filtered out by -not will be captured by the subsequent -regex arguments.
For completeness the ripgrep version:
rg --files-without-match "pattern"
You can combine with file type and search path, e.g.
rg --files-without-match -t ruby "frozen_string_literal: true" app/
another alternative when grep doesn't have the -L option (IBM AIX for example), with nothing but grep and the shell :
for file in * ; do grep -q 'my_pattern' $file || echo $file ; done
My grep does not have any -L option. I do find workaround to achieve this.
The ideas are :
to dump all the file name containing the deserved string to a txt1.txt.
dump all the file name in the directory to a txt2.txt.
make the difference between the 2 dump file with diff command.
grep 'foo' *.log | cut -c1-14 | uniq > txt1.txt
grep * *.log | cut -c1-14 | uniq > txt2.txt
diff txt1.txt txt2.txt | grep ">"
find *20161109* -mtime -2|grep -vwE "(TRIGGER)"
You can specify the filter under "find" and the exclusion string under "grep -vwE". Use mtime under find if you need to filter on modified time too.
Open bug report
As commented by #tukan, there is an open bug report for Ag regarding the -L/--files-without-matches flag:
ggreer/the_silver_searcher: #238 - --files-without-matches does not work properly
As there is little progress to the bug report, the -L option mentioned below should not be relied on, not as long as the bug has not been resolved. Use different approaches presented in this thread instead. Citing a comment for the bug report [emphasis mine]:
Any updates on this? -L completely ignores matches on the first line of the file. Seems like if this isn't going to be fixed soon, the flag should be removed entirely, as it effectively does not work as advertised at all.
The Silver Searcher - Ag (intended function - see bug report)
As a powerful alternative to grep, you could use the The Silver Searcher - Ag:
A code searching tool similar to ack, with a focus on speed.
Looking at man ag, we find the -L or --files-without-matches option:
...
OPTIONS
...
-L --files-without-matches
Only print the names of files that don´t contain matches.
I.e., to recursively search for files that do not match foo, from current directory:
ag -L foo
To only search current directory for files that do not match foo, simply specify --depth=0 for the recursion:
ag -L foo --depth 0
This may help others. I have mix of files Go and with test files. But I only need .go files. So I used
ls *.go | grep -v "_test.go"
-v, --invert-match select non-matching lines see https://stackoverflow.com/a/3548465
Also one can use this with vscode to open all the files from terminal
code $(ls *.go | grep -v "_test.go")
grep -irnw "filepath" -ve "pattern"
or
grep -ve "pattern" < file
above command will give us the result as -v finds the inverse of the pattern being searched
The following command could help you to filter the lines which include the substring "foo".
cat file | grep -v "foo"