I am working with a large C project that has many source files. Here is a line from one of the makefiles:
!$(TOOLSDIRECTORY)unifdef $(UNIFDEF_ARGUMENTS) $** > $(TARGET)\$**
The Unifdef tool referenced by this line is open source and available here:
http://dotat.at/prog/unifdef/
In this case, the last argument to Unifdef is the group of files to process. It is my understanding that this code is using the symbol "$**" to say "every file in this folder", then piping all of the output to the TARGET directory.
My confusion is that I don't understand how Unifdef receives multiple files in one command. Does a makefile package all of the files into one file stream when it sees "$**"? I understand how Unifdef handles the input it receives, but how do multiple files turn into the single argument that Unifdef receives?
Other note: this makefile is being run on Windows in MSVS 2010.
I suspect that this doesn't actually do what you want.
$** is not a single construct; it is the special Makefile variable $* followed immediately by the shell-glob wildcard *. The line is going to be rewritten twice: first, by Make, to something like
!../path/to/tools/unifdef --opt1 --opt2 foo* > ../path/to/target/foo\*
and then, by /bin/sh, to something like
!../path/to/tools/unifdef --opt1 --opt2 foo.c foo1.c foo2.c fooquux.c \
> ../path/to/target/foo*
and only then executed.
You didn't quote any of the context, so I can't be any more specific than that. Here's why I think this can't be right, though:
Unless maybe you set .RECIPEPREFIX, which is a feature I had never heard of before now, the ! at the beginning of the command doesn't make any sense and should be causing the command to fail because there is no executable named literally !../path/to/tools/unifdef.
The backslash on the second occurrence of $** does not escape the $* (you would do that by writing $$*); it is preserved, and escapes the shell-glob star, so the output is being written to a file literally named ../path/to/target/foo*, which is sufficiently weird that I don't think it can be what was intended.
If the target-directory glob weren't being escaped, there would be two more problems:
Glob expansion happens independently in the source and target directories, and so would (at least potentially) match unrelated sets of files.
The output redirection (>) only applies to the very next thing on the command line; all the other things matched by the glob in the target directory would be provided as input to unifdef.
Based on a wild-assed guess about what you're trying to do, I think you probably want something more like this:
# Resist the temptation to use wildcards. It will be less grief in the long run
# to list each file explicitly.
GENERIC_SOURCES := foo.c foo1.c foo2.c fooquux.c barblurf.c barbaz.c
UNIFDEFED_SOURCES := $(patsubst %.c,$(TARGET)/%-u.c,$(GENERIC_SOURCES))
# The indented lines below must be indented using exactly one hard tab character.
$(UNIFDEFED_SOURCES): %-u.c: %.c
$(TOOLSDIRECTORY)unifdef $(UNIFDEF_ARGUMENTS) $< > $#T
mv -f $#T $#
This does not attempt to batch invocations of unifdef; Make is generally much happier if each rule creates only one output file.
$(patsubst ...) and the static pattern rule are features specifically of GNU make. I normally advocate portability, but in the case of Make, the GNU incarnation is so much more powerful than the portable feature set that it's worth carrying around as a dependency.
The Windows CMD prompt expands the $** into multiple individual calls to Unifdef, each one with one of the files in the directory as the argument, and pipes the output to a file of the same name in the target directory. Therefore, each call to Unifdef is only receiving one file name as its input.
Related
For example, there is a C file a.c, there are three functions in this file: funA(), funB() and funC().
I want to get all the function names from this file.
Additionally, I also want to get the start line number and end line number of each function.
Is there any solution?
Can I use clang to implement it?
You can compile the file and use nm http://en.wikipedia.org/wiki/Nm_(Unix) on the generated binary. You can then just parse the output of nm to get the function names.
If you want to get line numbers, you can use the function names to parse the source file for line numbers.
All the this can be accomplished with a short perl script that makes system calls to gcc and nm.
This is assuming you are using a *nix system of course...
One solution that works well for the job is cproto. It will scan source files (in K&R or ANSI-C format) and output the function prototypes. You can process entire directories of source files with a find command similar to:
find "$dirname" -type f -name "*.c" \
-exec /path/to/cproto -s \
-I/path/to/extra/includes '{}' >> "$outputfile" \;
While the cproto project is no longer actively developed, the cproto application continues to work very, very well. It provides function output in a reasonable form that can be fairly easily parsed/formatted as you desire.
Note: this is just one option based on my use. There are many others available.
I have been trying to pipe in the results from ls into the command line for a C program I am writing (in Unix). I want to be able to have an index of the files and so I was planning on using argv. This is how I thought it should work:
./foo &(ls ~/path)
It doesn't work — what's the correct way to pass the output of ls as arguments to the command?
Your syntax is a bit off...
./foo $(ls ~/path)
Do note that this will choke on files with certain characters in them. Use an array instead to fix this.
pushd ~/path
files=(*)
popd
./foo "${files[#]}"
The notation you specified does two things:
./foo &
runs the program foo in the background (with no arguments other than its command name). Then:
(ls ~/path)
runs the ls command in a sub-shell (which, in this context, is the same as running it in the main shell). The problem is you intended (or need) to use $ in place of &.
./foo $(ls ~/path)
This runs the command ls ~/path and captures the output, which is split into words (using the separators listed in the $IFS variable). Each word is then supplied as an argument to the command ./foo, as you required.
We can then debate the wisdom of using the output of ls like that, but unless you have file names containing spaces (tabs, newlines etc,) you will be OK.
You know how Unix tools accept glob patterns, so you can do cat *.txt or rm ~/Pictures/Vacation*.jpg, without having to pipe/expand ls?
That's an ability your shell gives your program for free!
Just use ./foo ~/path/* and argv[1] will contain /home/you/path/fileone, argv[2] will contain /home/you/path/filetwo, and so forth.
These filenames may be relative or absolute, but can always be passed directly to open/fopen/execve or whichever function you want to use.
Using ls as you describe will only give you the last part of the filename with no directory, so you won't know where the files are to do anything with them (though if that's what you want, just use basename(argv[1])).
I'm a student and this is my first exposure to bash scripting, besides messing with a simple Makefile for c.
#!/usr/bin/sh
gcc -g -std=c99 -Wall -c field.c
gcc -g -std=c99 -Wall -c testField.c
gcc -g field.o testField.o -o testField
#testField get 0xa 0 1 > PA1output.txt
#testField get 0xaa 0 3 >> PA1output.txt
is my script.I want to compile field.c and testField.c into the executable testField.
No matter if I leave the last 2 lines commented out or not, they linux terminal hangs and after 10 seconds of nothing happening I press ctrl+c to stop it. Ultimately I want to redirect output to PA1output.txt, then concatenate things on the end of the file, but I want to rewrite the file contents each time.
As far as I understand it, > rewrites the contents of the specified file, and >> concatenates onto the end.
This is not my homework, I want to automate testing of other homework I have. 'testField get 0xaa 0 3 are arguments into my c program.'
I tried Bash script hangs
but that didn't answer my question totally.
My script is called 'as' to make it easy to type.
Why does the terminal hang and how do I get the script to do what I described above?
Thanks.
Your system has another program called ‘as’ which is an assembler. You are likely running this rather than your script, and it hangs because the assembler is waiting for input from your terminal.
If you insist on keeping the name, you should run your script with a full or partial pathname (like ‘./as’) so that the correct program is run.
You will probably find that your script will not run without the ‘#’ at the beginning of your first line. However, another way to run your script is ‘sh ./as’ from the command line, which does not depend on having the #! line.
As Jeremy described, it's most likely a conflict of names.
If you are running your script from the command line (I really hope you are), you don't have to be afraid of giving your scripts (and all file names for that matter) longer, but more specific, names. Most (if not all) command line interfaces on linux have some form of tab-expansion. All you have to do is type enough of the name to make it unique, then press [Tab], and the shell should complete the name for you.
Here's a more thorough explanation for Bash.
I'd like to add an operator ( e.g. ^> ) to handle prepend instead append (>>). Do I need to modify Bash source or is there an easier way (plugin, etc)?
First of all, you'd need to modify bash sources and quite heavily. Because, above all, your ^> would be really hard to implement.
Note that bash redirection operators usually do a very simple writes, and work on a single file (or program in case of pipes) only. Excluding very specific solutions, you usually can't write to a beginning of a file for the very simple reason you'd need to move all remaining contents forward after each write. You could try doing that but it will be hard, very ineffective (since every write will require re-writing the whole file) and very unsafe (since with any error you will end up with random mix of old and new version).
That said, you are indeed probably better off with a function or any other solution which would use a temporary file, like others suggested.
For completeness, my own implementation of that:
prepend() {
local tmp=$(tempfile)
if cat - "${1}" > "${tmp}"; then
mv "${tmp}" "${1}"
else
rm -f "${tmp}"
# some error reporting
fi
}
Note that you unlike #jpa suggested, you should be writing the concatenated data to a temporary file as that operation can fail and if it does, you don't want to lose your original file. Afterwards, you just replace the old file with new one, or delete the temporary file and handle the failure any way you like.
Synopsis the same as with the other solution:
echo test | prepend file.txt
And a bit modified version to retain permissions and play safe with symlinks (if that is necessary) like >> does:
prepend() {
local tmp=$(tempfile)
if cat - "${1}" > "${tmp}"; then
cat "${tmp}" > "${1}"
rm -f "${tmp}"
else
rm -f "${tmp}"
# some error reporting
fi
}
Just note that this version is actually less safe since if during second cat something else will write to disk and fill it up, you'll end up with incomplete file.
To be honest, I wouldn't personally use it but handle symlinks and resetting permissions externally, if necessary.
^ is a poor choice of character, as it is already used in history substitution.
To add a new redirection type to the shell grammar, start in parse.y. Declare it as a new %token so that it may be used, add it to STRING_INT_ALIST other_token_alist[] so that it may appear in output (such as error messages), update the redirection rule in the parser, and update the lexer to emit this token upon encountering the appropriate characters.
command.h contains enum r_instruction of redirection types, which will need to be extended. There's a giant switch statement in make_redirection in make_cmd.c processing redirection instructions, and the actual redirection is performed by functions throughout redir.c. Scattered throughout the rest of source code are various functions for printing, copying, and destroying pipelines, which may also need to be updated.
That's all! Bash isn't really that complex.
This doesn't discuss how to implement a prepending redirection, which will be difficult as the UNIX file API only provides for appending and overwriting. The only way to prepend to a file is to rewrite it entirely, which (as other answers mention) is significantly more complex than any existing shell redirections.
Might be quite difficult to add an operator, but perhaps a function could be enough?
function prepend { tmp=`tempfile`; cp $1 $tmp; cat - $tmp > $1; rm $tmp; }
Example use:
echo foobar | prepend file.txt
prepends the text "foobar" to file.txt.
I think bash's plugin architecture (loading shared objects via the 'enable' built-in command) is limited to providing additional built-in commands. The redirection operators are part of they syntax for running simple commands, so I think you would need to modify the parser to recognize and handle your new ^> operator.
Most Linux filesystems do not support prepending. In fact, I don't know of any one that has a stable userspace interface for it. So, as stated by others already, you can only rely on overwriting, either just the initial parts, or the entire file, depending on your needs.
You can easily (partially) overwrite initial file contents in Bash, without truncating the file:
exec {fd}<>"$filename"
printf 'New initial contents' >$fd
exec {fd}>&-
Above, $fd is the file descriptor automatically allocated by Bash, and $filename is the name of the target file. Bash opens a new read-write file descriptor to the target file on the first line; this does not truncate the file. The second line overwrites the initial part of the file. The position in the file advances, so you can use multiple commands to overwrite consecutive parts in the file. The third line closes the descriptor; since there is only a limited number available to each process, you want to close them after you no longer need them, or a long-running script might run out.
Please note that > does less than you expected:
Remove the > and the following word from the commandline, remembering the redirection.
When the commandline is processed and the command can be launched, calling fork(2) (or clone(2)), to create a new process.
Modify the new process according to the command. That includes things like modified environment variables (SOMEVAR=foo yourcommand), but also changed filedescriptors. At this point, a > yourfile from the cmdline will have the effect that the file is open(2)'ed at the stdout filedescriptor (that is #1) in write-only mode truncating the file to zero bytes. A >> yourfile would have the effect that the file is oppend at stdout in write-only mode and append mode.
(Only now launch the program, like execv(yourprogram, yourargs)
The redirections could, for a simple example, be implemented like
open(yourfile, O_WRONLY|O_TRUNC);
or
open(yourfile, O_WRONLY|O_APPEND);
respectively.
The program then launched will have the correct environment set up, and can happily write to fd1. From here, the shell is not involved. The real work is not done by the shell, but by the operating system. As Unix doesn't have a prepend mode (and it would be impossible to integrate that feature correctly), everything you could try would end up in a very lousy hack.
Try to re-think your requirements, there's always a simpler way around.
My C program (on Linux) needs to delete a file, say, /home/me/myfile, here is how I do it in my program
...
system ("rm -f /home/me/myfile");
...
When running this program, I got a message saying permission denied. BTW, ls -al /home/me/myfile returns -rw-r--r--
However, under the same user account and in the same shell I execute the C program, I can simple delete the file by typing rm -f /home/me/myfile
What did I miss here?
Thanks,
Update: Using remove(/home/me/myfile) or unlink(/home/me/myfile), the file can be deleted in my program.
For a start, it's the permissions on the directory that control whether you can delete a file.
But, having said that, there are numerous things that could be different between the two situations. Your program might be running as a different user (such as with the SETUID bit), the path may be different, leading to a different rm being run, the program may set up a chroot jail so that it can no longer even see the file (though that may manifest as a different error), and so forth. The possibilities are rather large.
However, C provides a call to delete files, called unlink - you should use that in preference and then check errno.
I would suggest checking the output of which rm in both cases, along with the full details of the file and executable, owner and permissions.