A program "execinput" reads input lines from stdin
and stores them into a character array "buffer" and then does this:
system(buffer);
Let "command" be any valid set of programs, parameters, and bash syntax. Ideally the next two lines would give the same result (neglecting for the moment the handling of any double quotes within "command"):
command
echo "command" | execinput
That is indeed the case on a linux system running in a bash shell. However, in bash in an "Msys2 MingW 32bit" shell this happens (trailing semicolon after the 3 is intentional):
echo 1; echo 2; echo 3;
1
2
3
echo "echo 1; echo 2; echo 3;" | execinput
1 ; echo 2; echo 3;
echo "echo 1 & echo 2 & echo 3;" | execinput
1
2
3;
The "&" character is what cmd.exe uses to separate sub-commands. That last command on a linux system, either directly in bash or through system() gives:
3
1
2
In the linux bash environment all the commands come out the same, whether run directly or via system() in execinput(). In the MSYS2 environment they don't.
I believe that COMSPEC and PATH are involved somehow but having set the former like so:
export COMSPEC="C:\progs\msys32\usr\bin\bash.exe -c "
instead of the default COMSPEC value of:
C:\Windows\system32\cmd.exe
the results still were not the same.
Can somebody please explain what is going on here, and hopefully, how to make "command" come out the same directly on a bash command line and when invoked with system()?
More info. In a bash command line in MSYS2:
echo 'set' | execinput > short.txt
echo 'bash -c "set"' | execinput > long.txt
then compare the file contents. key differences are:
long.txt has 11 BASH *symbols plus DIRSTACK, EUID, GROUPS,
IFS, MACHTYPE, OPTERR, OPTIND, OSTYPE, PPID, SHELLOPTS,UID.
Short.txt does not have these.
SYSTEMROOT, COMSPEC, CONTITLE, HOMEPATH, and many others are
in single quotes in long.txt, no quotes in short.txt.
The strings are otherwise the same.
PWD is in fully linux/posix syntax in long.txt and has its root
at the top of the MSYS2 file system. PWD is in hybrid syntax
in short.txt (starts with C: then has a / delimited path) and is
the full Windows path.
PATH in long.txt starts with linux/posix syntax entries,
root at the top of the MSYS2 file system, then followed by entries
like /c/Windows/System32. Short.txt has entries which are in full
Windows syntax.
Long.txt has SHLVL=2, short.txt has SHLVL=1.
In your question, you assume that command stands for "Bash syntax".
However, the system() call is calling sh on your Gnu/Linux and cmd.exe on your Windows system. Being interpreted by different shells leads to different results of the two command lines passed into the system() call:
echo 1; echo 2; echo 3;
echo 1 & echo 2 & echo 3;
I don't know if there is any way to make system() call a different command processor on Windows, if COMPSPEC does not seem to affect it.
If you have control over the source code of execinput, I would suggest implementing a more portable execution of command there, e.g. using one of the exec*() system calls together with a defined path to the desired shell.
"Msys2 MingW 32bit" means that you open a MSYS2 Bash shell where the path is set so that gcc resolves to a mingw-w64 build targeting native Win32. The native Win32 executable has nothing to do with MSYS2 Bash; the command processor is cmd.exe. The MSYS2 shell is only a development tool.
If you open "Msys2 MSYS2" then gcc resolves to a mingw-w64 build targeting the MSYS2 system. You can check the target with gcc -v and it will say Target: x86_64-pc-msys or similar. I tested this and it did use /usr/bin/bash as the command processor as expected.
Note that the different targets use different gcc installations; the MSYS2 target is pacman -S msys2/gcc and the native Win32 target is pacman -S mingw32/mingw-w64-i686-gcc. It does not use a single compiler which selects target via a switch, like some gcc builds.
Of course, if you build targeting MSYS2 then the resulting executable must be run under MSYS2.
Related
I am calculating the compiling time for a C project using CMake for compilation statistics.
Below you can see the CMake cmd's which I am using for printing the compilation time:
set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE "${CMAKE_COMMAND} -E time")
set_property(TARGET ${MAIN_TARGET} PROPERTY RULE_LAUNCH_COMPILE "${CMAKE_COMMAND} -E time")
This lines of code printing out in windows cmd prompt (e.g.):
Elapsed time: 0 s. (time), 0.000672 s. (clock)
So the calculation works as expected for me. But I would like to have the calculated time listed in a .txt file instead of printing it out in cmd prompt.
I found a solution here:
How to save CMake output to file?
cmake ... >> output_file.txt 2>&1
cmake ... 2>> output_file.txt
But I didn't understand how to use it for my CMake cmd's which is listed above.
I have tried it as below, but unfortunately it is not working:
set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE "${CMAKE_COMMAND} -E time" >> output_file.txt 2>&1)
set_property(TARGET ${MAIN_TARGET} PROPERTY RULE_LAUNCH_COMPILE "${CMAKE_COMMAND} -E time" >> output_file.txt 2>&1)
What I am doing wrong?
Since RULE_LAUNCH_COMPILE specifies a command prefix, and stream redirection goes at the end of a shell command, using it like this cannot work.
Also, commands as specified in CMake are not shell commands. This means that special characters will not be interpreted, and will just be arguments for the command.
To summerize, what you're trying to do will be expanded into something like:
cmake -E time >> output_file.txt 2>&1 gcc...
This will not be executed by a shell, but by CMake itself, so it is going to fail as time will try to execute a command named ">>".
The solution for you is to use an intermediate script, for example:
#echo off
time %* >> output_file.txt
Let's name this trace.cmd.
The use of %* means that we take all arguments to the script and pass them as arguments to the time command.
Then you can use it like this:
set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE "${CMAKE_COMMAND} -E trace.cmd")
Disclaimer: I haven't tested the solution.
I have a very basic shell script containing
#!/bin/sh
NAME[0]="Hello"
echo ${NAME[0]}
So when I run this script an error occurs stating
./test.sh: 2: ./test.sh: NAME[0]=Hello: not found
./test.sh: 3: ./test.sh: Bad substitution
So basically I looked through a few tutorials and found this to be the basic way to declare arrays. So I am confused as to why this is an error. Any ideas?
You are starting your script as #!/bin/sh, which has a soft link to dash (The current version of sh which conforms with the POSIX 1003.2 and 1003.2a specifications for the shell) and dash doesn't support arrays. In debian 8 onwards dash has become default shell, so if you run ls -la /bin/sh the output will be /bin/sh -> dash
However bash still remains the default login shell, only the default /bin/sh used in shell scripts has been changed. So if you run your code on terminal, it will work just fine. More information on why this switch was made in Ubuntu can be found here.
If you want to use arrays in your script then you must start your script with #!/bin/bash
So your script works perfectly if modified like this
#!/bin/bash
NAME[0]="Hello"
echo ${NAME[0]}
More information on Dash as Sh DashAsBinSh
I used to have a server running CentOS, and I used to execute shell files this way:
sudo sh /folder/script.sh
Now I have an Ubuntu server. When I'm executing the same command line, I now have the following error message:
/folder/script.sh: ID[0]=ID: not found
I had a look on the internet and it says I need to use:
sudo /bin/bash /folder/script.sh
But when I do so I got the same error message.
The first line of my script is:
ID[0]="ID"
/bin/sh is often a POSIX shell, which does not support arrays.
I suggest you install another shell which does support them, like mksh (disclaimer: I’m its developer), ksh93, zsh, or just use GNU bash instead, and call your script with, for example, sudo mksh /folder/script.sh instead. This will give you more consistent behaviour across systems, too (note that to behave consistent on all platforms is actually an mksh design goal).
Hm… this works for me:
$ cat >x
#!/bin/bash
ID[0]="ID"
echo works for me
$ mksh x
works for me
Do you have any weird characters in your script, like embedded Carriage Return (^M)? Check with: cat -v /folder/script.sh
i created a script that converts a text file into utf8 encoding. I can run it in vim. The problem is that i need to run it by cmd in windows and i cant figure out how. Help me
Sorry for my english. Im from south america, i speak spanish.
Alternatives
Unless you really need special Vim capabilities, you're probably better off using non-interactive tools like sed, awk, or Perl / Python / Ruby / your favorite scripting language here. For simple character set conversion, look into the iconv tool in particular.
That said, you can use Vim non-interactively:
Silent Batch Mode
For very simple text processing (i.e. using Vim like an enhanced 'sed' or 'awk', maybe just benefitting from the enhanced regular expressions in a :substitute command), use Ex-mode.
REM Windows
call vim -N -u NONE -n -es -S "commands.ex" "filespec"
Note: silent batch mode (:help -s-ex) messes up the Windows console, so you may have to do a cls to clean up after the Vim run.
# Unix
vim -T dumb --noplugin -n -es -S "commands.ex" "filespec"
Attention: Vim will hang waiting for input if the "commands.ex" file doesn't exist; better check beforehand for its existence! Alternatively, Vim can read the commands from stdin. You can also fill a new buffer with text read from stdin, and read commands from stderr if you use the - argument.
Full Automation
For more advanced processing involving multiple windows, and real automation of Vim (where you might interact with the user or leave Vim running to let the user take over), use:
vim -N -u NONE -n -c "set nomore" -S "commands.vim" "filespec"
Here's a summary of the used arguments:
-T dumb Avoids errors in case the terminal detection goes wrong.
-N -u NONE Do not load vimrc and plugins, alternatively:
--noplugin Do not load plugins.
-n No swapfile.
-es Ex mode + silent batch mode -s-ex
Attention: Must be given in that order!
-S ... Source script.
-c 'set nomore' Suppress the more-prompt when the screen is filled
with messages or output to avoid blocking.
My program gets executed like:
$./sort 1 < test.txt
sort is the program name
1 is the argument (argv[1])
and test.txt is the file I am inputting from
Is it possible to extract the name file from this? if so how?
The problem is I already wrote my whole program as if I could extract the name from the input line, so I need to be able to pass it into arguments.
Any help is appreciated,
Thanks!
You can't. The shell opens (open(2)) that file and sets up the redirect (most likely using dup2).
The only possible way would be for the shell to explicitly export the information in an environment variable that you could read via getenv.
But it doesn't always make sense. For example, what file name would you expect from
$ echo "This is the end" | ./sort 1
Though this can't be done portably, it's possible on Linux by calling readlink on /proc/self/fd/0 (or /proc/some_pid/fd/0).
eg, running:
echo $(readlink /proc/self/fd/0 < /dev/null)
outputs:
/dev/null
No you can't: the shell sends the content of test.txt to the standard input of your program.
Look at this:
sort << _EOF
3
1
2
_EOF
The < > | operators are processed by the shell, they alter standard input,output,error of the programs in the cmd line.
If you happen to run Solaris, you could parse pfiles output to get the file associated, if any, with stdin.
$ /usr/bin/sleep 3600 < /tmp/foo &
[1] 8430
$ pfiles 8430
8430: /usr/bin/sleep 3600
Current rlimit: 65536 file descriptors
0: S_IFREG mode:0600 dev:299,2 ino:36867886 uid:12345 gid:67890 size=123
O_RDONLY|O_LARGEFILE
/tmp/foo
1: S_IFCHR mode:0600 dev:295,0 ino:12569206 uid:12345 gid:67890 rdev:24,2
...
On most Unix platforms, you will also get the same information from lsof -p if this freeware is installed.