I've had a CGI program that I wrote in C and have used for some 20 years, and for the first time in a long time, I'm running in Windows, specifically windows 10, and I can't get a system() call to execute from the CGI program. It seems like the system call is just ignored and doesn't create my output on the server side that I expect.
The CGI program is called from a form and using sprintf makes up the html page response into a giant buffer (szbuffer) and then prints to stdout which in the apache world goes back to the client. Here is the routine that outputs the built up html page to the user:
/** Print the CGI response header, required for all HTML output. **/
/** Note the extra \n, to send the blank line. **/
printf("Content-type: text/html\n\n") ;
printf("%s\n<!--JL 7/12/01 11:00am -->\n",szBuffer);
free ( szBuffer);
Now this has worked fine for years, but in this version I wanted to run some system calls to search a csv file and create an answer set to be read with an fopen(). The way I've done this in linux is to just run commands using the system() command:
getvaluecgivar(szcustomer_chain, "cust_field3", globals);
getvaluecgivar(szcustomer_id, "cust_field4", globals);
//sprintf(szdoscommand,"./awklookup_customer_id.bat %s %s "
sprintf(szdoscommand,"awklookup_customer_id.bat %s %s"
,szcustomer_chain
,szcustomer_id
);
system(szdoscommand);
system("cat lunch.trace >hw.lis");
system("cat lunch.trace >/temp/hwt.lis");
sprintf(szTempstring,"system command:%s: return code=%d\n"
,szdoscommand
,return_from_system_call
);
strcat(szBuffer, szTempstring);
} // then lookup by customer id
for reasons I need explaining and fixing, it appears as if the system() call (all three of them) never execute. The two "cat lunch.trace" system calls were just a test to see if I could get ANY system call to run and those commands also seem to fail. Now I know I've done this on Linux without any issue before:
FILE * tfile;
system( "cat /tmp/relaystatus20*.txt >/tmp/rs20all.txt");
tfile = fopen( "/tmp/rs20all.txt" , "r");
if (tfile) {
strcat(szBuffer, "<pre>");
while(!feof(tfile)) {
and this creates the file rs20all.txt in the /tmp directory.
Is there something special I have to do on Windows? I did capture the return code from the awklookup_customer_id.bat and it was 127.
Here is awklookup_customer_id.bat, it runs fine from either the dos prompt cmd.exe or a cygwin64 terminal:
call \jon\bat\local >t.lis
rem echo %1%, %2%
set arg1=%1%
set arg2=%2%
gawk -F',' '{ if($3==%arg1%) print $0 }' t5_clean_customer.csv >awk01.lis
gawk -F',' '{ if($4==%arg2%) print $0 }' awk01.lis >awk02.lis
\jon\bat\grep.exe -nf awk02.lis t5_clean_customer.csv >awk03.lis
cat awk03.lis|wc -l >awk03_c.lis
set LINE_COUNT=0
set /p LINE_COUNT=<awk03_c.lis
cut -d: -f1 awk03.lis >awk03_ln.lis
set LINE_NUMBER=0
set /p LINE_NUMBER=<awk03_ln.lis
SET /A Result = %LINE_COUNT% * 10000000 + %LINE_NUMBER%
echo number of lines that match :%LINE_COUNT% line number: %LINE_NUMBER%
rem pause
exit %Result%
I'm running Apache 2.4, here are the environment variables that the cgi program gets:
Print_environ::
[HTTP_HOST=localhost]
[HTTP_CONNECTION=keep-alive]
[CONTENT_LENGTH=380]
[HTTP_CACHE_CONTROL=max-age=0]
[HTTP_ORIGIN=http://localhost]
[HTTP_UPGRADE_INSECURE_REQUESTS=1]
[CONTENT_TYPE=application/x-www-form-urlencoded]
[HTTP_USER_AGENT=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36]
[HTTP_ACCEPT=text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8]
[HTTP_REFERER=http://localhost/cgi-bin/bf_customer.cgi]
[HTTP_ACCEPT_ENCODING=gzip, deflate, br]
[HTTP_ACCEPT_LANGUAGE=en-US,en;q=0.9]
[PATH=/cygdrive/c/Program Files (x86)/Common Files/Oracle/Java/javapath:/cygdrive/c/Program Files (x86)/Intel/iCLS Client:/cygdrive/c/Program Files/Intel/iCLS Client:/cygdrive/c/Windows/system32:/cygdrive/c/Windows:/cygdrive/c/Windows/System32/Wbem:/cygdrive/c/Windows/System32/WindowsPowerShell/v1.0:/cygdrive/c/Program Files (x86)/Intel/Intel(R) Management Engine Components/DAL:/cygdrive/c/Program Files/Intel/Intel(R) Management Engine Components/DAL:/cygdrive/c/Program Files (x86)/Intel/Intel(R) Management Engine Components/IPT:/cygdrive/c/Program Files/Intel/Intel(R) Management Engine Components/IPT:/cygdrive/c/Program Files (x86)/NVIDIA Corporation/PhysX/Common:/cygdrive/c/Windows/system32/config/systemprofile/.dnx/bin:/cygdrive/c/Program Files/Microsoft DNX/Dnvm:/cygdrive/c/Program Files (x86)/Windows Kits/8.1/Windows Performance Toolkit:/cygdrive/c/Program Files/Intel/WiFi/bin:/cygdrive/c/Program Files/Common Files/Intel/WirelessCommon:/cygdrive/c/Program Files (x86)/2Printer:/cygdrive/c/Program Files/MATLAB/MATLAB Runtime/v93/runtime/win64:/cygdrive/c/Program Files (x86)/Pico Technology/PicoScope6:/cygdrive/c/Program Files (x86)/IVI Foundation/VISA/WinNT/Bin:/cygdrive/c/Program Files/IVI Foundation/VISA/Win64/Bin:/cygdrive/c/Users/jleslie/AppData/Local/Microsoft/WindowsApps]
[SYSTEMROOT=C:\Windows]
[COMSPEC=C:\Windows\system32\cmd.exe]
[PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC]
[WINDIR=C:\Windows]
[SERVER_SIGNATURE=]
[SERVER_SOFTWARE=Apache/2.4.23 (Win64)]
[SERVER_NAME=localhost]
[SERVER_ADDR=::1]
[SERVER_PORT=80]
[REMOTE_ADDR=::1]
[DOCUMENT_ROOT=C:/jon/programs/Apache24/htdocs]
[REQUEST_SCHEME=http]
[CONTEXT_PREFIX=/cgi-bin/]
[CONTEXT_DOCUMENT_ROOT=c:/jon/programs/Apache24/cgi-bin/]
[SERVER_ADMIN=admin#example.com]
[SCRIPT_FILENAME=C:/jon/programs/Apache24/cgi-bin/bf_customer.cgi]
[REMOTE_PORT=52945]
[GATEWAY_INTERFACE=CGI/1.1]
[SERVER_PROTOCOL=HTTP/1.1]
[REQUEST_METHOD=POST]
[QUERY_STRING=]
[REQUEST_URI=/cgi-bin/bf_customer.cgi]
[SCRIPT_NAME=/cgi-bin/bf_customer.cgi]
[TERM=cygwin]
[HOME=/home/jleslie]
cgi vars:
[colora] = [maroon]
[cust_field1] = []
[cust_field2] = []
[cust_field3] = [0]
[cust_field4] = [16]
I figured it out. the answer has to do with httpd.conf. I needed to tell apache that its ok to run .bat file:
# (You will also need to add "ExecCGI" to the "Options" directive.)
#
#AddHandler cgi-script .cgi
AddHandler cgi-script .bat # add .exe if you wish
Options +ExecCGI +FollowSymlinks
sorry about the confusion with the return_from_system_call, I was adding and removing it so often I posted a version of the code where it wasn't set. This is the final code section that works:
getvaluecgivar(szcustomer_chain, "cust_field3", globals);
getvaluecgivar(szcustomer_id, "cust_field4", globals);
sprintf(szdoscommand,"awklookup_customer_id.bat %s %s"
,szcustomer_chain
,szcustomer_id
);
return_from_system_call =system(szdoscommand);
sprintf(szTempstring,"system command:%s: return code=%d\n"
,szdoscommand
,return_from_system_call
);
strcat(szBuffer, szTempstring);
Related
The summary of this question is I have a command I'm running in batch, Some.exe args but it is not outputting to the console. It does however output to a text file if used like Some.exe args > test.txt. I've tried stuff like #Some.exe args and Some.exe args > CON to get it to output to the console, but neither seems to work.
Are there any other approaches which might work?
This follows on from a previous question I asked DOORS make console the interactive window.
I'm calling a program called DOORS through a batch script. It runs a simple script, hello.dxl that looks like
cout << "Hello world"
The batch script, Run.bat looks like
"C:\Program Files\IBM\Rational\DOORS\9.6\bin\doors.exe" -u test -pass testPass -b hello.dxl
When this is run, no output appears on the screen and there are no popup windows or anything (if hello.dxl said print("Hello World") an interactive window would pop-up, but not with cout)
If I add > test.txt to the end of the command
"C:\Program Files\IBM\Rational\DOORS\9.6\bin\doors.exe" -u test -pass testPass -b hello.dxl > test.txt
It outputs the Hello World to test.txt successfully. Something I noticed is when using print("Hello World") there was no output sent to the test.txt file and an interactive window popped up so it looks like cout is the way to go.
So I though the output might just not be being output anywhere, so I tried adding > CON instead to try to force it to go to the console.
"C:\Program Files\IBM\Rational\DOORS\9.6\bin\doors.exe" -u test -pass testPass -b hello.dxl > CON
But that still resulted in a blank output.
I also tried adding an #, before the command, as suggested in this Batch - redirect program output to current console, like
#"C:\Program Files\IBM\Rational\DOORS\9.6\bin\doors.exe" -u test -pass testPass -b hello.dxl
or
#"C:\Program Files\IBM\Rational\DOORS\9.6\bin\doors.exe" -u test -pass testPass -b hello.dxl > CON
But no luck there either
I would have tried to reproduce this issue without DOORS but I don't know what is causing it in the first place.
Edit: I'm not really looking to use > test.txt & type test.txt as that is the current workaround I am using. But ideally I don't want it outputting to a text file
Turns out this approach has limited functionality, as soon as I tried opening a module, it made the interactive window pop up and no output was visible in the interactive window or the the console window
This is not an issue with the > test.txt & type test.txt workaround
Based on #PA.'s suggestion that the program is somehow preventing output to stdout but not if it was redirected, so I wrote a small C# console app which looks like
using System;
using System.Diagnostics;
using Process process = new();
process.EnableRaisingEvents = true;
process.ErrorDataReceived += ErrOut;
process.OutputDataReceived += StdOut;
process.StartInfo.FileName = args[0].Trim();
process.StartInfo.Arguments = args[1].Trim();
process.StartInfo.UseShellExecute = false;
process.StartInfo.RedirectStandardOutput = true;
process.StartInfo.RedirectStandardError = true;
process.Start();
process.BeginErrorReadLine();
process.BeginOutputReadLine();
process.WaitForExit();
static void ErrOut(object sender, DataReceivedEventArgs dataReceivedEventArgs)
{
if (dataReceivedEventArgs.Data != null)
{
Console.WriteLine(dataReceivedEventArgs.Data);
}
}
static void StdOut(object sender, DataReceivedEventArgs dataReceivedEventArgs)
{
if (dataReceivedEventArgs.Data != null)
{
Console.WriteLine(dataReceivedEventArgs.Data);
}
}
I then exported this to a single exe using the command
dotnet publish /p:DebugType=None /p:DebugSymbols=false /p:PublishReadyToRun=true /p:PublishSingleFile=true /p:PublishReadyToRunShowWarnings=true /p:PublishTrimmed=false /p:IncludeNativeLibrariesForSelfExtract=true /p:IncludeAllContentForSelfExtract=true "RedirectOutput.csproj" -o "." -c release
which I added then just added the folder with that exe in to my path environment variable.
Now I can just do
RedirectOutput "C:\Program Files\IBM\Rational\DOORS\9.6\bin\doors.exe" "-u test -pass testPass -b hello.dxl"
and I get the desired output
This is more a workaround.
The idea is to a use a pipe program in order to redirect the output of the Windows program to the console:
SomeWinProg SomeArgs 2>>&1 | SomePipeProg PipeProgArgs
As a pipe program you may use a program that passes throug everything, like:
SomeWinProg SomeArgs 2>>&1 | findstr /r "/c:.*"
If this works or not depends on the Windows program.
Concerning the timing:
There may be some trouble when you have a long time running Windows program which produces sporadic output or when the output is done always to the same line (no line feeds, only carriage returns) and you want to have the output in real time.
In these cacses Windows programs like findstr or find are a little bit weak.
On github there is an implementation of mtee thats fits better:
SomeWinProg SomeArgs 2>>&1 | mtee nul
I'm using the 'qwinsta' cmd command to get the session ID of a remote computer and output it to a textfile, so I create a new batch file and write the command then I try running the batch file through python but it only returns the first line of the output. When I run the batch file by simply double-clicking it it works properly.
Using python 2.7:
def run_qwinsta(self, computerName):
qwinsta_check = open("q.bat", "w")
qwinsta_check.write('PsExec -u <username> -p <password> \\\\' + computerName + ' qwinsta' + ' > "q.txt" ')
qwinsta_check.close()
os.system("q.bat")
Expected results:
SESSIONNAME USERNAME ID STATE TYPE DEVICE
>services 0 Disc
console <username> 1 Active
rdp-tcp 65536 Listen
Actual results:
SESSIONNAME USERNAME ID STATE TYPE DEVICE
I would recommend you to avoid writing the batchfile, If you can. You can execute your batch command from os.system(). Also you can try using subprocess (documentation here) and then redirecting the stdout and stderr to file.
EDIT:
PsExec is a good choice, but If you want another way, you can also use ssh.
You can run PsExec from os.system() and then write the response to text file on the remote machine. The default PsExec working directory is System32 there you can find your text file.
Tested code:
import os
os.system('Psexec \\\\SERVER cmd.exe /c "tasklist > process_list.txt"')
I used tasklist, because I don't have qwinsta on my remote machine.
If you want to store the PsExec response on your machine you can use subprocess and then redirect the stdout to text file.
Tested code:
import subprocess
process_list = open("process_list.txt", "w")
subprocess.call(['Psexec', '\\\\SERVER', 'tasklist'], stdout=process_list)
process_list.close()
Actually I used Python 3.x, because I don't have Python 2.x, but it should work on both.
If this still didn't solve your problem, please provide more details to your question!
A program "execinput" reads input lines from stdin
and stores them into a character array "buffer" and then does this:
system(buffer);
Let "command" be any valid set of programs, parameters, and bash syntax. Ideally the next two lines would give the same result (neglecting for the moment the handling of any double quotes within "command"):
command
echo "command" | execinput
That is indeed the case on a linux system running in a bash shell. However, in bash in an "Msys2 MingW 32bit" shell this happens (trailing semicolon after the 3 is intentional):
echo 1; echo 2; echo 3;
1
2
3
echo "echo 1; echo 2; echo 3;" | execinput
1 ; echo 2; echo 3;
echo "echo 1 & echo 2 & echo 3;" | execinput
1
2
3;
The "&" character is what cmd.exe uses to separate sub-commands. That last command on a linux system, either directly in bash or through system() gives:
3
1
2
In the linux bash environment all the commands come out the same, whether run directly or via system() in execinput(). In the MSYS2 environment they don't.
I believe that COMSPEC and PATH are involved somehow but having set the former like so:
export COMSPEC="C:\progs\msys32\usr\bin\bash.exe -c "
instead of the default COMSPEC value of:
C:\Windows\system32\cmd.exe
the results still were not the same.
Can somebody please explain what is going on here, and hopefully, how to make "command" come out the same directly on a bash command line and when invoked with system()?
More info. In a bash command line in MSYS2:
echo 'set' | execinput > short.txt
echo 'bash -c "set"' | execinput > long.txt
then compare the file contents. key differences are:
long.txt has 11 BASH *symbols plus DIRSTACK, EUID, GROUPS,
IFS, MACHTYPE, OPTERR, OPTIND, OSTYPE, PPID, SHELLOPTS,UID.
Short.txt does not have these.
SYSTEMROOT, COMSPEC, CONTITLE, HOMEPATH, and many others are
in single quotes in long.txt, no quotes in short.txt.
The strings are otherwise the same.
PWD is in fully linux/posix syntax in long.txt and has its root
at the top of the MSYS2 file system. PWD is in hybrid syntax
in short.txt (starts with C: then has a / delimited path) and is
the full Windows path.
PATH in long.txt starts with linux/posix syntax entries,
root at the top of the MSYS2 file system, then followed by entries
like /c/Windows/System32. Short.txt has entries which are in full
Windows syntax.
Long.txt has SHLVL=2, short.txt has SHLVL=1.
In your question, you assume that command stands for "Bash syntax".
However, the system() call is calling sh on your Gnu/Linux and cmd.exe on your Windows system. Being interpreted by different shells leads to different results of the two command lines passed into the system() call:
echo 1; echo 2; echo 3;
echo 1 & echo 2 & echo 3;
I don't know if there is any way to make system() call a different command processor on Windows, if COMPSPEC does not seem to affect it.
If you have control over the source code of execinput, I would suggest implementing a more portable execution of command there, e.g. using one of the exec*() system calls together with a defined path to the desired shell.
"Msys2 MingW 32bit" means that you open a MSYS2 Bash shell where the path is set so that gcc resolves to a mingw-w64 build targeting native Win32. The native Win32 executable has nothing to do with MSYS2 Bash; the command processor is cmd.exe. The MSYS2 shell is only a development tool.
If you open "Msys2 MSYS2" then gcc resolves to a mingw-w64 build targeting the MSYS2 system. You can check the target with gcc -v and it will say Target: x86_64-pc-msys or similar. I tested this and it did use /usr/bin/bash as the command processor as expected.
Note that the different targets use different gcc installations; the MSYS2 target is pacman -S msys2/gcc and the native Win32 target is pacman -S mingw32/mingw-w64-i686-gcc. It does not use a single compiler which selects target via a switch, like some gcc builds.
Of course, if you build targeting MSYS2 then the resulting executable must be run under MSYS2.
I am using the Sift implementation by Vlfeat.org. It has the follwing function which is supposed to save the features to a file.
def process_image(imagename,resultname,params="--edge-thresh 10 --peak-thresh 5"):
""" process an image and save the results in a file"""
if imagename[-3:] != 'pgm':
#create a pgm file
im = Image.open(imagename).convert('L')
im.save('tmp.pgm')
imagename = 'tmp.pgm'
cmmd = str("sift "+imagename+" --output="+resultname+
" "+params)
os.system(cmmd)
print 'processed', imagename, 'to', resultname
Here how the line "os.system(cmmd)" is supposed to write the results in a file?
I am on an ubuntu machine and if I execute "sift" as command in terminal, I am getting the result as "not found". On linux, which process is this command trying to invoke? I need to save these Sift features into a file so that later I can use it to create Bag of Words descriptor for clustering.
A similar sift implementation at https://github.com/jesolem/PCV/blob/master/PCV/localdescriptors/sift.py also uses the same line to save the result into file.
On linux, which process is this command trying to invoke?
This refers to the command-line tool provided by vlfeat (see vlfeat/src/sift.c):
$ ./sift
Usage: ./sift [options] files ...
Options include:
--verbose -v Be verbose
--help -h Print this help message
--output -o Specify output file
...
You can either use the binary distribution from vlfeat download page (see bin/glnxa64/sift for Linux 64-bit), or directly clone the repo and compile it from sources.
Make sure to adjust the path with cmmd = str("/path/to/sift " ...) or install (= copy it) under a common path like /usr/local/bin.
i created a script that converts a text file into utf8 encoding. I can run it in vim. The problem is that i need to run it by cmd in windows and i cant figure out how. Help me
Sorry for my english. Im from south america, i speak spanish.
Alternatives
Unless you really need special Vim capabilities, you're probably better off using non-interactive tools like sed, awk, or Perl / Python / Ruby / your favorite scripting language here. For simple character set conversion, look into the iconv tool in particular.
That said, you can use Vim non-interactively:
Silent Batch Mode
For very simple text processing (i.e. using Vim like an enhanced 'sed' or 'awk', maybe just benefitting from the enhanced regular expressions in a :substitute command), use Ex-mode.
REM Windows
call vim -N -u NONE -n -es -S "commands.ex" "filespec"
Note: silent batch mode (:help -s-ex) messes up the Windows console, so you may have to do a cls to clean up after the Vim run.
# Unix
vim -T dumb --noplugin -n -es -S "commands.ex" "filespec"
Attention: Vim will hang waiting for input if the "commands.ex" file doesn't exist; better check beforehand for its existence! Alternatively, Vim can read the commands from stdin. You can also fill a new buffer with text read from stdin, and read commands from stderr if you use the - argument.
Full Automation
For more advanced processing involving multiple windows, and real automation of Vim (where you might interact with the user or leave Vim running to let the user take over), use:
vim -N -u NONE -n -c "set nomore" -S "commands.vim" "filespec"
Here's a summary of the used arguments:
-T dumb Avoids errors in case the terminal detection goes wrong.
-N -u NONE Do not load vimrc and plugins, alternatively:
--noplugin Do not load plugins.
-n No swapfile.
-es Ex mode + silent batch mode -s-ex
Attention: Must be given in that order!
-S ... Source script.
-c 'set nomore' Suppress the more-prompt when the screen is filled
with messages or output to avoid blocking.