Write a File from Jenkins Groovy Script-Console - file

I'm trying to find a way to write some content to a file using Jenkins Groovy Script-Console.
The use-case: Our CI manages some state-machine using a volume shared between all the nodes (which is in turn mapped to EFS). However - following the discovery of a bug in our CI groovy shared libs I found that some state files gone corrupt, and needed to write to them the corrected values, together with fixing the bug.
I could do that using ssh connection, however, as we're in process of abstracting out the workers we're trying to back off from that and manage ourselves only from the script-console and/or ci jobs.
I tried all these forms, all of which failed:
"echo 'the text' > /mnt/efs-ci-state/path/to/the-state-file.txt".execute().text
"""
cat <<<EOF > /mnt/efs-ci-state/path/to/the-state-file.txt
the text
EOF
""".execute().text
"bash -c 'echo the text > /mnt/efs-ci-state/path/to/the-state-file.txt'".execute().text
"echo 'the text' | tee /mnt/efs-ci-state/path/to/the-state-file.txt"
Can anybody show me the way to do that?
I'd also appreciate an explanation why the forms above won't work and/or a hint on how to execute commands that include piping and/or stdio directing from that script console.
Thanks :)

["bash", "echo the text > /mnt/efs-ci-state/path/to/the-state-file.txt"].execute().text
or use plain groovy:
new File('/mnt/efs-ci-state/path/to/the-state-file.txt').text = "echo the text"
why not working:
options 1, 2, 4 : echo and piping is a feature of shell/bash - it will not work without bash
option 3 you have c echo and c is not a valid command
use array to execute complex commands and to separate bash from main part
i suggest you to use this kind of code if you want to capture and validate stderr
["bash", 'echo my text > /222/12345.txt'].execute().with{proc->
def out=new StringBuilder(), err=new StringBuilder()
proc.waitForProcessOutput(out, err)
assert !err.toString().trim()
return out.toString()
}

Related

When executing a batch file from python, the output is only the first line

I'm using the 'qwinsta' cmd command to get the session ID of a remote computer and output it to a textfile, so I create a new batch file and write the command then I try running the batch file through python but it only returns the first line of the output. When I run the batch file by simply double-clicking it it works properly.
Using python 2.7:
def run_qwinsta(self, computerName):
qwinsta_check = open("q.bat", "w")
qwinsta_check.write('PsExec -u <username> -p <password> \\\\' + computerName + ' qwinsta' + ' > "q.txt" ')
qwinsta_check.close()
os.system("q.bat")
Expected results:
SESSIONNAME USERNAME ID STATE TYPE DEVICE
>services 0 Disc
console <username> 1 Active
rdp-tcp 65536 Listen
Actual results:
SESSIONNAME USERNAME ID STATE TYPE DEVICE
I would recommend you to avoid writing the batchfile, If you can. You can execute your batch command from os.system(). Also you can try using subprocess (documentation here) and then redirecting the stdout and stderr to file.
EDIT:
PsExec is a good choice, but If you want another way, you can also use ssh.
You can run PsExec from os.system() and then write the response to text file on the remote machine. The default PsExec working directory is System32 there you can find your text file.
Tested code:
import os
os.system('Psexec \\\\SERVER cmd.exe /c "tasklist > process_list.txt"')
I used tasklist, because I don't have qwinsta on my remote machine.
If you want to store the PsExec response on your machine you can use subprocess and then redirect the stdout to text file.
Tested code:
import subprocess
process_list = open("process_list.txt", "w")
subprocess.call(['Psexec', '\\\\SERVER', 'tasklist'], stdout=process_list)
process_list.close()
Actually I used Python 3.x, because I don't have Python 2.x, but it should work on both.
If this still didn't solve your problem, please provide more details to your question!

Sqlite Database Doesn't "Reset"

I have an Rspec file that runs the following method before and after each "describe" test block:
def self.reset
commands = [
"del #{CATS_DB_FILE}", #the "del" was changed from "rm" for windows
"cat #{CATS_SQL_FILE} > sqlite3 #{CATS_DB_FILE}" # | was used before (pipe in linux)
]
commands.each { |command| `#{command}` }
DBConnection.open(CATS_DB_FILE)
end
I'm running all of this in Windows command line. For some reason, the database overpopulates itself repeatedly; it seems that the database doesn't "clear" itself, so when another test block is run, the same attributes are added again. Does the syntax in the commands array above look okay? Kind of new to all this, would appreciate any suggestions!

Sift Implementation in Vlfeat

I am using the Sift implementation by Vlfeat.org. It has the follwing function which is supposed to save the features to a file.
def process_image(imagename,resultname,params="--edge-thresh 10 --peak-thresh 5"):
""" process an image and save the results in a file"""
if imagename[-3:] != 'pgm':
#create a pgm file
im = Image.open(imagename).convert('L')
im.save('tmp.pgm')
imagename = 'tmp.pgm'
cmmd = str("sift "+imagename+" --output="+resultname+
" "+params)
os.system(cmmd)
print 'processed', imagename, 'to', resultname
Here how the line "os.system(cmmd)" is supposed to write the results in a file?
I am on an ubuntu machine and if I execute "sift" as command in terminal, I am getting the result as "not found". On linux, which process is this command trying to invoke? I need to save these Sift features into a file so that later I can use it to create Bag of Words descriptor for clustering.
A similar sift implementation at https://github.com/jesolem/PCV/blob/master/PCV/localdescriptors/sift.py also uses the same line to save the result into file.
On linux, which process is this command trying to invoke?
This refers to the command-line tool provided by vlfeat (see vlfeat/src/sift.c):
$ ./sift
Usage: ./sift [options] files ...
Options include:
--verbose -v Be verbose
--help -h Print this help message
--output -o Specify output file
...
You can either use the binary distribution from vlfeat download page (see bin/glnxa64/sift for Linux 64-bit), or directly clone the repo and compile it from sources.
Make sure to adjust the path with cmmd = str("/path/to/sift " ...) or install (= copy it) under a common path like /usr/local/bin.

Create a vim script, function or macro and run in windows by command line

i created a script that converts a text file into utf8 encoding. I can run it in vim. The problem is that i need to run it by cmd in windows and i cant figure out how. Help me
Sorry for my english. Im from south america, i speak spanish.
Alternatives
Unless you really need special Vim capabilities, you're probably better off using non-interactive tools like sed, awk, or Perl / Python / Ruby / your favorite scripting language here. For simple character set conversion, look into the iconv tool in particular.
That said, you can use Vim non-interactively:
Silent Batch Mode
For very simple text processing (i.e. using Vim like an enhanced 'sed' or 'awk', maybe just benefitting from the enhanced regular expressions in a :substitute command), use Ex-mode.
REM Windows
call vim -N -u NONE -n -es -S "commands.ex" "filespec"
Note: silent batch mode (:help -s-ex) messes up the Windows console, so you may have to do a cls to clean up after the Vim run.
# Unix
vim -T dumb --noplugin -n -es -S "commands.ex" "filespec"
Attention: Vim will hang waiting for input if the "commands.ex" file doesn't exist; better check beforehand for its existence! Alternatively, Vim can read the commands from stdin. You can also fill a new buffer with text read from stdin, and read commands from stderr if you use the - argument.
Full Automation
For more advanced processing involving multiple windows, and real automation of Vim (where you might interact with the user or leave Vim running to let the user take over), use:
vim -N -u NONE -n -c "set nomore" -S "commands.vim" "filespec"
Here's a summary of the used arguments:
-T dumb Avoids errors in case the terminal detection goes wrong.
-N -u NONE Do not load vimrc and plugins, alternatively:
--noplugin Do not load plugins.
-n No swapfile.
-es Ex mode + silent batch mode -s-ex
Attention: Must be given in that order!
-S ... Source script.
-c 'set nomore' Suppress the more-prompt when the screen is filled
with messages or output to avoid blocking.

How to save all console output to file in R?

I want to redirect all console text to a file. Here is what I tried:
> sink("test.log", type=c("output", "message"))
> a <- "a"
> a
> How come I do not see this in log
Error: unexpected symbol in "How come"
Here is what I got in test.log:
[1] "a"
Here is what I want in test.log:
> a <- "a"
> a
[1] "a"
> How come I do not see this in log
Error: unexpected symbol in "How come"
What am I doing wrong? Thanks!
You have to sink "output" and "message" separately (the sink function only looks at the first element of type)
Now if you want the input to be logged too, then put it in a script:
script.R
1:5 + 1:3 # prints and gives a warning
stop("foo") # an error
And at the prompt:
con <- file("test.log")
sink(con, append=TRUE)
sink(con, append=TRUE, type="message")
# This will echo all input and not truncate 150+ character lines...
source("script.R", echo=TRUE, max.deparse.length=10000)
# Restore output to console
sink()
sink(type="message")
# And look at the log...
cat(readLines("test.log"), sep="\n")
If you have access to a command line, you might prefer running your script from the command line with R CMD BATCH.
== begin contents of script.R ==
a <- "a"
a
How come I do not see this in log
== end contents of script.R ==
At the command prompt ("$" in many un*x variants, "C:>" in windows), run
$ R CMD BATCH script.R &
The trailing "&" is optional and runs the command in the background.
The default name of the log file has "out" appended to the extension, i.e., script.Rout
== begin contents of script.Rout ==
R version 3.1.0 (2014-04-10) -- "Spring Dance"
Copyright (C) 2014 The R Foundation for Statistical Computing
Platform: i686-pc-linux-gnu (32-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
[Previously saved workspace restored]
> a <- "a"
> a
[1] "a"
> How come I do not see this in log
Error: unexpected symbol in "How come"
Execution halted
== end contents of script.Rout ==
If you are able to use the bash shell, you can consider simply running the R code from within a bash script and piping the stdout and stderr streams to a file. Here is an example using a heredoc:
File: test.sh
#!/bin/bash
# this is a bash script
echo "Hello World, this is bash"
test1=$(echo "This is a test")
echo "Here is some R code:"
Rscript --slave --no-save --no-restore - "$test1" <<EOF
## R code
cat("\nHello World, this is R\n")
args <- commandArgs(TRUE)
bash_message<-args[1]
cat("\nThis is a message from bash:\n")
cat("\n",paste0(bash_message),"\n")
EOF
# end of script
Then when you run the script with both stderr and stdout piped to a log file:
$ chmod +x test.sh
$ ./test.sh
$ ./test.sh &>test.log
$ cat test.log
Hello World, this is bash
Here is some R code:
Hello World, this is R
This is a message from bash:
This is a test
Other things to look at for this would be to try simply pipping the stdout and stderr right from the R heredoc into a log file; I haven't tried this yet but it will probably work too.
You can't. At most you can save output with sink and input with savehistory separately. Or use external tool like script, screen or tmux.
Run R in emacs with ESS (Emacs Speaks Statistics) r-mode. I have one window open with my script and R code. Another has R running. Code is sent from the syntax window and evaluated. Commands, output, errors, and warnings all appear in the running R window session. At the end of some work period, I save all the output to a file. My own naming system is *.R for scripts and *.Rout for save output files.
Here's a screenshot with an example.
You can print to file and at the same time see progress having (or not) screen, while running a R script.
When not using screen, use R CMD BATCH yourscript.R & and step 4.
When using screen, in a terminal, start screen
screen
run your R script
R CMD BATCH yourscript.R
Go to another screen pressing CtrlA, then c
look at your output with (real-time):
tail -f yourscript.Rout
Switch among screens with CtrlA then n
To save text from the console: run the analysis and then choose (Windows) "File>Save to File".
Set your Rgui preferences for a large number of lines, then timestamp and save as file at suitable intervals.
If you want to get error messages saved in a file
zz <- file("Errors.txt", open="wt")
sink(zz, type="message")
the output will be:
Error in print(errr) : object 'errr' not found
Execution halted
This output will be saved in a file named Errors.txt
In case, you want printed values of console to a file you can use 'split' argument:
zz <- file("console.txt", open="wt")
sink(zz, split=TRUE)
print("cool")
print(errr)
output will be:
[1] "cool"
in console.txt file. So all your console output will be printed in a file named console.txt
This may not work for your needs, but one solution might be to run your code from within an Rmarkdown file. You could write both the code and console output to HTML/PDF/Word.

Resources