i am writing a short bat file that contours a xyz file with GMT utilities (generic mapping tool) i want to read the max and min file and use it later in the bat file what i did is
set max_color=gawk "BEGIN {max = 0} {if ($3>max) max=$3} END {print max}" %file%
set min_color=gawk "BEGIN {min = %max_color%} {if ($3'<'min) min=$3} END {print min}" %file%
but when i try reading it later
makecpt -Crainbow -T%min_color%/%max_color%/10 > conc.cpt
instead of reding the value it has the whole gawk one liner
how can i set a value
use a for loop to get the results of the gawk command, eg
for /f %%a in ('your gawk command') do (
set var=%%a
)
Related
I have a huge file, and at some point it goes like this:
Bla bla bla
LAST ITERATION: 1780 6 12 0.689655172413793
-8708.81862246834 -8698.33572943212 -2003.09638506407
-9.912281246897692E-003
Bla bla bla
I would like to get all the numbers after "LAST ITERATION:" and put it in a line in a file.
I have managed to get the two first lines with this:
awk 'a && NR==n{ print a,b,c,d,$1,$3 } /LAST ITERATION:/{ a=$3; b=$4; c=$5; d=$6; n=NR+1 }' ./$FOLDER/$NAMEDATA >> $NAMEOUTPUT
But I can't seem to find a way to get the last number which is on the 3rd line. Could anyone help me?
Thanks!
If you are using gawk, where RS can be more than one character, an easier approach would be to use "LAST ITERATION:" as the record separator, and print the first 8 fields of the second record, with a space as the output record seperator:
gawk 'BEGIN{RS="LAST ITERATION:";ORS=" "}NR==2{for(n=1;n<=8;++n)print$n}'
Demo: https://ideone.com/UzwkdN
You might use getline Variable to access future lines, I would use GNU AWK for this task following way, let file.txt content be
Bla bla bla
LAST ITERATION: 1780 6 12 0.689655172413793
-8708.81862246834 -8698.33572943212 -2003.09638506407
-9.912281246897692E-003
Bla bla bla
then
awk '/LAST ITERATION/{getline x1;getline x2;line=$0 " " x1 " " x2;sub(/LAST ITERATION:[[:space:]]+/,"",line);gsub(/[[:space:]]+/," ",line);print line}' file.txt
gives output
1780 6 12 0.689655172413793 -8708.81862246834 -8698.33572943212 -2003.09638506407 -9.912281246897692E-003
Explanation: when LAST ITERATION is encountered I save next line to variable x1 and next next line to variable x2, then construct line from space-sheared current line, next line and next next line, then remove LAST ITERATION: and following whitespaces characters using sub function and alter multiple whitespace characters to single spaces using gsub, after doing that I print said line. Disclaimer: this solution assumes there are always at least 2 lines after line with LAST ITERATION.
(tested in gawk 4.2.1)
If you are using regular awk, you can concatenate the 3 lines starting with the line with LAST ITERATION: to a variable, and split it in the end:
awk 'BEGIN{ORS=" "}/LAST ITERATION:/{n=NR+3}NR<n{s=s$0}END{split(s,a);for(i=3;i<11;++i)print(a[i])}'
If I read your question correctly, you say you want to see an entry within a file, followed by two following lines.
Next to that, you seem to want to work with awk.
My answer is: "Why awk? This can very easily be done using grep.", as follows:
grep has three switches to add more lines, next to the matching one:
-A n : add "n" lines after the match
-B n : add "n" lines before the match
-C n : add "n" lines, combined after and before
So, you can simply use grep -A 2 "match" filename.
Good luck
For batch you can target lines using findstr:
The below can target single lines, consecutive lines or a list of lines.
#Echo off
REM :: CALL %0 <filepath> <integer|integer list>
Set "File.Name="
If exist "%~f1" (
Set "File.Name=%~f1"
)Else (
1>&2 Echo(File Not Found
Exit /B 1
)
If "%~2" == "" (
1>&2 Echo(Missing Arg/s for target Line/s.
Exit /B 2
)
Set Target.Lines=%*
Set "Target.Lines=%Target.Lines:"=%"
For /f "Delims=" %%G in ('CMD /V:ON /C "Echo(!Target.Lines:%~1 =!"')Do Set "Target.Lines=%%G"
Set "Line.Count=0"
Set "Highest.Target=0"
Setlocal EnableExtensions EnableDelayedExpansion
For %%G in (%Target.Lines%)Do if !Highest.Target! LSS %%G Set "Highest.Target=%%G"
Endlocal & Set "Highest.Target=%Highest.Target%"
For /f "Tokens=1* Delims=:" %%G in ('%SystemRoot%\System32\findstr.exe /VNLC:"{false.string[%~n0]}" "%File.Name%"')Do (
Set "Line.Count=%%G"
For %%i in (%Target.Lines%)Do if %%G EQU %%i Echo(%%H
)
If %Line.Count% LSS %Highest.Target% (
1>&2 Echo(Lines missing. Lines in file: %Line.Count%. Highest Target: %Highest.Target%
)
awk '
gsub(/LAST ITERATION:/,""){
gsub(/^ */,"");
o=sprintf("%s ",$0);
while(getline && $1 ~ /^[-]?[0-9]+(\.[0-9]*)?/) {
o=o sprintf("%s ", $0)
}
print gensub(/ */," ", "g", o) > "output_file"
}
' input_file
$ cat output_file
1780 6 12 0.689655172413793 -8708.81862246834 -8698.33572943212 -2003.09638506407 -9.912281246897692E-003
I want to write a batch utility to copy the output of a command prompt window to a file. I run my command prompt windows with the maximum depth of 9999 lines, and occasionally I want to grab the output of a command whose output is off-screen. I can do this manually with the keys Ctrl-A, Ctrl-Cand then pasting the result into Notepad - I just want to automate it in a batch file with a call to:
SaveScreen <text file name>
I know I can do it with redirection, but that would involve knowing that I will need to save the output of a batch command sequence beforehand.
So if I had a batch script:
call BuildPhase1.bat
if "%ErrorLevel% gtr 0 goto :ErrorExit
call BuildPhase2.bat
if "%ErrorLevel% gtr 0 goto :ErrorExit
call BuildPhase3.bat
if "%ErrorLevel% gtr 0 goto :ErrorExit
I could write:
cls
call BuildPhase1.bat
if "%ErrorLevel% gtr 0 call SaveScreen.bat BuildPhase1.err & goto :ErrorExit
call BuildPhase2.bat
if "%ErrorLevel% gtr 0 call SaveScreen.bat BuildPhase2.err & goto :ErrorExit
call BuildPhase3.bat
if "%ErrorLevel% gtr 0 call SaveScreen.bat BuildPhase3.err & goto :ErrorExit
or I could just type SaveScreen batch.log when I see that a run has failed.
My experiments have got me this far:
<!-- : Begin batch script
#cscript //nologo "%~f0?.wsf" //job:JS
#exit /b
----- Begin wsf script --->
<package>
<job id="JS">
<script language="JScript">
var oShell = WScript.CreateObject("WScript.Shell");
oShell.SendKeys ("hi folks{Enter}") ;
oShell.SendKeys ("^A") ; // Ctrl-A (select all)
oShell.SendKeys ("^C") ; // Ctrl-C (copy)
oShell.SendKeys ("% ES") ; // Alt-space, E, S (select all via menu)
oShell.SendKeys ("% EY") ; // Alt-space, E, Y (copy via menu)
// ... invoke a notepad session, paste the clipboard into it, save to a file
WScript.Quit () ;
</script>
</job>
</package>
My keystrokes are making it to the command prompt so presumably I have the correct window focused - it just seems to be ignoring the Ctrl and Alt modifiers. It also recognises Ctrl-C but not Ctrl-A. Because it has ignored the Ctrl-A to select all the text, the Ctrl-C causes the batch file to think it has seen a break command.
I've seen the other answers like this one but they all deal with methods using redirection, rather than a way of doing it after the fact "on demand".
* UPDATE *
On the basis of #dxiv's pointer, here is a batch wrapper for the routine:
Get-ConsoleAsText.bat
:: save the contents of the screen console buffer to a disk file.
#set "_Filename=%~1"
#if "%_Filename%" equ "" #set "_Filename=Console.txt"
#powershell Get-ConsoleAsText.ps1 >"%_Filename%"
#exit /b 0
The Powershell routine is pretty much as was presented in the link, except that:
I had to sanitise it to remove some of the more interesting character substitutions the select/copy/paste operation introduced.
The original saved the trailing spaces as well. Those are now trimmed.
Get-ConsoleAsText.ps1
# Get-ConsoleAsText.ps1 (based on: https://devblogs.microsoft.com/powershell/capture-console-screen/)
#
# The script captures console screen buffer up to the current cursor position and returns it in plain text format.
#
# Returns: ASCII-encoded string.
#
# Example:
#
# $textFileName = "$env:temp\ConsoleBuffer.txt"
# .\Get-ConsoleAsText | out-file $textFileName -encoding ascii
# $null = [System.Diagnostics.Process]::Start("$textFileName")
#
if ($host.Name -ne 'ConsoleHost') # Check the host name and exit if the host is not the Windows PowerShell console host.
{
write-host -ForegroundColor Red "This script runs only in the console host. You cannot run this script in $($host.Name)."
exit -1
}
$textBuilder = new-object system.text.stringbuilder # Initialize string builder.
$bufferWidth = $host.ui.rawui.BufferSize.Width # Grab the console screen buffer contents using the Host console API.
$bufferHeight = $host.ui.rawui.CursorPosition.Y
$rec = new-object System.Management.Automation.Host.Rectangle 0,0,($bufferWidth - 1),$bufferHeight
$buffer = $host.ui.rawui.GetBufferContents($rec)
for($i = 0; $i -lt $bufferHeight; $i++) # Iterate through the lines in the console buffer.
{
$Line = ""
for($j = 0; $j -lt $bufferWidth; $j++)
{
$cell = $buffer[$i,$j]
$line = $line + $cell.Character
}
$line = $line.trimend(" ") # remove trailing spaces.
$null = $textBuilder.Append($line)
$null = $textBuilder.Append("`r`n")
}
return $textBuilder.ToString()
The contents of the console buffer can be retrieved with the PS script from PowerShell's team blog Capture console screen mentioned in a comment, now edited into OP's question.
The last line could also be changed to copy the contents to the clipboard instead of returning it.
Set-Clipboard -Value $textBuilder.ToString()
As a side note, the reasons for using a StringBuilder rather than direct concatenation are discussed in How does StringBuilder work internally in C# and How the StringBuilder class is implemented.
i am trying to make output of a command be read by a for loop, but in such a way that the loop variable would be an array. is that possible?
this is what I've been trying so far:
function samplevals() {
echo '"aa bb"'
echo '"cc dd"'
echo '"ee ff" "gg hh"'
}
samplevar='"aa bb"
"cc dd"
"ee ff" "gg hh"'
echo call function samplevals:
for x in `samplevals `; do echo ">$x<"; done
echo read variable samplevar:
echo $samplevar
for x in $samplevar; do echo ">$x<"; done
echo process output of 'echo samplevar:'
for x in `echo $samplevar`; do echo ">$x<"; done
echo "the thing with set"
for x in $samplevar; do set -- $x ; echo "\$1=>$1<,\$2=>$2<"; done
but the output is always the same:
>"aa<
>bb"<
>"cc<
>dd"<
>"ee<
>ff"<
>"gg<
>hh"<
Can I somehow prevent bash from splitting the elements into smaller pieces?
Along the lines of chepner's answer, using read command. The -a flag part of the command lets the output to written out to an array.
IFS=$'\t' read -ra arrayDef < <(echo -ne '"abc def"\t"ghi jkl"')
for x in "${arrayDef[#]}"; do
echo ">$x<"
done
You can replace the echo part with some command that generates such a string. Remember to update the IFS appropriately as to how the string is de-limited. In my case, I just a have the string de-limited by \t
The way to define an array in bash is
samplevars=("aa bb" "cc dd" "ee ff" "gg hh")
for x in "${samplevars[#]}"; do
echo ">$x<"
done
I have a textfile (file1.txt) with multiple lines of data.
This textfile I'm using to copy data from a directory A to another B. My script looks if an expression is included in a filename of a file stored in A.
In the directory A I grep another textfile (file2.txt) to get information (rows like [bla][0-9][0-9][bla][0-9][0-9]) that I want to exclude in my script.
set x = `grep '[bla][0-9][0-9][bla][0-9][0-9]' file1.txt`
foreach i ( $x )
cp A/*$i* B/.
end
For example rows in file1.txt:
bla11bla11
bla12bla12
bla13bla13
bla14bla14
bla15bla15
and grep result from file2.txt that has to be excluded for the loop
bla11bla11
bla12bla12
My script should finally only use the following lines
bla13bla13
bla14bla14
bla15bla15
How can I do this?
A simple nested loop does it completely in csh:
#!/bin/csh -f
set f = `grep 'bla[0-9][0-9]bla[0-9][0-9]' file1.txt`
set x = `grep 'bla[0-9][0-9]bla[0-9][0-9]' file2.txt`
echo "Files: $f"
echo "To be excluded from Files: $x"
set r = ( )
foreach i ( $f )
set skip = 0
foreach j ( $x )
if ("$i" == "$j") then
set skip = 1
break
endif
end
if ($skip == 0) set r = ($r $i)
end
echo "Result: $r"
The output when run on your above example files:
Files: bla11bla11 bla12bla12 bla13bla13 bla14bla14 bla15bla15
To be excluded from Files: bla11bla11 bla12bla12
Result: bla13bla13 bla14bla14 bla15bla15
I have a bunch of files named like so:
output_1.png
output_2.png
...
output_10.png
...
output_120.png
What is the easiest way of renaming those to match a convention, e.g. with maximum four decimals, so that the files are named:
output_0001.png
output_0002.png
...
output_0010.png
output_0120.png
This should be easy in Unix/Linux/BSD, although I also have access to Windows. Any language is fine, but I'm interested in some really neat one-liners (if there are any?).
Python
import os
path = '/path/to/files/'
for filename in os.listdir(path):
prefix, num = filename[:-4].split('_')
num = num.zfill(4)
new_filename = prefix + "_" + num + ".png"
os.rename(os.path.join(path, filename), os.path.join(path, new_filename))
you could compile a list of valid filenames assuming that all files that start with "output_" and end with ".png" are valid files:
l = [(x, "output" + x[7:-4].zfill(4) + ".png") for x in os.listdir(path) if x.startswith("output_") and x.endswith(".png")]
for oldname, newname in l:
os.rename(os.path.join(path,oldname), os.path.join(path,newname))
Bash
(from: http://www.walkingrandomly.com/?p=2850)
In other words I replace file1.png with file001.png and file20.png with file020.png and so on. Here’s how to do that in bash
#!/bin/bash
num=`expr match "$1" '[^0-9]*\([0-9]\+\).*'`
paddednum=`printf "%03d" $num`
echo ${1/$num/$paddednum}
Save the above to a file called zeropad.sh and then do the following command to make it executable
chmod +x ./zeropad.sh
You can then use the zeropad.sh script as follows
./zeropad.sh frame1.png
which will return the result
frame001.png
All that remains is to use this script to rename all of the .png files in the current directory such that they are zeropadded.
for i in *.png;do mv $i `./zeropad.sh $i`; done
Perl
(from: Zero pad rename e.g. Image (2).jpg -> Image (002).jpg)
use strict;
use warnings;
use File::Find;
sub pad_left {
my $num = shift;
if ($num < 10) {
$num = "00$num";
}
elsif ($num < 100) {
$num = "0$num";
}
return $num;
}
sub new_name {
if (/\.jpg$/) {
my $name = $File::Find::name;
my $new_name;
($new_name = $name) =~ s/^(.+\/[\w ]+\()(\d+)\)/$1 . &pad_left($2) .')'/e;
rename($name, $new_name);
print "$name --> $new_name\n";
}
}
chomp(my $localdir = `pwd`);# invoke the script in the parent-directory of the
# image-containing sub-directories
find(\&new_name, $localdir);
Rename
Also from above answer:
rename 's/\d+/sprintf("%04d",$&)/e' *.png
Fairly easy, although it combines a few features not immediately obvious:
#echo off
setlocal enableextensions enabledelayedexpansion
rem iterate over all PNG files:
for %%f in (*.png) do (
rem store file name without extension
set FileName=%%~nf
rem strip the "output_"
set FileName=!FileName:output_=!
rem Add leading zeroes:
set FileName=000!FileName!
rem Trim to only four digits, from the end
set FileName=!FileName:~-4!
rem Add "output_" and extension again
set FileName=output_!FileName!%%~xf
rem Rename the file
rename "%%f" "!FileName!"
)
Edit: Misread that you're not after a batch file but any solution in any language. Sorry for that. To make up for it, a PowerShell one-liner:
gci *.png|%{rni $_ ('output_{0:0000}.png' -f +($_.basename-split'_')[1])}
Stick a ?{$_.basename-match'_\d+'} in there if you have other files that do not follow that pattern.
I actually just needed to do this on OSX. Here's the scripts I created for it - single line!
> for i in output_*.png;do mv $i `printf output_%04d.png $(echo $i | sed 's/[^0-9]*//g')`; done
For mass renaming the only safe solution is mmv—it checks for collisions and allows renaming in chains and cycles, something that is beyond most scripts. Unfortunately, zero padding it ain't too hot at. A flavour:
c:> mmv output_[0-9].png output_000#1.png
Here's one workaround:
c:> type file
mmv
[^0-9][0-9] #1\00#2
[^0-9][0-9][^0-9] #1\00#2#3
[^0-9][0-9][0-9] #1\0#2#3
[^0-9][0-9][0-9][^0-9] #1\0#2#3
c:> mmv <file
Here is a Python script I wrote that pads zeroes depending on the largest number present and ignores non-numbered files in the given directory. Usage:
python ensure_zero_padding_in_numbering_of_files.py /path/to/directory
Body of script:
import argparse
import os
import re
import sys
def main(cmdline):
parser = argparse.ArgumentParser(
description='Ensure zero padding in numbering of files.')
parser.add_argument('path', type=str,
help='path to the directory containing the files')
args = parser.parse_args()
path = args.path
numbered = re.compile(r'(.*?)(\d+)\.(.*)')
numbered_fnames = [fname for fname in os.listdir(path)
if numbered.search(fname)]
max_digits = max(len(numbered.search(fname).group(2))
for fname in numbered_fnames)
for fname in numbered_fnames:
_, prefix, num, ext, _ = numbered.split(fname, maxsplit=1)
num = num.zfill(max_digits)
new_fname = "{}{}.{}".format(prefix, num, ext)
if fname != new_fname:
os.rename(os.path.join(path, fname), os.path.join(path, new_fname))
print "Renamed {} to {}".format(fname, new_fname)
else:
print "{} seems fine".format(fname)
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
$rename output_ output_0 output_? # adding 1 zero to names ended in 1 digit
$rename output_ output_0 output_?? # adding 1 zero to names ended in 2 digits
$rename output_ output_0 output_??? # adding 1 zero to names ended in 3 digits
That's it!
with bash split,
linux
for f in *.png;do n=${f#*_};n=${n%.*};mv $f $(printf output_"%04d".png $n);done
windows(bash)
for f in *.png;do n=${f#*_};mv $f $(printf output_"%08s" $n);done
I'm following on from Adam's solution for OSX.
Some gotchyas I encountered in my scenario were:
I had a set of .mp3 files, so the sed was catching the '3' in the '.mp3' suffix. (I used basename instead of echo to rectify this)
My .mp3's had spaces within their names, E.g., "audio track 1.mp3", this was causing basename+sed to screw up a little bit, so I had to quote the "$i" parameter.
In the end, my conversion line looked like this:
for i in *.mp3 ; do mv "$i" `printf "track_%02d.mp3\n" $(basename "$i" .mp3 | sed 's/[^0-9]*//g')` ; done
Using ls + awk + sh:
ls -1 | awk -F_ '{printf "%s%04d.png\n", "mv "$0" "$1"_", $2}' | sh
If you want to test the command before runing it just remove the | sh
I just want to make time lapse movie using
ffmpeg -pattern_type glob -i "*.jpg" -s:v 1920x1080 -c:v libx264 output.mp4
and got a similar problem.
[image2 # 000000000039c300] Pattern type 'glob' was selected but globbing is not supported by this libavformat build
glob not support on Windows 7 .
Also if file list like below, and uses %2d.jpg or %02d.jpg
1.jpg
2.jpg
...
10.jpg
11.jpg
...
[image2 # 00000000005ea9c0] Could find no file with path '%2d.jpg' and index in the range 0-4
%2d.jpg: No such file or directory
[image2 # 00000000005aa980] Could find no file with path '%02d.jpg' and index in the range 0-4
%02d.jpg: No such file or directory
here is my batch script to rename flies
#echo off
setlocal enabledelayedexpansion
set i=1000000
set X=1
for %%a in (*.jpg) do (
set /a i+=1
set "filename=!i:~%X%!"
echo ren "%%a" "!filename!%%~xa"
ren "%%a" "!filename!%%~xa"
)
after rename 143,323 jpg files,
ffmpeg -i %6d.jpg -s:v 1920x1080 -c:v libx264 output.mp4