If I create a file in ubuntu like this: "echo "asd" > file.txt" and I do a ls -l file.txt it says that it's size is 4 bytes, but I only wrote 3 (asd). If I do "cat file.txt" it shows the 3 chars that I have added. Why is the file 4 bytes large?
asd + new line character = 4 bytes
$ echo asd | wc -c
4
$ echo -n asd | wc -c
3
$ echo asd | hd
00000000 61 73 64 0a |asd.|
00000004
-n in echo switches newline off
hd shows you a hexdump of the stream. You see the 0a character at the end. That is the fourth character, newline.
Related
I have this C function that I am using to output any amount of integer that is input.
/*
This program will be called function.c
*/
#include <stdio.h>
int main (void) {
int num;
while(scanf("%d",&num)==1) {
printf("You entered: %d\n",num);
}
return 0;
}
In my shell the function works like this as expected with the pipe command
$echo "1 7 3 " | ./function
Output:
You entered: 1
You entered: 7
You entered: 3
Now what I'm trying to do is use the sed command on a csv file and pipe the output into my function.
Here is my CSV file
$cat file.csv
Output:
2,2,3,3,8
Using the sed command I remove the commas
$sed 's/,/ /g' file.csv
Output:
2 2 3 3 8
Now the issue I have is when I try to use the output of the sed command to pipe the numbers into my function:
$sed 's/,/ /g' file.csv | ./function
I get no output. I don't know if there is a syntax error, but I believe I should be able to do this with a csv file.
I say you have a BOM in "file.csv".
That will stop ./function from scanning the first number.
$ hexdump -C file.csv
00000000 ef bb bf 32 2c 32 2c 33 2c 33 2c 38 0a |...2,2,3,3,8.|
0000000d
$ sed 's/,/ /g' file.csv |hexdump -C
00000000 ef bb bf 32 20 32 20 33 20 33 20 38 0a |...2 2 3 3 8.|
0000000d
$ cat file.csv
2,2,3,3,8
$ cut -b4- file.csv |sed 's/,/ /g' |./function
You entered: 2
You entered: 2
You entered: 3
You entered: 3
You entered: 8
$ sed 's/,/ /g' file.csv |cut -b4- |./function
You entered: 2
You entered: 2
You entered: 3
You entered: 3
You entered: 8
Let's say we have a shell variable $x containing a space separated list of numbers from 1 to 30:
$ x=$(for i in {1..30}; do echo -n "$i "; done)
$ echo $x
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
We can print the first three input record fields with AWK like this:
$ echo $x | awk '{print $1 " " $2 " " $3}'
1 2 3
How can we print all the fields starting from the Nth field with AWK? E.g.
4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
EDIT: I can use cut, sed etc. to do the same but in this case I'd like to know how to do this with AWK.
Converting my comment to answer so that solution is easy to find for future visitors.
You may use this awk:
awk '{for (i=3; i<=NF; ++i) printf "%s", $i (i<NF?OFS:ORS)}' file
or pass start position as argument:
awk -v n=3 '{for (i=n; i<=NF; ++i) printf "%s", $i (i<NF?OFS:ORS)}' file
Version 4: Shortest is probably using sub to cut off the first three fields and their separators:
$ echo $x | awk 'sub(/^ *([^ ]+ +){3}/,"")'
Output:
4 5 6 7 8 9 ...
This will, however, preserve all space after $4:
$ echo "1 2 3 4 5" | awk 'sub(/^ *([^ ]+ +){3}/,"")'
4 5
so if you wanted the space squeezed, you'd need to, for example:
$ echo "1 2 3 4 5" | awk 'sub(/^ *([^ ]+ +){3}/,"") && $1=$1'
4 5
with the exception that if there are only 4 fields and the 4th field happens to be a 0:
$ echo "1 2 3 0" | awk 'sub(/^ *([^ ]+ +){3}/,"")&&$1=$1'
$ [no output]
in which case you'd need to:
$ echo "1 2 3 0" | awk 'sub(/^ *([^ ]+ +){3}/,"") && ($1=$1) || 1'
0
Version 1: cut is better suited for the job:
$ cut -d\ -f 4- <<<$x
Version 2: Using awk you could:
$ echo -n $x | awk -v RS=\ -v ORS=\ 'NR>=4;END{printf "\n"}'
Version 3: If you want to preserve those varying amounts of space, using GNU awk you could use split's fourth parameter seps:
$ echo "1 2 3 4 5 6 7" |
gawk '{
n=split($0,a,FS,seps) # actual separators goes to seps
for(i=4;i<=n;i++) # loop from 4th
printf "%s%s",a[i],(i==n?RS:seps[i]) # get fields from arrays
}'
Adding one more approach to add all value into a variable and once all fields values are done with reading just print the value of variable. Change the value of n= as per from which field onwards you want to get the data.
echo "$x" |
awk -v n=3 '{val="";for(i=n; i<=NF; i++){val=(val?val OFS:"")$i};print val}'
With GNU awk, you can use the join function which has been a built-in include since gawk 4.1:
x=$(seq 30 | tr '\n' ' ')
echo "$x" | gawk '#include "join"
{split($0, arr)
print join(arr, 4, length(arr), "|")}
'
4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30
(Shown here with a '|' instead of a ' ' for clarity...)
Alternative way of including join:
echo "$x" | gawk -i join '{split($0, arr); print join(arr, 4, length(arr), "|")}'
Using gnu awk and gensub:
echo $x | awk '{ print gensub(/^([[:digit:]]+[[:space:]]){3}(.*$)/,"\\2",$0)}'
Using gensub, split the string into two sections based on regular expressions and print the second section only.
How to convert the following go code to bash
data, _ := base64.StdEncoding.DecodeString("nJpGBA==")
fmt.Println(data)
//Output
[156 154 70 4]
I got up to here
echo nJpGBA== |base64 -d
https://play.golang.org/p/OfyztKQINg9
Not a exact match, but:
echo nJpGBA== |base64 -d | od -A n -t u1
Output: 156 154 70 4
Note leading space and multiple spaces between.
Other solution. Assign it to an array:
val_array=( $(echo nJpGBA== |base64 -d | od -A n -t u1) )
echo "${val_array[#]}"
Output: 156 154 70 4
The command od dumps any binary files, by default in octal values. Here it reads from stdin, as no file is given.
-A n suppresses the output of byte addresses
-t u1 prints one byte unsigned decimals
I have to print the middle line of any text file without sed nor awk.
For example, the following file.txt:
line 1
line 2
line 3
line 4
line 5
I need something like:
$ command -flags file.txt
line 3
Is there any command?
Thanks.
Not the most efficient, but works in bash.
Use wc -l to count the lines, and divide by two. Then use tail -n +N | head -n 1 to print just the Nth line (where N starts at 1).
$ cat input.txt
A
B
C
D
E
$ tail -n +$(((`cat input.txt | wc -l` / 2) + 1)) input.txt | head -n 1
C
Note that a file with an even number of lines has no single "middle line".
I cat-ed the file to wc -l so it wouldn't print the filename.
sed -n $(((`cat input.txt| wc -l`/ 2) + 1))p input.txt
I'm trying to delete specific lines based on the argument passed in.
My data.txt file contains
Cpu 500 64 6
Monitor 22 42 50
Game 32 64 128
My del.sh contains
myvar=$1
sed'/$myvar/d' data.txt > temp.txt
mv temp.txt > data.txt
but it just prints every line in temp.txt to data.txt....however
sed '/64/d' data.txt > temp.txt
will do the correct data transfer (but I don't want to hardcode 64), I feel like there's some kind of syntax error with the argument. Any input please
It's because of the single quotes, change them to double quotes. Variables inside single quotes are not interpolated, so you are sending the literal string $myvar to sed, instead of the value of $myvar.
Change:
sed '/$myvar/d' data.txt
to:
sed "/$myvar/d" data.txt
Note: You will run into issues when $myvar contains regular expression meta characters or forward slashes as pointed out in this response from Ed Morton. If you are not in complete control of your input you will need to find another avenue to accomplish this.
Assuming this is undesirable behavior:
$ cat file
Cpu 500 64 6
Monitor 22 42 50
Game 32 64 128
$ myvar=6
$ sed "/$myvar/d" file
Monitor 22 42 50
$ myvar=/
$ sed "/$myvar/d" file
sed: -e expression #1, char 3: unknown command: `/'
$ myvar=.
$ sed "/$myvar/d" file
$
Try this instead:
$ myvar=6
$ awk -v myvar="$myvar" '{for (i=1; i<=NF;i++) if ($i == myvar) next }1' file
Monitor 22 42 50
Game 32 64 128
$ myvar=/
$ awk -v myvar="$myvar" '{for (i=1; i<=NF;i++) if ($i == myvar) next }1' file
Cpu 500 64 6
Monitor 22 42 50
Game 32 64 128
$ myvar=.
$ awk -v myvar="$myvar" '{for (i=1; i<=NF;i++) if ($i == myvar) next }1' file
Cpu 500 64 6
Monitor 22 42 50
Game 32 64 128
and if you think you can just escape the /s and use sed, you can't because you might be adding a 2nd backslash to one already present:
$ foo='\/'
$ myvar=${foo//\//\\\/}
$ sed "/$myvar/d" file
sed: -e expression #1, char 5: unknown command: `/'
$ awk -v myvar="$myvar" '{for (i=1; i<=NF;i++) if ($i == myvar) next }1' file
Cpu 500 64 6
Monitor 22 42 50
Game 32 64 128
This is simply NOT a job you can in general do with sed due to it's syntax and it's restriction of only allowing REs in it's search.
You can also use awk to do the same,
awk '!/'$myvar'/' data.txt > temp.txt && mv temp.txt data.txt
Use -i option in addition to what #SeanBright proposed. Then you won't need > temp.txt and mv temp.txt data.txt.
sed -i "/$myvar/d" data.txt