I have a file with |*| delimiter. In bcp, using -t|*| does not work. can we set delimiter with more than 1 character?
You can use -t '|*|' and that will work fine for that delimiter. Single quotes work for multiple character delimiters for both row and field delimiter.
Related
I try to solve a problem in shell.
Im trying to find a way to delete all newlines from each element of an array. I tried to do this with a for loop.
The Strings look like this (always three numbers, separated with dots)
"14.1.3\n" and I need to get rid of the newline at the end.
This is what i tried to do:
As a single-liner
for i in ${backup_versions[*]}; do backup_versions[$i]=echo "$i" | tr '\n' ' ' ; done
Easier to read
for i in ${backup_versions[*]};
do
backup_versions[$i]=echo "$i" | tr '\n' ' '
done
I think I try to reassign the element with the wrong syntax, but I tried every kind of writing i which I found or knew myself.
The deletion of the newline works just fine and just the reassigning is my Problem.
If the strings are always of that form and don't contain any whitespace or wildcard characters, you can just use the shell's word-splitting to remove extraneous whitespace characters from the values.
backup_versions=(${backup_versions[*]})
If you used mapfile to create the array, you can use the -t option to prevent it from including the newline in the value in the first place.
Use Bash's string substitution expansion ${var//old/new} to delete all newlines, and dynamically create a declaration for a new array, with elements stripped of newlines:
#!/usr/bin/env bash
backup_versions=(
$'foo\nbar\n'
$'\nbaz\ncux\n\n'
$'I have spaces\n and newlines\n'
$'It\'s a \n\n\nsingle quote and spaces\n'
$'Quoted "foo bar"\n and newline'
)
# shellcheck disable=SC2155 # Dynamically generated declaration
declare -a no_newlines="($(
printf '%q ' "${backup_versions[#]//$'\n'/}"
))"
# Debug print original array declaration
declare -p backup_versions
# Debug print the declaration of no_newlines
declare -p no_newlines
declare -a no_newlines="($(: Creates a dynamically generated declaration for the no_newlines array.
printf '%q ': Print each argument with quotes if necessary and add a trailing space.
"${backup_versions[#]//$'\n'/}": Expand each element of the backup_versions array, // replacing all $'\n' newlines by nothing to delete them.
Finally the no_newlines array will contain all entries from backup_versions, with newlines stripped-out.
Debug output match expectations:
declare -a backup_versions=([0]=$'foo\nbar\n' [1]=$'\nbaz\ncux\n\n' [2]=$'I have spaces\n and newlines\n' [3]=$'It\'s a \n\n\nsingle quote and spaces\n' [4]=$'Quoted "foo bar"\n and newline')
declare -a no_newlines=([0]="foobar" [1]="bazcux" [2]="I have spaces and newlines" [3]="It's a single quote and spaces" [4]="Quoted \"foo bar\" and newline")
You can use a modifier when expanding the array, then save the modified contents. If the elements just have a single trailing newline, use substring removal to trim it:
backup_versions=("${backup_versions[#]%$'\n'}")
(Note: when expanding an array, you should almost always use [#] instead of [*], and put double-quotes around it to avoid weird parsing. Bash doesn't generally let you combine modifiers, but you can combo them with [#] to apply the modifier to each element as it's expanded.)
If you want to remove all newlines from the elements (in case there are multiple newlines in some elements), use a substitution (with an empty replacement string) instead:
backup_versions=("${backup_versions[#]//$'\n'/}")
(But as several comments have mentioned, it'd probably be better to look at how the array's being created, and see if it's possible to just avoid putting newlines in the array in the first place.)
Using Bash I am extracting multiple strings from a binary file. Those strings are filenames, so only NUL and slash can not appear. I use a function that outputs those filenames to an array. I know, I can use IFS separator newline to get filenames with spaces. I hope it is possible to separate functions multiline strings with NUL to save in array, so any *nix legal filename can be worked with. If I set IFS to '' or '\0' I get some numbers instead of names. Not sure why, and maybe I have overseen something pretty basic :)
How do I achieve getting all possible filename strings including not just spaces, but newlines and other characters/byte values as well?
Here is my simplified example.
#! /bin/bash
binaryFile=$1
getBinaryList () {
fileNameAddresses=( 123 456 789 ) #Just a mock example for simplicity
for currAddr in "${fileNameAddresses[#]}"
do
fileNameStart=$((currAddr)) #Just a mock example for simplicity
fileNameLength=48 #Just a mock example for simplicity
currFileName=$( dd status=none bs=1 skip=$fileNameStart count=$fileNameLength if=$binaryFile )
printf "%s\n" "$currFileName"
done
}
IFS=$'\n'
allFileNames=($(getBinaryList $binaryFile))
echo ${#allFileNames[#]}
printf "%s\n" "${allFileNames[#]}"
Your idea is right, but with a couple of slight modifications you can achieve what you are looking for. In the getBinaryList() function instead of using printf() emitting output with newline formatting, use a NULL byte separator, i.e.
printf "%s\0" "$currFileName"
and now instead of modifying IFS to newline and slurping the result into an array. Use a command like mapfile which puts the results directly into array. The command provides an option to delimit results on the NULL byte with -d '' and to store in array specified by -t. So your result can look like
mapfile -t -d '' allFileNames < <(getBinaryList "$binaryFile")
I am using below command to write data to csv file in isql
$ISQL -S DSA1_PROD -U emer_r_gh5432 -X
Query -
Select * from SecDb..LoginOwnerTb where SvrId= 45566 and OwnerRitsId = '1001167635';
OUTPUT TO '/tmp/sometable.csv' FORMAT ASCII DELIMITED BY ';' QUOTE '';
go
it says
Server 'ABC', Line 1:
Incorrect syntax near ';'.
Please help
NOTE: I'm assuming you're working with Sybase ASE and the isql command line tool. There may be other ways to accomplish what you're trying to do when going against the SQLAnywhere, IQ and/or Advantage database products ... *shrug* ...
The OUTPUT TO clause is used with the dbisql GUI tool.
To perform a somewhat-similar OUTPUT operation with the isql command line tool:
-- once logged in via isql ...
-- to write to new file; to overwrite existing file:
select ....
go > /path/to/local/file/accessible/by/user/running/isql
-- to append to existing file:
select ...
go >> /path/to/local/file/accessible/by/user/running/isql
To set the column delimiter you can use the -s flag when invoking isql from the command line, eg:
# set the column delimiter to a semi-colon:
$ isql ... -s ';' ...
# set the column delimiter to a pipe:
$ isql ... -s '|' ...
Keep in mind that the output will still be generated using fixed-width columns, with each column's width determined by either a) the column's datatype 'width' or b) the column's title/label width, whichever is wider.
I'm not aware of any way to perform the following with the isql command line tool:
designate a column delimiter on-the-fly while inside a isql session
designate a quote character
remove extra spaces (ie, ouput data in true delimited format as opposed to fixed-width format)
To generate true delimited files you have a few options:
see if the dbisql GUI tool serves your purpose [I don't use dbisql so I'm *assuming* the OUTPUT TO clause works as expected]
use the bcp (command line) utility to place the data into a delimited file [bcp options, and how to handle subsets of tables, is a much larger discussion, ie, too much to address in this response]
see if you can find another (3rd party) tool that can extract the desired data set to a delimited file
I have a file called output.txt and it caontains a line "test[1,2,3]: [x+y=0]"
I want to delete all the characters upto '['. and delete ']' as well. I want to have only x+y=0 in output.txt using shell script. I can not delete some fixed no. of characters from the beginning as this output.txt is generated dynamically. Please provide some solution.
Powershell:
$teststring=teststring.replace("[", "").replace("]", "") -replace ".* "
This will split your row based on the ':' delimiter and take the second token and with that I then replace blank space and '[' and ']' characters with nothing:
cat output.txt | cut -d':' -f2 | sed -e "s/ //g; s/\[//g; s/\]//g"
Prints:
x+y=0
Unix 'file' command has a -0 option to output a null character after a filename. This is supposedly good for using with 'cut'.
From man file:
-0, --print0
Output a null character ‘\0’ after the end of the filename. Nice
to cut(1) the output. This does not affect the separator which is
still printed.
(Note, on my Linux, the '-F' separator is NOT printed - which makes more sense to me.)
How can you use 'cut' to extract a filename from output of 'file'?
This is what I want to do:
find . "*" -type f | file -n0iNf - | cut -d<null> -f1
where <null> is the NUL character.
Well, that is what I am trying to do, what I want to do is get all file names from a directory tree that have a particular MIME type. I use a grep (not shown).
I want to handle all legal file names and not get stuck on file names with colons, for example, in their name. Hence, NUL would be excellent.
I guess non-cut solutions are fine too, but I hate to give up on a simple idea.
Just specify an empty delimiter:
cut -d '' -f1
(N.B.: The space between the -d and the '' is important, so that the -d and the empty string get passed as separate arguments; if you write -d'', then that will get passed as just -d, and then cut will think you're trying to use -f1 as the delimiter, which it will complain about, with an error message that "the delimiter must be a single character".)
This works with gnu awk.
awk 'BEGIN{FS="\x00"}{print$1}'
ruakh's helpful answer works well on Linux.
On macOS, the cut utility doesn't accept '' as a delimiter argument (bad delimiter):
Here is a portable workaround that works on both platforms, via the tr utility; it only makes one assumption:
The input mustn't contain \1 control characters (START OF HEADING, U+0001) - which is unlikely in text.
You can substitute any character known not to occur in the input for \1; if it's a character that can be represented verbatim in a string, that simplifies the solution because you won't need the aux. command substitution ($(...)) with a printf call for the -d argument.
If your shell supports so-called ANSI C-quoted strings - which is true of bash, zsh and ksh - you can replace "$(printf '\1')" with $'\1'
(The following uses a simpler input command to demonstrate the technique).
# In zsh, bash, ksh you can simplify "$(printf '\1')" to $'\1'
$ printf '[first field 1]\0[rest 1]\n[first field 2]\0[rest 2]' |
tr '\0' '\1' | cut -d "$(printf '\1')" -f 1
[first field 1]
[first field 2]
Alternatives to using cut:
C. Paul Bond's helpful answer shows a portable awk solution.