I am working with php and trying to concatenate within the GLOB function. I can't seem to get it to work. My code is as follows:
$dir = "$images/*";
$images = glob( $dir . "{jpg,png,gif}", GLOB_BRACE );
This does work:
$images = glob( "images/*.{jpg,png,gif}", GLOB_BRACE);
For perspective, I am using the glob function to post images to a webpage from within a directory "images." What am I doing wrong? Is it possible to concatenate within glob?
You have a php $ variable symbol hiding out in your $dir variable:
$dir = "$images/*";
In PHP, the dollar sign within double quotes will be interpreted as a proper PHP variable. So $images/* will get interpreted as the variable $images plus the string /*.
Probably what you're looking for instead is this:
$dir = "images/*";
As a general habit, I prefer single quotes in PHP where possible, as it tends to surface this kind of easily-masked error more efficiently, because the single quoted $ symbol will be interpreted as a string.
Related
I want to replace the text in the file and overwrite file.
use strict;
use warnings;
my ($str1, $str2, $i, $all_line);
$str1 = "AAA";
$str2 = "bbb";
open(RP, "+>", "testfile") ;
$all_line = $_;
$i = 0;
while(<RP>) {
while(/$str1/) {
$i++;
}
s/$str1/$str2/g;
print RP $_;
}
close(RP);
A normal process is to read the file line by line and write out each line, changed as/if needed, to a new file. Once that's all done rename that new file, atomically as much as possible, so to overwrite the orgiinal.
Here is an example of a library that does all that for us, Path::Tiny
use warnings;
use strict;
use feature 'say';
use Path::Tiny;
my $file = shift || die "Usage: $0 file\n";
say "File to process:";
say path($file)->slurp;
# NOTE: This CHANGES THE INPUT FILE
#
# Process each line: upper-case a letter after .
path($file)->edit_lines( sub { s/\.\s+\K([a-z])/\U$1/g } );
say "File now:";
say path($file)->slurp;
This upper-cases a letter following a dot (period), after some spaces, on each line where it is found and copies all other lines unchanged. (It's just an example, not a proper linguistic fix.)
Note: the input file is edited in-place so it will have been changed after this is done.
This capability was introduced in the module's version 0.077, of 2016-02-10. (For reference, Perl version 5.24 came in 2016-08. So with the system Perl 5.24 or newer, a Path::Tiny installed from an OS package or as a default CPAN version should have this method.)
Perl has a built-in in-place editing facility: The -i command line switch. You can access the functionality of this switch via $^I.
use strict;
use warnings;
my $str1 = "AAA";
my $str2 = "bbb";
local $^I = ''; # Same as `perl -i`. For `-i~`, assign `"~"`.
local #ARGV = "testfile";
while (<>) {
s/\Q$str1/$str2/g;
print;
}
I was not going to leave an answer, but I discovered when leaving a comment that I did have some feedback for you, so here goes.
open(RP, "+>", "testfile") ;
The mode "+>" will truncate your file, delete all it's content. This is described in the documentation for open:
You can put a + in front of the > or < to indicate that you want both read and write access to the file; thus +< is almost always preferred for read/write updates--the +> mode would clobber the file first. You can't usually use either read-write mode for updating textfiles, since they have variable-length records. See the -i switch in perlrun for a better approach.
So naturally, you can't read from the file if you first delete it's content. They mention here the -i switch, which is described like this in perl -h:
-i[extension] edit <> files in place (makes backup if extension supplied)
This is what ikegami's answer describes, only in his case it is done from within a program file rather than on the command line.
But, using the + mode for both reading and writing is not really a good way to do it. Basically it becomes difficult to print where you want to print. The recommended way is to instead read from one file, and then print to another. After the editing is done, you can rename and delete files as required. And this is exactly what the -i switch does for you. It is a predefined functionality of Perl. Read more about it in perldoc perlrun.
Next, you should use a lexical file handle. E.g. my $fh, instead of a global. And you should also check the return value from the open, to make sure there was not an error. Which gives us:
open my $fh, "<", "testfile" or die "Cannot open 'testfile': $!";
Usually if open fails, you want the program to die, and report the reason it failed. The error is in the $! variable.
Another thing to note is that you should not declare all your variables at the top of the file. It is good that you use use strict; use warnings, Perl code should never be written without them, but this is not how you handle it. You declare your variable as close to the place you use the variable as possible, and in the smallest scope possible. With a my declared variable, that is the nearest enclosing block { ... }. This will make it easy to trace your variable in bigger programs, and it will encapsulate your code and protect your variable.
In your case, you would simply put the my before all the variables, like so:
my $str1 = "AAA";
my $str2 = "bbb";
my $all_line = $_;
my $i = 0;
Note that $_ will be empty/undefined there, so that assignment is kind of pointless. If your intent was to use $all_line as the loop variable, you would do:
while (my $all_line = <$fh>) {
Note that this variable is declared in the smallest scope possible, and we are using a lexical file handle $fh.
Another important note is that your replacement strings can contain regex meta characters. Sometimes you want them to be included, like for example:
my $str1 = "a+"; # match one or more 'a'
Sometimes you don't want that:
my $str1 = "google.com"; # '.' is meant as a period, not the "match anything" character
I will assume that you most often do not want that, in which case you should use the escape sequence \Q ... \E which disables regex meta characters inside it.
So what do we get if we put all this together? Well, you might get something like this:
use strict;
use warnings;
my $str1 = "AAA";
my $str2 = "bbb";
my $filename = shift || "testfile"; # 'testfile', or whatever the program argument is
open my $fh_in, "<", $filename or die "Cannot open '$filename': $!";
open my $fh_out, ">", "$filename.out" or die "Cannot open '$filename.out': $!";
while (<$fh_in>) { # read from input file...
s/\Q$str1\E/$str2/g; # perform substitution...
print $fh_out $_; # print to output file
}
close $fh_in;
close $fh_out;
After this finishes, you may choose to rename the files and delete one or the other. This is basically the same procedure as using the -i switch, except here we do it explicitly.
rename $filename, "$filename.bak"; # save backup of input file in .bak extension
rename "$filename.out", $filename; # clobber the input file
Renaming files is sometimes also facilitated by the File::Copy module, which is a core module.
With all this said, you can replace all your code with this:
perl -i -pe's/AAA/bbb/g' testfile
And this is the power of the -i switch.
Consider the code (the variable $i is there because it was in a loop, adding several conditions to the pattern, e.g. *.a and *.b, ... but to illustrate this problem only one wildcard pattern is enough):
#!/bin/bash
i="a"
PATTERN="-name bar -or -name *.$i"
find . \( $PATTERN \)
If ran on a folder containing files bar and foo.a, it works, outputting:
./foo.a
./bar
But if you now add a new file to the folder, namely zoo.a, then it no longer works:
find: paths must precede expression: zoo.a
Presumably, because the wildcard in *.$i gets expanded by the shell to foo.a zoo.a, which leads to an invalid find command pattern. So one attempt at a fix is to put quotes around the wildcard pattern. Except it does not work:
with single quotes -- PATTERN="-name bar -or -name '*.$i'" the find command outputs only bar. Escaping the single quotes (\') yields the same result.
idem with double quotes: PATTERN="-name bar -or -name \"*.$i\"" -- only bar is returned.
in the find command, if $PATTERN is replaced with "$PATTERN", out comes an error (for single quotes same error, but with single quotes around the wildcard pattern):
find: unknown predicate -name bar -or -name "*.a"'
Of course, replacing $PATTERN with '$PATTERN' also does not work... (no expansion whatsoever takes place).
The only way I could get it to work was to use... eval!
FINDSTR="find . \( $PATTERN \)"
eval $FINDSTR
This works properly:
./zoo.a
./foo.a
./bar
Now after a lot of googling, I saw it mentioned several times that to do this kind of thing, one should use arrays. But this doesn't work:
i="a"
PATTERN=( -name bar -or -name '*.$i' )
find . \( "${PATTERN[#]}" \)
# result: ./bar
In the find line the array has to be enclosed in double quotes, because we want it to be expanded. But single quotes around the wildcard expression don't work, and neither does not quotes at all:
i="a"
PATTERN=( -name bar -or -name *.$i )
find . \( "${PATTERN[#]}" \)
# result: find: paths must precede expression: zoo.a
BUT DOUBLE QUOTES DO WORK!!
i="a"
PATTERN=( -name bar -or -name "*.$i" )
find . \( "${PATTERN[#]}" \)
# result:
# ./zoo.a
# ./foo.a
# ./bar
So I guess my question are actually two questions:
a) in this last example using arrays, why are double quotes required around the *.$i?
b) using an array in this way is supposed to expand «to all elements individually quoted». How would do this with a variable (cf my first attempt)? After getting this to function, I went back and tried using a variable again, with blackslashed single quotes, or \\', but nothing worked (I just got bar). What would I have to do to emulate "by hand" as it were, the quoting done when using arrays?
Thank you in advance for your help.
Required reading:
BashFAQ — I'm trying to put a command in a variable, but the complex cases always fail!
a) in this last example using arrays, why are double quotes required around the *.$i?
You need to use some form of quoting to prevent the shell from performing glob expansion on *. Variables are not expanded in single quotes so '*.$i' doesn't work. It does inhibit glob expansion but it also stops variable expansion. "*.$i" inhibits glob expansion but allows variable expansion, which is perfect.
To really delve into the details, there are two things you need to do here:
Escape or quote * to prevent glob expansion.
Treat $i as a variable expansion, but quote it to prevent word splitting and glob expansion.
Any form of quoting will do for item 1: \*, "*", '*', and $'*' are all acceptable ways to ensure it's treated as a literal asterisk.
For item 2, double quoting is the only answer. A bare $i is subject to word splitting and globbing -- if you have i='foo bar' or i='foo*' the whitespace and globs will cause problems. \$i and '$i' both treat the dollar sign literally, so they're out.
"$i" is the only quoting that does everything right. It's why common shell advice is to always double quote variable expansions.
The end result is, any of the following would work:
"*.$i"
\*."$i"
'*'."$i"
"*"."$i"
'*.'"$i"
Clearly, the first is the simplest.
b) using an array in this way is supposed to expand «to all elements individually quoted». How would do this with a variable (cf my first attempt)? After getting this to function, I went back and tried using a variable again, with blackslashed single quotes, or \\', but nothing worked (I just got bar). What would I have to do to emulate "by hand" as it were, the quoting done when using arrays?
You'd have to cobble together something with eval, but that's dangerous. Fundamentally, arrays are more powerful than simple string variables. There's no magic combination of quotes and backslashes that will let you do what an array can do. Arrays are the right tool for the job.
Could you explain in a little more detail, why ... PATTERN="-name bar -or -name \"*.$i\"" does not work? The quoted double quotes should, when the find command is actually ran, expand the $i but not the glob.
Sure. Let's say we write:
i=a
PATTERN="-name bar -or -name \"*.$i\""
find . \( $PATTERN \)
After the first two line runs, what is the value of $PATTERN? Let's check:
$ i=a
$ PATTERN="-name bar -or -name \"*.$i\""
$ printf '%s\n' "$PATTERN"
-name bar -or -name "*.a"
You'll notice that $i has already been replaced with a, and the backslashes have been removed.
Now let's see how exactly the find command is parsed. In the last line $PATTERN is unquoted because we want all the words to be split apart, right? If you write a bare variable name Bash ends up performing an implied split+glob operation. It performs word splitting and glob expansion. What does that mean, exactly?
Let's take a look at how Bash performs command-line expansion. In the Bash man page under the "Expansion" section we can see the order of operations:
Brace expansion
Tilde expansion, parameter and variable expansion, arithmetic expansion, command substitution, and process substitution
Word splitting
Pathname (AKA glob) expansion
Quote removal
Let's run through these operations by hand and see how find . \( $PATTERN \) is parsed. The end result will be a list of strings, so I'll use a JSON-like syntax to show each stage. We'll start with a list containing a single string:
['find . \( $PATTERN \)']
As a preliminary step, the command-line as a whole is subject to word splitting.
['find', '.', '\(', '$PATTERN', '\)']
Brace expansion -- No change.
Variable expansion
['find', '.', '\(', '-name bar -or -name "*.a"', '\)']
$PATTERN is replaced. For the moment it is all a single string, whitespace and all.
Word splitting
['find', '.', '\(', '-name', 'bar', '-or', '-name', '"*.a"', '\)']
The shell scans the results of variable expansion that did not occur within double quotes for word splitting. $PATTERN was unquoted, so it's expanded. Now it is a bunch of individual words. So far so good.
Glob expansion
['find', '.', '\(', '-name', 'bar', '-or', '-name', '"*.a"', '\)']
Bash scans the results of word splitting for globs. Not the entire command-line, just the tokens -name, bar, -or, -name, and "*.a".
It looks like nothing happened, yes? Not so fast! Looks can be deceiving. Bash actually performed glob expansion. It just happened that the glob didn't match anything. But it could...†
Quote removal
['find', '.', '(', '-name', 'bar', '-or', '-name', '"*.a"', ')']
The backslashes are gone. But the double quotes are still there.
After the preceding expansions, all unquoted occurrences of the characters \, ', and " that did not result from one of the above expansions are removed.
And that's the end result. The double quotes are still there, so instead of searching for files named *.a it searches for ones named "*.a" with literal double quotes characters in their name. That search is bound to fail.
Adding a pair of escaped quotes \" didn't at all do what we wanted. The quotes didn't disappear like they were supposed to and broke the search. Not only that, but they also didn't inhibit globbing like they should have.
TL;DR — Quotes inside a variable aren't parsed the same way as quotes outside a variable.
† The first four tokens have no special characters. But the last one, "*.a", does. That asterisk is a wildcard. If you read the "pathname expansion" section of the man page carefully you'll see that there's no mention of quotes being ignored. The double quotes do not protect the asterisk.
Hang on! What? I thought quotes inhibit glob expansion!
They do—normally. If you write quotes out by hand they do indeed stop glob expansion. But if you put them inside an unquoted variable, they don't.
$ touch 'foobar' '"foobar"'
$ ls
foobar "foobar"
$ ls foo*
foobar
$ ls "foo*"
ls: foo*: No such file or directory
$ var="\"foo*\""
$ echo "$var"
"foo*"
$ ls $var
"foobar"
Read that over carefully. If we create a file named "foobar"—that is, it has literal double quotes in its filename—then ls $var prints "foobar". The glob is expanded and matches the (admittedly contrived) filename!
Why didn't the quotes help? Well, the explanation is subtle, and tricky. The man page says:
After word splitting ... bash scans each word for the characters *, ?, and [.
Any time Bash performs word splitting it also expands globs. Remember how I said unquoted variables are subject to an implied split+glob operator? This is what I meant. Splitting and globbing go hand in hand.
If you write ls "foo*" the quotes prevent foo* from being subject to splitting and globbing. However if you write ls $var then $var is expanded, split, and globbed. It wasn't surrounded by double quotes. It doesn't matter that it contains double quotes. By the time those double quotes show up it's too late. Word splitting has already been performed, and so globbing is done as well.
I have one ini configuration file, I need to create a shell script using this configuration. What is the easiest method to access all variable, Can be used effectively from the shell script.
Can I use an array or something? Now planing to find the count of [] brackets then through awk get all variables one by one. Please suggest if any easiest way to effectively
cat app.ini
Below the output of my sample configuration file. Can be N no of Blocks.
[APP1]
name=Application1
StatusScript=/home/status_APP1.sh
startScript=/home/start_APP1.sh
stopScript=/home/stop_APP1.sh
restartScript=/home/restart.APP1.sh
logdir=/log/APP1/
[APP2]
name=Application2
StatusScript=/home/status_APP2.sh
startScript=/home/start_APP2.sh
stopScript=/home/stop_APP2.sh
restartScript=/home/restart.APP2.sh
logdir=/log/APP2/
.
.
.
.
.
[APPN]
name=ApplicationN
StatusScript=/home/status_APPN.sh
startScript=/home/start_APPN.sh
stopScript=/home/stop_APPN.sh
restartScript=/home/restart.APPN.sh
logdir=/log/APPN
/
Consider using a library like bash-ini-parser https://github.com/albfan/bash-ini-parser. It covers a lot of nuances like indentation, whitespaces, comments etc.
The example for your case may look like this:
#!/bin/bash
. bash-ini-parser
cfg_parser app.ini
cfg_section_APP1
echo $name
cfg_section_APP2
echo $logdir
cfg_section_APPN
echo $logdir
Below line help us to locate particular values in each section.
sed -nr "/^\[APP1\]/ { :l /^name[ ]*=/ { s/.*=[ ]*//; p; q;}; n; b l;}" app.ini
In bash I have an array with path names, and I would like to replace each of them with different ones using sed, like so:
sed 's#^(.*?)master_repo(.*?)#\1"${SOME_REPO_NAME}"\2#g' <<< ${FULL_TGT_DIRS[${i}]}
A sample path name which is an element of the array would be:
/Volumes/munki/master_repo/pkgs/apps
I would like to replace the path name "master_repo" with for example "somedir", which is stored in $SOME_REPO_NAME, so I get:
/Volumes/munki/somedir/pkgs/apps
Or with built in string substitution:
for i in ${FULL_TGT_DIRS[#]}
do
FULL_TGT_DIRS[$i]=${FULL_TGT_DIRS[$i]/master_repo/$SOME_REPO_NAME}
#sed 's#^(.*?)master_repo(.*?)#\1"${SOME_REPO_NAME}"\2#g' <<< ${FULL_TGT_DIRS[${i}]}
done
I always get the following error when running my script:
> /usr/local/bin/repomgr: line 135:
> /Volumes/munki/master_repo/pkgs/apps: syntax error: operand expected
> (error token is "/Volumes/munki/master_repo/pkgs/apps")
I've tried using different separaters and sed options, as well as shuffling through different quote constellations. I don't write bash scripts on a daily basis so perhaps I'm missing something?
BTW, I run this on a Mac and therefore only have bash 3.2 at my disposal.
There's no need to use sed for this, bash has built-in string replacement in its parameter expansion.
var=/Volumes/munki/master_repo/pkgs/apps
$SOME_REPO_NAME=somedir
newvar=${var/master_repo/$SOME_REPO_NAME}
In a for-in loop, the variable gets set to the array elements, not the array indexes, so you shouldn't be using FULL_TGT_DIRS[$i] -- $i contains the pathname. So the loop should be:
for file in ${FULL_TGT_DIRS[#]}
do
file=${file/master_repo/$SOME_REPO_NAME}
# Do something with $file here
done
If you need to modify the array in place, you need a different loop for the indexes:
for ((i = 0; i < ${#FULL_TGT_DIRS[#]}; i++))
do
FULL_TGT_DIRS[$i]=${FULL_TGT_DIRS[$i]/master_repo/"$SOME_REPO_NAME"}
done
You can even go a step further using bashes own replacement:
for file in "${FULL_TGT_DIRS[#]/master_repo/somedir}"
do
...work on file variable here...
done
I have a simple enough problem I think, I have recently ran a script which extracted specific information from the string in each element in an array. I have written this before and it functions well however when trying the very simple version of it right now it will not presen data only the same response uninitialized value argument! I am getting really frustrated as my previous code works. I am clearly doing something STUPID and would love some help!
#!/usr/bin/env perl
use strict;
use warnings;
my#histone;
my$line;
my$idea;
my$file="demo_site.txt";
open(IN, "<$file")||die"\ncannot be opend\n";
#histone=<IN>;
print #histone;
foreach $line(#histone)
{
$line=~ m/([a-zA-Z0-9]+)\t[0-9]+\t[0-9]+\t/;
print$1."\n";
print$2."\n";
print$3."\n";
}
The infile "demo_site.txt" takes the format of a tab delimited .txt file:
chr9 1234 5678 . 200 . 14.0 -1
This file has multiple lines as above and I wish to extract the first three items of data so the output looks as follows.
chr9
1234
5678
Cheers!
You don't really need a regular expression since it's tab delimited.
foreach $line(#histone)
{
#line_data = split(/\t/,$line)
print $line_data[0]."\n";
print $line_data[1]."\n";
print $line_data[2]."\n";
}
Edit:
If you want to assign the values to specific named variables, assign it in a temporary array.
($varA, $varB, $varC .... ) = split(/\t/,$line)
The actual problem here is that you're trying to print the values of $1, $2 and $3, but you only have one set of capturing parenthesis in your regex, so only $1 gets a value. $2 and $3 will remain undefined and hence give you that error when you try to print them.
The solution is to add two more sets of capturing parenthesis. I expect you want something like this:
$line=~ m/([a-zA-Z0-9]+)\t([0-9]+)\t([0-9]+)\t/;
Let's assume, that file.txt have what you want: (file.txt eq demo_site.txt )
chr9 1234 5678 . 200 . 14.0 -1
you can use simple thing:
perl -ane '$" = "\n"; print "#F[0..2]"' file.txt 1>output.txt
One-liners in Perl are powerful. And you don't need to write your scripts for simple tasks;)
Just open Terminal sometimes;)
P.S:
This is not very good one-liner, I know, but It do what It must.
$line=~ m/([a-zA-Z0-9]+)\t[0-9]+\t[0-9]+\t/)
First of all, the parens are not balanced.
Second, I haven't checked this, but don't you need a set of parens for each capture?
Third, as misplacedme said split() is definitely the way to go. ;)
If I may self-promote, you can use Tie::Array::CSV to give direct read-write access to the file as a Perl array of arrayrefs.
use strict;
use warnings;
use Tie::Array::CSV;
tie my #file, 'Tie::Array::CSV', 'demo_site.txt', sep_char => "\t";
print $file[0][0]; # first line before first tab
$file[2][1] = 10; # set the third line between the first and second tabs