Creating a JSON array containing multiple variables - arrays

I want to modify my Perl script to output a list of variables using the json_encode function, but I'm not sure how.
Here's the output of my unmodified Perl script:
Vicia_sativa = Vicia_sativa.png
Geranium_maculatum = Geranium_maculatum.png
Narcissus_pseudonarcissus = Narcissus_pseudonarcissus1.png Narcissus_pseudonarcissus2.png
Polygonum_persicaria = Polygonum_persicaria1.png Polygonum_persicaria2.png
Corylus_americana = Corylus_americana1.png Corylus_americana2.png
The variables to the left of the equal signs are plant names, and the one or more file names to the right of the equal signs are plant photos. Notice that these file names are not separated by commas.
Here's my Perl script that generated the above output:
#!/usr/bin/perl
use strict;
use warnings;
use English; ## use names rather than symbols for special variables
my $dir = '/Users/jdm/Desktop/xampp/htdocs/cnc/images/plants';
opendir my $dfh, $dir or die "Can't open $dir: $OS_ERROR";
my %genus_species; ## store matching entries in a hash
for my $file (readdir $dfh)
{
next unless $file =~ /.png$/; ## entry must have .png extension
my $genus = $file =~ s/\d*\.png$//r;
push(#{$genus_species{$genus}}, $file); ## push to array,the #{} is to cast the single entry to a reference to an list
}
for my $genus (keys %genus_species)
{
print "$genus = ";
print "$_ " for sort #{$genus_species{$genus}}; # sort and loop though entries in list reference
print "\n";
}
Please advise how to output these variables in a JSON array. Thanks.
Update... Here's the revised script with the recommended changes per a forum member:
#!/usr/bin/perl
use strict;
use warnings;
use JSON::PP;
use English; ## use names rather than symbols for special variables
my $dir = '/Users/jdm/Desktop/xampp/htdocs/cnc/images/plants';
opendir my $dfh, $dir or die "Can't open $dir: $OS_ERROR";
my %genus_species; ## store matching entries in a hash
for my $file (readdir $dfh)
{
next unless $file =~ /.png$/; ## entry must have .png extension
my $genus = $file =~ s/\d*\.png$//r;
push(#{$genus_species{$genus}}, $file); ## push to array,the #{} is to cast the single entry to a reference to an list
}
print(encode_json(\%genus_species));
This revised code works! However, the file names are no longer sorted. Any ideas how to incorporate sort into the encode_json function?

Pretty straight forward actually... You can use the JSON::PP module and pass a reference of your hash to encode_json().
#!/usr/bin/perl
use strict;
use warnings;
use JSON::PP;
# your other code goes here...
# instead of `for my $genus (keys %genus_species) { ... }` do:
print(encode_json(\%genus_species));

Related

Removing file extension from an array variable

I'm trying to remove the .png file extension that appears in many (but not all) of the variables of my outputted array. The array variables that show the extension are doing so because they weren't generated from file names in the format of "Genus_species#.png" where "#" is a number. Rather, they were generated from an un-numbered file name in the format of "Genus_species.png". I believe this line of code is creating this issue: "$genus = $file =~ s/\d.png$//r;". How do I resolve this? Please advise.
Here's my Perl script:
#!/usr/bin/perl
use strict;
use warnings;
use English; ## use names rather than symbols for special varables
my $dir = '/Users/jdm/Desktop/xampp/htdocs/cnc/images/plants';
opendir my $dfh, $dir or die "Can't open $dir: $OS_ERROR";
my %genus_species; ## store matching entries in a hash
for my $file (readdir $dfh)
{
next unless $file =~ /.png$/; ## entry must have .png extension
my $genus = $file =~ s/\d\.png$//r;
push(#{$genus_species{$genus}}, $file); ## push to array,the #{} is to cast the single entry to a referance to an list
}
for my $genus (keys %genus_species)
{
print "$genus = ";
print "$_, " for sort #{$genus_species{$genus}}; # sort and loop though entries in list referance
print "\n";
}
Here's the outputted array:
Euonymus_fortunei = Euonymus_fortunei1.png, Euonymus_fortunei2.png, Euonymus_fortunei3.png,
Polygonum_persicaria = Polygonum_persicaria1.png, Polygonum_persicaria2.png,
Polygonum_cuspidatum.png = Polygonum_cuspidatum.png,
Notice that the variable "Polygonum_cuspidatum.png" unwantingly includes the file extension because this variable was generated from a file that lacked a number in its name. Specifically, this variable should read:
Polygonum_cuspidatum = Polygonum_cuspidatum.png
Again, please advise how to resolve this issue. Thanks.
You're going to see the same issue if you ever have a multi-digit number in a filename. This is all due to the choice of regular expression:
s/\d\.png$//r
This looks for exactly one digit followed by .png. If you want no digit, or any number of digits before .png modify your regular expression as such:
s/\d*\.png$//r
That says "zero or more digits followed by .png at the end of the string".

Creating array of file names using grep

I'm having difficulty outputting file names as an array using grep. Specifically, I want to create an array of file names (plant photos) formatted like this:
Ilex_verticillata= Ilex_verticillata1.png, Ilex_verticillata2.png, Ilex_verticillata3.png
Asarum_canadense= Asarum_canadense1.png, Asarum_canadense2.png
Ageratina_altissi= Ageratina_altissi1.png, Ageratina_altissi2.png
Here's my original Perl script that I'm attempting to modify. It returns, as intended, ONE file name per plant as "Genus_species", printing a list of those plants:
#!/usr/bin/perl
use strict;
use warnings;
my $dir = '/Users/jdm/Desktop/xampp/htdocs/cnc/images/plants';
opendir my $dfh, $dir or die "Can't open $dir: $!";
my #files =
map { s/1\.png\z/.png/r } # Removes "1" from end of file names
grep { /^[^2-9]*\.png\z/i && /_/ } # Finds "Genus_species.png" & "Genus_species1.png" and returns one file name per plant as "Genus_species.png"
readdir $dfh;
foreach my$file (#files) {
$file =~s/\.png//; # Removes ".png" extension
print "$file\n"; #Prints list of file names (plant names)
}
Here's the output:
Ilex_verticillata
Asarum_canadense
Ageratina_altissima
However, since each plant often has MULTIPLE photos (e.g.-- "Genus_species1.png, Genus_species2.png, etc.), I need to re-grep the directory using the above output to find their file names, then output the results in the form of an array as previously illustrated.
I know the solution likely involves modifying the "foreach" statement, using grep to return ALL file names with "Genus_species" in their name. Here's what I tried:
foreach my$file (#files) {
$file =~s/\.png//;
grep ($file,readdir(DIR));
print "$file = $file\n";
But the output was this:
Ilex_verticillata = Ilex_verticillata
Asarum_canadense = Asarum_canadense
Ageratina_altissima = Ageratina_altissima
Again, I want to output an array formatted as:
"Genus_species= Genus_species1.png, Genus_species2.png, etc.," meaning I want it to look like this:
Ilex_verticillata= Ilex_verticillata1.png, Ilex_verticillata2.png, Ilex_verticillata3.png
Asarum_canadense= Asarum_canadense1.png, Asarum_canadense2.png
Ageratina_altissi= Ageratina_altissi1.png, Ageratina_altissi2.png
Notice that I also want to add back the ".png" extension ONLY to the file names to the right of the equals sign.
Please advise. Thanks.
Readdir returns a list of files in the folder. You've put them on one line, which is compact. However, if you loop them you can process the items further.
#!/usr/bin/perl
use strict;
use warnings;
use English; ## use names rather than symbols for special varables
my $dir = '/Users/jdm/Desktop/xampp/htdocs/cnc/images/plants';
opendir my $dfh, $dir or die "Can't open $dir: $OS_ERROR";
my %genus_species; ## store matching entries in a hash
for my $file (readdir $dfh)
{
next unless $file =~ /\d\.png$/; ## skip entry if not a png file ending with a number
my $genus = $file =~ s/\d\.png$//r;
push(#{$genus_species{$genus}}, $file); ## push to array,the #{} is to cast the single entry to a referance to an list
}
for my $genus (keys %genus_species)
{
print "$genus = ";
print "$_ " for sort #{$genus_species{$genus}}; # sort and loop though entries in list referance
print "\n";
}

Need json_encode to return sorted array

I need json_encode to return a sorted array, but can't figure out how.
Here's my Perl script:
#!/usr/bin/perl
use strict;
use warnings;
use JSON::PP;
use English; ## use names rather than symbols for special variables
my $dir = '/Users/jdm/Desktop/xampp/htdocs/cnc/images/plants';
opendir my $dfh, $dir or die "Can't open $dir: $OS_ERROR";
my %genus_species; ## store matching entries in a hash
for my $file (readdir $dfh)
{
next unless $file =~ /.png$/; ## entry must have .png extension
my $genus = $file =~ s/\d*\.png$//r;
push(#{$genus_species{$genus}}, $file); ## push to array,the #{} is to cast the single entry to a reference to an list
}
print "var galleryarray = "; ## HTML Variable element ( <var> )
print (encode_json(\%genus_species)); ## define array in Javascript outputting elements containing image file names
Here's part of the output, showing the unsorted elements:
var galleryarray = {"Polygonum_pensylvanicum":["Polygonum_pensylvanicum2.png","Polygonum_pensylvanicum3.png","Polygonum_pensylvanicum1.png"]
Notice that the indexed file names are unsorted numerically.
Before posting, I tried adding the following sort function below the push function:
sort(#{$genus_species{$genus}}, $file);
Unfortunately, that caused an error; specifically "Useless use of sort in void context."
Please advise how I can output a sorted array using json_encode. Thanks.
The following is the proper syntax:
#{$genus_species{$genus}} = sort #{$genus_species{$genus}};
It's inefficient to repeatedly sort the arrays. Instead, you should create a second loop after the first.
for my $genus (keys(%genus_species)) {
#{$genus_species{$genus}} = sort #{$genus_species{$genus}};
}
or
#{$genus_species{$_}} = sort #{$genus_species{$_}}
for keys(%genus_species);
But your strings have a mix of text and numbers, so a natural sort would be better.
use Sort::Key::Natural qw( natsort );
#{$genus_species{$_}} = natsort #{$genus_species{$_}}
for keys(%genus_species);
If you want to the keys to be sorted too, replace
encode_json(...)
with
JSON::PP->new->utf8->canonical->encode(...)
or the faster
Cpanel::JSON::XS->new->utf8->canonical->encode(...)
---UPDATE--- One of our forum members provided a sort expression that allows the json_encode function to return a sorted array. I want to share his solution to help others.
Here's my revised script that returns a sorted array:
#!/usr/bin/perl
use strict;
use warnings;
use JSON::PP;
use English; ## use names rather than symbols for special variables
my $dir = '/Users/jdm/Desktop/xampp/htdocs/cnc/images/plants';
opendir my $dfh, $dir or die "Can't open $dir: $OS_ERROR";
my %genus_species; ## store matching entries in a hash
for my $file (readdir $dfh)
{
next unless $file =~ /.png$/; ## entry must have .png extension
my $genus = $file =~ s/\d*\.png$//r;
push(#{$genus_species{$genus}}, $file); ## push to array,the #{} is to cast the single entry to a reference to an list
}
#{$genus_species{$_}} = sort #{$genus_species{$_}}
for keys(%genus_species);
print (JSON::PP->new->utf8->canonical->encode(\%genus_species)); ## define array in Javascript outputting elements containing image file names

Advanced sorting of arrays

I'm having difficulty alphabetically sorting the output of my Perl script.
Here's the script:
#!/usr/bin/perl
use strict;
use warnings;
use English; ## use names rather than symbols for special variables
my $dir = '/Users/jdm/Desktop/xampp/htdocs/cnc/images/plants';
opendir my $dfh, $dir or die "Can't open $dir: $OS_ERROR";
my %genus_species; ## store matching entries in a hash
for my $file (readdir $dfh)
{
next unless $file =~ /.png$/; ## entry must have .png extension
my $genus = $file =~ s/\d*\.png$//r;
push(#{$genus_species{$genus}}, $file); ## push to array,the #{} is to cast the single entry to a reference to an list
}
for my $genus (keys %genus_species)
{
print "$genus = ";
print "$_ " for sort #{$genus_species{$genus}}; # sort and loop though entries in list reference
print "\n";
}
Here's the un-sorted output:
Veronica_chamaedrys = Veronica_chamaedrys.png
Cardamine_douglassii = Cardamine_douglassii1.png Cardamine_douglassii2.png
Filipendula_rubra = Filipendula_rubra1.png Filipendula_rubra2.png
Taxodium_distichum = Taxodium_distichum.png
Asplenium_platyneuron = Asplenium_platyneuron1.png Asplenium_platyneuron2.png Asplenium_platyneuron3.png
Here's the desired sorted output:
Asplenium_platyneuron = Asplenium_platyneuron1.png Asplenium_platyneuron2.png Asplenium_platyneuron3.png
Cardamine_douglassii = Cardamine_douglassii1.png Cardamine_douglassii2.png
Filipendula_rubra = Filipendula_rubra1.png Filipendula_rubra2.png
Taxodium_distichum = Taxodium_distichum.png
Veronica_chamaedrys = Veronica_chamaedrys.png
Please advise. Thanks.
Replace
for my $genus (keys %genus_species)
with
for my $genus (sort keys %genus_species)
If your numbers are going to reach 10, you'll want to use natsort from Sort::Keys::Natural (1, 2, ..., 9, 10, ...) instead of the builtin sort (1, 10, 11, ..., 2, ...), at least for the one sorting the file names.

Perl combine multiple file contents to single file

I have multiple log files say file1.log file2.log file3.log etc. I want to combine these files contents and put it into single file called result_file.log
Is there any Perl module which can achieve this?
Update: Here is my code
#!/usr/bin/perl
use strict;
use warnings;
use Data::Dumper;
use File::Copy;
my #files;
my $dir = "/path/to/directory";
opendir(DIR, $dir) or die $!;
while (my $file = readdir(DIR)) {
# We only want files
next unless (-f "$dir/$file");
# Use a regular expression to find files ending in .log
next unless ($file =~ m/\.log$/);
print "$file\n";
push( #files, $file);
}
closedir(DIR);
print Dumper(\#files);
open my $out_file, ">result_file.log" ;
copy($_, $out_file) foreach ( #files );
exit 0;
Do you think it is feasible solution?
CPAN 'File::Copy' should do the work, you will have to open the output file youself.
use File::Copy ;
open my $out, ">result.log" ;
copy($_, $out) foreach ('file1.log', 'file2.log', );
close $out ;
Update 1:
Based on additional information posted to answer, looks like the ask is to concatenate (in Perl) list of files match a pattern (*.log). Below extends the above solution to include additional logic, using glob, avoiding the readdir and filtering.
use File::Copy ;
open my $out, ">result.log" ;
copy($_, $out) foreach glob('/path/to/dir/*.log' );
close $out ;
Important notes:
* Using glob will SORT the file name alphabetically, while readdir does NOT guarantee any order.
* The output file 'result.log' match '*.log', should not execute the code in the current directory.
Do you think it is feasible solution?
I'm afraid not. Your code is the equivalent of typing these commands at your prompt:
$ cp file1.log result_file.log
$ cp file2.log result_file.log
$ cp file3.log result_file.log
$ ... etc ...
The problem with this is that it copies each file, in turn over the top of the previous one. So you end up with a copy of the final file in the list.
As I said in a comment, this is most easily done using cay - no need for Perl at all.
$ cat file1.log file2.log file3.log > result_file.log
If you really want to do it in Perl, then something like this would work (the first section is rather similar to yours).
#!/usr/bin/perl
use strict;
use warnings;
use Data::Dumper;
my #files;
my $dir = "/path/to/directory";
opendir(my $dh, $dir) or die $!;
while (my $file = readdir($dh)) {
# We only want files
next unless (-f "$dir/$file");
# Use a regular expression to find files ending in .log
next unless ($file =~ m/\.log$/);
print "$file\n";
push( #files, "$dir/$file");
}
closedir($dh);
print Dumper(\#files);
open my $out_file, '>', 'result_file.log';
foreach my $fn (#files) {
open my $in_file, '<', $fn or die "$fn: $!";
print $out_file while <$fn>);
}

Resources