There are so many modules for serializing data for Perl, and I don't know which one to choose.
I've the following data that I need to serialize as a string so I can put it in the database:
my #categories = (
["Education", "Higher Education", "Colleges"],
["Schooling", "Colleges"]
);
How could I turn it into text, and then later when I need it, turn back into an array of array-references?
I vote for JSON (or Data::Serializer as mentioned in another answer, in conjunction with JSON).
The JSON module is plenty fast and efficient (if you install JSON::XS from cpan, it will compile the C version for you, and use JSON will automatically use that).
It works great with Perl data structures, is standardized, and the Javascript syntax is so similar to Perl syntax. There are options you can set with the JSON module to improve human readability (linebreaks, etc.)
I've also used Storable. I don't like it--the interface is weird, and the output is nonsensical, and it is a proprietary format. Data::Dumper is fast and quite readable but is really meant to be one-way (evaling it is slightly hackish), and again, it's Perl only. I've also rolled my own as well. In the end, I concluded JSON is the best, is fast, flexible, and robust.
You can use Data::Serializer:
Examples/Information from OnPerl.net
Data::Serializer Module from CPAN
You could roll your own, but you have to worry about tricky issues such as escaping quotes and backslashes or the separators you choose.
The program below shows how you can use standard Perl modules Data::Dumper and Storable to serialize and deserialize your data in a way that is suitable for storing in a database.
#! /usr/bin/env perl
use strict;
use warnings;
use Data::Dumper;
use Storable qw/ nfreeze thaw /;
use Test::More tests => 2;
my #categories = (
["Education", "Higher Education", "Colleges"],
["Schooling", "Colleges"]
);
{
local $Data::Dumper::Indent = 0;
local $Data::Dumper::Terse = 1;
my $serialized = Dumper \#categories;
print $serialized, "\n";
my $restored = eval($serialized) || die "deserialization failed: $#";
is_deeply $restored, \#categories;
}
{
my $serialized = unpack "H*", nfreeze \#categories;
print $serialized, "\n";
my $restored = thaw pack "H*", $serialized;
die "deserialization failed: $#" unless defined $restored;
is_deeply $restored, \#categories;
}
Data::Dumper has the nice property of being human readable, but the severe negative of requiring eval to deserialize. Storable is nice and compact but opaque.
Related
Getting an error when I attempt to dump out part of a multi dimensional hash array. Perl spits out
Can't use string ("somedata") as an ARRAY ref while "strict refs" in use at ./myscript.pl
I have tried multiple ways to access part of the array I want to see but I always get an error. I've used Dumper to see the entire array and it looks fine.
#!/usr/bin/perl
use strict;
use warnings;
use Data::Dumper qw(Dumper);
use String::Util qw(trim);
my %arrHosts;
open(my $filehdl, "<textfile.txt") || die "Cannot open or find file textfile.txt: $!\n";
while( my $strInputline = <$filehdl> ) {
chomp($strInputline);
my ($strHostname,$strOS,$strVer,$strEnv) = split(/:/, $strInputline);
$strOS = lc($strOS);
$strVer = trim($strVer);
$strEnv = trim($strEnv);
$strOS = trim($strOS);
$arrHosts{$strOS}{$strVer}{$strEnv} = $strHostname;
}
# If you want to see the entire database, remove the # in front of Dumper
print Dumper \%arrHosts;
foreach my $machine (#{$arrHosts{solaris}{10}{DEV}}) {
print "$machine\n";
}
close($filehdl);
The data is in the form
machine:OS:OS version:Environment
For example
bigserver:solaris:11:PROD
smallerserver:solaris:11:DEV
I want to print out only the servers that are solaris, version 11, in DEV. Using hashes seems the easiest way to store the data but alas, Perl barfs when attempting to print out only a portion of it. Dumper works great but I don't want to see everything. Where did I go wrong??
You have the following:
$arrHosts{$strOS}{$strVer}{$strEnv} = $strHostname;
That means the following contains a string:
$arrHosts{solaris}{10}{DEV}
You are treating it as if it contains a reference to an array. To group the hosts by OS+ver+env, replace
$arrHosts{$strOS}{$strVer}{$strEnv} = $strHostname;
with
push #{ $arrHosts{$strOS}{$strVer}{$strEnv} }, $strHostname;
Iterating over #{ $arrHosts{solaris}{10}{DEV} } will then make sense.
My previous code also had the obvious problem whereby if the combo of OS, Version, and Environment were the same it wrote over previous data. Blunderful. Push is the trick
I'm trying Perl's PDL in the following code:
#!/usr/bin/perl -w
use strict;
use PDL::Core qw(pdl);
use PDL::Math qw(isfinite);
use PDL::Primitive qw(statsover);
my $div = 4;
my #array1 = (0..10);
my $pdl_array = log(pdl(#array1)/$div);
$pdl_array->where(!isfinite($pdl_array)) .= 0;
my($mean,$stdev) = statsover($pdl_array);
die $pdl_array,"\n",$mean," ",$stdev,"\n";
and I'm getting this error:
Undefined subroutine &PDL::divide called at ./compare_const.pl line 10.
Any hint, please?Thanks a lot.
PDL is unusual in its design, and therefore has an unusual and somewhat fragile import mechanism. Each PDL module adds functionality to PDL by inserting a new method directly into PDL's package. This decision was made very early in the design of PDL v2, and has not been changed in the intervening decade. (There's no reason it couldn't be changed, even in a backward compatible way, but none of the PDL developers have set aside the time to do so.)
As a result, you must load a handful of modules to ensure that PDL has its requisite, basic functionality. If you take a look at PDL's import function, you'll notice that it explicitly loads a number of packages into the caller's namespace. The reason for this is good---splitting functionality across multiple modules to keep the distribution sane---but the implementation is not aligned with common Perl practices. This is why your attempt to import specific functions into your namespace failed.
The solution has already been explained. Either replace all of your use PDL::... statements with a single use PDL:
use strict;
use warnings;
use PDL;
my $div = 4;
...
or say use PDL::Lite (to ensure that PDL's package is complete) and then import the specific functions into your (main) package
use strict;
use warnings;
use PDL::Lite;
use PDL::Core qw(pdl);
use PDL::Math qw(isfinite);
use PDL::Primitive qw(statsover);
my $div = 4;
...
Just add use PDL; and your code will work:
#!/usr/bin/perl -w
use strict;
use PDL;
use PDL::Core qw(pdl);
use PDL::Math qw(isfinite);
use PDL::Primitive qw(statsover);
my $div = 4;
my #array1 = (0..10);
my $pdl_array = log( pdl(#array1) / $div );
$pdl_array->where(!isfinite($pdl_array)) .= 0;
my ($mean, $stdev) = statsover($pdl_array);
die $pdl_array, "\n", $mean, " ", $stdev, "\n";
Outputs:
[0 -1.3862944 -0.69314718 -0.28768207 0 0.22314355 0.40546511 0.55961579 0.69314718 0.81093022 0.91629073]
0.112860814716055 0.696414187766251
PDL has a minimum set of things that must be loaded. To get these all properly loaded, you must either use PDL (which also exports a bunch of stuff) or use PDL::Lite.
(I for some reason thought you were explicitly calling PDL::divide directly and getting that error, hence my original answer below.)
Original answer:
I'm wondering why you think that should work?
Yes, PDL exports a bunch of stuff (if you use it, which you do not), but that doesn't give you any guarantee about where it is exporting it from. (In point of fact, it appears to export from a number of different places directly into the use`ing package.)
If you are trying to avoid namespace pollution, I would recommend either importing into a designated package and using stuff from there:
{
package My::PDL;
use PDL;
}
...
My::PDL::divide...
or using the OO interface (see PDL::Lite, I think?)
I am having problems trying to read an Outlook Email (Unicode) .msg file in Perl. Every other day i get an E-Mail with information which i have to put into another File. I'd like to automate this process.
Basically i have limited programming skills and have just started to learn Perl specifically for this task.
This is the part of my program trying to read the mail. So far did the Email::Outlook::Message from perlmonks get me:
use warnings;
use strict;
use Email::Outlook::Message;
use Email::MIME;
my $sourceDir = "c:/temp";
open_msg("test.msg");
sub open_msg {
my $verbose = 0;
my $msgFile = shift;
my $origMsg = new Email::Outlook::Message "$sourceDir/$msgFile", $verbose or die "$!";
my $mime = $origMsg->to_email_mime;
$mime->as_string;
return ($origMsg);
}
I am able to print the encoded hash, but i don't know how i can store the decoded text of the body in an array?
I am glad for every bit of help i can get.
Edit: i figured i change my initial question a bit to avoid posting a nearly equal question.
$mime->as_string returns the value you want to print, but you don't do anything with it, and then instead print the unchanged $mime handle. You want print $mime->as_string;
I believe the code should also display Useless use of as_string in void context or similar with use warnings;
I'm a beginner in programming, this is my first language. And in my class we are using a slightly out of date book to learn with (Book copyrighted '02). Doubt this would affect you helping me much, but worth noting.
The problem
I don't know how to format a simple foreach statement using/combined with an array. I'm getting mixed up and my book doesn't provide examples. I'm trying to get it so the Uses/Primary_Uses are shown when the user checks multiple checkboxes.
#!/usr/bin/perl
#c04ex5.cgi - creates a dynamic Web page that acknowledges
#the receipt of a registration form
print "Content-type: text/html\n\n";
use CGI qw(:standard -debug);
use strict;
#declare variables
my ($name, $serial, $modnum, $sysletter, $primary_uses, $use, #primary_uses, #uses);
my #models = ("Laser JX", "Laser PL", "ColorPrint XL");
my #systems = ("Windows", "Macintosh", "UNIX");
my #primary_uses = ("Home", "Business", "Educational", "Other");
#assign input items to variables
$name = param('Name');
$serial = param('Serial');
$modnum = param('Model');
$sysletter = param('System');
#primary_uses = param('Use');
#create Web page
print "<HTML><HEAD><TITLE>Juniper Printers</TITLE></HEAD>\n";
print "<BODY><H2>\n";
print "Thank you , $name, for completing \n";
print "the registration form.<BR><BR>\n";
print "We have registered your Juniper $models[$modnum] printer, \n";
print "serial number $serial.\n";
print "You indicated that the printer will be used on the\n";
print "$systems[$sysletter] system. <BR>\n";
print "The primary uses for this printer will be the following:\n";
#The part I'm having trouble with.
foreach $use (#primary_uses) {
print "$use [#use]<BR>\n";
}
print "</H2></BODY></HTML>\n";
My naming of variables might be a bit off, I was getting desperate and making sure I declare more than I should.
If you wanted to print a simple list of items, you should just use the $use variable:
foreach $use (#primary_uses) {
print "$use<BR>\n";
}
Note that this will also remove the fatal error that comes from not declaring #use. Perhaps that was also a point of confusion for you. $use and #use are two completely different variables, despite having the same name.
Note that you can print a list with the CGI module very easily:
my $cgi = CGI->new;
print $cgi->li(\#primary_uses);
Outputs the list interpolated in a list html entity, like so:
<li>Home</li> <li>Business</li> <li>Educational</li> <li>Other</li>
Some other pointers:
Note that it is a good idea to declare your variables in the smallest scope possible
foreach my $use (#primary_uses) { # note the use of "my"
print "$use<BR>\n";
}
That also goes with the other variables. A good idea is to declare them right as you initialize them:
my $name = param('Name');
Then people who read your code don't have to scan backwards in the file to see where the variable has "been" before.
Note that you should never, ever use the content of data from a web form without sanitizing it first, because it is a huge security risk, especially when you print it. It allows a web user to execute arbitrary code on your system.
You should know that for and foreach are aliases for the same function.
Also, you should always, always use warnings:
use warnings;
There really is no good reason to ever not turn warnings on.
foreach my $myuse (#primary_uses) {
print $myuse;
}
You need to declare the variable.
I'm a noob.I need some basic knowledge about how data should be saved and read under perl. Say to save a hash and an array. What format (extension) of file should be used? txt? So far I can only save all the things as stringprint FILE %hash and read them back as stringprint <FILE>. What should I do if I need my function hash and array inputs from a file. How to put them back to hash and array?
You're looking for data serialisation. Popular choices that are robust are Sereal, JSON::XS and YAML::XS. Lesser known formats are: ASN.1, Avro, BERT, BSON, CBOR, JSYNC, MessagePack, Protocol Buffers, Thrift.
Other often mentioned choices are Storable and Data::Dumper (or similar)/eval, but I cannot recommend them because Storable's format is Perl version dependent, and eval is unsafe because it executes arbitrary code. As of 2012, the parsing counter-part Data::Undump has not progressed very far yet. I also cannot recommend using XML because it does not map Perl data types well, and there exists multiple competing/incompatible schemas how to translate between XML and data.
Code examples (tested):
use JSON::XS qw(encode_json decode_json);
use File::Slurp qw(read_file write_file);
my %hash;
{
my $json = encode_json \%hash;
write_file('dump.json', { binmode => ':raw' }, $json);
}
{
my $json = read_file('dump.json', { binmode => ':raw' });
%hash = %{ decode_json $json };
}
use YAML::XS qw(Load Dump);
use File::Slurp qw(read_file write_file);
my %hash;
{
my $yaml = Dump \%hash;
write_file('dump.yml', { binmode => ':raw' }, $yaml);
}
{
my $yaml = read_file('dump.yml', { binmode => ':raw' });
%hash = %{ Load $yaml };
}
The next step up from here is object persistence.
Also read: Serializers for Perl: when to use what
Perlmonks has two good discussions on serialization.
How to save and reload my hash
How can I visualize my complex data structure?
This really depends upon how you'd like store your data in your file. I will try writing some basic perl code to enable you to read a file into an array and or write back a hash into a file.
#Load a file into a hash.
#My Text file has the following format.
#field1=value1
#field2=value2
#<FILE1> is an opens a sample txt file in read-only mode.
my %hash;
while (<FILE1>)
{
chomp;
my ($key, $val) = split /=/;
$hash{$key} .= exists $hash{$key} ? ",$val" : $val;
}
If you new I just suggest make to string from array/hash with join() and they write it with "print" and then read and use split() to make array/hash again. That would be more simple way like Perl teaching text book example.