Perl: Multiply loops, 1 hash and regex - arrays

I got stuck with logic behind loops (while & foreach) and AoH. I have basic knowledge about loops and arrays of hashes, but I can't quite understand how to combine them into 1 single and simple solution. My task is to check regular user's password age, if it is older than n-days (last part is OK for me, I know how to solve it, using GetOptions etc,.).
To accomplish that I figured out a solution:
1 Load file /etc/passwd into script, preform regex search to find out regular users. Regular users in Linux like systems have IDs from 1000 and above, so I use this regex to find out those:
/(\w+)[:]x[:]1[0-9]{3}/
2 Load results of regex serch in to array:
my (#Usernames, %pwdsettings);
while (my $pwdsettings = <$fh2>) {
if ($pwdsettings =~ /(\w+)[:]x[:]1[0-9]{3}/) {
$pwdsettings{"Username"} = $1;
push (#Usernames, \%pwdsettings);
}
}
3 Preform chage check for every entry in array:
my $pwdsett_dump = "tmp/pwdsett-dump.txt";
...
foreach (#Usernames) {
system("chage -l $_ > $pwdsett_dump")
}
4 Open $pwdsett_dump and then preform second regex search to get date of last password change. After, load results into existing hash inside array (AoH):
open (my $fh3, "<", $pwdsett_dump) or die "Could not open file '$pwdsett_dump': $!";
while (my $array = <$fh3>) {
if ($array =~ /^Last\s+password\s+change\s+:\s(\w{3})\s+(\d{2}),\s+(\d{4})/) {
$pwdsettings{"Month"} = $1;
$pwdsettings{"Day"} = $2;
$pwdsettings{"Year"} = $3;
}
}
But, somewhere it went terribly wrong. My script loads only 1 user in to AoH, second user is never loaded and I get $VAR1->[0].
What I want is to understand how AoH and loops are created in right way.
Full script:
#!/usr/bin/perl
use strict;
use warnings;
use Data::Dumper;
my $pwdsett_dump = "tmp/pwdsett-dump.txt";
my $usernames_dump = "tmp/usernames-dump.txt";
system("cat /etc/passwd > $usernames_dump");
open (my $fh2, "<", $usernames_dump) or die "Could not open file '$usernames_dump': $!";
my (#Usernames, %pwdsettings);
while (my $pwdsettings = <$fh2>) {
if ($pwdsettings =~ /(\w+)[:]x[:]1[0-9]{3}/) {
$pwdsettings{"Username"} = $1;
push (#Usernames, \%pwdsettings);
}
}
foreach (#Usernames) {
system("chage -l $_ > $pwdsett_dump")
}
open (my $fh3, "<", $pwdsett_dump) or die "Could not open file '$pwdsett_dump': $!";
while (my $array = <$fh3>) {
if ($array =~ /^Last\s+password\s+change\s+:\s(\w{3})\s+(\d{2}),\s+(\d{4})/) {
$pwdsettings{"Month"} = $1;
$pwdsettings{"Day"} = $2;
$pwdsettings{"Year"} = $3;
}
}
print Dumper \#Usernames;

you need to append the file when you output meaning use ">>" instead of ">" which will overwrite the file.
system("chage -l $_ >> $pwdsett_dump") as you are running it in loop you are overwriting each time the loop executes.
Use:
foreach (#Usernames) {
system("chage -l $_ >> $pwdsett_dump")
}
########sample script
#!/usr/bin/perl
use strict;
use warnings;
my $usernames_dump = "/etc/passwd";
open (my $fh2, "<", $usernames_dump) or die "Could not open file '$usernames_dump': $!";
my #pwdsettings;
my $i =0;
my #pwdsett_dump;
while (<$fh2>) {
if ($_ =~ /(\w+)[:]x[:]1[0-9]{3}/) {
my #user = split(/:/, $_);
$pwdsettings[$i] = $user[0];
$pwdsett_dump[$i] = `chage -l $user[0]|grep Last`;
$pwdsett_dump[$i] =~ s/Last.*://;
$pwdsett_dump[$i] =~ s/,//;
my #m = split(/ /,$pwdsett_dump[$i]);
print "$user[0]\t Date: $m[2] Month: $m[1] Year: $m[3]\n";
$i++;
}
}
Output: testuser Date: 12 Month: May Year: 2015

Related

Reading a line from a file using perl

First off, I have to find the existence of the pass and fail files in the subdirectories. Then, I need to read the first line of the pass/fail file. I thought of separating the $file1 and $file to differentiate it. I'm very new to perl so I know my approach is very bad.
I trying to figure out how to combine my current code to read the files I checked exists.
use strict;
use File::Find 'find';
my $file = 'pass.txt';
my $file1 = 'fail.txt';
my #directory = ('ram1','ram2');
sub check
{
if ( -e $_ && $_ eq $file )
{
print "Found file '$_' in directory '$File::Find::dir'\n";
}
elsif ( -e $_ && $_ eq $file1 )
{
print "Found file '$_' in directory '$File::Find::dir'\n";
}
}
find (\&check,#directory);
Is it possible I use the code below for the first if condition? I know it doesn't work but I'm not sure what to do next as the fail and pass text are inside the directories.
if (open my $File::Find::dir, '<', $file){
my $firstLine = <$File::Find::dir>;
close $firstLine;
Any suggestions would be helpful!
If you just want to look just in ram1 and ram2, there's no point in using File::Find. That is used for recursive searches, meaning if you want to search all the subdirectories of ram1 and ram2. (And for that, I'd use File::Find::Rule over File::Find; it's much cleaner.)
my #dir_qfns = ( 'ram1', 'ram2' );
for my $dir_qfn (#dir_qfns) {
for my $fn ('pass.txt', 'fail.txt') {
my $file_qfn = "$dir_qfn/$fn";
open(my $fh, '<', $file_qfn)
or warn("Can't open \"$file_qfn\": $!\n"), next;
defined( my $first_line = <$fh> )
or warn("\"$file_qfn\" is empty\n"), next;
print("$file_qfn: $first_line");
}
}
If it's ok for a file to be missing, then you can ignore that error (ENOENT).
Similarly, you don't need to output an error message if the file is empty.
my #dir_qfns = ( 'ram1', 'ram2' );
for my $dir_qfn (#dir_qfns) {
for my $fn ('pass.txt', 'fail.txt') {
my $file_qfn = "$dir_qfn/$fn";
my $fh;
if (!open($fh, '<', $file_qfn)) {
warn("Can't open \"$file_qfn\": $!\n") if $!{ENOENT};
next;
}
defined( my $first_line = <$fh> )
or next;
print("$file_qfn: $first_line");
}
}
if (open my $f, '<', 'pass.txt') {
my $firstLine = <$f>;
close $f;
}
OP's code does not make much sense. Perhaps OP is looking for something of next kind
use strict;
use warnings;
use feature 'say';
my $dir = shift || 'some_dir_to_start_from';
my #files = qw/pass.txt fail.txt/;
my $match = join '|', #files;
my $regex = qr/\b($match)\b/;
files_lookup($dir,$regex);
exit 0;
sub files_lookup {
my $dir = shift;
my $re = shift;
for ( glob("$dir/*") ) {
files_lookup($_) if -d;
next unless /$re/;
if( -f ) {
say "File: $_";
open my $fh, '<', $_
or die "Couldn't open $_";
my $line = <$fh>;
say $line;
close $fh;
}
}

How to read array line in sequence from Input

I have some trouble
with array, how to read an array line in sequence from Input.
Here's my code.
sites.txt: (input file)
site1
site2
site3
program
#!/usr/bin/perl
my $file = 'sites.txt';
open(my $fh, '<:encoding(UTF-8)', $file)
or die "Couldn't open file !'$file' $!";
my #rows = <$fh>;
chomp #rows;
foreach my $site (#rows) {
$sitename = $site;
#domains = qw(.com .net .org);
foreach $dns (#domains){
$domain = $dns;
print "$site$dns\n";
}
}
and the output is like this
site1.com
site1.net
site1.org
site2.com
site2.net
site2.org
site3.com
site3.net
site3.org
I understand until that point,but i want make it the first element of array from #domains
reading until the end of Input line first,
then looping back to the 1st line of Input and going to the next element of array so
the output would be like this,
site1.com
site2.com
site3.com
site1.net
site2.net
site3.net
site1.org
site2.org
site3.org
it possible doing it? or need another module.sorry for basic question
I'll be really appreciated for the Answers.
thanks :)
You are iterating over all your sites and then (for each site) add the domain to the current site.
In pseudocode this is:
foreach site
foreach domain
print site + domain
Swap your loops so that the logic is
foreach domain
foreach site
print site + domain
Note that this is pseudocode, not Perl.
In "real" Perl this would look like:
#!/usr/bin/env perl
use strict;
use warnings;
my $file = 'sites.txt';
open( my $fh, '<:encoding(UTF-8)', $file )
or die "Couldn't open file !'$file' $!";
my #rows = <$fh>;
chomp #rows;
my #domains = qw(.com .net .org);
foreach my $dns (#domains) {
foreach my $site (#rows) {
print "$site$dns\n";
}
}
Output
site1.com
site2.com
site3.com
site1.net
site2.net
site3.net
site1.org
site2.org
site3.org
Please always include use strict; and use warnings; on top of your scripts. These two statements will show you the most common errors in your code.

Compare two hashes in perl and list which records are extra?

I have two text files that contain user records. I have to compare these two files and figure out which users are missing from File1. And delete these Orphans from file2.
#!/usr/local/bin/perl -w
use strict;
use warnings;
use autodie;
use Text::Diff;
use List::Compare;
use Data::Dumper;
my $Users1 = "Users1.txt";
my $Users2 ="Users2.txt";
my %hash1;
my %hash2;
my %new_hash;
my #sorted_1;
my #sorted_2;
my #list_keys1;
my #list_keys2;
open(my $fh1, '<:encoding(UTF-8)', $Users1) or die "Colud not open the file!";
while(my $record1 = <$fh1>)
{
chomp $record1;
my #list1 = split( '/', $record1);
foreach my $item(#list1)
{
$new_hash{$list1[1]} = $list1[0];
$hash1{$list1[1]} = $list1[0];
}
while ( my ($key, $value) = each(%hash1) ) {
push (#list_keys1, $key);
#sorted_1 = sort #list_keys1;
}
}
print "\t\tHash values for USERS1:\n";
print Dumper \%hash1;
open(my $fh2, '<:encoding(UTF-8)', $Users2) or die "Colud not open the file!";
while(my $record2 = <$fh2>)
{
chomp $record2;
my #list2 = split( '/', $record2);
foreach my $item(#list2)
{
$hash2{$list2[1]} = $list2[0];
}
while ( my ($key, $value) = each(%hash2) )
{
push (#list_keys2, $key);
#sorted_2 = sort #list_keys2;
}
}
print "\n\n\t\tHash values for Users2:\n";
print Dumper \%hash2;
#hash1{#list_keys1} = 1;
#hash2{#list_keys2} = 1;
foreach(keys %hash2)
{
print "\nThis user does not exist(to be deleted): $_\n" unless exists $hash1{$_};
}
foreach (keys %hash1)
{
print "\nNew User (to be added):$_\n" unless exists $hash2{$_};
}
close ($fh1);
close ($fh2);
Questions:
I am not able to sort the user ID (String) alphabetically(here, USER IDs are random strings of length 7). Is there any limitations when it comes to sorting array/hashes in Perl?
I am not able to compare two hashes and get the differences. What would be the most efficient way to do that?
Are there any additional libraries that I need to install in order to handle this part of code?
Sample records from file:
File1:
ASIA/ASEDF46
INDIA/PSDfT5V
CHINA/FSDfT5V
INDIA/AA44TYB
USA/BBRTT67
File 2:
INDIA/PSDfT5V
CHINA/FSDfT5V
INDIA/AA44TYB
USA/BBRTT67
UK/ZK9EELO
use strict;
use warnings;
use autodie;
open my $in, '<', 'in.txt';
open my $in2, '<', 'in_2.txt';
my (%data1, %data2);
while(<$in>){
chomp;
my #split = split/\//;
$data1{$split[0]} = $split[1];
}
while(<$in2>){
chomp;
my #split = split/\//;
$data2{$split[0]} = $split[1];
}
foreach(sort keys %data1){
print "User: $_ Value: $data1{$_}\n" if $data2{$_};
}

Compare MD5 from files in a directory against an array (perl)

I was checking out this link here: How could I write a Perl script to calculate the MD5 sum of every file in a directory?
It gets the md5 of each file in a specified directory. What i want to do is take those md5's and compare them against an array. This is what i have so far.
use warnings;
use strict;
use Digest::MD5 qw(md5_hex);
my $dirname = "./";
opendir( DIR, $dirname );
my #files = readdir(DIR);
closedir(DIR);
print "#files\n";
foreach my $file (#files) {
if ( -d $file || !-r $file ) { next; }
open( my $FILE, $file );
binmode($FILE);
print Digest::MD5->new->addfile($FILE)->hexdigest, " $file\n";
my #array = ('667fc8db8e5519cacbf8f9f2af2e0b08');
if (#array ~~ $FILE) {
print "matches array", "\n";
} else {
print "doesnt match array", "\n";
}
}
system ( 'pause' )
But with this, i always get doesnt match array no matter if it does match the array perfectly. I can print #array and it will even show the same md5 values of the file. But like i said it just always says "doesnt match array". ive never got it to say "matches array" on any file. Thank you for looking :)
EDIT:
This is what i have now.
use warnings;
use strict;
use Digest::MD5 qw(md5_hex);
my $dirname = "./";
opendir( DIR, $dirname );
my #files = readdir(DIR);
closedir(DIR);
print "#files\n";
foreach my $file (#files) {
next if -d $file || !-r $file;
open( my $FILE, $file );
binmode($FILE);
#print digest::MD5->new->addfile($FILE)->hexdigest, " $file\n";
Sdigest = Digest::MD5->new->addfile($FILE)->hexdigest, " $file\n";
my #array = ('667fc8db8e5519cacbf8f9f2af2e0b08');
if($digest eq $array[0]) {
print "matches array", "\n";
} else {
print "doesnt match array", "\n";
}
}
system ( 'pause' );
Thanks to all for your help. You guys are awesome ;)
Please do not use smartmatch ~~. It was declared experimental in the latest release of Perl, and the semantics are likely to change in the future.
The best solution is to create a hash of the fingerprints you know:
my %fingerprints;
$fingerprints{"667fc8db8e5519cacbf8f9f2af2e0b08"} = undef;
If you want to load a whole array of fingerprints into the hash so that we can easily test for existence, you can use a hash slice:
#fingerprints{#array} = ();
Next, we store the fingerprint of the current file in a variable:
my $digest = Digest::MD5->new->addfile($FILE)->hexdigest;
Then we test if that $digest exists in the hash of fingerprints:
if (exists $fingerprints{$digest}) {
print "$digest for <$file> -- FOUND\n";
}
else {
print "$digest for <$file>\n";
}
Using a hash is usually faster than looping through an array (If you do multiple lookups).
Suggested complete program:
use strict;
use warnings;
use feature qw< say >;
use autodie; # automatic error handling
use Digest::MD5;
my ($dirname, $fingerprint_file) = #ARGV; # takes two command line arguments
length $dirname or die "First argument must be a directory name\n";
length $fingerprint_file or die "Second argument must be a file with fingerprints\n";
# load the fingerprints
my %fingerprints;
open my $fingerprints_fh, "<", $fingerprint_file;
while (<$fingerprints_fh>) {
chomp;
$fingerprints{$_} = undef;
}
close $fingerprints_fh;
opendir my $directory, $dirname;
while(my $file = readdir $directory) {
next if not -f $file;
open my $fh, "<:raw", "$dirname/$file";
my $digest = Digest::MD5->new->addfile($fh)->hexdigest;
close $fh;
if (exists $fingerprints{$digest}) {
say qq($digest "$file" -- FOUND);
}
else {
say qq($digest "$file");
}
}
closedir $directory;
Example invocation
> perl script.pl . digests.txt
Perhaps the following will be helpful:
use warnings;
use strict;
use Digest::MD5 qw(md5_hex);
use File::Basename;
my $dirname = './';
my %MD5s = (
'667fc8db8e5519cacbf8f9f2af2e0b08' => 1,
'8c0452b597bc2c261ded598a65b043b9' => 1
);
for my $file ( grep { !-d and -r } <$dirname*> ) {
open my $FILE, '<', $file or die $!;
binmode $FILE;
my $md5hexdigest = Digest::MD5->new->addfile($FILE)->hexdigest;
close $FILE;
print basename ($file), " md5hexdigest $md5hexdigest ";
if ( $MD5s{$md5hexdigest} ) {
print "matches hash", "\n";
}
else {
print "doesn't match hash", "\n";
}
}
Sample output:
XOR_String_Match.pl md5hexdigest 8c0452b597bc2c261ded598a65b043b9 matches hash
zipped.txt md5hexdigest d41d8cd98f00b204e9800998ecf8427e doesn't match hash
Like this:
my $digest = Digest::MD5->new->addfile($FILE)->hexdigest, " $file\n";
then
if($digest eq $array[0])
By the way, it would maybe be slightly more idiomatic to say (earlier on in your code):
next if -d $file || !-r $file;

problem with the code in perl

My problem is that I am not able to figure out that why my code is taking each of the line from the file as one element of an array instead of taking the whole record starting from AD to SS as one element of the array. As you can see that my file is starting from AD and ending at SS which is same for all the followed lines in the data. But I want to make the array having elements starting from AD to SS which will be having all the lines in between AD to SS that is BC....,EG...., FA.....etc.not each line as an element. I tried my way and get the same file as such.Could anyone check my code. Thanks in advance.
AD uuu23
BC jjj
EG iii
FA vvv
SS
AD hhh25
BC kkk
EG ppp
FA aaa
SS
AD ttt26
BC xxx
FA rrr
SS
#!/usr/bin/env perl
use strict;
use warnings;
my $ifh;
my $line = '';
my #data;
my $ifn = "fac.txt";
open ($ifh, "<$ifn") || die "can't open $ifn";
my $a = "AD ";
my $b = "SS ";
my $_ = " ";
while ($line = <$ifh>)
{
chomp
if ($line =~ m/$a/g); {
$line = $_;
push #data, $line;
while ($line = <$ifh>)
{
$line .= $_;
push #data, $line;
last if
($line =~ m/$b/g);
}
}
push #data, $line; }
print #data;
If I understand correctly your problem, the fact is that the way you are reading the file:
while ($line = <$ifh>)
is inherently a line-by-line approach. It uses the content of the "line termination variable" ($/) to understand where to split lines. One easy way to change this behavior is un-defining the $/:
my $oldTerminator = $/;
undef $/;
....... <your processing here>
$/ = $oldTerminator;
so, your file would be just one line, but I am not sure what would happen of your code.
Another approach is the following (keeping in mind what I said about the fact that you are reading the file line-by-line): instead of doing
`push #data, $line;`
at each iteration of your loop, just accumulate the lines you read in a variable
$line .= $_;
(like you already do), and do the push only at the end, just once. Actually, this second approach will be more easily applicable to your code (you only have to remove the two push statements you have and put one outside of the loop).
I believe part of your problem is here
chomp
if ($line =~ m/$a/g);
it should be
chomp;
if ($line =~ m/$a/g)
otherwise the if statement is always executed. Please update your question if this has helped you advance
Here's a way to accomplish reading the records into an array, with newlines removed:
Code:
use strict;
use warnings;
use autodie;
my #data;
my $record;
my $file = "fac.txt";
open my $fh, '<', $file;
while (<$fh>) {
chomp;
if (/^AD /) { # new record starts
$record = $_;
while (<$fh>) {
chomp;
$record .= $_;
last if /^SS\s*/;
}
push #data, $record;
} else { die "Data outside record: $_" }
}
use Data::Dumper;
print Dumper \#data;
Output:
$VAR1 = [
'AD uuu23BC jjjEG iiiFA vvvSS',
'AD hhh25BC kkkEG pppFA aaaSS',
'AD ttt26BC xxxFA rrrSS'
];
This is another version, using the input record separator $/:
use strict;
use warnings;
use autodie;
my $file = "fac.txt";
open my $fh, '<', $file;
my #data;
$/ = "\nSS";
while (<$fh>) {
s/\n//g;
push #data, $_;
}
use Data::Dumper;
print Dumper \#data;
Produces the same output with this data. It does not care about the record start characters, only the end, which is SS at the beginning of a line.

Resources