Simple Perl Script (Total Perl Noob, just starting out) [closed] - database

It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 11 years ago.
I'm not in school or anything, but I have this example test that I want to complete. I'm not 100% sure where to begin (as said, I am completely new to Perl but really, really want to get into it).
Given a table 'mailing':
CREATE TABLE mailing (
addr VARCHAR(255) NOT NULL
);
The mailing table will initially be empty. New addresses will be added on a daily basis. It is expected that the table will store at least 10,000,000 email addresses and 100,000 domains.
Write a perl script that updates another table which holds a daily count of email addresses by their domain name.
Use this table to report the top 50 domains by count sorted by percentage growth of the last 30 days compared to the total.
NOTE
- You MUST use the provided DB.pm for all database interaction, and you must use it as it is (DB.pm cannot be modified except for the connection settings).
The original mailing table should not be modified.
All processing must be done in Perl (eg. no complex queries or sub-queries)
And here is the DB.pm
package GUI::DB;
use strict;
use DBI;
use vars qw(#ISA #EXPORT);
use Exporter;
#ISA = qw(Exporter);
#EXPORT = qw(dbConnect query);
#
# dbConnect - connect to the database, get the database handle
#
sub dbConnect {
# Read database settings from config file:
my $dsn = "DBI:mysql:database=test";
my $dbh = DBI->connect( $dsn,
'',
'',
{ RaiseError => 1 }
);
return $dbh;
}
#
# query - execute a query with parameters
# query($dbh, $sql, #bindValues)
#
sub query {
my $dbh = shift;
my $sql = shift;
my #bindValues = #_; # 0 or several parameters
my #returnData = ();
# issue query
my $sth = $dbh->prepare($sql);
if ( #bindValues ) {
$sth->execute(#bindValues);
} else {
$sth->execute();
}
if ( $sql =~ m/^select/i ) {
while ( my $row = $sth->fetchrow_hashref ) {
push #returnData, $row;
}
}
# finish the sql statement
$sth->finish();
return #returnData;
}
__END__
I work with PHP and fun stuff like that regularly, but Perl is just out there for me.

Here is an example of Perl DBI usage that inserts a record into mailing table:
use DBI;
$dbh = DBI->connect('DBI:mysql:databasename', 'username', 'password'
) || die "Could not connect to database: $DBI::errstr";
$dbh->do('INSERT INTO mailing VALUES(?)', 'test#test.com');
$dbh->disconnect();

Given that the rules state that you must use the query method from DB.pm, then:
my $dbh = dbConnect() or die "A horrible death";
query($dbh, "INSERT INTO Mailing(addr) VALUES(?)", 'someone#example.com');
The DB.pm module is defective in that it does not provide a error return indication - it relies on the DBI RaiseError to generate the error. It also does not provide a way to terminate the database connection.
The code in DB.pm is flabby. The dbConnect() method is:
sub dbConnect {
# Read database settings from config file:
my $dsn = "DBI:mysql:database=test";
my $dbh = DBI->connect( $dsn,
'',
'',
{ RaiseError => 1 }
);
return $dbh;
}
It could be:
sub dbConnect
{
# Read database settings from config file:
my $dsn = "DBI:mysql:database=test";
return DBI->connect($dsn, '', '', { RaiseError => 1 });
}
That's 6 lines instead of 13, and is actually easier to read than the original.

Related

Perl - Get the structure of a sqlite database using DBI

I need to test the structure of my SQLite database which is composed by a unique table with let's say 2 columns (id, name). I can't figure out the SQL query to get the table schema of my database.
I am able to get all the content of the database using the DBI method selectall_arrayref(). However it only returns an array containing the values inside my database. This information is useful but I would like to have a SQL query which returns something like id, name (Basically, the table schema).
I tried the following queries : SHOW COLUMNS FROM $tablename but also SELECT * from $tablename (This one returns all the table content).
Here is my implementation so far :
# database path
my $db_path = "/my/path/to/.database.sqlite";
my $tablename = "table_name";
sub connect_to_database {
# Connect to the database
my $dbh = DBI->connect ("dbi:SQLite:dbname=$db_path", "", "",
{ RaiseError => 1, AutoCommit => 0 },
)
or confess $DBI::errstr;
return $dbh;
}
sub get_database_structure {
# Connect to the database
my $dbh = &connect_to_database();
# Get the structure of the database
my $sth = $dbh->prepare("SHOW COLUMNS FROM $tablename");
$sth->execute();
while (my $inphash = $sth->fetrow_hashref()) {
print $inphash."\n";
}
# Disconnect from the database
$dbh->disconnect();
}
# Call the sub to print the database structure
&get_database_structure();
I expect the output to be the structure of my table so id, name but I raise an error : DBD::SQLite::db prepare failed: near "SHOW": syntax error
I can't find the good query. Any comments or help would be greatly appreciated.
Thanks !
What you're looking for is really just the SQL lite query for the table and column information. This answer SQLite Schema Information Metadata has the full details if this query doesn't work for you, but under the assumption you're using whatever the 'recent' version mentioned in one of the answers is, you can do something like this:
# Get the structure of the database
my $sth = $dbh->prepare("<<END_SQL");
SELECT
m.name as table_name,
p.name as column_name
FROM sqlite_master AS m
JOIN pragma_table_info(m.name) AS p
ORDER BY m.name, p.cid
END_SQL
$sth->execute();
my $last = '';
while (my $row = $sth->fetchrow_arrayref()) {
my ($table, $column) = #$row;
if ($table ne $last) {
print "=== $table ===\n";
$last = $table;
}
print "$column\n";
}
After digging through the community answers I finally find a solution using the pragma table_info.
sub get_database_structure {
# Connect to the database
my $dbh = &connect_to_database ();
# Return the structure of the table execution_host
my $sth = $dbh->prepare('pragma table_info(execution_host)');
$sth->execute();
my #struct;
while (my $row = $sth->fetchrow_arrayref()) {
push #struct, #$row[1];
}
# Disconnect from the database
$dbh->disconnect ();
return #struct;
}
It returns a list of the columns name present in the table execution_host.
Thanks for the help !

sqlsrv does not support named parameters to queries

I'm using PHP7 (with Droctrine 2.5.13) on Ubuntu 16.04 x64.
I'm trying to do a prepared statment to a Microsoft SQL database using a named variable like so:
$sql = 'SELECT DISTINCT
"PRODUCT"."NUMBER"
FROM
"PRODDB"."PRODUCT"
WHERE
"PRODUCT"."NUMBER" LIKE :productNumber
ORDER BY
"PRODUCT"."NUMBER" DESC';
$query = $this->db->prepare($sql);
$query ->bindParam(':productNumber' , $ofSearch);
$status = $query->execute();
$result = $query->fetchAll();
But SQL tell me that I need to use question mask instead of the named parameter
Error : sqlsrv does not support named parameters to queries, use question mark (?)
If I replace the named variable by a question mark it work fine.
The example below work fine:
$sql = 'SELECT DISTINCT
"PRODUCT"."NUMBER"
FROM
"PRODDB"."PRODUCT"
WHERE
"PRODUCT"."NUMBER" LIKE ?
ORDER BY
"PRODUCT"."NUMBER" DESC';
$query = $this->db->prepare($sql);
$query ->bindParam(1 , $ofSearch);
$status = $query->execute();
$result = $query->fetchAll();
Using question mark is a terrible way to do prepared statement as when you have very complex query with over 20 parameters it become a nightmare to maintain.
Do you have any idea if there is a solution to this problem or at least a workaround?

How to automate a large number of txt files to be imported from Perl to a SQL Server using DBI?

I have attempted to transfer one txt file into a SQL Server from Perl, but I now need to create a DB in SQL that combines hundreds of txt files. Is there an automated way of doing this to save manually transferring all the txt files? I will insert my attempt of coding one txt file. I believe I need to use Bulk Insert to achieve this but I am unsure what code to use. Any advice on the problem would be really appreciated, and if there is anymore code you wish to see or a specific thing I forgot to mention then please comment and I will edit. Thank you.
#!/usr/bin/perl -w
use strict;
use DBI;
use Carp;
# Connect to the data source and get a handle for that connection.
my $dbh = DBI->connect('dbi:ODBC:RegressionDBS') ||
confess "Connection Failed: $DBI::errstr";
# This query generates a result set with one record in it.
my $sql = "SELECT 1 AS test_col";
my $tablename = "RegressionTable";
open(FH, "newregression.txt") || die "cant open file";
my $Sqlh = $dbh->prepare("INSERT INTO RegressionTable (Ticker, TradeDate, HighPrice, LowPrice, TradePrice, TotalVolume, TotalValue) VALUES (?,?,?,?,?,?,?);") || confess $DBI::errstr;
while (my $line = <FH> )
{
$line =~/^(.*)(..)(..)$/;
my ($curTicker, $wTradeDate, $wHighPrice, $wLowPrice, $wTradePrice, $wTotalVolume, $wTotalValue);
$curTicker=$1
#$wTradeDate=
$wHighPrice=$2;
$wLowPrice=$1;
#$wTradePrice=$1;
#$wTotalVolume=$1;
#$wTotalValue=$4;
$Sqlh->execute($curTicker, $wTradeDate, $wHighPrice, $wLowPrice, $wTradePrice, $wTotalVolume, $wTotalValue) || confess $DBI::errstr;
}
$Sqlh->finish();
$dbh->disconnect();
Ok, so you run
INSERT INTO RegressionTable (
Ticker, TradeDate, HighPrice, LowPrice, TradePrice, TotalVolume, TotalValue
)
VALUES (?, ?, ?, ?, ?, ?, ?)
for every line in newregression.txt, and now you want to do it for multiple files. Why not wrap your code above in a subroutine, say insert_file(), then call it per file in a loop...
my #files = qw(red.txt green.txt blue.txt);
for my $file (#files) {
insert_file($file);
}
Or if the target table varies per file, define a hash:
my %files = (
'red.txt' => 'RedTable',
'blue.txt' => 'BlueTable'
);
while ( my ($file, $table) = each %files ) {
insert_file($file, $table);
};
Not sure how you'd use Bulk Insert given the field separator in your data file seems to be indeterminate (i.e. the regex /(.*)(..)(..)/ defines field boundaries). You may need to use the FORMATFILE option. Also, if you generate a BULK INSERT statement using DBI, be aware that the user your SQL Server runs as must be able to read your data files over the filesystem (networked, local or otherwise).

Using Perl to select 1 from#db_link where db_link comes from each entries of a lookup table

I want to read a table in Oracle which contains database links for checking availability of each database link and return only the bad results for each database link and error message.
I want to fetch this lookup table into an array, and pass the entries of db_link to a select from dualQdb_link, test all the entries of lookup to test for success or failure. This seems very hard to achieve in perl.
Any ideas?
Seems pretty straightforward, something like this:
# Or whatever the column is really named ;)
my $dblinks = $dbh->selectcol_arrayref("select dbname from db_link");
for my $dblink (#$dblinks) {
my $success = eval {
my ($ret) = $dbh->selectrow_array("select 1 from "
. $dbh->quote_identifier($dblink, undef, "dual") );
$ret;
};
if ($success) {
say "$dblink is up";
} else {
say "$dblink is down";
}
}

how to connect SQL Server with perl

I know there is a similar question: Connect to SQL Server 2005 from Perl and do a SELECT , but I tried the accepted answer and am unable to get it to work.
Assuming I have a db named test, and would love to do a select from mytable
(select id, name from mytable)
Code is from the link above with updated dsn:
use strict;
use warnings;
use DBI;
# Insert your DSN's name here.
my $dsn = 'database=test'
# Change username and password to something more meaningful
my $dbh = DBI->connect("DBI::ODBC::$dsn", 'username', 'password')
# Prepare your sql statement (perldoc DBI for much more info).
my $sth = $dbh->prepare('select id, name from mytable');
# Execute the statement.
if ($sth->execute)
{
# This will keep returning until you run out of rows.
while (my $row = $sth->fetchrow_hashref)
{
print "ID = $row->{id}, Name = $row->{name}\n";
}
}
# Done. Close the connection.
$dbh->disconnect;
This is what I got when running the script:
Can't connect to data source 'ODBC::database=test' because I can't work out what
driver to use (it doesn't seem to contain a 'dbi:driver:' prefix and the DBI_DR
IVER env var is not set) at script.pl line 9
Looks like the problem is in the dsn but I have no idea how to fix it (I am on sql 2005, active perl 5.10 and windows xp).
Edit:
I used the following code to verified whether ODBC is installed.
use DBI;
print join (", ", DBI->installed_versions);
Output:
It looks like ODBC is indeed in the list.
ADO, CSV, DBM, ExampleP, File, Gofer, ODBC, SQLite, Sponge, mysql
What am I missing?
I got the same error with SQLite just now, and it looks like you did the same thing wrong as me. Note the number of colons in the first argument - this is the wrong format:
my $db = DBI->connect('DBI::SQLite::dbname=testing.db', '', '', {RaiseError => 1, AutoCommit => 1});
There should actually only be two colons, not two pairs of colons in the first argument:
my $db = DBI->connect('DBI:SQLite:dbname=testing.db', '', '', {RaiseError => 1, AutoCommit => 1});
Question answered despite its age because it's still top of the results in Google for this particular error message
Try setting your DSN to something like:
my $dbh = DBI->connect("dbi:ODBC:test", 'username', 'password')
If that doesn't work, ensure you have DBD::ODBC installed by running:
perl -MDBI -e 'DBI->installed_versions;'
Assume SQL server is located on local server, connection below can be right:
my $DSN = "driver={SQL Server};Server=127.0.0.1;Database=test;UID=sa;PWD=123456";
my $dbh = DBI->connect("dbi:ODBC:$DSN");

Resources