I have a contact form that uses a CGI script to get form data and send through email. The script works fine except for the fact that it seems to change the order of form elements. I think I have pinpointed the block of code responsible for this.
Is there a way to alter this so that it sends the form data as-is, without re-ordering?
sub get_data {
use CGI qw/:standard/;
my $query = new CGI;
foreach $key ($query->param()){
$data{$key} = $query->param($key);
}
%data; # return associative array of name=value
}
From perldoc CGI
If the script was invoked with a parameter list (e.g. "name1=value1&name2=value2&name3=value3"), the param() method will return the parameter names as a list. If the script was invoked as an script and contains a string without ampersands (e.g. "value1+value2+value3") , there will be a single parameter named "keywords" containing the "+"-delimited keywords.
NOTE: As of version 1.5, the array of parameter names returned will be in the same order as they were submitted by the browser. Usually this order is the same as the order in which the parameters are defined in the form (however, this isn't part of the spec, and so isn't guaranteed).
So you can keep order of keys by storing them in array,
my #ordered = $query->param();
or don't use hash at all,
my #data;
foreach $key ($query->param()){
push #data, [ $key, $query->param($key) ];
}
return #data;
Well, you're putting your parameters into a hash. And hashes have no intrinsic ordering. The only way to get key/value pairs out for a hash in the same order as you put them in, is to keep a separate array containing the order of the keys.
Do you really need the parameters in a hash? Can't you just use your foreach loop at the point when you're creating the email?
Related
I need to compare the 2 arrays declared here to return records that exist only in the filtered_apps array. I am using the contents of previous_apps array to see if an ID in the record exists in filtered_apps array. I will be outputting the results to a CSV and displaying records that exist in both arrays to the console.
My question is this: How do I get the records that only exist in filtered_apps? Easiest for me would be to put those unique records into a new array to work with on the csv.
start_date = Date.parse("2022-02-05")
end_date = Date.parse("2022-05-17")
valid_year = start_date.year
dupe_apps = []
uniq_apps = []
# Finding applications that meet my criteria:
filtered_apps = FinancialAssistance::Application.where(
:is_requesting_info_in_mail => true,
:aasm_state => "determined",
:submitted_at => {
"$exists" => true,
"$gte" => start_date,
"$lte" => end_date })
# Finding applications that I want to compare against filtered_apps
previous_apps = FinancialAssistance::Application.where(
is_requesting_info_in_mail: true,
:submitted_at => {
"$exists" => true,
"$gte" => valid_year })
# I'm using this to pull the ID that I'm using for comparison just to make the comparison lighter by only storing the family_id
previous_apps.each do |y|
previous_apps_array << y.family_id
end
# This is where I'm doing my comparison and it is not working.
filtered_apps.each do |app|
if app.family_id.in?(previous_apps_array) == false
then #non_dupe_apps << app
else "No duplicate found for application #{app.hbx_id}"
end
end
end
So what am I doing wrong in the last code section?
Let's check your original method first (I fixed the indentation to make it clearer). There's quite a few issues with it:
filtered_apps.each do |app|
if app.family_id.in?(previous_apps_array) == false
# Where is "#non_dupe_apps" declared? It isn't anywhere in your example...
# Also, "then" is not necessary unless you want a one-line if-statement
then #non_dupe_apps << app
# This doesn't do anything, it's just a string
# You need to use "p" or "puts" to output something to the console
# Note that the "else" is also only triggered when duplicates WERE found...
else "No duplicate found for application #{app.hbx_id}"
end # Extra "end" here, this will mess things up
end
end
Also, you haven't declared previous_apps_array anywhere in your example, you just start adding to it out of nowhere.
Getting the difference between 2 arrays is dead easy in Ruby: just use -!
uniq_apps = filtered_apps - previous_apps
You can also do this with ActiveRecord results, since they are just arrays of ActiveRecord objects. However, this doesn't help if you specifically need to compare results using the family_id column.
TIP: Getting the values of only a specific column/columns from your database is probably best done with the pluck or select method if you don't need to store any other data about those objects. With pluck, you only get an array of values in the result, not the full objects. select works a bit differently and returns ActiveRecord objects, but filters out everything but the selected columns. select is usually better in nested queries, since it doesn't trigger a separate query when used as a part of another query, while pluck always triggers one.
# Querying straight from the database
# This is what I would recommend, but it doesn't print the values of duplicates
uniq_apps = filtered_apps.where.not(family_id: previous_apps.select(:family_id))
I highly recommend getting really familiar with at least filter/select, and map out of the basic array methods. They make things like this way easier. The Ruby docs are a great place to learn about them and others. A very simple example of doing a similar thing to what you explained in your question with filter/select on 2 arrays would be something like this:
arr = [1, 2, 3]
full_arr = [1, 2, 3, 4, 5]
unique_numbers = full_arr.filter do |num|
if arr.include?(num)
puts "Duplicates were found for #{num}"
false
else
true
end
end
# Duplicates were found for 1
# Duplicates were found for 2
# Duplicates were found for 3
=> [4, 5]
NOTE: The OP is working with ruby 2.5.9, where filter is not yet available as an array method (it was introduced in 2.6.3). However, filter is just an alias for select, which can be found on earlier versions of Ruby, so they can be used interchangeably. Personally, I prefer using filter because, as seen above, select is already used in other methods, and filter is also the more common term in other programming languages I usually work with. Of course when both are available, it doesn't really matter which one you use, as long as you keep it consistent.
EDIT: My last answer did, in fact, not work.
Here is the code all nice and working.
It turns out the issue was that when comparing family_id from the set of records I forgot that the looped record was a part of the set, so it would return it, too. I added a check for the ID of the array to match the looped record and bob's your uncle.
I added the pass and reject arrays so I could check my work instead of downloading a csv every time. Leaving them in mostly because I'm scared to change anything else.
start_date = Date.parse(date_from)
end_date = Date.parse(date_to)
valid_year = start_date.year
date_range = (start_date)..(end_date)
comparison_apps = FinancialAssistance::Application.by_year(start_date.year).where(
aasm_state:'determined',
is_requesting_voter_registration_application_in_mail:true)
apps = FinancialAssistance::Application.where(
:is_requesting_voter_registration_application_in_mail => true,
:submitted_at => date_range).uniq{ |n| n.family_id}
#pass_array = []
#reject_array = []
apps.each do |app|
family = app.family
app_id = app.id
previous_apps = comparison_apps.where(family_id:family.id,:id.ne => app.id)
if previous_apps.count > 0
#reject_array << app
puts "\e[32mApplicant hbx id \e[31m#{app.primary_applicant.person_hbx_id}\e[32m in family ID \e[31m#{family.id}\e[32m has registered to vote in a previous application.\e[0m"
else
<csv fields here>
csv << [csv fields here]
end
end
Basically, I pulled the applications into the app variable array, then filtered them by the family_id field in each record.
I had to do this because the issue at the bottom of everything was that there were records present in app that were themselves duplicates, only submitted a few days apart. Since I went on the assumption that the initial app array would be all unique, I thought the duplicates that were included were due to the rest of the code not filtering correctly.
I then use the uniq_apps array to filter through and look for matches in uniq_apps.each do, and when it finds a duplicate, it adds it to the previous_applications array inside the loop. Since this array resets each go-round, if it ever has more than 0 records in it, the app gets called out as being submitted already. Otherwise, it goes to my csv report.
Thanks for the help on this, it really got my brain thinking in another direction that I needed to. It also helped improve the code even though the issue was at the very beginning.
I have extracted this variable (userID_ALL) that contains the key-value pairs of some users. I would like to make a foreach loop that will call an API that will use the ID of each user. Is there a way to access the ID of the user in the foreach loop from the varible?
extracted key-value pairs from Json Extractor
I believe there should be a better way to get the IDs from the response, however if you have to deal with the variables as per your screenshot - you could fetch the "id" attribute value using the following __groovy() function:
${__groovy(new groovy.json.JsonSlurper().parseText(vars.get('userID')).id,)}
Demo:
More information:
JsonSlurper Documentation
Apache Groovy - Parsing and producing JSON
Apache Groovy - Why and How You Should Use It
I've been pulling data from an API in JSON, and am currently stumbling over an elmementary problem
The data is on companies, like Google and Facebook, and is in an array or hashes, like so:
[
{"id"=>"1", "properties"=>{"name"=>"Google", "stock_symbol"=>GOOG, "primary_role"=>"company"}},
{"id"=>"2", "properties"=>{"name"=>"Facebook", "stock_symbol"=>FB, "primary_role"=>"company"}}
]
Below are two operations I'd like to try:
For each company, print out the name, ID, and the stock symbol (i.e. "Google - 1 - GOOG" and "Facebook - 2 - FB")
Remove "primary role" key/value from Google and Facebook
Assign a new "industry" key/value for Google and Facebook
Any ideas?
Am a beginner in Ruby, but running into issues with some functions / methods (e.g. undefined method) for arrays and hashes as this looks to be an array OF hashes
Thank you!
Ruby provides a couple of tools to help us comprehend arrays, hashes, and nested mixtures of both.
Assuming your data looks like this (I've added quotes around GOOG and FB):
data = [
{"id"=>"1", "properties"=>{"name"=>"Google", "stock_symbol"=>"GOOG", "primary_role"=>"company"}},
{"id"=>"2", "properties"=>{"name"=>"Facebook", "stock_symbol"=>"FB", "primary_role"=>"company"}}
]
You can iterate over the array using each, e.g.:
data.each do |result|
puts result["id"]
end
Digging into a hash and printing the result can be done in a couple of ways:
data.each do |result|
# method 1
puts result["properties"]["name"]
# method 2
puts result.dig("properties", "name")
end
Method #1 uses the hash[key] syntax, and because the first hash value is another hash, it can be chained to get the result you're after. The drawback of this approach is that if you have a missing properties key on one of your results, you'll get an error.
Method #2 uses dig, which accepts the nested keys as arguments (in order). It'll dig down into the nested hashes and pull out the value, but if any step is missing, it will return nil which can be a bit safer if you're handling data from an external source
Removing elements from a hash
Your second question is a little more involved. You've got two options:
Remove the primary_role keys from the nested hashes, or
Create a new object which contains all the data except the primary_role keys.
I'd generally go for the latter, and recommend reading up on immutability and immutable data structures.
However, to achieve [1] you can do an in-place delete of the key:
data.each do |company|
company["properties"].delete("primary_role")
end
Adding elements to a hash
You assign new hash values simply with hash[key] = value, so you can set the industry with something like:
data.each do |company|
company["properties"]["industry"] = "Advertising/Privacy Invasion"
end
which would leave you with something like:
[
{
"id"=>"1",
"properties"=>{
"name"=>"Google",
"stock_symbol"=>"GOOG",
"industry"=>"Advertising/Privacy Invasion"
}
},
{
"id"=>"2",
"properties"=>{
"name"=>"Facebook",
"stock_symbol"=>"FB",
"industry"=>"Advertising/Privacy Invasion"
}
}
]
To achieve the first operation, you can iterate through the array of companies and access the relevant information for each company. Here's an example in Ruby:
companies = [ {"id"=>"1", "properties"=>{"name"=>"Google", "stock_symbol"=>"GOOG", "primary_role"=>"company"}}, {"id"=>"2", "properties"=>{"name"=>"Facebook", "stock_symbol"=>"FB", "primary_role"=>"company"}}]
companies.each do |company|
name = company['properties']['name']
id = company['id']
stock_symbol = company['properties']['stock_symbol']
puts "#{name} - #{id} - #{stock_symbol}"
end
This will print out the name, ID, and stock symbol for each company.
To remove the "primary role" key/value, you can use the delete method on the properties hash. For example:
companies.each do |company|
company['properties'].delete('primary_role')
end
To add a new "industry" key/value, you can use the []= operator to add a new key/value pair to the properties hash. For example:
companies.each do |company|
company['properties']['industry'] = 'Technology'
end
This will add a new key/value pair with the key "industry" and the value "Technology" to the properties hash for each company.
I have an assignment that I cannot figure out where my mistake lies. I have a large array of hashes all under the method twitter_data. The hash is structured as such.
def twitter_data
[{"User"=>
{"description"=>
"Description here",
"last twenty tweets"=>
["tweets written out here"],
"number of followers"=>1000,
"number of friends"=>100,
"latest tweet"=>
"tweet written out here",
"number of tweets"=>1000,
"location"=>"Wherever, Wherever"}},]
end
Now if I wanted to for instance list all of the users and their descriptions I thought the code would read as such.
twitter_data.each do |twitter_data|
puts "#{twitter_data[:twitter_data]}: #{twitter_data[:description]}"
end
But the output for that just gives me about seven :, without the username in front of it or the description afterwards.
As you can see the description key is nested into another hash which key is User. I don't know which is the other key you want to print because data seems incomplete but if you wanted to print just the descriptions this one should work
twitter_data.each do |user_data|
description = user_data["User"]["description"]
puts description
end
There are a couple of reasons why this does not work:
1) The twitter_data element inside the each looks like this { 'User' => { 'description'.... On that hash, the value stored under the :description key is nil.
2) Even if you where to refer to the correct hash via twitter_data['User'] you would still be using symbols (e.g. :description) instead of strings. So even then, the value stored for the keys would be nil.
3) You are referencing elements that do not seem to exist in the hash even if one where to use strings (e.g. :twitter_data). Now this might simply be due to the example selected.
What will work is to correctly reference the hashes:
twitter_data.each do |data|
user_hash = data['User']
puts "#{user_hash['twitter_data']}: #{user_hash['description']}"
end
I have a queries that reside in multiple methods each (query) of which can contain multiple parameters. I am trying to reduce file size and line count to make it more maintainable. Below is such an occurrence:
$sql_update = qq { UPDATE database.table
SET column = 'UPDATE!'
WHERE id = ?
};
$sth_update = $dbh->prepare($sql_update);
if ($dbh->err) {
my $error = "Could not prepare statement. Error: ". $dbh->errstr ." Exiting at line " . __LINE__;
print "$error\n";
die;
}
$sth_rnupdate->execute($parameter);
if ($dbh->err) {
my $error = "Could not execute statement. Error: ". $dbh->errstr ." Exiting at line " . __LINE__;
print "$error\n";
die;
}
This is just one example, however, there are various other select examples that contain just the one parameter to be passed in, however there is also some with two or more parameters. I guess I am just wondering would it be possible to encapsulate this all into a function/method, pass in an array of parameters, how would the parameters be populated into the execute() function?
If this was possible I could write a method that you simply just pass in the SQL query and parameters and get back a reference to the fetched records. Does this sound safe at all?
If line-count and maintainable code is your only goal, your best bet would be to use any one of the several fine ORM frameworks/libraries available. Class::DBI and DBIx::Class are two fine starting points. Just in case, you are worried about spending additional time to learn these modules - dont: It took me just one afternoon to get started and productive. Using Class::DBI for example your example is just one line:
Table->retrieve(id => $parameter)->column('UPDATE!')->update;
The only down-side (if that) of these frameworks is that very complicated SQL statements required writing custom methods learning which may take you some additional time (not too much) to get around.
No sense in checking for errors after every single database call. How tedious!
Instead, when you connect to the database, set the RaiseError option to true. Then if a database error occurs, an exception will be thrown. If you do not catch it (in an eval{} block), your program will die with a message, similar to what you were doing manually above.
The "execute" function does accept an array containing all your parameters.
You just have to find a way to indicate which statement handle you want to execute and you're done ...
It would be much better to keep your statement handles somewhere because if you create a new one each time and prepare it each time you don't really rip the benefits of "prepare" ...
About returning all rows you can do that ( something like "while fetchrow_hashref push" ) be beware of large result sets that coudl eat all your memory !
Here's a simple approach using closures/anonymous subs stored in a hash by keyword name (compiles, but not tested otherwise), edited to include use of RaiseError:
# define cached SQL in hash, to access by keyword
#
sub genCachedSQL {
my $dbh = shift;
my $sqls = shift; # hashref for keyword => sql query
my %SQL_CACHE;
while (my($name,$sql) = each %$sqls) {
my $sth = $dbh->prepare($sql);
$SQL_CACHE{$name}->{sth} = $sth;
$SQL_CACHE{$name}->{exec} = sub { # closure for execute(s)
my #parameters = #_;
$SQL_CACHE{$name}->{sth}->execute(#parameters);
return sub { # closure for resultset iterator - check for undef
my $row; eval { $row = $SQL_CACHE{$name}->{sth}->fetchrow_arrayref(); };
return $row;
} # end resultset closure
} # end exec closure
} # end while each %$sqls
return \%SQL_CACHE;
} # end genCachedSQL
my $dbh = DBI->connect('dbi:...', { RaiseError => 1 });
# initialize cached SQL statements
#
my $sqlrun = genCachedSQL($dbh,
{'insert_table1' => qq{ INSERT INTO database.table1 (id, column) VALUES (?,?) },
'update_table1' => qq{ UPDATE database.table1 SET column = 'UPDATE!' WHERE id = ? },
'select_table1' => qq{ SELECT column FROM database.table1 WHERE id = ? }});
# use cached SQL
#
my $colid1 = 1;
$sqlrun->{'insert_table1'}->{exec}->($colid1,"ORIGINAL");
$sqlrun->{'update_table1'}->{exec}->($colid1);
my $result = $sqlrun->{'select_table1'}->{exec}->($colid1);
print join("\t", #$_),"\n" while(&$result());
my $colid2 = 2;
$sqlrun->{'insert_table1'}->{exec}->($colid2,"ORIGINAL");
# ...
I'm very impressed with bubaker's example of using a closure for this.
Just the same, if the original goal was to make the code-base smaller and more maintainable, I can't help thinking there's a lot of noise begging to be removed from the original code, before anyone embarks on a conversion to CDBI or DBIC etc (notwithstanding the great libraries they both are.)
If the $dbh had been instantiated with RaiseError set in the attributes, most of that code goes away:
$sql_update = qq { UPDATE database.table
SET column = 'UPDATE!'
WHERE id = ?
};
$sth_update = $dbh->prepare($sql_update);
$sth_update->execute($parameter);
I can't see that the error handling in the original code is adding much that you wouldn't get from the vanilla die produced by RaiseError, but if it's important, have a look at the HandleError attribute in the DBI manpage.
Furthermore, if such statements aren't being reused (which is often the main purpose of preparing them, to cache how they're optimised; the other reason is to mitigate against SQL injection by using placeholders), then why not use do?
$dbh->do($sql_update, \%attrs, #parameters);