I wrote the following script to load data from CSV file in numpy array form:
import numpy
sample = numpy.genfromtxt('sample_cor.csv', delimiter = ',')
print sample
and this sample looked like:
[[ 259463.392 2737830.062 ]
[ 255791.4823 2742050.772 ]
[ 249552.4949 2746152.328 ]
[ 247925.1228 2746422.143 ]
[ 262030.4697 2728966.229 ]
[ 260462.1936 2731412.856 ]
[ 260644.0281 2735003.027 ]
[ 268588.7974 2732835.097 ]]
now I want to extract every row in this array to string with a comma, for example, I expected row 1 to be converted to 259463.392,2737830.062, row 2 to be 255791.4823,2742050.772, and so on.
I tried the code below:
ss = numpy.array_str(sample[0])
print ss
print type(ss)
and got the result maybe not what I want,
[ 259463.392 2737830.062]
<type 'str'>
(I used coords = '259467.2,2737833.87' and got the string form which was what I want:
259467.2,2737833.87)
How to convert elements in a numpy array to string with a comma?
Here's an approach using join method -
[",".join(item) for item in a.astype(str)]
Sample run -
In [141]: a
Out[141]:
array([[ 259463.392 , 2737830.062 ],
[ 255791.4823, 2742050.772 ],
[ 249552.4949, 2746152.328 ],
[ 247925.1228, 2746422.143 ],
[ 262030.4697, 2728966.229 ],
[ 260462.1936, 2731412.856 ],
[ 260644.0281, 2735003.027 ],
[ 268588.7974, 2732835.097 ]])
In [142]: [",".join(item) for item in a.astype(str)]
Out[142]:
['259463.392,2737830.062',
'255791.4823,2742050.772',
'249552.4949,2746152.328',
'247925.1228,2746422.143',
'262030.4697,2728966.229',
'260462.1936,2731412.856',
'260644.0281,2735003.027',
'268588.7974,2732835.097']
Related
{
"pricexquant": [
[
"20233.54000000",
"0.00877000"
],
[
"20233.39000000",
"0.00999000"
]
]
}
for an array of arrays with 2 elements as strings, how would jq multiply element 1 * element 2 in each array? and from that generate a new array...
eg.
{
[
"177.4481458"
],
[
"200.310561"
]
}
and for that sum the total across all?
.pricexquant|map(map(tonumber)|.[0] * .[1])|add
yields
379.5797119
I'm working with mongodb and I'm dealing with aggregate to join two collections, now I would like to convert my result
"Users" : [
[
"Carl",
"John",
"Ever"
],
[
"Dani",
"GG",
"Sussan"
],
.... N arrays
]
to this output
"Users" : [
"Carl",
"John",
"Ever",
"Dani",
"GG",
"Sussan",
..... M elements from N arrays
]
Query
reduce starting from []
concat each array member to one flatten array
Test code here
aggregate(
[{"$set":
{"Users":
{"$reduce":
{"input":"$Users",
"initialValue":[],
"in":{"$concatArrays":["$$value", "$$this"]}}}}}])
Objective: Get all row in CSV file (minimum 1 row & more), add fix string & use the variable in JSON array.
Sorry, if the question is not related. I'm not sure what the correct question to be asked.
I want the "entryList" format to be like below, for successfully post the JSON object.
But, none of below approach works.
"entryList": [
{"entry": ["value_from","csv",""]},
{"entry": ["value_from","csv",""]},
{"entry": ["value_from","csv",""]}
]
Dumps JSON (full format):
x={
"act.addEntries": {
"act.authToken": "test123",
"act.resourceId": "asdasda=123asd1",
"act.entryList": {
"columns": [
"domainName","threatInfo","riskScore"
],
"entryList": [
VARIABLE
]
}
}
}
"""
Sample CSV file
domainName,threatInfo,riskScore
xxx.xxx.com,Test1,
yyy.yyy.com,Test2,
zzz.zzz.com,Test3,
"""
# CSV to List
domainList=[]
with open('Desktop/domainMatches.csv', 'r') as file:
reader = csv.reader(file)
col = next(reader)
Approach 1 - String Formatting
Problem: JSON dumps only show 1 entryList value. I couldn't get the correct way to escape {}, hence it lead to undesired JSON Object
for data in reader:
domainList.append(data)
line= '{{"entry": {}}}'.format(data)
print(line)
**OUTPUT:**
{"entry": ['xxx.xxx.com', 'Test1', '']}
{"entry": ['yyy.yyy.com', 'Test2', '']}
{"entry": ['zzz.zzz.com', 'Test3', '']}
...
...
"entryList": [
{{{}}}.format(line)
]
**OUTPUT - JSON Dumps:**
{
"act.addEntries": {
"act.authToken": "test123",
"act.resourceId": "asdasda=123asd1",
"act.entryList": {
"columns": [
"domainName",
"threatInfo",
"riskScore"
],
"entryList": [
"{{\"entry\": ['zzz.zzz.com', 'Test3', '']}}"
]
}
}
}
Aprroach 2 - + Concatenation
Too many problem here as well. Advice if needed
line = ""
for value in reader:
line+="{"
line+=""" "entry": """
line+=str(value)
line+="},\n"
**OUTPUT:**
{"entry": ['xxx.xxx.com', 'Test1', '']},
{"entry": ['yyy.yyy.com', 'Test2', '']},
{"entry": ['zzz.zzz.com', 'Test3', '']},
...
...
"entryList": [
"""+line+"""
]
Let me preface by saying I'm a total novice at perl.
I need to modify rules on a mail system. I can access the rules as an array and I believe the array contains subarrays. I need to modify one particular element and preserve the rest. My problem is I'm confused as to what the array type really is and how to consistently access the elements.
There may be more than one set of rules, but I'm only interested in processing rules with a priority of '1', which is $Rule[0]. Within $Rule[3] I need to parse the addresses.
use strict;
use Data::Dumper qw(Dumper);
my $Rules=$cli->GetAccountMailRules($account);
print Dumper \#$Rules;
foreach my $Rule (#$Rules) {
if($Rule->[0]=~/1/) {
my $pri=$Rule->[0];
my $type=$Rule->[1];
my $auto=$Rule->[2];
my $actions=$Rule->[3];
my $action1;
my $auto1;
my $auto2;
my #newRule;
my $oldDest;
print "\n";
print "Priority:\n";
print Dumper \$pri;
print "\n";
print "Rule Type:\n";
print Dumper \$type;
print "\n";
print "Forward Auto?:\n";
print Dumper \$auto;
print "\n";
print "Actions:\n";
print Dumper \$actions;
print "\n";
foreach my $ax (#$actions) {
$action1=$ax->[0];
$oldDest=$ax->[1];
}
my #addresses=split /[;,]|\\e/, $oldDest;
my #dests = grep(/corp.com|corp1.com|corp2.com|corp3.com/, #addresses);
my $newDest = join(",", #dests);
if (#$auto) {
foreach my $au (#$auto) {
$auto1=$au->[0];
$auto2=$au->[1];
}
#newRule=(
[ $pri, $type,
[[$auto1,$auto2]],
[[$action1,$newDest]]
]
);
} else {
#newRule=(
[ $pri, $type,
[],
[[$action1,$newDest]]
]
);
}
}
}
}
Output thusly:
# perl removeRules.pl
$VAR1 = [
[
'1',
'#Redirect',
[
[
'Human Generated',
'---'
]
],
[
[
'Mirror to',
'test10#corp.com\\etest10#gmail.com\\etest10#corp1.com'
]
]
]
];
Priority:
$VAR1 = \'1';
Rule Type:
$VAR1 = \'#Redirect';
Forward Auto?:
$VAR1 = \[
[
'Human Generated',
'---'
]
];
Actions:
$VAR1 = \[
[
'Mirror to',
'test10#corp.com\\etest10#gmail.com\\etest10#corp1.com'
]
];
The problem I'm running into is there is an option within $actions to discard emails after forwarding, which introduces new elements (or subarray?) into $actions:
# perl removeRules.pl
$VAR1 = [
[
'1',
'#Redirect',
[
[
'Human Generated',
'---'
]
],
[
[
'Mirror to',
'test10#corp.com\\etest10#gmail.com\\etest10#corp1.com'
],
[ <---- Begin new elements
'Discard',
'---'
] <---- End new elements
]
]
];
Priority:
$VAR1 = \'1';
Rule Type:
$VAR1 = \'#Redirect';
Forward Auto?:
$VAR1 = \[
[
'Human Generated',
'---'
]
];
Actions:
$VAR1 = \[
[
'Mirror to',
'test10#corp.com\\etest10#gmail.com\\etest10#corp1.com'
],
[
'Discard',
'---'
]
];
I tried testing to see if they can be referenced as additional elements in $actions but it throws off the index.
my $action2;
my $action3;
print "Actions:\n";
print Dumper \$actions;
print "\n";
foreach my $ax (#$actions) {
$action1=$ax->[0];
$oldDest=$ax->[1];
$action2=$ax->[2];
$action3=$ax->[3];
}
print " action1 $action1\n";
print " oldDest $oldDest\n";
print " action2 $action2\n";
print " action3 $action3\n";
Output:
Actions:
$VAR1 = \[
[
'Mirror to',
'test10#corp.com\\etest10#gmail.com\\etest10#corp1.com'
],
[
'Discard',
'---'
]
];
action1 Discard
oldDest ---
Use of uninitialized value $action2 in concatenation (.) or string at removeRules.pl line 107, <GEN0> line 4.
action2
Use of uninitialized value $action3 in concatenation (.) or string at removeRules.pl line 108, <GEN0> line 4.
action3
Thank you in advance.
Using this:
[
[
'Mirror to',
'test10#corp.com\\etest10#gmail.com\\etest10#corp1.com'
],
[
'Discard',
'---'
]
]
This is a reference to an array (the outer [..]) that has two items. Each item is again a reference to an array.
First item (position 0 of outer array reference) is
[
'Mirror to',
'test10#corp.com\\etest10#gmail.com\\etest10#corp1.com'
],
and second (position 1) is:
[
'Discard',
'---'
]
If $ractions is this outer array, then the above two items are respectively under $ractions->[0] and $ractions->[1].
Since they are both an array reference again you can access their items using the same construct, or using a Perl property, you can remove the second array.
In short:
'Mirror to' can be accessed by $ractions->[0]->[0] or shorter $ractions->[0][0]
'test10#corp.com\etest10#gmail.com\etest10#corp1.com' can be accessed by $ractions->[0]->[1]
'Discard' can be accessed by $ractions->[1]->[0]
'---' can be accessed by $ractions->[1]->[1]
Be aware however that $VAR1 = \[ shows that you have a reference over a reference. So you will need an extra step of derefencing:
DB<1> use Data::Dumper;
DB<2> #a=(1,2)
DB<3> print Data::Dumper::Dumper(#a);
$VAR1 = 1;
$VAR2 = 2;
DB<4> print Data::Dumper::Dumper(\#a);
$VAR1 = [
1,
2
];
DB<5> print Data::Dumper::Dumper(\\#a);
$VAR1 = \[
1,
2
];
PS: do not use corp.com or anything like that when you need to obfuscate domain names. See guidance in RFC2606 or TL;DR: use example.com
I have array values that is getting returned from SQL object.
my #keys = $db_obj->SelectAllArrayRef($sql);
print Dumper #keys;
gives
$VAR1 = [ [ '8853' ], [ '15141' ] ];
I need to create string from this array: 8853, 15141.
my $inVal = join(',', map { $_->[0] }, #$keys);
my $inVal;
foreach my $result (#$keys){
$inVal .= $result->[0];
}
my $inVal = join(',', #$keys);
Value i get is ARRAY(0x5265498),ARRAY(0x52654e0). I think its reference to the array. Any idea what am I missing here?
Don't pass arrays to Dumper; it leads to confusing output. $VAR1 is not a dump of #keys, it's a dump of $keys[0]. Instead, you should have done
print(Dumper(\#keys));
This would have given
$VAR1 = [ [ [ '8853' ], [ '15141' ] ] ];
The code you want is
join ',', map { $_->[0] }, #{ $keys[0] };
That said, it appears that ->SelectAllArrayRef returns a reference to the result, and so it should be called as follows:
my $keys = $db_obj->SelectAllArrayRef($sql);
For this,
print(Dumper($keys));
outputs
$VAR1 = [ [ '8853' ], [ '15141' ] ];
And you may use either of the methods you used in your question.
join ',', map { $_->[0] }, #$keys;
The first version should work for you:
my $arr = [ [ '8853' ], [ '15141' ] ];
my $values = join(',', map { $_->[0] } #$arr);
print $values . "\n";
8853,15141