Extract the contents from CSV into an array - arrays

I have a CSV file with contents:
John,1,2,4,67,100,41,234
Maria,45,23,67,68,300,250
I need to read this content and separate these data into two sections:
1.a Legend1 = John
1.b Legend2 = Maria
2.a Data_array1 = [1,2,4,67,100,41,234]
2.b Data_array2 = [45,23,67,a,67,300,250]
Here is my code; it reads the contents and separates the contents from ','.
testsample = CSV.read('samples/linechart.csv')
CSV.foreach('samples/linechart.csv') do |row|
puts row
end
Its output results in a class of array elements. I am stuck in pursuing it further.

I would recommend not using CSV.read for this it's too simple for that - instead, use File.open and read each line and treat it as a big string.
eg:
# this turns the file into an array of lines
# eg you now have: ["John,1,2,4,67,100,41,234", "Maria,45,23,67,a,67,300,250"]
lines = File.readlines('samples/linechart.csv')
# if you want to do this for each line, just iterate over this array:
lines.each do |line|
# now split each line by the commas to turn it into an array of strings
# eg you have: ["john","1","2","4","67","100","41","234"]
values = line.split(',')
# now, grab the first one as you name and the rest of them as an array of strings
legend = values[0] # "john"
data_array = values[1..-1] # ["1","2","4","67","100","41","234"]
# now do what you need to do with the name/numbers eg
puts "#{legend}: [#{data_array.join(',')}]"
# if you want the second array to be actual numbers instead of strings, you can convert them to numbers using to_i (or to_f if you want floats instead of integers)
# the following says "take each value and call to_i on it and return the set of new values"
data_array = data_array.map(&:to_i)
end # end of iterating over the array

First get the data out of csv like:
require 'csv'
csv_text = File.read('/tmp/a.csv')
csv = CSV.parse(csv_text)
# => [["John", "1", "2", "4", "67", "100", "41", "234"], ["Maria", "45", "23", "67", "a", "67", "300", "250"]]
Now you can format output as per your requirements. Eg:
csv.each.with_index(1){ |a, i|
puts "Legend#{i.to_s} = #{a[0]}"
}
# Legend1 = John
# Legend2 = Maria

You may looking for this,
csv = CSV.new(body)
csv.to_a
You can have a look at http://technicalpickles.com/posts/parsing-csv-with-ruby/
Reference this, too, if needed.

Over-engineered version ;)
class Lines
class Line
attr_reader :legend, :array
def initialize(line)
#line = line
parse
end
private
def parse
#legend, *array = #line.strip.split(",")
#array = array.map(&:to_i)
end
end
def self.parse(file_name)
File.readlines(file_name).map do |line|
Line.new(line)
end
end
end
Lines.parse("file_name.csv").each do |o|
p o.legend
p o.array
puts
end
# Result:
#
# "John"
# [1, 2, 4, 67, 100, 41, 234]
#
# "Maria"
# [45, 23, 67, 68, 300, 250]
Notes:
Basically, Lines.parse("file_name.csv") will give you an array of objects that will respond to the methods: legend and array; which holds the name and array of numbers respectively.
Jokes aside, I think OO will help maintainability.

Related

Working with Transpose functions result in error

consider the following array
arr = [["Locator", "Test1", "string1","string2","string3","string4"],
["$LogicalName", "Create Individual Contact","value1","value2"]]
Desired result:
[Test1=>{"string1"=>"value1","string2"=>"value2","string3"=>"","string4"=>""}]
When I do transpose, it gives me the error by saying second element of the array is not the length of the first element in the array,
Uncaught exception: element size differs (2 should be 4)
so is there any to add empty string in the place where there is no element and can perform the transpose and then create the hash as I have given above? The array may consist of many elements with different length but according to the size of the first element in the array, every other inner array has to change by inserting empty string and then I can do the transpose. Is there any way?
It sounds like you might want Enumerable#zip:
headers, *data_rows = input_data
headers.zip(*data_rows)
# => [["Locator", "$LogicalName"], ["Test1", "Create Individual Contact"],
# ["string1", "value1"], ["string2", "value2"], ["string3", nil], ["string4", nil]]
If you wish to transpose an array of arrays, each element of the array must be the same size. Here you would need to do something like the following.
arr = [["Locator", "Test1", "string1","string2","string3","string4"],
["$LogicalName", "Create Individual Contact","value1","value2"]]
keys, vals = arr
#=> [["Locator", "Test1", "string1", "string2", "string3", "string4"],
# ["$LogicalName", "Create Individual Contact", "value1", "value2"]]
idx = keys.index("Test1") + 1
#=> 2
{ "Test1" => [keys[idx..-1],
vals[idx..-1].
concat(['']*(keys.size - vals.size))].
transpose.
to_h }
#=> {"Test1"=>{"string1"=>"value1", "string2"=>"value2", "string3"=>"", "string4"=>""}}
It is not strictly necessary to define the variables keys and vals, but that avoids the need to create those arrays multiple times. It reads better as well, in my opinion.
The steps are as follows. Note keys.size #=> 6 and vals.size #=> 4.
a = vals[idx..-1]
#=> vals[2..-1]
#=> ["value1", "value2"]
b = [""]*(keys.size - vals.size)
#=> [""]*(4 - 2)
#=> ["", ""]
c = a.concat(b)
#=> ["value1", "value2", "", ""]
d = keys[idx..-1]
#=> ["string1", "string2", "string3", "string4"]
e = [d, c].transpose
#=> [["string1", "value1"], ["string2", "value2"], ["string3", ""], ["string4", ""]]
f = e.to_h
#=> {"string1"=>"value1", "string2"=>"value2", "string3"=>"", "string4"=>""}
f = e.to_h
#=> { "Test1" => f }
Find the longest Element in your Array and make sure every other element has the same length - loop and add maxLength - element(i).length amount of "" elements.

Passing elements from one array to another, generating CSV

I'm trying to write a program in Ruby that allows one array to receive information from another array. Basically, I have a multidimensional array called "student_array" that contains information on a few students
student_array = [["Mike", 13, "American", "male"],
["Grace", 12, "Canadian", "female"],
["Joey", 13, "American", "male"],
["Lily", 13, "American", "female"]
]
I also initialized two other arrays that will count nationalities:
nationality_array = Array.new
nationality_count = Array.new
The purpose of this program is to loop through student array, count the different nationalities of the students, and create a CSV file that will contain the headers of the different nationalities, and a count for each one.
Expected output.csv
American, Canadian
3, 1
Here is the code I have so far
student_array.each do |student|
#pushes the nationality string into the nationality array
nationality_array.push(student[2])
end
so the nationality_array should currently look like this:
nationality_array = ["American", "Canadian", "American", "American"];
nationality_array.uniq = ["American", "Canadian"];
So I will have two headers - "American" and "Canadian"
Now I need a way to loop through the student_array, count up each instance of "American" and "Canadian", and somehow assign it back to the nationality array. I'm having a hard time visualizing how to go about this. This is what I have so far--
american_count = 0;
canadian_count = 0;
student_array.each do |student|
if student[2] = "American"
american_count++
elsif student[2] = "Canadian"
canadian_count++
end
end
nationality_count.push(american_count);
nationality_count.push(canadian_count);
Okay, now I have those counts in the nationality_count array, but how can I pass it to a CSV, making sure that they are assigned to the right headers? I have a feeling that my code is very awkward and could be much more streamlined as well.
It would probably look something like this?
CSV.open("output/redemptions.csv", "wb") do |csv|
csv << [nationality_array]
csv << [nationality_count]
end
Can anyone provide any insight into a cleaner way to go about this?
You could use a Hash to group the counts by nationality instead of different arrays.
nationalities_count = student_array.each_with_object(Hash.new(0)) do |student, hash|
nationality = student[2]
hash[nationality] += 1
end
That will give you a Hash that would look like
{ "American" => 2, "Canadian" => 1 }
You could then use Hash#to_a and Array#transpose like so:
hsh = { "American" => 2, "Canadian" => 1 }
=> {"American"=>2, "Canadian"=>1}
2.4.2 :002 > hsh.to_a
=> [["American", 2], ["Canadian", 1]]
2.4.2 :003 > hsh.to_a.transpose
=> [["American", "Canadian"], [2, 1]]
Finally, to output the CSV file all you need to do is write the arrays into the file
nationalities_with_count = hash.to_a.transpose
CSV.open("output/redemptions.csv", "wb") do |csv|
csv << nationalities_with_count[0]
csv << nationalities_with_count[1]
end
Array#group_by in Ruby core and Hash#transform_values in ActiveSupport are two very versitile methods that can be used here:
require 'active_support/all'
require 'csv'
student_array = [
["Mike", 13, "American", "male"],
["Grace", 12, "Canadian", "female"],
["Joey", 13, "American", "male"],
["Lily", 13, "American", "female"]
]
counts = student_array.group_by { |attrs| attrs[2] }.transform_values(&:length)
# => => {"American"=>3, "Canadian"=>1}
CSV.open("output/redemptions.csv", "wb") do |csv|
csv << counts.keys
csv << counts.values
end
puts File.read "output/redemptions.csv"
# => American,Canadian
# 3,1
.group_by { |attrs| attrs[2] } turns the array into a hash, where keys are the unique values for attrs[2], and values are a list of elements that have that attrs[2]. At this point you can use transform_values to turn those values into numbers representing their length (meaning, how many elements have that specific attrs[2]). The keys and values can then be extracted from the hash as separate arrays.
You even don’t need a CSV tool here:
result =
student_array.
map { |a| a[2] }. # get nationalities
group_by { |e| e }. # hash
map { |n, c| [n, c.count] }. # map values to count
transpose. # put data in rows
map { |row| row.join ',' }. # join values in a row
join($/) # join rows
#⇒ American,Canadian
# 3,1
Now you have a string that is valid CSV, just spit it out to the file.

ruby split initial file contents into array

I have a file that I am reading that follows the following format:
12345,500,500
23456,100,150
34567,99,109
What I'm trying to do is read up until the first comma of the file and then map them into an array.
test = File.read('results.txt').split(',')[0]
p test
=> "12345"
would return me back the first value before the comma but I want to put all of them into an array
test = File.read('results.txt').split(',')[0].map(&:strip)
I have tried the following above and other similar permutations but unfortunately it's not quite the right it seems.
my desired result is to have an array of the following
[12345,23456,34567]
Here are a couple of ways to do that. First create the file.
txt =<<_
12345,500,500
23456,100,150
34567,99,109")
_
FName = "tmp"
File.write(FName, txt)
#=> 43
#1
File.foreach(FName).map { |line| line[0, line.index(',')] }
#=> ["12345", "23456", "34567"]
#2
File.foreach(FName).map { |line| line.to_i.to_s }
#=> ["12345", "23456", "34567"]
IO#foreach reads the file line-by-line, as contrasted by IO#readlines, which "gulps" the entire file into an array. foreach is therefore less demanding of memory than readlines. You can write either IO.foreach... or File.foreach... as File is a subclass of IO (File < IO #=> true).
File.readlines('results.txt').map { |line| line.split(',') }.map(&:first)
=> ["12345", "23456", "34567"]

When I try to make an array in Ruby from a file, it makes multiple arrays

sorry if this is a simple solution, but I'm trying to grab all the numbers from a file into an array. The file I'm importing reads
3 5 10
2 7 15
This is my code
grab = Array.new
IO.foreach("test.txt") do |line|
grab = line.chomp(" " + "/n").split
end
p grab
When I do this, it prints the last line of numbers, yet when I print it from the IO block, it prints 2 arrays. First and second line.
If you want to consolidate this into a singular array:
grab = [ ]
IO.foreach("test.txt") do |line|
grab.concat(line.chomp.split)
end
Another way:
grab = IO.readlines("test.txt").flat_map do |line|
line.chomp.split
end
You should do something like
grab << line.chomp(" " + "/n").split
This gives you what you're looking for:
grab = IO.foreach("test2.txt").map {|line| line.split(/\D/) }.flatten
puts "#{grab}"
This produces the single array output:
["3", "5", "10", "2", "7", "15"]
It can all be wrapped up into a nice, tidy one-liner if you prefer it that way:
grab = IO.foreach("test2.txt").map {|line| line.split(/\D/) }.flatten
puts "#{grab}"
If you want to do an integer conversion in the process, you can modify the line.split to this:
grab = line.split(/\D/).map(&:to_i)
That change will produce this output:
[3, 5, 10, 2, 7, 15]

Why does my array read from file contain bytes

I'm writing a little system that parses lines of data in a txt separated by commas,
so to be basic about it I read the file lines into an array, then use .each on the array and split everything by "'" then push it into the holding array which is returned as the database made from the file, I have made two, the first works fine but its data is stored line by line with a keyword, this one works fine, access and return all good.
I'm using a file containing text data like this
476,TACKLE,40,25,30,0,0,1,A3F,move description string with, punctuation and t's
477,ANOTHERATTACK,BLAHBLAHBLAH,1,2,3,4
This would be data parsing kind of right, well
so I go:
$fs = File_SYstem.new
#path = Dir.getwd.to_S + "/desktop/file.txt"
#data_lines = $fs.file_read_lines(#path)
#data = []
#data_lines.each do |line|
#data >> line.split(',')
end
return #data
#this would make an array of the lines, each line being an array of its elements, right?
#data = The_Code_Above_In_A_Class.new(#path)
=>#data
#data[0]
=>"354,FISSURE,10,40,50,blah blah blah, the second half of the text."
#hmmmm
#data[0][0]
=>"354"
So it seems to work fine, but some times, the numbers at the beginning come back as bytes :O
And for example:
#data.each do |line|
puts line[1].to_S #return second element which is name of move
end
This would print a list of the expected names, fine and dandy, but then I get the remaining data I didn't ask for returned below it in an unrecognizable pattern.
Maybe I can do this?
array = [1,2,3]
array = [array,array,array]
array[2][0] = "Hello!"
array.each do |item|
puts item[2]
end
=>"3"
"3"
"Hello!"
=>:
Seems to me this should work since I'm already using close variations of this style somewhere else with success.
Now this is a sample of the real 580 line file:
1,MEGAHORN,Megahorn,000,120,BUG,Physical,85,10,0,00,0,abef,Cool,"Using its tough and impressive horn, the user rams into the target with no letup."
2,ATTACKORDER,Attack Order,000,90,BUG,Physical,100,15,0,00,0,befh,Smart,The user calls out its underlings to pummel the target. Critical hits land more easily.
3,BUGBUZZ,Bug Buzz,046,90,BUG,Special,100,10,10,00,0,bek,Cute,The user vibrates its wings to generate a damaging sound wave. It may also lower the target's Sp. Def stat.
Now this is the class I use to load it up:
class Move_Data_Extracter
def initialize(path)
load $path.to_s + "/source/string_helper.rb"
#load "/mnt/sdcard/pokemon/system/source/string_helper.rb"
#path = path.to_s
#file_lines = $file_system.file_read_lines(#path.to_s)
$movedata = []
#file_lines.each do |line|
$movedata << line.split(",")
end
end
def get_move_id(move_name)
$movedata.each do |move|
if move[1].upcase.to_s == move_name.upcase.to_s
return move[0].to_i
else
return "Move Doesnt Exist In The System!"
end
end
end
end
This is the feedback I got when I accessing the first item in the returned array(s?):
irb(main):002:0> $movedata[0]
=> ["\xEF\xBB\xBF1", "MEGAHORN", "Megahorn", "000", "120", "BUG", "Physical", "8
5", "10", "0", "00", "0", "abef", "Cool", "\"Using its tough and impressive horn
", " the user rams into the target with no letup.\"\n"]
irb(main):003:0> $movedata[0][0]
=> "\xEF\xBB\xBF1"
irb(main):004:0>
Access worked ok this time but the first element is bytes and that each method I'm trying is going so wrong.
Can anyone figure out whats wrong here?
First thing, that's obviously not the code you're using as things like to_S aren't part of ruby and would instantly fail anyway.
Let's clean up the code a bit:
# $fs = File_SYstem.new # this is just not needed
path = File.expand_path "/desktop/file.txt" # instance variables *only* within explicit objects
data_lines = File.read( path ).split ","
I've no idea what the rest of what you've written really means.
This outputs:
# => ["476", "TACKLE", "40", "25", "30", "0", "0", "1", "A3F", "move description string with", " punctuation and t's\n477", "ANOTHERATTACK", "BLAHBLAHBLAH", "1", "2", "3", "4"]
This bit of code - what is it?
array = [1,2,3]
array = [array,array,array] # pure craziness!
array[2][0] = "Hello!"
array.each do |item|
puts item[2]
end
=>"3"
"3"
"Hello!"
=>:
As to why you're getting back bytes, it's because the file is (likely) encoded as UTF-8. Try File.read( path, "r:UTF-8") to get Ruby to use the correct encoding.

Resources