Appending to an array value in a hash - arrays

Question

I'm parsing multiple website and trying to build a hash that looks something like:
"word" => [[01.html, 2], [02.html, 7], [03.html, 4]]
where word is a given word in the index, the first value in each sublist is the file it was found in, and the second value is the number of occurrences in that given file.
I'm running into an issue where, rather than appending ["02.html", 7] inside the values list, it creates a whole new entry for "word" and puts ["02.html", 7] at the end of the hash. This results in basically giving me individual indexes for all of my websites appended after each other rather than giving me one master index.
Here is my code:
for token in tokens
if !invindex.include?(token)
invindex[token] = [[doc_name, 1]] #adds the word to the hash with the doc name and occurrence of 1
else
for list in invindex[token]
if list[0] == doc_name
list[1] += 1 #adds one to the occurrence with the same doc_name
else
invindex[token].insert([doc_name, 1]) #this SHOULD append the doc name and initial occurrence inside the word's value list since the word is already in the hash
end
end
end
end
end
Hopefully it's something simple and I just missed something when I traced it on paper.

Answer 1

I'm running into an issue where, rather than appending ["02.html", 7]
inside the values list, it creates a whole new entry for "word" and
puts ["02.html", 7] at the end of the hash.
I'm not seeing that:
invindex = {
word1: [
['01.html', 2],
]
}
tokens = %i[
word1
word2
word3
]
doc_name = '02.html'
tokens.each do |token|
if !invindex.include?(token)
invindex[token] = [[doc_name, 1]] #adds the word to the hash with the doc name and occurrence of 1
else
invindex[token].each do |list|
if list[0] == doc_name
list[1] += 1 #adds one to the occurrence with the same doc_name
else
invindex[token].insert([doc_name, 1]) #this SHOULD append the doc name and initial occurrence inside the word's value list since the word is already in the hash
end
end
end
end
p invindex
--output:--
{:word1=>[["01.html", 2]], :word2=>[["02.html", 1]], :word3=>[["02.html", 1]]}
invindex[token].insert([doc_name, 1]) #this SHOULD append the doc name
Nope:
invindex = {
word: [
['01.html', 2],
]
}
token = :word
doc_name = '02.html'
invindex[token].insert([doc_name, 7])
p invindex
invindex[token].insert(-1, ["02.html", 7])
p invindex
--output:--
{:word=>[["01.html", 2]]}
{:word=>[["01.html", 2], ["02.html", 7]]}
Array#insert() requires that you specify an index as the first argument. Typically when you want to append something to the end, you use <<:
invindex = {
word: [
['01.html', 2],
]
}
token = :word
doc_name = '02.html'
invindex[token] << [doc_name, 7]
p invindex
--output:--
{:word=>[["01.html", 2], ["02.html", 7]]}
for token in tokens
Rubyists don't use for-in loops because for-in loops call each(), so rubyists call each() directly:
tokens.each do |token|
...
end
Finally, indenting in ruby is 2 spaces--not 3 spaces, not 1 space, not 4 spaces. It's 2 spaces.
Applying all that to your code:
invindex = {
word1: [
['01.html', 2],
]
}
tokens = %i[
word1
word2
word3
]
doc_name = '01.html'
tokens.each do |token|
if !invindex.include?(token)
invindex[token] = [[doc_name, 1]] #adds the word to the hash with the doc name and occurrence of 1
else
invindex[token].each do |list|
if list[0] == doc_name
list[1] += 1 #adds one to the occurrence with the same doc_name
else
invindex[token] << [doc_name, 1] #this SHOULD append the doc name and initial occurrence inside the word's value list since the word is already in the hash
end
end
end
end
p invindex
--output:--
{:word1=>[["01.html", 3]], :word2=>[["01.html", 1]], :word3=>[["01.html", 1]]}
However, there is still a problem, which is due to the fact that you are changing an Array that you are stepping through--a big no-no in computer programming:
invindex[token].each do |list|
if list[0] == doc_name
list[1] += 1 #adds one to the occurrence with the same doc_name
else
invindex[token] << [doc_name, 1] #***PROBLEM***
Look what happens:
invindex = {
word1: [
['01.html', 2],
]
}
tokens = %i[
word1
word2
word3
]
%w[ 01.html 02.html].each do |doc_name|
tokens.each do |token|
if !invindex.include?(token)
invindex[token] = [[doc_name, 1]] #adds the word to the hash with the doc name and occurrence of 1
else
invindex[token].each do |list|
if list[0] == doc_name
list[1] += 1 #adds one to the occurrence with the same doc_name
else
invindex[token] << [doc_name, 1] #this SHOULD append the doc name and initial occurrence inside the word's value list since the word is already in the hash
end
end
end
end
end
p invindex
--output:--
{:word1=>[["01.html", 3], ["02.html", 2]], :word2=>[["01.html", 1], ["02.html", 2]], :word3=>[["01.html", 1], ["02.html", 2]]}
Problem 1: You don't want to insert [doc_name, 1] every time the sub Array you are examining doesn't contain the doc_name--you only want to insert [doc_name, 1] after ALL the sub Arrays have been examined, and the doc_name wasn't found. If you run the example above with the starting hash:
invindex = {
word1: [
['01.html', 2],
['02.html', 7],
]
}
...you will see that the output is even worse.
Problem 2: Appending [doc_name, 1] to the Array while you are stepping through the Array means that [doc-name, 1] will be examined, too, when the loop gets to the end of the Array--and then your loop will increment its count to 2. The rule is: don't change an Array you are stepping through because bad things will happen.

Answer 2

Do you actually need to have a hash that contains an array of arrays?
This can be much better described with a nested hash
invindex = {
"word" => { '01.html' => 2, '02.html' => 7, '03.html' => 4 },
"other" => { '01.html' => 1, '02.html' => 17, '04.html' => 4 }
}
which can be easily populated by using a Hash factory like
invindex = Hash.new { |h,k| h[k] = Hash.new {|hh,kk| hh[kk] = 0} }
tokens.each do |token|
invindex[token][doc_name] += 1
end
now if you absolutely need to have the format you mention you can get it from the described invindex with a simple iteration
result = {}
invindex.each {|k,v| result[k] = v.to_a }

Answer 3

Suppose:
arr = %w| 01.html 02.html 03.html 02.html 03.html 03.html |
#=> ["01.html", "02.html", "03.html", "02.html", "03.html", "03.html"]
is an array of your files for a given word in the index. Then the value of that word in the hash is given by constructing the counting hash:
h = arr.each_with_object(Hash.new(0)) { |s,h| h[s] += 1 }
#=> {"01.html"=>1, "02.html"=>2, "03.html"=>3}
and then converting it to an array:
h.to_a
#=> [["01.html", 1], ["02.html", 2], ["03.html", 3]]
so you could write:
arr.each_with_object(Hash.new(0)) { |s,h| h[s] += 1 }.to_a
Hash::new is given a default value of zero. That means that if the hash being constructed, h, does not have a key s, h[s] returns zero. In that case:
h[s] += 1
#=> h[s] = h[s] + 1
# = 0 + 1 = 1
and when the same value of s in arr is passed to the block:
h[s] += 1
#=> h[s] = h[s] + 1
# = 1 + 1 = 2
You may consider whether it would be better to make the value of each word in the index the hash h.