How to get all different part in two array? - arrays

Question

Suppose I have a sentence:
When Grazia Deledda submitted a short story to a fashion magazine at the age of 13
this sentence is split into two list:
# list 1
[When] [Grazia Deledda] [submitted a short story] [to] [a] [fashion magazine] [at] [the age of] [13]
# list 2
[When] [Grazia Deledda] [submitted] [a short story] [to] [a fashion] [magazine at] [the age of] [13]
Now I want to get the different parts in this two array, this example's result should be:
[
([[submitted a short story]],[[submitted] [a short story]]),
([[a] [fashion magazine] [at]], [[a fashion] [magazine at]])
]
so it should meet these requirements:
every pair should have the same content, for example: [[submitted a short story]] can be joined into 'submitted a short story', and [[submitted] [a short story]] can also be joined into 'submitted a short story'
every pair should have the same start position and end position, for example: [[submitted a short story]] the starts at 3, and ends with 6. [[submitted] [a short story] are the same.
the most important is that every one should be the shortest, for example [[submitted a short story] [to]] and [[submitted] [a short story] [to]] also meets the first two requirements, but it is not the shortest.
Any way to avoid O(n^2) complexity？

Answer 1

I may got wrong direction at beginning, this question can be easy, I have think a good idea:
#!/usr/bin/env python
# encoding: utf-8
# list 1
llist = [["When"], ["Grazia", "Deledda"], ["submitted", "a", "short", "story"], ["to"], ["a"], ["fashion", "magazine"], ["at"], ["the", "age", "of",], ["13"],]
# list 2
rlist = [["When"], ["Grazia", "Deledda"], ["submitted"], ["a", "short", "story"], ["to"], ["a", "fashion"], ["magazine", "at"], ["the", "age", "of",], ["13"],]
loffset = -1
roffset = 0
rindex = 0
lstart = -1
rstart = -1
for lindex, litem in enumerate(llist):
if loffset == roffset and litem != rlist[rindex]:
lstart = lindex
rstart = rindex
loffset += len(litem)
while roffset < loffset:
roffset += len(rlist[rindex])
rindex += 1
if loffset == roffset and lstart >= 0:
print(llist[lstart:lindex+1], rlist[rstart:rindex])
lstart = -1

Answer 2

I tokenize all the words and pad them as a sequence as a list of list. Then I compare the first list against the second building string buffers and match when the index length counts differ. I then remove the duplicate indexe values for out1 and out2 at the end
from keras.preprocessing.text import Tokenizer
tokenizer=Tokenizer()
# list 1
list1 = [["When"], ["Grazia Deledda"], ["submitted a short story"], ["to"],
["a"], ["fashion magazine"], ["at"], ["the age of"], ["13"],["EOS"]]
# list 2
list2 = [["When"], ["Grazia Deledda"], ["submitted"], ["a short story"], ["to"],
["a fashion"], ["magazine at"], ["the age of"], ["13"],["EOS"]]
tokenizer.fit_on_texts([" ".join(item) for item in list1])
tokenizer.fit_on_texts([" ".join(item) for item in list2])
seq1=[]
seq2=[]
for item1,item2 in zip(list1,list2):
seq1.append(tokenizer.texts_to_sequences(item1))
seq2.append(tokenizer.texts_to_sequences(item2))
out1=[]
out2=[]
out1_buffer=[]
out2_buffer=[]
current_index=0
string1=""
for seq1_index in range(len(seq1)-1):
string1=""
index=0
out1_buffer=[]
found=False
#check each seq1 string accumulation until a match is found or the end of queue is detect 16 - maps to eos
while seq1[seq1_index+index][0] != [16] and found==False:
out1_buffer.append(seq1_index+index)
seq_string=" ".join([str(token) for token in seq1[seq1_index+index][0]])
if string1=="":
string1=seq_string
else:
string1+=" "+seq_string
string2=""
out2_buffer=[]
for seq2_index in range(current_index,len(seq2)-1):
seq_string=" ".join([str(token) for token in seq2[seq2_index][0]])
if string2=="":
string2=seq_string
else:
string2+=" "+seq_string
out2_buffer.append(seq2_index)
count_seq1=len(out1_buffer)
count_seq2=len(out2_buffer)
if string1==string2 and count_seq1!=count_seq2:
print("string_a", [list1[int(index)] for index in out1_buffer])
print("string_b",[list2[int(index)] for index in out2_buffer])
current_index=seq2_index+1
print("match",count_seq1,count_seq2)
for index1 in out1_buffer:
out1.append(index1)
for index2 in out2_buffer:
out2.append(index2)
out1_buffer=[]
out2_buffer=[]
found=True
break
index+=1
tuple1=[]
tuple2=[]
result1=[]
for item1 in out1:
found=False
for item2 in out2:
if list1[item1]==list2[item2]:
found=True
break
if found==True:
out2 = list(filter(lambda item2: list1[item1]!=list2[item2],out2))
if found==False:
result1.append(item1)
for item1 in result1:
tuple1.append(list1[item1])
for item2 in out2:
tuple2.append(list2[item2])
tuple1=tuple(tuple1)
tuple2=tuple(tuple2)
print("{}\n{}\n".format(tuple1,tuple2))
output
(['submitted a short story'], ['a'], ['fashion magazine'], ['at'])
(['submitted'], ['a short story'], ['a fashion'], ['magazine at'])