Incorporating fuzzy search in a matcher object - c

Question

My task is querying medical texts for institute names using a rule as below:
[{'ENT_TYPE': 'institute_name'}, {'TEXT': 'Hospital'}]
The rule will only idenify a match if both terms are included therein. Thus, it will accept "Mount Sinai Hospital", but not "Mount Sinai". I've tried spaczz that wraps spaCy and works great for single term or phrase. However neither spaCy not spaczz allow for a fuzzy multi-words rule with more than one typo as in "Moung Sinai Mospital."
Therefore, I'm trying to re-write the Matcher object by incorporating a fuzzy similarity algorithm such as RapidFuzz but I'm having some difficulty with its Cython component.
The Matcher's Class call method finds all token sequences matching the supplied patterns on doclike, the document to match over or a Span (Type: Doc/Span), returning
a list of (match_id, start, end) tuples, describing the matches:
matches = find_matches (&self.patterns[0], self.patterns.size(), doclike, length,
extensions=self._extensions, predicates=self._extra_predicates)
for i, (key, start, end) in enumerate(matches):
on_match = self._callbacks.get(key, None)
if on_match is not None:
on_ma
return matches
find_matches is a cython class that returns the matches in a doc, with a compiled array of patterns as a list of (id, start, end) tuples and has main loop that seems to match the doc against the pre-defined patterns:
# Main loop
cdef int nr_predicate = len(predicates)
for i in range(length):
for j in range(n):
states.push_back(PatternStateC(patterns[j], i, 0))
transition_states(states, matches, predicate_cache,
doclike[i], extra_attr_values, predicates)
extra_attr_values += nr_extra_attr
predicate_cache += len(predicates)
Can you help me locate the actual matching operation (pattern against string) in the python/C-level objects as attributes? I hope to be able to extend this operation with the fuzzy matching algorithm. You can find the code for the Matcher class, the call method and the find_matches class here.
You can follow a more pythonic effort to achieve this goal by spaczz here.

Answer 1

I think the easiest way would be to add an additional predicate type called something like FUZZY. Look at how the regex, set, and comparison predicates are defined and do something similar for FUZZY with your custom code to match on strings with small edit differences:
https://github.com/explosion/spaCy/blob/master/spacy/matcher/matcher.pyx#L687-L781
The predicate classes are standard python classes, no cython required. You'll also need to add the predicate to the schema in spacy/matcher/_schemas.py.
Remember that like the rest of the Matcher predicates, it matches over tokens, so your definition of fuzziness will have to be at the token level.