What does the error "Exceeded maximum allowed number of Trackers" mean? - ios11

Question

I'm following this tutorial to implement object tracking on iOS 11. I'm able to track objects perfectly, until a certain point, then this error appears in the console.
Throws: Error Domain=com.apple.vis Code=9 "Internal error: Exceeded maximum allowed number of Trackers for a tracker type: VNObjectTrackerType" UserInfo={NSLocalizedDescription=Internal error: Exceeded maximum allowed number of Trackers for a tracker type: VNObjectTrackerType}
Am I using the API incorrectly, or perhaps Vision has trouble handling too many consecutive object tracking tasks? Curious if anyone has insight into why this is happening.

Answer 1

It appears that you hit the limit on the number of trackers that can be active in the system. First thing to note is that a new tracker is created every time a new observation, with new -uuid property is used. You should be recycling the initial observation you use when you started the tracker, until you no longer want to use it, by feeding what you got from “results” for time T into the subsequent request you make for time T+1. When you no longer want to use that tracker (maybe the confidence score gets too low), there is a “lastFrame” property that can be set, which lets the Vision framework know that you are done with that tracker. Trackers also get released when the sequence request handler is released.

Answer 2

To track the rectangle you feed consequent observations to the same VNSequenceRequestHandler instance, say, handler. When the rectangle is lost, i.e. the new observation is nil in your handler function / callback, or you are getting some other tracking error, just re-instantiate the handler and continue, e.g. (sample code to show the idea):
private var handler = VNSequenceRequestHandler()
// <...>
func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
guard
let pixelBuffer: CVPixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer),
let lastObservation = self.lastObservation
else {
self.handler = VNSequenceRequestHandler()
return
}
let request = VNTrackObjectRequest(detectedObjectObservation: lastObservation, completionHandler: self.handleVisionRequestUpdate)
request.trackingLevel = .accurate
do {
try self.handler.perform([request], on: pixelBuffer)
} catch {
print("Throws: \(error)")
}
}
Note that handler is var, not a constant.
Also, you may re-instantiate the handler in actual handler function (like func handleVisionRequestUpdate(_ request: VNRequest, error: Error?)) in case new observation object is invalid.

Answer 3

My problem with this was that I had a function that called perform... on the same VNSequenceRequestHandler that the tracking was also calling perform on, because of that I was processing too many try self.visionSequenceHandler.perform(trackRequests, on: ciimage) concurrently. Make sure the VNSequenceRequestHandler is not getting hit at the same time by multiple performs....