Tensorflow.js BodyPix very slow in browser for single image - tensorflow.js

Question

I'm trying to create an in-browser background removal using tensorflow.js body-pix
While the online demo reaches ~10 frames per second on my computer, my code takes ~3 seconds for a single frame.
I mostly followed the example on the official github
Here is my code:
const remove_background = async (img) => {
console.time("ML");
console.time("loadModel");
const net = await bodyPix.load({
architecture: 'MobileNetV1',
outputStride: 16,
multiplier: 0.5,
quantBytes: 2
});
console.timeEnd("loadModel");
console.time("segmentPerson");
const segmentation = await net.segmentPerson(img.imageData, {
segmentationThreshold: 0.7,
internalResolution: 0.25
});
console.timeEnd("segmentPerson");
console.time("ApplyMask");
for (var i=0;i<segmentation.data.length;i++) {
if(segmentation.data[i] === 0) {
img.imageData.data[i*4] = 255;
img.imageData.data[i*4+1] = 0;
img.imageData.data[i*4+2] = 0;
}
}
console.timeEnd("ApplyMask");
console.timeEnd("ML");
return img;
}
And here are the times:
loadModel: 129.35498046875 ms
segmentPerson: 2755.817138671875 ms
ApplyMask: 4.910888671875 ms
ML: 2890.7060546875 ms
I started with a 1700x1700 Image with internal resolution of 0.1, and worked down to a 200x200 image with 0.25 resolution, with no significant improvement.
What am I doing wrong?
I'm using tfjs#1.2 & body-pix#2.0

Answer 1

I have found in my implementation using bodypix that the first detection takes quite a bit longer than subsequent ones. I believe they do one-time initialization on the first call to segmentPerson. We are segmenting video. I found I could save about 500ms of startup time of the first detection just by passing a blank image to segmentPerson while the stream is being acquired before passing the video frame by frame when the stream has started.