TFJS: loadGraphModel from HTTP server - reactjs

Question

I am implementing an object-detection web application using React and Tensorflow JS. I converted my model to a tensorflow JS model, such that I can load it into my React application. I want to load the model using a simple HTTP endpoint, which is a Flask server currently hosting on my local machine. The Flask main file looks as follows:
from flask import Flask
from flask_cors import CORS, cross_origin
import os
app = Flask(__name__)
cors = CORS(app)
#app.route('/')
def hello_world():
return 'Hello, World!'
#app.route('/model', methods=['GET'])
def get_modeljson():
"""
Get the model.json file and return it's contents.
"""
current_dir = os.getcwd()
file_path = os.path.join(current_dir, "models", "model.json")
with open(file_path, "r") as f:
return f.read()
if __name__ == '__main__':
app.run(debug=True, host="0.0.0.0", threaded=True)
I have written a function in my React application that loads the graph model using the endpoint /model that is defined in the code above. The React function looks as follows:
import {useEffect, useState} from 'react';
import * as tf from '#tensorflow/tfjs';
import {loadGraphModel} from '#tensorflow/tfjs-converter';
function Model(props) {
const [model, setModel] = useState();
async function loadModel() {
try {
const model_url = "http://127.0.0.1:5000/model";
const result = await fetch(model_url);
const result_json = await result.json();
const model = await loadGraphModel(result_json);
console.log('model loaded...')
setModel(model);
console.log("Model correctly loaded");
} catch (err) {
console.log(err);
console.log("failed load model");
}
}
useEffect(() => {
tf.ready().then(() => {
loadModel();
});
}, []);
async function predictFunction() {
// use model to make predictions
}
return (
<Button onClick={() => {
predictFunction();
}}
/>
);
}
export default Model;
The FLASK API returns correctly the model.json file, however loadGraphModel returns the following error:
TypeError: url.startsWith is not a function
at indexedDBRouter (indexed_db.ts:215)
at router_registry.ts:95
at Array.forEach (<anonymous>)
at Function.getHandlers (router_registry.ts:94)
at Function.getLoadHandlers (router_registry.ts:84)
at Module.getLoadHandlers (router_registry.ts:110)
at GraphModel.findIOHandler (graph_model.ts:107)
at GraphModel.load (graph_model.ts:126)
at loadGraphModel (graph_model.ts:440)
at loadModel (Model.js:16)
I can not find any documentation about url.startsWith. Who sees what is going wrong here?

Answer 1

Going through the code I see a major issue with it, where you are trying to basically send a model.json from the backend to the frontend and then load the model from that model.json and perform inference on it. It would work but it is not efficient at all. Imagine having to do this a couple hundred times and I know the model.json file can be big in size. Instead there are two routes that you could go with:
Host the model on the backend, send the data to the backend through a POST request and then make predictions on the data from the request.
Use the model on the frontend and then make predictions on the input data from there.
There are some errors in the code which are causing the error but this is the issue that you need to fix first. If you could give me more information about the inputs you are working with I could draft up a workable solution.