React app not refreshing the log streamed through socket.io - reactjs

In my project I am using flask_socketio as the server and socket.io-client with react as the client. My main.py(flask server) is constantly reading a log file(console.log) that is continuously updating. When the start button is clicked in the UI, the data of the log file is displayed, but as my log file is updated, the updated data is not displayed in the UI. I have to refresh the page or click the button again to see the updated data.
I want the data of the log file to live stream on the UI with a button click. How to fix this?
flask code
from flask import Flask, jsonify
# Needed for localhost testing.
from flask_cors import CORS, cross_origin
from flask_socketio import SocketIO, emit
from time import sleep
import pdb
import json
app = Flask(__name__)
# Socket io setup.
app.config['SECRET_KEY'] = 'secret!'
# |cors_allowed_origins| is required for localhost testing.
socket = SocketIO(app, cors_allowed_origins="*")
# For localhost testing.
CORS(app)
#socket.on('event_stream', namespace='/logConsole')
def test_connect():
def generate():
fname = "./src/console.log"
with open(fname) as f:
yield f.read()
emit_data = next(generate())
socket.sleep(0)
emit('custom-server-msg', {'data': emit_data})
if __name__ == '__main__':
socket.run(app)
React code
import React from 'react'
import io from 'socket.io-client'
class App extends React.Component {
state = { startVar: true, setVar: false };
setSocketListeners() {
let socket = io.connect('ws://localhost:5000/logConsole');
socket.emit('event_stream', () => {
console.log("Websocket connected: " + socket.connected)
})
socket.on('custom-server-msg', (data) => {
console.log("Data received: " + data.data)
const setup_logs = data.data;
this.setState({ setup_logs });
})
}
render() {
return (
<div className="App">
<h1>Data from log file</h1>
<button type="button" onClick={() => this.setSocketListeners()}>Start</button>
<p>{this.state.setup_logs}</p>
</div>
);
}
}
export default App;
This is how my browser console looks like-->
And this is my backend console-->

In your flask code, if you want to stream continuously, the next() needs to be called in a loop, now either that can be done by putting a infinite loop with a sleep time,
#socket.on('event_stream')
def test_connect():
def generate():
fname = "./src/console.log"
with open(fname, "r+") as f:
yield f.read()
while True:
emit_data = next(generate())
socket.sleep(2)
emit('custom-server-msg', {'data':emit_data})
or else, if the log file is too being updated continuously, os.stat(FILE_NAME).st_mtime can be used which will check the time stamp of the file being updated, and if any change is there in the log file, next() will be called to stream it:
#socket.on('event_stream')
def test_connect():
cached_stamp = 0
def generate():
fname = "./src/console.log"
with open(fname, "r+") as f:
yield f.read()
while True:
stamp = os.stat('./src/console.log').st_mtime
if stamp != cached_stamp:
cached_stamp = stamp
emit_data = next(generate())
emit('topo-server-msg', {'data':emit_data})

Related

How to upload files with DRF and React?

I'm learning Django Rest Framework, everything was going well until I had to create a service to upload files to my application.As much as I read the documentation, I can't understand it.
First I want to clarify that I am not a programming expert, I am a
newbie but I am here learning more every day.
From what I've managed to understand so far:
Documents and photos are not stored in the database. These files are stored in a folder.
This is correct ?
I have a form where it allows me to upload multiple files
example:
file.txt, document.doc, photo.png etc...
My view (Frontend):
import { useState } from "react";
import axios from "axios";
const Form = () => {
const [state_files, setState_Files] = useState(null);
const UploadFiles = function (event) {
setState_Files(event);
};
const InsertFiles = async function () {
const formData = new FormData();
for (let index = 0; index < state_files.length; index++) {
formData.append("files", state_files[index]);
}
await axios
.post("http://127.0.0.1:8000/api/v1/upload/", formData)
.then((response) => {
console.log(response.data);
})
.catch((error) => {
console.log(error);
});
};
return (
<>
<input
type="file"
name="files"
multiple
onChange={() => InsertFiles(event.target.files)}
/>
<button>Upload All files</button>
</>
);
};
export default Form;
Backend
url.py
path("upload/", Storage_View.as_view(), name="storage-index"),
storage/view.py
from rest_framework.views import APIView
from rest_framework.response import Response
from rest_framework.parsers import MultiPartParser
class Storage_View(APIView):
parser_classes = [MultiPartParser]
def put(self, request, filename, format=None):
file_obj = request.data['file']
# ...
# What do I have to do here?
# ...
return Response({'received data': request.data})
Questions:
Why don't I see the button to upload files using the DFR tool? see attached files
The documentation does not explain what I have to do inside the function comment
https://www.django-rest-framework.org/api-guide/parsers/
def put(...):
How do I specify the path where you have to store the files you upload?
Please I need guidance.
1: show your models for more detail. you have to use filefield or
imagefield in your model. You can google and read more about these
fields.
2: put is for update, write your post method to save data. You don't
have to really do anything serious there. just check if
serializer.is_Valid() for request.data and if valid do
serializer.save(). the file or image will be uploaded to upload_to
folder which you define in your model and you will get the link.
3: read more about upload_to in Django. you can define this with the
model field.
I used ModelViewSet and this is how the create method looks like -
def create(self, request, format=None):
data = request.data
if isinstance(data, list): # <- is the main logic
serializer = self.get_serializer(data=request.data, many=True)
else:
serializer = self.get_serializer(data=request.data)
if serializer.is_valid():
serializer.save()
return Response(serializer.data, status=status.HTTP_201_CREATED)
return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)
You are passing a list from frontend and by default, Django works on dictionary, so you will have to manage this. Think of this as your go to code and write your post method.
Feel free to write more if you have doubts, Hope this helps :)

Can't get file from POST request using Flask and Bootstrap-Vue Form File Input

I'm trying to upload and image using Bootstrap-Vue Form File Input and send it to Flask Backend via POST using Axios library, then store in a folder.
My problem is that Flask can't find "file" in "request.files". Pretty sure I'm falling in a rookie mistake.
That's my code:
Frontend:
<template>
<div class="mx-5 container-fluid">
<div class="mx-5 row">
<div class="col-sm-10">
<b-form-file
type="file"
id="file"
v-model="file"
:state="Boolean(file)"
ref="file"
placeholder="Choose a file or drop it here..."
drop-placeholder="Drop file here..."
v-on:change="submitFile"
></b-form-file>
</div>
</div>
</div>
</template>
<script>
import axios from 'axios';
export default {
data() {
return {
file: null,
};
},
methods: {
submitFile() {
/* Initialize the form data */
const path = 'http://localhost:5000/single-file';
const formData = new FormData();
/* Add the form data we need to submit */
formData.append('file', this.file);
/* Make the request to the POST /single-file URL */
axios.post(path,
formData,
{
headers: {
'Content-Type': 'multipart/form-data',
},
}).then(() => {
// console.log('SUCCESS!!');
})
.catch(() => {
// console.log('FAILURE!!');
});
},
},
};
Backend:
from flask import Flask, jsonify, request, send_file, redirect, url_for
from werkzeug.utils import secure_filename
import os
# configuration
DEBUG = True
UPLOAD_FOLDER = '/images'
ALLOWED_EXTENSIONS = {'png', 'jpg', 'jpeg', 'gif'}
#app.route('/single-file', methods=['GET', 'POST'])
def upload_file():
if request.method == 'POST':
# check if the post request has the file part
if 'file' not in request.files:
print('No file part')
return redirect(request.url)
file = request.files['file']
# If the user does not select a file, the browser submits an
# empty file without a filename.
if file.filename == '':
print('No selected file')
return redirect(request.url)
if file and allowed_file(file.filename):
filename = secure_filename(file.filename)
file.save(os.path.join(app.config['UPLOAD_FOLDER'], filename))
return redirect(url_for('download_file', name=filename))
return ''
if __name__ == '__main__':
app.run()
I get HTTP code 302 (redirect) and print in console 'No file part'.
Any help would be very apreciated.
I can't see an obvious mistake in your code, it seems that the request is correctly passed through from your frontend and backend.
What I would suggest is to use Postman to decouple your frontend and backend in this case. If you get a correct response when using Postman, you can narrow down that the error is in the frontend, the browser, or something about axios which is meddling with the request data.
Also, try to get an error message, or print why flask thinks "file" isnt in request.files, it should be there if everything works as intended
I followed the response for Get the data received in a Flask request to get to the flask documentation for the Request class: Each key in files is the name from the <input type="file" name="">, which means that most likely you have to change 'file' from file = request.files['file'] to point to the actual filename that was selected from the file selection menu.

TFJS: loadGraphModel from HTTP server

I am implementing an object-detection web application using React and Tensorflow JS. I converted my model to a tensorflow JS model, such that I can load it into my React application. I want to load the model using a simple HTTP endpoint, which is a Flask server currently hosting on my local machine. The Flask main file looks as follows:
from flask import Flask
from flask_cors import CORS, cross_origin
import os
app = Flask(__name__)
cors = CORS(app)
#app.route('/')
def hello_world():
return 'Hello, World!'
#app.route('/model', methods=['GET'])
def get_modeljson():
"""
Get the model.json file and return it's contents.
"""
current_dir = os.getcwd()
file_path = os.path.join(current_dir, "models", "model.json")
with open(file_path, "r") as f:
return f.read()
if __name__ == '__main__':
app.run(debug=True, host="0.0.0.0", threaded=True)
I have written a function in my React application that loads the graph model using the endpoint /model that is defined in the code above. The React function looks as follows:
import {useEffect, useState} from 'react';
import * as tf from '#tensorflow/tfjs';
import {loadGraphModel} from '#tensorflow/tfjs-converter';
function Model(props) {
const [model, setModel] = useState();
async function loadModel() {
try {
const model_url = "http://127.0.0.1:5000/model";
const result = await fetch(model_url);
const result_json = await result.json();
const model = await loadGraphModel(result_json);
console.log('model loaded...')
setModel(model);
console.log("Model correctly loaded");
} catch (err) {
console.log(err);
console.log("failed load model");
}
}
useEffect(() => {
tf.ready().then(() => {
loadModel();
});
}, []);
async function predictFunction() {
// use model to make predictions
}
return (
<Button onClick={() => {
predictFunction();
}}
/>
);
}
export default Model;
The FLASK API returns correctly the model.json file, however loadGraphModel returns the following error:
TypeError: url.startsWith is not a function
at indexedDBRouter (indexed_db.ts:215)
at router_registry.ts:95
at Array.forEach (<anonymous>)
at Function.getHandlers (router_registry.ts:94)
at Function.getLoadHandlers (router_registry.ts:84)
at Module.getLoadHandlers (router_registry.ts:110)
at GraphModel.findIOHandler (graph_model.ts:107)
at GraphModel.load (graph_model.ts:126)
at loadGraphModel (graph_model.ts:440)
at loadModel (Model.js:16)
I can not find any documentation about url.startsWith. Who sees what is going wrong here?
Going through the code I see a major issue with it, where you are trying to basically send a model.json from the backend to the frontend and then load the model from that model.json and perform inference on it. It would work but it is not efficient at all. Imagine having to do this a couple hundred times and I know the model.json file can be big in size. Instead there are two routes that you could go with:
Host the model on the backend, send the data to the backend through a POST request and then make predictions on the data from the request.
Use the model on the frontend and then make predictions on the input data from there.
There are some errors in the code which are causing the error but this is the issue that you need to fix first. If you could give me more information about the inputs you are working with I could draft up a workable solution.

net::ERR_EMPTY_RESPONSE: only on deployed version

I get a net::ERR_EMPTY_RESPONSE error:
only on deployed version, not localhost
other GET-requests work
It downloads all my files from the ftp-server but it returns an error.:(
Flask Backend:
app = Flask(__name__)
cors = CORS(app)
#app.route('/downloadftp', methods=['POST'])
#cross_origin()
def download_all_ftp_data():
# connect to sever...
# download files...
for f in ftp.nlst():
fhandle = open(f, 'wb')
ftp.retrbinary('RETR ' + f, fhandle.write)
ftp.quit()
return 'OK 200'
React Frontend:
useEffect(() => {
axios
.post(`${process.env.REACT_APP_HOST}/downloadftp`, { content: "post" })
.then(res => {
console.log(res)
setError(false)
})
.catch(function (err) {
console.log(err)
setError(true)
})
}, [])
The error is due to the amount of data to download as it is described here: getting this net::ERR_EMPTY_RESPONSE error in browser
The solution for my problems was creating a daemon thread. The response is sent directly after starting the background thread.
from multiprocessing import Process
#app.route('/downloadftp')
#cross_origin()
def get_all_ftp_data():
"""creates a daemon thread for downloading all files while the response it sent directly after starting the background thread."""
daemon_process = Process(
target=download_data_multiprocessing,
daemon=True
)
daemon_process.start()
return Response(status=200)
def download_data_multiprocessing():
"""triggers download current data from ftp server"""
# connect to sever
# download
for f in ftp.nlst():
if (f != '.' and f != '..' and "meteo" not in f):
fhandle = open(f, 'wb')
ftp.retrbinary('RETR ' + f, fhandle.write)
ftp.quit()

React fine-uploader upload the file

I have a simple react app. I've added react-fine-uploader to it. I have a Flask server that accepts files and puts them into MongoDB database. The code for server looks like this:
from flask import Flask, render_template, request
from pymongo import MongoClient
import os
import time
from json import dumps, loads
app = Flask(__name__)
global index
map_dir = 'maps'
client = MongoClient(
'<connection_string>')
db = client.business
#app.route('/maps', methods=['GET'])
def process():
cursor = db.maps.find({}, {'_id': False})
maps = {'maps': []}
for doc in cursor:
try:
maps['maps'].append(doc['filename'])
except Exception:
pass
return dumps(maps)
#app.route('/map/<map_name>', methods=['GET'])
def get_map(map_name):
doc = db.maps.find_one({'filename': map_name}, {'_id': False})
return dumps(doc)
#app.route('/uploader', methods=['GET', 'POST'])
def upload_file():
if request.method == 'POST':
f = request.files['file']
parsed = loads(f.read())
filename = ''.join(f.filename.split('.')[:-1]) + str(index) + '.json'
parsed['filename'] = filename
try:
result = db.maps.insert_one(parsed)
return 'File uploaded successfully'
except Exception:
return 'Error while uploading a file'
if __name__ == '__main__':
global index
index = len(os.listdir('maps')) + 1
app.run(debug=True)
It works with standard HTML input form with specifying target as localhost:5000/uploader. Now I want my fine-uploader form to do the same. In code it looks like this:
const uploader1 = new FineUploader({
options: {
request: {
endpoint: "localhost:5000/uploader"
},
resume: {
enabled: true
},
retry: {
enableAuto: true,
showButton: true
}
}
});
And somewhere in the render() method I got: <Gallery uploader={uploader1} />
Now I can select file, but when it is selected the form tries to upload it and it fails. Server is running and I can see no request on him in the terminal. Is there something I am doing wrong?
#Edit I've enabled debug mode and it throws something like this into dev console:
Cross origin requests are only supported for protocol schemes: http, data, chrome, chrome-extension, https

Resources