Passing base64 to google vision gives 400 error - reactjs

I am grabbing frames from the webcam, converting each image bitmap into a base64 string then passing that to the Google vision API. When i do this i am catching an error but it only logs as true. Im new to react and am struggling to see what i am missing.
grabFrame() {
let mediaStreamTrack = this.state.mediaStream.getVideoTracks()[0];
let imageCapture = new window.ImageCapture(mediaStreamTrack);
return imageCapture.grabFrame();
}
uploadFrame() {
this.grabFrame()
.then(function(bitmapImage) {
var canvas = document.createElement("canvas")
canvas.width = bitmapImage.width;
canvas.height = bitmapImage.height;
let context = canvas.getContext("2d")
context.drawImage(bitmapImage, 0, 0);
let base64Image = canvas.toDataURL("image/png")
const request = new vision.Request({
image: new vision.Image({
base64: base64Image,
}),
features: [ new vision.Feature('FACE_DETECTION') ]
})
vision.annotate(request)
.then((response) => {
console.log(`Response: ${response}`)
})
.catch((error) => {
console.log(`Error: ${error}`) >>>> "Error: true"
});
}).catch((error) => {
console.log('grabFrame() error: ', error)
});
}
In the console, all I can see is POST https://vision.googleapis.com/v1/images:annotate?key=xxxxxxxxxxxxxxxxxxx 400
Logging base64Image gives data:image/png;base64,iVBORw0KGgoAA...
Am I missing something?

Related

Choppy sound on MediaRecorder when playing video along side the recording

I'm trying to build a website with React that let me record myself using MediaRecorder with another video running in the background, but when the other video volume is up the sound of the record is choppy and scrappy, when I turn the video volume down the sound of the record is good, at first I was thinking the problem happends when the sound of the other video is also recorded and disturbe the the actual record, but even when I'm using a microphone and headphones the problem seems to happend, I think its something in my code.
This is the function that init the recorder:
const initVideoStream = async () => {
const stream = await navigator.mediaDevices.getUserMedia(constraints).then(async function (stream) {
const options = {
mimeType: 'audio/webm'
}
recorder.current = new MediaRecorder(stream, options)
recorder.current.ondataavailable = e => {
recordedChunks.current.push(e.data)
}
recorder.current.onstop = async e => {
setisBlob(true)
setVideoBlob(new Blob(recordedChunks.current, { 'type': 'video/mp4' }))
const videoUrlObj = window.URL.createObjectURL(new Blob(recordedChunks.current, { 'type': 'video/mp4' }))
recorderRef.current.src = videoUrlObj
recordedChunks.current = []
}
return stream
}).catch(function (error) {
console.log(error);
setErrorMsg('Cannot find any camera,please check if your camera is connected and try again')
setIsPopupOpen(true)
});
recordedChunks.current = []
setStream(stream)
}
I've tried to change the mimeType to "video/mp4" and that didn't help.
I've upload my project to netlidy so you can see the problem:
https://recorder-poc.netlify.app/

Imgly plugin is not exporting captured image

I have an Ionic app built with angular. I have integrated Imgly in my app. But it is not working properly. After editing an image the plugin is not exporting that image. I have looked into many documentation's but haven't found any solution on this.
Method capturing image and opening Imgly plugin
captureImage2() {
if (this.usepicker) {
this.filePickerRef.nativeElement;
// this.filePickerRef.nativeElement.click();
// return;
}
const options: CaptureImageOptions = { limit: 1 };
this.mediaCapture.captureImage(options).then(
async (data: MediaFile[]) => {
const fullPath = data[0].fullPath;
const directoryPath = fullPath.substring(0, fullPath.lastIndexOf("/"));
const fileName = fullPath.substring(fullPath.lastIndexOf("/") + 1);
console.log(
"File Details ======>>>>>>",
fullPath,
directoryPath,
fileName
);
PESDK.openEditor(
async (data1) => {
console.log("REEEEEE =====> ", JSON.stringify(data1));
await this.toProcessImage(data1.image);
const check = await this.displayImage(data1.image);
// console.log(check);
this.onButtonClick(data1.image);
// const dataURL = await this.file.readAsDataURL(data1.image, fileName);
// console.log("DATA URL ======>>>>>>" + dataURL);
data1
.save()
.then(async (sceneAsString) => {
const file = {
dataURI: sceneAsString,
directory: this.dic,
};
const fileResponse = await UriFileUpload(file);
if (fileResponse) {
console.log("image UPLOADED", JSON.stringify(fileResponse));
this.toastSer.success("uploaded");
// this.uploadedList = [...this.uploadedList , fileResponse1];
// console.log(this.uploadedList);
this.updateFiles.next(fileResponse);
} else {
this.toastSer.error("Somthing went wrong!");
}
})
.catch((error) => {
console.error("Save failed", error);
});
},
(error) => console.log(error),
fullPath
);
},
(err: CaptureError) => console.error(err)
);
}
IMG.LY team here!
We would like to provide a correct answer, but keep in mind that for this type of issue, a minimal example project is required in order to test what the different outputs are from the used camera and image picker plugin.
You can find Stack Overflow's suggestions here for a minimal project.
Feel free to submit your further queries to our Support Desk, we're always happy to help.

How to use custom fonts in #react-pdf/pdfkit

Can anyone say how to add custom fonts and use it with #react-pdf/pdfkit in React.js.
I tried to register custom font like this.
import utils from "./utils";
const SVGtoPDF = require("svg-to-pdfkit");
const PDFDocument = require("#react-pdf/pdfkit").default;
const { Base64Encode } = require("base64-stream");
const generatePDF = async (width, height, canvas, fonts) => {
const canvasSvg = canvas.toSVG();
return await new Promise((resolve, reject) => {
try {
var doc = new PDFDocument({
bufferPages: true,
size: [width * 0.75, height * 0.75],
});
utils.getFont(fonts).forEach(async (font) => {
doc.registerFont(font.family, font.path, font.family);
});
doc.font("Pacifico").text("Hello Message");
let finalString = "";
const stream = doc.pipe(new Base64Encode());
SVGtoPDF(doc, canvasSvg, 0, 0);
stream.on("data", (chunk) => (finalString += chunk));
stream.on("end", () => {
resolve(finalString);
});
doc.end();
} catch (err) {
reject(err);
}
});
};
export default generatePDF;
getFont() function will return an array of objects having props like
{
path: "#fonts/Pacifico-Regular.ttf",
family: "Pacifico",
weight: "regular",
style: "normal",
}
I have added #font as alias to the path where I have saved the .ttf fonts.
So technically, the following is happening
doc.registerFont("Pacifico", "src/fonts/Pacifico-Regural.ttf");
And I tried to use the font like this:
doc.font("Pacifico").text("Hello Message");
But I got the following error:
Unhandled Rejection (Error): fontkit.openSync unavailable for browser build
Is there is any other method to add font and use it in react-pdf/pdfkit?

How would I use Promises to fix this issue of synchronization in React?

I am working on a group project where we have to make a weather app. I am doing the details section, which displays values such as temperature, humidity, chance of rain, etc. from an API. I am using two different APIs in this project as OpenWeatherMap does not have some of the data I need. A different section of our app is getting the location and it is passed down to my section, where it is put into the OpenWeatherMap URL without issue.
constructor(props){
super(props);
}
componentDidMount() {
if(this.props.location) {
this.fetchWeatherData1();
this.fetchWeatherData2();
}
}
componentDidUpdate(prevProps) {
if(this.props.location != prevProps.location) {
this.fetchWeatherData1();
this.fetchWeatherData2();
}
}
fetchWeatherData1 = () => {
let urlOWM = "http://api.openweathermap.org/data/2.5/weather?q=" + this.props.location + "&units=metric&APPID=" + API_KEY_OWM;
$.ajax({
url: urlOWM,
dataType: "jsonp",
success : this.parseFirstResponse,
error : function(req, err){ console.log('API call failed ' + err); }
})
}
parseFirstResponse = (parsed_json) => {
var feelsLike = this.formatTemp(parsed_json['main']['feels_like']);
var humidity = this.formatPercentage(parsed_json['main']['humidity']);
var wind = this.formatSpeed(parsed_json['wind']['speed']);
var visib = this.formatVis(parsed_json['visibility']);
var cloud = this.formatPercentage(parsed_json['clouds']['all']);
var lat = this.formatString(parsed_json['coord']['lat']);
var long = this.formatString(parsed_json['coord']['lon']);
// set states for fields so they could be rendered later on
this.setState({
feelsLike: feelsLike,
humidity: humidity,
windSpeed: wind,
visibility: visib,
cloudCover: cloud,
latitude: lat,
longitude: long
});
}
In parseFirstResponse(), I store the latitude and longitude values for the location. I have to do this because the second API (DarkSky) URL can only take the coordinates for some reason. Here is the code below, where I am placing the state values for latitude and longitude into the URL:
fetchWeatherData2 = () => {
let urlDS = "https://api.darksky.net/forecast/" + API_KEY_DS + "/" + this.state.latitude + "," + this.state.longitude + "?exclude=minutely,hourly,daily,alerts,flags";
$.ajax({
url: urlDS,
dataType: "jsonp",
success : this.parseSecondResponse,
error : function(req, err){ console.log('API call failed ' + err); }
})
}
parseSecondResponse = (parsed_json) => {
var precipChance = parsed_json['currently']['precipProbability'];
var precipType = "";
if (precipChance == 0.0) {
precipType = "Precipitation";
}
else {
precipType = this.capitalize(parsed_json['currently']['precipType']);
}
precipChance = this.formatDecimaltoPercentage(precipChance);
var uv = parsed_json['currently']['uvIndex'];
var dew = this.formatTemp(this.fToC(parsed_json['currently']['dewPoint']));
// set states for fields so they could be rendered later on
this.setState({
precipChance: precipChance,
precipType: precipType,
uvIndex: uv,
dewPoint: dew
});
}
When I run this code and put a location in for the first time, I get an error in the console that says "Failed to load resource: the server responded with a status of 400 ()" and the URL looks like this: https://api.darksky.net/forecast/767ed401d519be925156b6c885fce737/undefined,undefined?exclude=minutely,hourly,daily,alerts,flags&callback=jQuery34106961395668750288_1585829837010&_=1585829837011
When I put a second, different location in, however, without refreshing the page, the URL and API call works without issue.
The coordinates of the location are supposed to be where the words "undefined, undefined" are. I have tried to console.log() the latitude and longitude values in my parseSecondResponse function and gotten the right values. I think this is a synchronization issue, but I'm not too sure.
Putting the fetchWeatherData() functions in the explicit ordering of 1 then 2 in my componentDidMount() function does not seem to help. I read about using Promises but I am not very familiar with React, so I am unsure how to implement them/if they will fix this issue.
I made small changes:
use fetch;
remove fetchWeatherData2 from componentDidMount;
use componentDidUpdate;
import React from 'react';
class Component extends React.Component {
constructor(props) {
super(props);
this.state = {
latitude: null,
longitude: null,
};
}
componentDidMount() {
const { location } = this.props;
if (location) {
this.fetchWeatherData1(location);
//fetchWeatherData2() your cannot call this because latitude and longitude === null
}
}
componentDidUpdate(prevProps, prevState) {
const { location: prevLocation } = prevState;
const { location: currLocation } = this.props;
const {
latitude: prevLatitude,
longitude: prevLongitude,
} = prevState;
const {
latitude: currLatitude,
longitude: currLongitude,
} = this.state;
if (prevLocation !== currLocation) {
this.fetchWeatherData1(currLocation);
}
if (prevLatitude !== currLatitude || prevLongitude !== currLongitude) {
this.fetchWeatherData2(currLatitude, currLongitude);
}
}
fetchWeatherData1(location) {
fetch(http://api.openweathermap.org/data/2.5/weather?q=${location}&units=metric&APPID=${API_KEY_OWM}`)
.then(res => res.json())
.then((data) => {
// parse you data here
this.setState(prevState => ({
latitude: data.coord.lat,
longitude: data.coord.long,
}));
})
.catch(({ message }) => {
console.error(`API call failed: ${message}`);
});
}
fetchWeatherData2(latitude, longitude) {
fetch(https://api.darksky.net/forecast/${API_KEY_DS}/${latitude},${longitude}?exclude=minutely,hourly,daily,alerts,flags`)
.then(res => res.json())
.then((data) => {
// parse you data here
})
.catch(({ message }) => {
console.error(`API call failed: ${message}`);
});
}
}

Confused on blob:url and converting it to base64 in react-dropzone

I am using the package react-dropzone (https://github.com/okonet/react-dropzone) to get images from the user. The user uploads their image and everything is fine, but Im only getting something like "blob:http//blahblah" from it and I need the image to be in base64 png.
my dropzone component:
<Dropzone ref="dropzone" multiple={false} onDrop={this.onDrop.bind(this)} >
{this.state.files ?<img className="img-responsive" src={this.state.files[0].preview}></img>
: <div>Upload Photo</div> }
</Dropzone>
and the drop function that gets the blob url :
onDrop (files ) {
if ( files.length === 0 ) {
alert("upload img please")
return;
}
console.log('Received files: ', files);
this.setState({files:files})
var blobURL = files[0].preview
var reader = new FileReader();
reader.readAsDataURL(blobURL)
}
I would get an error :Uncaught TypeError: Failed to execute 'readAsDataURL' on 'FileReader': parameter 1 is not of type 'Blob'.
I think it is because im trying to pass in an object-url that points to a blob, but where would I be able to get blob so I can convert to base64?
I would suggest to use a promise to get the result of async convertion by FileReader.readAsDataURL method. Here's the sample how it can be done:
const promise = new Promise((resolve, reject) => {
const reader = new FileReader()
reader.readAsDataURL(files[0])
reader.onload = () => {
if (!!reader.result) {
resolve(reader.result)
}
else {
reject(Error("Failed converting to base64"))
}
}
})
promise.then(result => {
// dispatch or do whatever you need with result
}, err => {
console.log(err)
})

Resources