can't append to sourceBuffer because mediaSource Element (Parent) was removed - reactjs

I'm having problem with video Streaming.
I'm using socket io for video Streaming and react for front end.
Front End code looks like this:-
class VideoElement extends React.Component {
constructor(props) {
/* props is having Id, projectDesignSocket */
super();
this.state = { hasError: false };
this.videoRef = React.createRef();
this.mediaSource = null;
this.sourceBuffer = null;
}
componentDidMount() {
this.mediaSource = new MediaSource();
this.videoRef.current.src = URL.createObjectURL(this.mediaSource);
this.mediaSource.onsourceopen = (e) => {
URL.revokeObjectURL(this.videoRef.current.src);
this.sourceBuffer = this.mediaSource.addSourceBuffer("video/webm; codecs=\"vp8, opus\"");
this.sourceBuffer.mode = 'sequence';
this.props.projectDesignSocket.on(`${this.props.Id}VideoData`, ({ blob }) => {
try {
this.sourceBuffer.appendBuffer(blob);
}
catch (err) {
console.log(err);
}
finally {
console.log(this.sourceBuffer);
}
});
}
}
render() {
if (this.state.hasError) {
return <h1>error</h1>;
}
else {
return <video autoPlay ref={this.videoRef} id={this.props.Id}></video>;
}
}
componentWillUnmount() {
this.mediaSource.onsourceopen = null;
this.props.projectDesignSocket.off(`${this.props.Id}VideoData`);
}
static getDerivedStateFromError(error) {
return { hasError: true };
}
componentDidCatch(error, info) {
console.log(error, info);
}
}
class VideoConfrence extends React.Component {
/* prop is having projectDesignSocket, isAdmin, userList, currentRoom, userName, userId */
constructor(props) {
super();
this.state = {};
this.userVideoRef = React.createRef();
this.mediaRecorder = null;
this.videoStream = null;
}
static getDerivedStateFromProps(nextProp, prevState) {
if (nextProp.userList !== prevState.userList) {
return { userList: nextProp.userList }
}
else {
return {};
}
}
componentDidMount() {
navigator.mediaDevices.getUserMedia({
audio: true,
video: true
}).then(stream => {
this.videoStream = stream;
this.userVideoRef.current.srcObject = stream;
this.mediaRecorder = new MediaRecorder(stream);
this.mediaRecorder.start(100);
this.mediaRecorder.ondataavailable = (e) => {
if (e.data.size) {
this.props.projectDesignSocket.emit('video', {
room: this.props.currentRoom,
data: { userId: this.props.userId, blob: e.data }
});
}
}
}).catch(err => {
console.log(err);
});
}
render() {
return (
<div className='videoConfrencing'>
<video autoPlay ref={this.userVideoRef} />
{
this.state.userList.map((item, index) => {
if (item !== this.props.userId) {
return <VideoElement key={index} Id={item} projectDesignSocket={this.props.projectDesignSocket} />
}
else {
return null;
}
})
}
</div>
);
}
componentWillUnmount() {
this.videoStream.getTracks().forEach(function (track) {
track.stop();
});
this.mediaRecorder.ondataavailable = null;
this.props.projectDesignSocket.off('newMemberAdded');
}
}
And Backend Like This :-
socket.on('video', function ({ room, data }) {
// data has userid and blob as it's properties
if (room && data ) {
socket.broadcast.to(room).emit(`${data.userId}VideoData`, {blob:data.blob});
}
})
error is occuring at try/catch block of componentDidMount of VideoElement
err:-Media resource blob:http://localhost:3000/493042c5-431b-40a1-aff6-5fafe218a677 could not be decoded, error: Error Code: NS_ERROR_FAILURE (0x80004005)
basicaly i'm designing an app for video confrencing and,
i've used mediaRecorder to record stream obtained by getUserMedia into Blobs and send those blobs to server throgh socket ('video event')
and the server receives roomName from which event was emmited, userId and blob of video and broadcast an event ${userId}VideoData to the specific room
when a user connects the server start receiving blobs and start broadcasting event ${userId}VideoData with {userId, blob} as values to be passed to client
when another user connects his webcam feeds are also sent to server...the user connected before him receives his video blobs and is able to play them , this user also receives video blobs of previously connected user but is unable to play them , on first append to sourceBuffer above warning is logged after which if appended again error is thrown!!
the error is that user which connects cant see the video of other users which were connected before him but is able to see the video of users connected after him!!
but users connected before him are able to see his video without any error

Related

unable to switch camera (front to rear) in react application using webrtc MediaDevices

This is the sample demonstration of what I'm intended to do.
If anyone has any idea about this fix to make it work or any new logic please do share.
This demonstration is implemented by using mediaStream API and
using react-webcam library which actually gives option to manage the camera view with the help of the props named videoConstraints={facingMode: 'user' or 'environment'} but it doesn't seems to be working.
when I click the camera switch ICON screen just hangs and nothing shows and also sometime it is working unexpectedly So ultimately I had to jumps to this native API solution which shows the code right below.
with all regards thanks in anticipation.
start() {
if (window.stream) {
console.log('found stream and clearing that', window.stream)
window.stream.getTracks().forEach(function(track) {
track.stop()
})
}
const constraints = {
video: true,
audio: false
}
return navigator.mediaDevices
.getUserMedia(constraints)
.then(this.gotStream)
.then(this.gotDevices)
.catch(this.handleError);
}
gotStream(stream) {
window.stream = stream // make stream available to console
// video.srcObject = stream;
// Refresh button list in case labels have become available
console.log('enumerating media devices ')
return navigator.mediaDevices.enumerateDevices()
}
gotDevices(mediaDevices) {
const { availableVideoInputs, videoConstraints } = this.state
mediaDevices.forEach(mediaDevice => {
// console.log(mediaDevice)
if (mediaDevice.kind === 'videoinput') {
console.log('found new video input ', mediaDevice)
availableVideoInputs.push({
deviceId: mediaDevice.deviceId,
label: mediaDevice.label
})
// availableVideoInputs.push('mediaDevice.deviceId.availableVideoInputs.push(mediaDevice.deviceId)')
}
})
console.log('aggregated availableVideoInputs new ', availableVideoInputs)
if (availableVideoInputs.length > 0) {
// there are accessible webcam
// setting first device as default to open
const tempVideoConstraint = {...videoConstraints}
if (availableVideoInputs[0].deviceId) {
console.log('availableVideoInputs[0] = ', availableVideoInputs[0])
tempVideoConstraint.deviceId = availableVideoInputs[0].deviceId
}
// console.log('putting tempVideoConstraint.facingMode ', tempVideoConstraint)
// if (availableVideoInputs[0].label.includes('back')) {
// tempVideoConstraint.facingMode = { exact: 'environment'}
// } else {
// // it is now turn to set front active
// tempVideoConstraint.facingMode = 'user'
// }
console.log('setting new video constrains ', tempVideoConstraint)
// this.setState({
// availableVideoInputs,
// // activeVideoInputID: availableVideoInputs[0].deviceId,
// // videoConstraints: tempVideoConstraint
// })
this.updateAvailableVideoStream(availableVideoInputs)
return Promise.resolve('done setting updateAvailableVideoStream')
} else {
// no webcam is available or accessible
console.error('ERR::VIDEO_STREAM_NOT_AVAILABLE')
}
}
updateAvailableVideoStream(availableVideoInputs) {
this.setState({ availableVideoInputs })
}
componentDidMount() {
this.start()
.then(data => {
console.log('data ', data)
console.log('update state ', this.state)
this.setState({
videoConstraints: {
...this.state.videoConstraints,
facingMode: 'user'
}
})
})
}
handleCameraSwitch() {
const { videoConstraints, availableVideoInputs, activeVideoInputID } = this.state
console.log('current video constraints ', videoConstraints)
const tempVideoConstraint = { ...videoConstraints }
// now check if it is possible to change camera view
// means check for another webcam
console.log({ availableVideoInputs })
console.log({ activeVideoInputID })
console.log({ remainingVideoStreams })
if (availableVideoInputs.length === 1) {
// cannot change the webcam as there is only 1 webcam available
console.error('ERR - cannot change camera view [Available Video Inputs: 1]')
return
}
// now change the view to another camera
// get the current active video input device id and filter then from available video stream
const remainingVideoStreams = availableVideoInputs.filter(videoStream => videoStream.deviceId !== activeVideoInputID)
// now check if in remainingVideoStreams there is more than 1 stream available to switch
// if available then show the Stream Selection List to user
// else change the stream to remainingVideoStreams[0]
console.log({ availableVideoInputs })
console.log({ activeVideoInputID })
console.log({ remainingVideoStreams })
if (remainingVideoStreams && remainingVideoStreams.length === 1) {
tempVideoConstraint.deviceId = remainingVideoStreams[0].deviceId
console.log('new video constraints ', {...tempVideoConstraint})
console.log('webcam ref ', this.webCamRef.current)
// if (remainingVideoStreams[0].label.includes('back') || tempVideoConstraint.facingMode === 'user') {
// tempVideoConstraint.facingMode = { exact: 'environment' }
// } else {
// // it is now turn to set front active
// tempVideoConstraint.facingMode = 'user'
// }
console.log('new video constraints with facing mode', tempVideoConstraint)
// const constraints = {
// video: tempVideoConstraint
// }
// navigator.mediaDevices.getUserMedia(constraints)
// .then((stream) => {
// console.log('stream -> ', stream)
// })
// .catch((error) => {
// console.error('Some error occured while changing the camera view ', error)
// console.log(error)
// })
this.setState({ videoConstraints: tempVideoConstraint, activeVideoInputID: remainingVideoStreams[0].deviceId })
} else {
// show the remaining stream list to user
}
}
<script src="https://cdnjs.cloudflare.com/ajax/libs/react/16.6.3/umd/react.production.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/react-dom/16.6.3/umd/react-dom.production.min.js"></script>
This is the little variation of your Implementation.
But this will work exactly you've wished for.
Please see the below implementation for switching the camera front/back.
I have also added error validation like:
It will throw an error if there is no video stream available.
It will throw an error if there is only 1 video stream available when trying to access back camera.
Please do like and comment back if you have any other approach or want more clarification
componentDidMount() {
const gotDevices = (mediaDevices) =>
new Promise((resolve, reject) => {
const availableVideoInputs = []
mediaDevices.forEach(mediaDevice => {
if (mediaDevice.kind === 'videoinput') {
availableVideoInputs.push({
deviceId: mediaDevice.deviceId,
label: mediaDevice.label
})
}
})
if (availableVideoInputs.length > 0) {
resolve(availableVideoInputs)
} else {
reject(new Error('ERR::NO_MEDIA_TO_STREAM'))
}
})
navigator.mediaDevices.enumerateDevices().then(gotDevices)
.then((availableVideoInputs) => this.setState({ availableVideoInputs }))
.catch((err) => this.setState({ hasError: err }))
}
updateFileUploadView(newActiveView) {
this.setState({ activeFileUploadView: newActiveView })
const { hasError } = this.state
if (newActiveView === 'clickFromWebcam' && hasError) {
return console.error(hasError)
}
if (newActiveView === '') {
// means no view is active and clear the selected image
this.setState({ captureImageBase64: '', videoConstraints: defaultVideoConstraints })
}
}
changeCameraView() {
const { availableVideoInputs } = this.state
if (availableVideoInputs.length === 1) {
return console.error('ERR::AVAILABLE_MEDIA_STREAMS_IS_1')
}
this.setState({ resetCameraView: true })
setTimeout(() => {
const { videoConstraints: { facingMode } } = this.state
const newFacingMode = facingMode === 'user' ? { exact: 'environment' } : 'user'
this.setState({
videoConstraints: { facingMode: newFacingMode },
resetCameraView: false
})
}, 100)
}
<script src="https://cdnjs.cloudflare.com/ajax/libs/react/16.6.3/umd/react.production.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/react-dom/16.6.3/umd/react-dom.production.min.js"></script>
!resetCameraView ?
<Webcam
audio={false}
height='100%'
ref={this.webCamRef}
screenshotFormat="image/png"
minScreenshotWidth={screenShotWidth}
minScreenshotHeight={screenShotHeight}
screenshotQuality={1}
width='100%'
videoConstraints={videoConstraints}
/>
: 'Loading...'
As you can see this implementation is using react-webcam library
In componentDidMount you will first check for the available media stream of kind video input, then in other methods like changing cameraView i.e switching the camera to front/back.
I'm unmounting Webcam for 100ms only and then mounting it back with new videoConstraints either { facingMode: 'user' } or { facingMode: { exact: 'environment' } }
This approach will give your code a head start and you can play around the code and have fun.
Thank you!

Mapping array value in API response for conditional check

I have an api which loads a response with an array value I need to map, in order to both check if available, and if so, use the id value of the offers to display later. I'm currently able to simply get the available option, but need to be able to map down to the offers and grab the id value within the FETCH OFFER DETAILS fetchContent.
API RESPONSE
{
"available": true,
"offers": [
{
"days_free": 30,
"id": 1,
}
]
}
React Component
class CancelOffer extends React.Component {
constructor (props) {
super(props)
this.state = {
user: []
}
this.processData = this.processData.bind(this)
this.cancelAccount = this.cancelAccount.bind(this)
this.acceptPromo = this.acceptPromo.bind(this)
}
componentDidMount () {
this.fetchContent(this.processData)
}
/**
* Fetch offer details
*/
fetchContent (cb) {
superagent
.get('/api/user/offers')
.then(function(res) {
/**
* If res.body.available === true, get offers.id and use in
acceptPromo.
* Else if not true, redirect to
.then(this.props.setStep(ACCEPTANCE))
*/
if(res.body.available){
const offerCode = res.body.offers[0].id
alert(offerCode)
} else {
this.props.setStep(ACCEPTANCE)
}
})
}
/**
* Set state after user have been fetched
* #param data
*/
processData (data) {
this.setState({
user: data.body.offers
})
}
/**
* Send accept promo offer code
*/
acceptPromo (e) {
e.preventDefault()
superagent
.post('/api/user/offers')
.send({
offerId: offerCode <==THIS NEEDS TO COME FROM THE API RESPONSE
})
.then(this.props.setStep(ACCEPTANCE))
}
render () {
const content = this.props.config.contentStrings
const daysFree = this.state.user.map((daysFree, i) => {
return (
<span>{daysFree.days_free}</span>
)
})
return (
<div className='offer'>
<h2 className='offer-heading md'>Heading</h2>
<p className='offer-subpara'>sub paragraph {daysFree}</p>
<div className='footer-links'>
<a onClick={this.acceptPromo} className='btn btn--primary btn--lg'>accept promo</a>
<a onClick={this.cancelAccount} className='cancel-link'>cancel</a>
</div>
</div>
)
}
}
export default CancelOffer
EDIT - ANSWER:
I needed to set the promo code in state
fetchContent (cb) {
superagent
.get('/api/user/offers')
.then((res) => {
if(res.body.available){
this.setState({
offerId: res.body.offers[0].id
})
} else {
this.props.setStep(OFFER)
}
})
}
and pass it to the accept promo
acceptPromo (e) {
e.preventDefault()
const { offerId } = this.state
superagent
.post('/api/user/offers')
.send({offerId})
.then(this.props.setStep(ACCEPTANCE))
}

Cannot see firebase data in application - react native

I have set up a call to fetch data from my firebase database using react native.
Database structure
Code inside FirebaseList.js
constructor(props) {
super(props);
this.state = {
data: []
};
}
componentWillMount() {
firebase.database().ref('/signposts/items').on('value', snapshot => {
const dataArray = [];
const result = snapshot.val();
for (const data in result) {
dataArray.push(data);
}
this.setState({ data: dataArray });
console.log(this.state.data);
});
}
render() {
return (
<FlatList
data={this.state.data}
renderItem={({ item }) => (
<Text>{item}</Text>
)}
keyExtractor={item => item}
/>
);
}
I believe the connection to firebase is successful as I can build and run the application. However, when the component renders, I do not see my two rows of data 'row1' and 'row2'.
You said your code is right then also check that this rules are set
{
"rules": {
"foo": {
".read": true,
".write": false
}
}
}
Note : - When you use the above rule your database is open for all Read more here. Make you use update the rules once you push to production.
If console.log(dataArray) shows an empty array (assuming that console.log() works...), try checking your connection:
componentDidMount() {
const ref = firebase.database().ref('/signposts');
const checkConnection = firebase.database().ref(`.info/connected`);
checkConnection.on('value', function(snapshot) {
if (snapshot.val() === true) { /* we're connected! */
firebase.database().ref('/signposts').on('value', snapshot => {
const dataArray = [];
const result = snapshot.val();
for (const data in result) {
dataArray.push(data);
}
if (dataArray.length === 0)
console.log("No data.")
else
this.setState({ listViewData: dataArray });
});
} else { /* we're disconnected! */
console.error("Check your internet connection.")
}
}
}

Angular2 EventEmitter subscription does not work after navigate and come back

I emitted the data from MerchantComponent
and subscribed via EventEmitterService from MerchantPaymentChannelComponent, its OK when route directly opens this page. But you see there is other tabs which every tab has its own components. When I change the tab to different one, then come back to this MerchantPaymentChannelComponent its not subscribe again.
NOTE: I'm doing unsubscription on NgOnDestroy event
Here is codes;
MerchantListDetailService (SHARING SERVICE VIA EVENTEMITTER)
export class MerchantListDetailService {
#Output() emittedMerchant: EventEmitter<any> = new EventEmitter();
constructor() {}
emitMerchant(data) {
this.emittedMerchant.emit(data);
}
getEmittedValue() {
return this.emittedMerchant;
}
}
MerchantComponent (TRIGGERING EMIT FROM THIS COMPONENT)
private getMerchantDetail() {
let data = {
Id: this.merchantId,
}
this.merchantsService.getMerchants(data)
.then((res) => {
// console.log(res)
if (res.Success) {
this.merchant = res.Data[0];
this.merchantListDetailService.emitMerchant(res.Data[0]);
}
})
.catch((err) => { })
}
MerchantPaymentChannelComponent (SUBSCRIBING FROM THIS COMPONENT)
ngOnInit() {
this.merchantSubscribe = this.merchantListDetailService.getEmittedValue()
.subscribe(merchant => {
this.merchant = merchant;
this.getMerchantPaymentChannels();
})
}
ngOnDestroy() {
this.merchantSubscribe.unsubscribe();
}

Using hello.js with React.js

I'd like to understand how to make Hello.js work with React.js , especially the custom event handler hello.on
As I'm new to React.js, I don't understand how to bind non React events into the app flow.
I tried putting the event handler in the componentDidMount handler
handleClick(){
hello('twitter').login();
}
componentDidMount(){
hello.on('auth.login', function(auth) {
// Call user information, for the given network
hello(auth.network).api('/me').then(function(r) {
console.log(r);
});
});
hello.init({
'twitter' : 'J1jqqO50tcLtLx8Js0VDitjZW'
},
{
redirect_uri:'/',
oauth_proxy: 'https://auth-server.herokuapp.com/proxy'
});
}
thanks
And 3 years later:
You need a class for authentication, for example:
import * as React from "react";
import * as hello from "hellojs";
import { Event } from "../interfaces/Event";
export class Authentication extends React.Component<{}, { sendEvent: boolean }> {
constructor(public props, public context) {
super(props, context);
this.state = {
sendEvent: true
};
}
public login(network) {
hello.init({
aad: {
name: "Azure Active Directory",
oauth: {
version: 2,
auth: "https://login.microsoftonline.com/common/oauth2/v2.0/authorize",
grant: "https://login.microsoftonline.com/common/oauth2/v2.0/token"
},
// Authorization scopes
scope: {
// you can add as many scopes to the mapping as you want here
profile: "user.read",
offline_access: ""
},
scope_delim: " ",
login: p => {
if (p.qs.response_type === "code") {
// Let's set this to an offline access to return a refresh_token
p.qs.access_type = "offline_access";
}
},
base: "https://www.graph.microsoft.com/v1.0/",
get: {
me: "me"
},
xhr: p => {
if (p.method === "post" || p.method === "put") {
JSON.parse(p);
} else if (p.method === "patch") {
hello.utils.extend(p.query, p.data);
p.data = null;
}
return true;
},
// Don't even try submitting via form.
// This means no POST operations in <=IE9
form: false
}
});
hello.init(
{
aad: "ClientID"
},
{
redirect_uri: "YOUR REDIRECT_URI",
//redirect_uri: 'https://localhost:4321/temp/workbench.html',
scope: "user.read"
}
);
// By defining response type to code, the OAuth flow that will return a refresh token to be used to refresh the access token
// However this will require the oauth_proxy server
hello(network)
.login({ display: "none" })
.then(
authInfo => {
console.log(authInfo);
localStorage.setItem("logged", authInfo.authResponse.access_token);
},
e => {
console.error("Signin error: " + e.error.message);
}
);
}
//when the component is mounted you check the localstorage
//logged ==> undefined you call login and save a token in localstorage
//logged ==> with a token -> setEvent call a function that use graph api
public componentDidMount() {
let logged = localStorage["logged"];
if (logged === undefined) this.login("aad");
else {
if (this.state.sendEvent) {
this.props.setEvent(null);
this.props.setEvent(Event.GET_ALL_USERS);
}
}
}
public render() {
return null;
}
}
the file name is auth.tsx and you can call this class in the main react class:
export class mainClass extends React.Component{
......
......
private getEvent = (event) => {
this.setState({ event: event });
//HERE YOU recive the event when auth is ready
}
public render(){
<Authentication setEvent={this.getEvent} />
}
}

Resources