UseEffect/useCallback is not triggering on very fast changes - reactjs

I'm a Backend Dev and having limited knowledge in React still have to fix the problem
My project uses WebRTC for video calls. For signaling I'm using SignalR on my .NET backend.
On the frontend I have 2 classes:
signalRContext.tsx which holds an instance of HubConnection and listeners, onmessage is the relevant one.
const [currentSignal, setCurrentSignal] = useState<TCurrentSignal>
(InitialSignalR.currentSignal);
const initializeSignalListeners = (connection: HubConnection): void => {
console.log('START SIGNAL_R', connection);
connection.on('master', function (RoundInfo: IRoundInfo) {
console.log('MASTER', RoundInfo);
setCurrentSignal({ type: 'master', payload: RoundInfo });
});
connection.on('slave', function (RoundInfo: IRoundInfo) {
console.log('SLAVE', RoundInfo);
setCurrentSignal({ type: 'slave', payload: RoundInfo });
});
connection.on('message', function (message: TSignalRMessage) {
console.log('MESSAGE', message);
setCurrentSignal({ type: 'message', payload: message });
});
connection.on('endround', (payload) => {
console.log('END_ROUND');
setCurrentSignal({ type: 'endround', payload });
});
useRTCPeerConnection.ts which has the whole WebRtc relevant logic
import { useSignalRContext } from '../../../../core/contexts';
const {
signalrRRef,
currentSignal: { type, payload },
} = useSignalRContext();
useCallback(() => { //tried UseEffect as well
if (type === 'message') {
console.log('PAYLOAD', payload);
onMessage(payload as TSignalRMessage);
return;
}
}, [type, payload]);
My problem starts when WebRTC starts exchanging the ICE candidates and sends them sometimes twice per millisecond (see the last column).
The connection.on('message'... listener seems to be fast enough, I'm seeing all console.log('MESSAGE'... outputs in the console.
My problem is that the useCallback/useEffect logic is not firing on every payload change, like for 20 MESSAGE outputs I'm seeing 4-7 PAYLOAD outputs.
My assumption is that useEffect is simply not designed for such quick changes.
Is there any other concept better suitable to solve this problem or any improvement I could do here? Thinking on .NET I would just use the composition pattern and call the relevant method from peer connection class within the event handler in signalR class but not sure how to fix it here.
P.S. I've tried to wait until ICE candidates are gathered and sending them at once but the performance becomes not acceptable.

Related

Invalid value for transfer with ipcRenderer.postMessage in Electron

I'm getting this error message : Invalid value for transfer when trying to use, for the very first time, the message-ports-reply-streams.
In preload.js I defined this api:
contextBridge.exposeInMainWorld(
"api", {
electronIpcPostMessage: (channel: string, message: any, transfer?: MessagePort[]) => {
ipcRenderer.postMessage(channel, message, transfer)
},
}
)
declare global {
interface Window {
api: {
electronIpcPostMessage: (channel: string, message: any, transfer?: MessagePort[]) => void;
}
}
And , following the example found here: https://www.electronjs.org/docs/latest/tutorial/message-ports#reply-streams , in the renderer React component I defined the streaming request as follows:
const Layers = () => {
const componentIsMounted = React.useRef(true)
React.useEffect(() => {
const cb = (event, args) => {
try {
if (componentIsMounted.current) {
console.log("react-leaflet-layers-args: ", args)
}
} catch (err) {
console.log("err: ", err)
}
}
const makeStreamingRequest = (element, cb) => {
// MessageChannels are lightweight--it's cheap to create a new one for each request.
const { port1, port2 } = new MessageChannel()
// We send one end of the port to the main process ...
window.api.electronIpcPostMessage(
'give-me-a-stream',
{ element, count: 10 },
[port2]
)
// ... and we hang on to the other end.
// The main process will send messages to its end of the port,
// and close it when it's finished.
port1.onmessage = (event) => {
cb(event.data)
}
port1.onclose = () => {
console.log('stream ended')
}
}
makeStreamingRequest(42, (data) => {
console.log('got response data:', event.data)
})
// We will see "got response data: 42" 10 times.
return () => { // clean-up function
componentIsMounted.current = false
window.api.electronIpcRemoveListener(
"give-me-a-stream",
cb,
)
}
}, [])
As said, when running Electron-React app the error message I get when accessing the page rendered by that component, is : Invalid value for transfer .
From this StackOverflow question : Invalid value for transfer while using ipcRenderer.postMessage of electron, it seems that I'm not the only one stumbling on this type of error, but I didn't find any solutions yet.
What am I doing wrongly or missing? How to solve the problem?
My objective is to send, better in a streaming fashion, a very big geojson file from the main process to the renderer process. That's why I thought to try to use ipcRenderer.postMessage.
By the way, any other working solutions that accomplish this goal, are welcomed.
Other info:
electron: v. 16
node: v. 16.13.0
O.S: Ubuntu 20.04
Looking forward to hints and help
I also encountered the same problem. In https://www.electronjs.org/docs/latest/api/context-bridge, it is mentioned that the types of parameters, errors and return values in functions bound with contextBridge are restricted, and MessagePort is one of the types that cannot be transported, so it doesn't recognize the MessagePort you passed in and throw this error.
If you want to use MessageChannel for communication, you can provide some proxy functions through contextBridge in preload.js, call these functions in renderer.js and pass in copyable parameters.
Hope my answer helps you.

Multiple video call (n users) using Peerjs in React Native

I have an application in which I am trying to get video chatting to work in React Native.
Used packages like react-native-webrtc and react-native-peerjs.
Created peer js server using Node Js.
One to One Video call is working fine with react native Peerjs. But, Now I want more than 2 users to be connected upto n users.
Is it possible to convert one to one video call to Multiple video call. Kindly let me know how Multiple video call can be achieved using Peer js and web rtc.
Here is my code for one to one video call:
Initialize webrtc and PeerJS:
const initialize = async () => {
const isFrontCamera = true;
const devices = await mediaDevices.enumerateDevices();
const facing = isFrontCamera ? 'front' : 'environment';
const videoSourceId = devices.find(
(device: any) => device.kind === 'videoinput' && device.facing === facing,
);
const facingMode = isFrontCamera ? 'user' : 'environment';
const constraints: MediaStreamConstraints = {
audio: true,
video: {
mandatory: {
minWidth: 1280,
minHeight: 720,
minFrameRate: 30,
},
facingMode,
optional: videoSourceId ? [{ sourceId: videoSourceId }] : [],
},
};
const newStream = await mediaDevices.getUserMedia(constraints);
setLocalStream(newStream as MediaStream);
console.log("************ Started ************");
// const io = socketio(SERVER_URL);
// io.connect();
console.log(SERVER_URL);
const io = socketio.connect(SERVER_URL, {
reconnection: true,
autoConnect: true,
reconnectionDelay: 500,
jsonp: false,
reconnectionAttempts: Infinity,
// transports: ['websocket']
});
io.on('connect', () => {
console.log("----------- Socket Connected -----------");
setSocket(io);
io.emit('register', username);
});
io.on('users-change', (users: User[]) => {
console.log("----------- New User - " + JSON.stringify(users) + " -----------");
setUsers(users);
});
io.on('accepted-call', (user: User) => {
setRemoteUser(user);
});
io.on('rejected-call', (user: User) => {
setRemoteUser(null);
setActiveCall(null);
Alert.alert('Your call request rejected by ' + user?.username);
navigate('Users');
});
io.on('not-available', (username: string) => {
setRemoteUser(null);
setActiveCall(null);
Alert.alert(username + ' is not available right now');
navigate('Users');
});
const peerServer = new Peer(undefined, {
host: PEER_SERVER_HOST,
path: PEER_SERVER_PATH,
secure: false,
port: PEER_SERVER_PORT,
config: {
iceServers: [
{
urls: [
'stun:stun1.l.google.com:19302',
'stun:stun2.l.google.com:19302',
],
},
],
},
});
peerServer.on('error', (err: Error) =>
console.log('Peer server error', err),
);
peerServer.on('open', (peerId: string) => {
setPeerServer(peerServer);
setPeerId(peerId);
io.emit('set-peer-id', peerId);
});
io.on('call', (user: User) => {
peerServer.on('call', (call: any) => {
//Alert.alert("PeerServer Call");
setRemoteUser(user);
Alert.alert(
'New Call',
'You have a new call from ' + user?.username,
[
{
text: 'Reject',
onPress: () => {
io.emit('reject-call', user?.username);
setRemoteUser(null);
setActiveCall(null);
},
style: 'cancel',
},
{
text: 'Accept',
onPress: () => {
io.emit('accept-call', user?.username);
call.answer(newStream);
setActiveCall(call);
navigate('Call');
},
},
],
{ cancelable: false },
);
call.on('stream', (stream: MediaStream) => {
setRemoteStream(stream);
});
call.on('close', () => {
closeCall();
});
call.on('error', () => { });
});
});
};
When a user call another user:
const call = (user: User) => {
if (!peerServer || !socket) {
Alert.alert('Peer server or socket connection not found');
return;
}
if (!user.peerId) {
Alert.alert('User not connected to peer server');
return;
}
socket.emit('call', user.username);
setRemoteUser(user);
try {
const call = peerServer.call(user.peerId, localStream);
call.on(
'stream',
(stream: MediaStream) => {
setActiveCall(call);
setRemoteStream(stream);
},
(err: Error) => {
console.error('Failed to get call stream', err);
},
);
} catch (error) {
console.log('Calling error', error);
}
};
Now, how should I call multiple user from the code below and how multiple streams have to be handled.
const call = peerServer.call(user.peerId, localStream);
Is it possible to convert one to one video call to Multiple video call
It's not possible to "convert" a one to one video call to "multiple" in a peer-to-peer architecture. In a p2p architecture with n participants, each participant will have a separate, one-to-one connection with the rest n-1 other participants.
I may possibly be misunderstanding your question, but if you're asking whether it's possible to establish n-1 connections for each participant, then the answer is yes. Here's how I would implement:
Anytime a new participant joins a session, extract their peer information. This is the peerId provided by the peer.js library.
Next, let the rest of the participants know about the presence of this new user. For this, you'll share this new participant's name, peerID and any other metadata with the rest of the participants in the room. This can be done by the signalling logic that you have implemented using socket.io.
Now going forward, you have 2 options:
The new participant could initiate the one-to-one peer connection with others in the room, OR,
The rest of the participants could initiate a one-on-one connection with the new participant.
Personally I prefer the first. So continuing the process:
Using the same signalling logic via socket.io, the rest of the participants will let the new user know about their presence by providing their own peer information and other metadata.
Once the new participant gets everyone's peer information, initiate a new peer connection using call.on('stream', callback) and start broadcasting their video.
On the recipient side, when a call is received along with the stream, you'll create a new video element in react-native, and bind the received media stream to this element. Which means, each participant will have n-1 video elements for streaming the media of n-1 other participants. The recipient also starts to broadcast their own video to the initiator of the call.
Here's a tutorial showing how this can be done using vanilla JavaScript, along with the github repository with source code.
Now, to answer the next question:
Kindly let me know how Multiple video call can be achieved using Peer js and webrtc.
This depends on the number of participants, where they lie geographically, browser/device limits, device computational power, and network bandwidth. So there are multiple factors involved which makes it tricky to give any specific number.
Browsers can place their own upper limits on the maximum number of connections possible, and there might be other values for Android and iOS. On chrome, the max theoretical limit is 500. If you're developing for Android, you may want to check here. But I couldn't manage to find much info on this.
Most practical applications involving WebRTC don't rely on a mesh architecture. Common implementations involve using an SFU, which takes multiple media streams and forwards them. A slightly more sophisticated technique is an MCU architecture, which combines all those media streams from multiple participants into a single one, and send that single stream to the rest of the participants.
I discuss this in some detail here:
https://egen.solutions/articles/how-to-build-your-own-clubhouse-part-2/#architectures-scaling-and-costs
Here's a nice article that explains the difference between SFU and MCU.

SIP integration with call conference in JS

I am developing an Electron application with the integration of React.js as a front-end framework, which will be more like a calling application.
In that application-specific users can have multiple calls incoming, outgoing, mute | unmute calls, hold | unhold calls, etc.
For this functionality to be achieved we have our own sip server, and for integrating that SIP server, on the frontend we are using a library which is known as SIP.JS.
SIP.JS provides us mostly all the predefined functions to make a call, receive a call, mute, unmute, blind transfer, attended transfer, etc.
But when it comes to having a call conference, it doesn't have proper documentation for that.
SIP.JS specifies to us that we can use FreeSWITCH as well as ASTERISK in order to achieve the functionality, but with our specific requirements, no additional server needs to be integrated.
We have also referred to rfc documentation for the call conference, but no such progress was there.
So far what we did is:
Registered the userAgent
Code for Incoming call integrated
Code for outgoing calls integrated
multiple session handling is achieved, for multiple calls
mute | unmute, hold | unhold.
DTMF functionality
Blind Transfer, Attended Transfer
Ring all Devices
In this scenario of call conference, I guess we have to make changes in Incoming and outgoing session handling functions.
For registration and incoming call in context:
const getUAConfig = async (_extension, _name) => {
let alreadyLogin = '';
try {
alreadyLogin = 'yes';
if (alreadyLogin == 'yes') {
_displayname = _name;
_sipUsername = _extension;
_sipServer = 'SIP SERVER';
_sipPassword = 'SIP PASSWORD';
_wssServer = 'WSS SERVER;
const uri = UserAgent.makeURI('sip:' + _sipUsername + '#' + _sipServer);
const transportOptions = {
wsServers: 'WSS SERVER',
traceSip: true,
maxReconnectionAttempts: 1,
};
const userAgentOptions = {
uri: uri,
transportOptions: transportOptions,
userAgentString: 'App name',
authorizationPassword: _sipPassword,
sipExtension100rel: 'Supported',
sipExtensionReplaces: 'Supported',
register: true,
contactTransport: 'wss',
dtmfType: 'info',
displayName: _name,
sessionDescriptionHandlerFactoryOptions: {
peerConnectionOptions: {
rtcpMuxPolicy: 'negotiate',
iceCheckingTimeout: 1000,
iceTransportPolicy: 'all',
iceServers: [{ urls: 'stun:stun.l.google.com:19302' }],
},
},
};
userAgent = await new UserAgent(userAgentOptions);
const registerOptions = {
extraContactHeaderParams: [],
};
registerer = await new Registerer(userAgent, registerOptions);
registerer.stateChange.addListener((newState) => {
});
userAgent.start().then(async () => {
console.log('Connected with WebSocket.');
// Send REGISTER
await registerer
.register()
.then((request) => {
console.log('Successfully sent REGISTER, object is here');
dispatch({
type: USER_REGISTERED,
payload: true,
});
})
.catch((error) => {
console.log('Failed to send REGISTER');
});
});
return { userAgent, registerer };
} else {
return null;
}
} catch (error) {
console.log(error.message + '');
return null;
}
};
Outgoing functionality:
const dilaerFun = (inputNumber, userAgentInfo) => {
var session;
var uri = UserAgent.makeURI(
`URI which we wanna call (sip number)`
);
session = new Inviter(userAgentInfo, uri);
session
.invite()
.then((request) => {
console.log('Successfully sent INVITE');
sessionInfoAdd(session);
session.stateChange.addListener(async (state) => {
switch (state) {
case 'Established':
setMissedStatus(null);
console.log('established outgoing....');
//outgoing call log-----
const mediaElement = document.getElementById(
`mediaElement${session._id}`
);
const remoteStream = new MediaStream();
session.sessionDescriptionHandler.peerConnection
.getReceivers()
.forEach((receiver) => {
if (receiver.track) {
remoteStream.addTrack(receiver.track);
}
});
mediaElement.srcObject = remoteStream;
mediaElement.play();
break;
case 'Terminated':
console.log('terminated');
dispatch({
type: DEMO_STATE,
payload: session._id,
});
break;
default:
break;
}
});
})
.catch((error) => {
console.error(' Failed to INVITE');
console.error(error.toString());
});
};
Array of sessions are maintained by:
const sessionInfoAdd = (session) => {
dispatch({
type: SESSION_STORE,
payload: session,
});
};
Variable in which all sessions are stored is:
sessionInfo:[]
NOTE: getUAConfig() is called as soon as the application is started.
dialerFun() is called when we want to dial a specific number.
sessionInfoAdd() is called in both getUAConfig and dialerFun, as they are codes for incoming and outgoing calls.
when sessionInfoAdd() is triggered, the particular session which we get in return is added in the sessionInfo (Array) for the maintenance of sessions.
SIP.JS is just a library so you will have to get the conference setup on the FreeSWITCH or Asterisk (FreeSWITCH is the better in my opinion)
Doing this is fairly straight forward, at your app level you need a way to get calls across to the box after checking the details like access ID and any auth you want to add, (like a PIN.)
Once you have that done, you can forward that to an extension specifically set for conferencing or have a dynamic conference setup by send from the app towards a specific gateway/dialplan to do this.
The FreeSWITCH software has a steep learning curve on it but this helped me when I was doing something similar: https://freeswitch.org/confluence/display/FREESWITCH/mod_conference
You can also code you own conf if you wish.

RxJS Observable with multiple ajax responses

Brand noob at this and have gotten one post request working but hoping to chain several and process the response. I gather the way to do this is forkJoin(), however I am not getting the responses (although I see the requests and responses in the Network) and don't really get how to do the composition. I think I may need to subscribe to them?
const requests: Array<Observable<AjaxResponse>> = [];
fields.forEach((field: string) => {
const request: AjaxRequest = generateRequest(field);
requests.push(Observable.ajax(request));
});
Observable.forkJoin(requests).map(
responses => { // never stops here
responses.map((res, idx) => { // or here
})
});
Found it about 10 mins later
const forkJoin = Observable.forkJoin(requests);
forkJoin.subscribe(ajaxResponses => {
});

API requests as a result of changes to domain objects

I'm having trouble wrapping my head around how to handle api calls as a result
of updates to data only the store should know how to perform (business logic).
The basic flow is as follows:
AdComponent#_changeGeoTargeting calls the action creator
UnpaidIntents#updateTargeting which dispatches an
ActionTypes.UPDATE_TARGETS action which is handled like so:
AdStore.dispatchToken = Dispatcher.register(action => {
switch(action.type) {
case ActionTypes.UPDATE_TARGETS:
// Business logic to update targeting from an action payload.
// `payload` is an object, e.g. `{ geos: geos }`, `{ devices: devices }`,
// etc.
_unpaid.targeting = _calcTargeting(
_unpaid.targeting, action.payload);
// Ajax call to fetch inventory based on `Ad`s parameters
WebAPIUtils.fetchInventoryPredictions(
_unpaid.start, _unpaid.end, _unpaid.targeting)
.then((resp) => {
var key = _makeCacheKey(
_unpaid.start, _unpaid.end, _unpaid.targeting);
// Updates store's inventory cache
_updateInventoryCache(key, resp);
})
.catch((error) => {
// How can I handle this error? If this request
// was executed inside an action I could have my
// `NotiticationsStore` listening for
// `ActionTypes.INVENTORY_PREDICTIONS_ERROR`
// and updating itself, but I can't dispatch here.
});
break;
default:
return true;
}
AdStore.emitChange();
return true;
});
The problem being that this call can't dispatch other actions since it's in a
store.
I could make the call in the action creator, but that requires it to know how
to update the Ad. I was under the impression that action creators should
be dumb "dispatcher helper methods", and something like this would violate those
principles:
UnpaidIntents.updateTargeting = (ad, value) => {
var targeting = _calcTargeting(ad.targeting, value);
WebAPIUtils.fetchInventoryPredictions(ad.start, ad.end, targeting)
.then((resp) => {
Dispatcher.dispatch({
type: ActionTypes.UPDATE_TARGETING,
payload: {
targeting: targeting,
inventory: resp,
},
});
})
.catch((error) => {
Dispatcher.dispatch({
type: ActionTypes.INVENTORY_PREDICTIONS_ERROR,
payload: error,
});
});
};
Would breaking out _calcTargeting into an AdUtils module and using that as
my business logic layer be the way to do this? I'm afaid if I have business
logic in utils and possibly also stores that things will get messy very quickly.
Can anyone give some guidance here?

Resources