google local home 1500ms timeout issue - google-smart-home

I am working on a Google Home Local Fulfillment project following "Enable local fulfillment for smart home Actions" tutorial on the Google Codelabs.
The project is only focus on local control without accessing remote control.
I need to run a query on my firestore database at executeHandler before I run this.app.getDeviceManager().send(deviceCommand).
This will give me timeout errors as below.
statusType: "RESPONSE_UNAVAILABLE"
isSuccess: false
externalDebugString: "Pubsub failed with 1500ms timeout."
Is there a way I can set the timeout more than 1500ms?
The following is my code of executeHandler:
executeHandler(request: IntentFlow.ExecuteRequest):
Promise<IntentFlow.ExecuteResponse> {
// TODO: Implement local execution
console.log("EXECUTE intent: " + JSON.stringify(request, null, 2));
const command = request.inputs[0].payload.commands[0];
const execution = command.execution[0];
const response = new Execute.Response.Builder().setRequestId(request.requestId);
const promises: Array<Promise<void>> = command.devices.map(async (device) => {
console.log("Handling EXECUTE intent for device: " + JSON.stringify(device));
// Convert execution params to a string for the local device
const params = execution.params as IFireplaceParams;
const tcpMsg = 'MWIB2,02';
const payload = this.stringToHex(tcpMsg);
console.log("Sending request to the smart home device:", payload);
const firebaseDataObj = await this.queryFirebase(device.id);
console.log("firebaseDataObj:", JSON.stringify(firebaseDataObj));
// Construct a local device command over TCP
const deviceCommand = new DataFlow.TcpRequestData();
deviceCommand.requestId = request.requestId;
deviceCommand.deviceId = device.id;
deviceCommand.data = payload;
deviceCommand.port = SERVER_PORT;
deviceCommand.operation = Constants.TcpOperation.WRITE;
try {
const result = await this.app.getDeviceManager().send(deviceCommand);
console.log("Sending deviceCommand result:", JSON.stringify(result));
const state = { online: true };
response.setSuccessState(result.deviceId, Object.assign(state, params));
} catch (err) {
err.errorCode = err.errorCode || 'invalid_request';
response.setErrorState(device.id, err.errorCode);
console.error('An error occurred sending the command', err.errorCode);
}
});
return Promise.all(promises)
.then(() => {
return response.build();
})
.catch((e) => {
const err = new IntentFlow.HandlerError(request.requestId, 'invalid_request', e.message);
return Promise.reject(err);
});
}

Communicating directly with external service from the local home app is not recommended as it might not be supported across all supported execution environments:
https://developers.google.com/assistant/smarthome/concepts/local#execution-environment
The documentation recommends instead to pass additional data required for execution as part of the customData fields of the SYNC response:
https://developers.google.com/assistant/smarthome/develop/local#update_sync_response_in_the_cloud_fulfillment

Related

store data in firestore when Browser Tab is closed or the route is changed (react JS)

const handleDraftContracts = async () => {
console.log('/bruhhhhhhandleDraftContract');
const paragraphRef: string | any = document.getElementById('contract');
const contractDetails = {
contractName: 'House Rental',
states: {
amount: amount,
},
content: paragraphRef?.textContent,
};
await makeDraftContract(contractDetails);
};
useEffect(() => {
console.log('///////I am hreeeee');
window.addEventListener('onbeforeunload', (env) => {
handleDraftContracts();
});
return () => {
console.log('///////removing');
window.removeEventListener('onbeforeunload', handleDraftContracts);
};
// eslint-disable-next-line react-hooks/exhaustive-deps
}, []);
firestore.js
// make Draft Contracts
export async function makeDraftContract(contractDetails: object | any) {
try {
console.log("making a draft contract", contractDetails);
const draftContractRef: any = collection(db,"makeDraftContracts");
let contract = await addDoc(draftContractRef, contractDetails);
console.log("./////////makeDraftContract", contract);
} catch (error) {
console.log('////errror in contract Hanlder', error);
}
}
I want to call my handleDraftContracts method whenever user closes the tab or changes the route. I am using onbeforeunload event. The handleDraftContracts is getting called but the tab unloads before Firestore could update the collection. How can I get around this that as the user closes the tab or move to a new route, my firestore method get executed first then the tab gets unloaded ?
Try with Beacon api
https://developer.mozilla.org/en-US/docs/Web/API/Beacon_API
as 'onbeforeunload' cannot make sure you request to server has been made and requests can slow down the browser
componentWillUnmount is like that one, cannot to make long running script.

Saving an ID value from an API to a User with GraphQL

I'm working on a video game website where a user can save a game to a list. How this is supposed to work is when the user clicks "Complete Game", the ID of the game is saved to a state that holds the value. The value is then passed into the mutation, then the mutation runs, saving the ID of the game to the users list of completed games. However, all I'm seeing in the console is this:
"GraphQLError: Variable \"$addGame\" got invalid value { gameId: 740, name: \"Halo: Combat Evolved\",
The above error continues, listing the entirety of the API response, instead of just the gameId.
I was able to successfully add the game to the list in the explorer with the following mutation:
mutation completeGame($addGame: AddNewGame!) {
completeGame(addGame: $addGame) {
_id
completedGameCount
completedGames {
gameId
}
}
}
with the following variable:
{
"addGame": {"gameId": 740}
}
How can I trim down what is being passed into the mutation to just be the gameId?
Below is the entirety of the page, except the return statement at the bottom.
const [selectedGame, setSelectedGame] = useState([]);
const [savedGameIds, setSavedGameIds] = useState(getSavedGameIds());
const [completeGame, { error }] = useMutation(COMPLETE_GAME);
const { id: gameId } = useParams();
useEffect(() => {
return () => saveGameIds(savedGameIds);
});
useEffect(() => {
async function getGameId(gameId) {
const response = await getSpecificGame(gameId);
if (!response.ok) {
throw new Error('Something went wrong...');
}
const result = await response.json();
const gameData = result.map((game) => ({
gameId: game.id,
name: game.name,
cover: game.cover,
summary: game.summary,
platforms: game.platforms,
platformId: game.platforms,
genres: game.genres,
genreId: game.genres,
}));
setSelectedGame(gameData);
}
getGameId(gameId);
}, [])
const handleCompleteGame = async (gameId) => {
const gameToComplete = selectedGame.find((game) => game.gameId === gameId);
const token = Auth.loggedIn() ? Auth.getToken() : null;
if (!token) {
return false;
}
try {
const { data } = await completeGame({
variables: { addGame: { ...gameToComplete } },
});
console.log(data);
setSavedGameIds([...savedGameIds, gameToComplete]);
} catch (err) {
console.error(err);
}
};
With the mutation working in the explorer when I'm able to explicitly define the variable, I am led to believe that the issue is not with the resolver or the typedef, so I'm going to omit those from this post because I don't want it to get too long.
However, I'd be happy to attach any extra code (resolver, typeDef, getSavedGameIds function, etc) if it would allow anyone to assist. The issue (I think) lies in getting my response to match the syntax I used in the explorer, which means trimming down everything except the gameId.
I specifically am extremely suspicious of this line
const gameToComplete = selectedGame.find((game) => game.gameId === gameId)
but I have fiddled around with that for awhile to no avail.
Thank you to anyone who is able to help!
It sounds like you're trying to pass more into your mutation then your schema is defined to allow. In this part:
const { data } = await completeGame({
variables: { addGame: { ...gameToComplete } },
});
You're spreading gameToComplete here which means everything in the gameToComplete object is going to be sent as a variable. If your schema is setup to just expect gameId to be passed in, but your error message is showing that name is also being passed in, you just need to adjust your variables to exclude everything you can't accept. Try:
const { data } = await completeGame({
variables: { addGame: { gameId } },
});

React native: how do I wait for a state to be set, before I call another state related operation?

I am writing a chat app. Users can search for other users, and then press the "Message" button. Then I navigate to ChatScreen.js. If both users have been messaging each other, I set the chatId variable accordingly. If they have not messaged each other before I dont create chatId, until the ery first message has been sent. When the first message is sent, I first, create new chat, store its properties (user ids, chatId, etc) in my db and then I sent the first message. The problem is that I store chatId as a state variable, and when I create the chat I call setChatId(id). setChatId() is not synchronous call, so by the time when I need to send message with sendText(text, chatId); my chatId is undefined even though I have already created a chat and I have called setChatId.
How can I avoid this error? Ofc, I can check if chatId == undefined then calling sendText(text, id), otherwise calling sendText(text, chatId). Is there a better/neath way to avoid the undefined check?
Here is part of my code:
...
import {
createChat,
} from "./actions";
...
function ChatScreen(props) {
...
const [chatId, setChatId] = useState(props.route.params.chatId);
...
const setupChat = async () => {
try {
await createChat(user.id, setChatId);
props.fetchUserChats();
} catch (error) {
console.error("Error creating chat: ", error);
}
};
async function handleSend(messages) {
if (!chatId) {
// creating chat
await setupChat();
}
const text = messages[0].text ? messages[0].text : null;
const imageUrl = messages[0].image ? messages[0].image : null;
const videoUrl = messages[0].video ? messages[0].video : null;
const location = messages[0].location ? messages[0].location : null;
//assuming chatId is already setup but it is not
if (imageUrl) {
sendImage(imageUrl, chatId, setSendImageError);
} else if (location) {
sendLocation(location, chatId, setLocationError);
} else if (videoUrl) {
sendVideo(videoUrl, chatId, setSendImageError);
} else {
sendText(text, chatId);
}
}
...
}
My createChat function from actions.js file
export async function createChat(otherUid, setChatId) {
let chatId = firebase.auth().currentUser.uid + "_" + otherUid;
await firebase
.firestore()
.collection("Chats")
.doc(chatId)
.set({
users: [firebase.auth().currentUser.uid, otherUid],
lastMessage: "Send the first message",
lastMessageTimestamp: firebase.firestore.FieldValue.serverTimestamp(),
})
.then(() => {
console.log("doc ref for creatign new chat: ", chatId);
setChatId(chatId);
})
.catch((error) => {
console.error("Error creating chat: ", error);
});
}
Instead of using a state variable, I would advise you to use useRef(). This would be a good solution to your problem.Eg Define it this way
const chatId = useRef(null),
then set it this way chatId.current = yourChatId
and get it this way chatId.current. I hope this solves your problem

How to start WebRTC call with AWS Kinesis as Master

I'm trying to start a WebRTC call with AWS Kinesis, but the demo on The AWS Kinesis Javascript docs only shows how to join the call as a VIEWER not the MASTER.
I can't find a clear example anywhere online, and I've spent hours on it with my teammate.
I can see and hear myself, so I know I'm getting the hardware working correctly, but we can't see or hear each other. I know it's going to be something simple, but I just can't figure out where I'm going wrong with the connection.
const startKinesisCall = async () => {
const coachingSession = new AWS.KinesisVideo({
region,
accessKeyId,
secretAccessKey,
correctClockSkew: true
});
// Get Signaling Channel Endpoints
// Each signaling channel is assigned an HTTPS and WSS endpoint to connect to for
// data-plane operations. These can be discovered using the GetSignalingChannelEndpoint API.
const getSignalingChannelEndpointResponse = await coachingSession.getSignalingChannelEndpoint({
ChannelARN: channelARN,
SingleMasterChannelEndpointConfiguration: {
Protocols: ['WSS', 'HTTPS'],
Role: Role.VIEWER
}
}).promise();
const endpointsByProtocol = getSignalingChannelEndpointResponse?.ResourceEndpointList?.reduce((endpoints, endpoint) => {
endpoints[endpoint.Protocol] = endpoint?.ResourceEndpoint;
return endpoints;
}, {});
// Create KVS Signaling Client
// The HTTPS endpoint from the GetSignalingChannelEndpoint response is used with this client.
// This client is just used for getting ICE servers, not for actual signaling.
const kinesisVideoSignalingChannelsClient = new AWS.KinesisVideoSignalingChannels({
region,
accessKeyId,
secretAccessKey,
endpoint: endpointsByProtocol.HTTPS,
correctClockSkew: true,
});
// Get ICE server configuration
// For best performance, we collect STUN and TURN ICE server configurations.
// The KVS STUN endpoint is always stun:stun.kinesisvideo.${region}.amazonaws.com:443.
// To get TURN servers, the GetIceServerConfig API is used.
const getIceServerConfigResponse = await kinesisVideoSignalingChannelsClient
.getIceServerConfig({
ChannelARN: channelARN,
}).promise();
const iceServers = [{ urls: `stun:stun.kinesisvideo.${region}.amazonaws.com:443` }];
getIceServerConfigResponse.IceServerList.forEach(iceServer =>
iceServers.push({
urls: iceServer.Uris,
username: iceServer.Username,
credential: iceServer.Password,
}),
);
console.log('ICE SERVERS: ', iceServers);
// Create RTCPeerConnection
// The RTCPeerConnection is the primary interface for WebRTC communications in the Web.
const peerConnection = new RTCPeerConnection({ iceServers });
// Create WebRTC Signaling Client
// This is the actual client that is used to send messages over the signaling channel.
const signalingClient = new SignalingClient({
channelARN,
channelEndpoint: endpointsByProtocol.WSS,
role: Role.MASTER,
region,
clientId,
credentials: {
accessKeyId,
secretAccessKey,
},
systemClockOffset: coachingSession.config.systemClockOffset
});
// GET THE USER MEDIA DEVICES
const localStream = await navigator.mediaDevices.getUserMedia({
video: true,
audio: true
}).catch(e => {
console.log("COULD NOT FIND WEBCAM");
setShowErrorStartingVideoModal(true);
});
// *** AUDIO & VIDEO DEVICE COLLECTION ***
let audioInputDevices: MediaDeviceInfo[];
let audioOutputDevices: MediaDeviceInfo[];
let videoInputDevices: MediaDeviceInfo[];
try {
const mediaDevices = await navigator.mediaDevices.enumerateDevices();
audioInputDevices = mediaDevices.filter(device => device.kind === 'audioinput');
audioOutputDevices = mediaDevices.filter(device => device.kind === 'audiooutput');
videoInputDevices = mediaDevices.filter(device => device.kind === 'videoinput');
setMicrophoneList(audioInputDevices);
setSpeakerList(audioOutputDevices);
setCameraList(videoInputDevices);
} catch (e) {
console.log(e);
console.log("ERROR COLLECTING MEDIA DEVICE INFORMATION: MAKE SURE PERMISSIONS ARE ALLOWED AND TRY AGAIN");
};
// GRAB THE LOCAL PROVIDER AND PATIENT VIDEO TILES
const providerVideoTile: HTMLVideoElement = document.getElementById('provider-video-element') as HTMLVideoElement;
const patientVideoElement = document.getElementById('patient-video-element') as HTMLVideoElement;
// let dataChannel: RTCDataChannel
// Add Signaling Client Event Listeners
signalingClient.on('open', async () => {
if (!localStream || !peerConnection) return;
// Get a stream from the webcam, add it to the peer connection, and display it in the local view
try {
localStream.getTracks().forEach(track => peerConnection.addTrack(track, localStream));
providerVideoTile.srcObject = localStream;
} catch (e) {
// Could not find webcam
console.log(e);
return;
};
// Create an SDP offer and send it to the master
const offer = await peerConnection.createOffer({
offerToReceiveAudio: true,
offerToReceiveVideo: true
});
console.log('CREATED OFFER: ', offer);
await peerConnection.setLocalDescription(offer);
if (peerConnection.localDescription) signalingClient.sendSdpOffer(peerConnection.localDescription, patient.patientID);
});
// When the SDP answer is received back from the master, add it to the peer connection.
signalingClient.on('sdpAnswer', async answer => {
console.log('RECEIVED ANSWER: ', answer);
if (!peerConnection) return;
await peerConnection.setRemoteDescription(answer).catch(e => console.log(e));
});
signalingClient.on('sdpOffer', async (offer, senderClientID) => {
console.log({ offer });
if (!peerConnection) return;
await peerConnection.setRemoteDescription(offer).catch(e => console.log(e));
console.log('REMOTE DESCRIPTION SET: ', peerConnection);
const answer = await peerConnection.createAnswer().catch(e => console.log(e));
console.log({ answer });
if (answer) signalingClient.sendSdpAnswer(answer, senderClientID);
// dataChannel = peerConnection.createDataChannel(`data-channel-of-${senderClientID}`);
// dataChannel.addEventListener("open", (event) => {
// console.log(event);
// dataChannel.send('******HI ALEC*******');
// });
});
// When an ICE candidate is received from the master, add it to the peer connection.
signalingClient.on('iceCandidate', async (candidate, senderClientID) => {
if (!peerConnection) return;
console.log('new iceCandidate received:', candidate);
await peerConnection.addIceCandidate(candidate).catch(e => console.log(e));
console.log("ICE CANDIDATE ADDED: ", candidate);
});
signalingClient.on('close', async () => {
if (!localStream) return;
// Handle client closures
console.log("ENDING THE CALL");
localStream.getTracks().forEach(track => track.stop());
peerConnection.close();
if ('srcObject' in providerVideoTile) providerVideoTile.srcObject = null;
});
signalingClient.on('error', error => {
// Handle client errors
console.log(error);
});
signalingClient.on('chat', (dataMessage: any) => {
const decodedMessage = UTF8Decoder.decode(new Uint8Array(dataMessage.data));
console.log("GOT TEST MESSAGE:", decodedMessage);
});
signalingClient.on('SeriesData', (dataMessage: any) => {
const seriesFromMobile = JSON.parse(UTF8Decoder.decode(new Uint8Array(dataMessage.data)));
console.log("SERIES FROM MOBILE:", seriesFromMobile);
kickOffSeriesCreation(seriesFromMobile);
});
signalingClient.on('EffortMarker', (dataMessage: any) => {
const effortMarker = UTF8Decoder.decode(new Uint8Array(dataMessage.data));
console.log("EFFORT MARKER:", effortMarker);
setEffortMarker(effortMarker);
});
signalingClient.on('CoachingMessage', async (dataMessage: any) => {
const coachingMessage = UTF8Decoder.decode(new Uint8Array(dataMessage.data));
console.log("COACHING MESSAGE FROM MOBILE:", coachingMessage);
if (coachingMessage === 'EndSeries') {
await handleForceEndEffort(signalingClient);
await handleEndSeries(signalingClient);
};
});
// Add Peer Connection Event Listeners
// Send any ICE candidates generated by the peer connection to the other peer
peerConnection.addEventListener('icecandidate', ({ candidate }) => {
if (candidate) {
console.log(candidate);
signalingClient.sendIceCandidate(candidate, patient.patientID);
} else {
// No more ICE candidates will be generated
console.log('NO MORE ICE CANDIDATES WILL BE GENERATED');
}
});
// As remote tracks are received, add them to the remote view
peerConnection.addEventListener('track', event => {
// if (patientVideoElement.srcObject) return;
setNoPatientConnected(false);
console.log({ event });
try {
peerConnection.addTrack(event.track, event.streams[0]);
if (event.track.kind === 'video') patientVideoElement.srcObject = event.streams[0];
} catch (e) {
console.log(e);
}
});
// Open Signaling Connection
signalingClient.open();
};
Try this this page, You can use master on one computer and viewer on other.
https://awslabs.github.io/amazon-kinesis-video-streams-webrtc-sdk-js/examples/index.html
For anyone else with the same issue, I managed to find the master example on this github repo and was able to get it working

ReactJS: STOMP subscription to multiple topics

My React code creates a WebSocket connection to our company's ActiveMQ 5.15.5 server, and then subscribes to the following two topics: salary and decoding. The problem is that the code is only able to subscribe to one of the topics. It cannot subscribe to both.
const client = window.Stomp.client(`ws://${ipAddress}:61614`, 'aj6.stomp');
const headers = { id: 'username' };
client.debug = null;
client.connect('user', 'pass', () => {
client.subscribe(
'/topic/salary', //BREAKPOINT was set here
message => {
const body = JSON.parse(message.body);
if (body && body.pcId) {
salaries[body.pcId] = body;
setState({ salaries});
}
},
headers,
);
client.subscribe(
'/topic/decoding', //BREAKPOINT was set here
message => {
const newBody = JSON.parse(message.body);
if (newBody && newBody.PcID) {
consoleMessage[newBody.PcID] = newBody;
setState({ consoleMessage });
}
},
headers,
);
});
So in the code above I put a break-point at client.subscribe('/topic/decoding... and client.subscribe('/topic/salary.... I saw that it only subscribes to /topic/decoding but not /topic/salary.
Does anyone know how I can fix this issue so that it subscribes to both topics?
From Stomp documentation:
Since a single connection can have multiple open subscriptions with a server, an id header MUST be included in the frame to uniquely identify the subscription. The id header allows the client and server to relate subsequent MESSAGE or UNSUBSCRIBE frames to the original subscription.
Within the same connection, different subscriptions MUST use different subscription identifiers.
Stomp API definition:
subscribe(destination, callback, headers = {})
So for my understanding, You can't have the same username id for both of your subscriptions
Try creating 2 clients, e.g.:
const salaryClient = window.Stomp.client(`ws://${ipAddress}:61614`, 'aj6.stomp');
const salaryHeaders = { id: 'salary' };
salaryClient.debug = null;
salaryClient.connect('user', 'pass', () => {
salaryClient.subscribe(
'/topic/salary',
message => {
const body = JSON.parse(message.body);
if (body && body.pcId) {
salaries[body.pcId] = body;
setState({ salaries});
}
},
salaryHeaders,
);
});
const decodingClient = window.Stomp.client(`ws://${ipAddress}:61614`, 'aj7.stomp');
const decodingHeaders = { id: 'decoding' };
decodingClient.debug = null;
decodingClient.connect('user', 'pass', () => {
decodingClient.subscribe(
'/topic/decoding',
message => {
const newBody = JSON.parse(message.body);
if (newBody && newBody.PcID) {
consoleMessage[newBody.PcID] = newBody;
setState({ consoleMessage });
}
},
decodingHeaders,
);
});

Resources