React Web Audio API - Play, pause and export loaded audio file - reactjs

My purpose is to upload and listen to an audio file using the WEB Audio API. I have been able to listen to the audio file when selecting it, but having trouble pausing and playing it afterwards. I need to export the file to WAV format aswell. I have created a simple example, any help will be much appreciated
Loading Audio and playing it from file input
const onFileChange = (e) => {
let file = e.target.files[0];
console.log(file);
setFile(file);
let fileReader = new FileReader();
fileReader.onload = function (ev) {
audioContext.decodeAudioData(ev.target.result).then(function (buffer) {
playSound(buffer);
});
};
fileReader.readAsArrayBuffer(file);
};
Play sound using buffer source
const playSound = (buffer, time) => {
source = audioContext.createBufferSource();
source.buffer = buffer;
source.connect(audioContext.destination);
source.start(time);
setIsPlaying(true);
};
I'm facing problem here with pausing and playing:
const onPlayPause = (e) => {
console.log("audioState", audioContext.state);
console.log("duration", audioContext.currentTime);
if (!isPlaying) {
//source.start();
setIsPlaying(true);
} else if (audioContext.state === "running") {
setPlayDuration(audioContext.currentTime);
audioContext.suspend();
setIsPlaying(false);
} else if (audioContext.state === "suspended") {
audioContext.resume();
}
};
Export Audio:
const exportAudioFile = () => {
offlineContext.render().then((buffer) => {
setRenderState('encoding');
const handleMessage = ({ data }) => {
var blob = new window.Blob([new DataView(data)], {
type: 'audio/wav',
});
//blob = new Blob([buffer], { type: "audio/wav" });
const url = window.URL.createObjectURL(blob);
}
window.URL.revokeObjectURL(url);
})};
Codesandboxlink:
https://codesandbox.io/s/react-audiocontext-pause-play-fw20u?file=/src/App.js

I've had my fair share of headaches with persisting things in react functional components. Fortunately, useRef is an awesome tool for just that:
https://reactjs.org/docs/hooks-reference.html#useref
As the documentation says, it essentially returns a container which .current property persists across re-renders.
I forked your code to useRef in action:
https://codesandbox.io/s/react-audiocontext-pause-play-forked-si59u?file=/src/App.js:120-159
Basically, when you load the file, store your shiny new AudioContext in the ref's .current field, and reference that throughout the rest of your component. You can clean it up a bit, IE store the .current in a constant scoped to the function you're using it in.
Two key spots:
export default function App() {
const audioCtxContainer = useRef(null);
...
and
audioCtxContainer.current = new AudioContext();
audioCtxContainer.current
.decodeAudioData(ev.target.result)
.then(function (buffer) {
playSound(buffer);
});
useRef is useful for any mutable object that you want to persist for the lifetime of the component.
Let me know if that helps!

Related

H5P Instance is duplicated in reactjs

I'm developing with h5p standalone plugin in react (nextjs), passing the path as prop to a Modal Component which render the h5p activity.
useEffect(() => {
const initH5p = async (contentLocation) => {
const { H5P: H5PStandalone } = require('h5p-standalone')
const h5pPath = `https://cdn.thinkeyschool.com/h5p/${contentLocation}`
const options = {
id: 'THINKeyLesson',
h5pJsonPath: h5pPath,
frameJs: '/h5p/dist/frame.bundle.js',
frameCss: '/h5p/dist/styles/h5p.css',
}
let element = document.getElementById('h5p_container')
removeAllChildNodes(element)
await new H5PStandalone(element, options)
fireCompleteH5PTopic(H5P)
setIsLoaderVisible(false)
}
initH5p(location)
}, [location, session.data.user.id, course.slug, topic])
With that code, I get two h5p rendered in screen. So I'm using removeAllChildren() to eliminate them from the render.
function removeAllChildNodes(parent) {
console.log(parent)
while (parent.firstChild) {
parent.removeChild(parent.firstChild)
}
}
That hack is working fine, but when I try to send the xAPI statement to my database, it fires twice
const fireCompleteH5PTopic = async (H5P) => {
H5P.externalDispatcher.on("xAPI", (event) => {
// console.log('event fired')
if (event?.data?.statement?.result?.completion) {
setCounter(counter + 1)
completeH5PTopic(event, session.data.user.id, course.slug, topic)
return true
}
})
}
Any help regarding why it fires twice? I think it may be related to h5p rendering twice too.
Thanks in advance.
I tried using a state to render only once, but it is not working.

Is there any memory problem that "Too many active WebGL contents." keeps coming out in my web console

Now I'm using RTSP stream with jsmpeg, ffmpeg, node-rtsp-stream in React.js
I set multiple streaming in react component about 1 to maximum 25
and when I change route to many , It apeard my console
I'm a little afraid of if it caused by memory leak problem
I cannot figure out it is okay or critical problem
Could you tell me how to solve this problem or is it fine
// out of component
const streams: Streams = {};
...
const { preset } = props;
const elementsRef = useRef<any>(new Array(preset?.partition).fill(0).map((v) => createRef()));
// Mount
useEffect(() => {
const { JSMpeg } = window;
const sortedStream = _.sortBy(preset?.stream, ["position"]);
console.log(sortedStream);
sortedStream.forEach((v, i: number) => {
const player = new JSMpeg.Player(v.camera_monitoring.ws, {
canvas: elementsRef?.current[v.position]?.current, // Canvas should be a canvas DOM element
});
console.dir(elementsRef?.current[v.position]?.current);
streams[v.id] = player;
});
}, []);
// UnMount
useEffect(() => {
return () => {
Object.keys(streams).forEach((v) => {
console.log("unmount key:", v);
if (streams[v] !== null) {
streams[v].destroy();
streams[v] = null;
}
});
};
}, []);
...
https://github.com/phoboslab/jsmpeg
above library `jsmpeg.min.js` is set by global ( in public directory and set in my html )
actually my code are so many antipattern in contrast react style, To make an excuse, It is my limits of one's ability

How do I access the 'currentImageIdIndex' when using the stack scroll tool in Cornerstone.js in a React functional component?

I'm currently successfully displaying a stack of images in a React component but am unsure where to place an event listener in order to access the currentImageIdIndex when scrolling.
import React, { useEffect, useRef, useCallback } from "react";
import cornerstone from "cornerstone-core";
import cornerstoneMath from "cornerstone-math";
import cornerstoneTools from "cornerstone-tools";
import cornerstoneFileImageLoader from "cornerstone-file-image-loader";
import Hammer from "hammerjs";
function StackImageViewport(props) {
const viewerRef = useRef(null);
const base64StringToArrayBuffer = useCallback((base64) => {
var binary_string = window.atob(base64);
var len = binary_string.length;
var bytes = new Uint8Array(len);
for (var i = 0; i < len; i++) {
bytes[i] = binary_string.charCodeAt(i);
}
return bytes.buffer;
}, []);
const initializeCornerstone = () => {
// Initialise cornerstone and link to DOM element
cornerstoneTools.external.cornerstone = cornerstone;
cornerstoneTools.external.cornerstoneMath = cornerstoneMath;
cornerstoneFileImageLoader.external.cornerstone = cornerstone;
cornerstoneTools.external.Hammer = Hammer;
cornerstoneTools.init();
cornerstone.enable(viewerRef.current);
};
const setCornerstoneTools = () => {
// define Cornerstone Tools
const StackScrollTool = cornerstoneTools.StackScrollTool;
const StackScrollMouseWheelTool =
cornerstoneTools.StackScrollMouseWheelTool;
const WindowingTool = cornerstoneTools.WwwcTool;
// Add tools
cornerstoneTools.addTool(StackScrollTool);
cornerstoneTools.addTool(StackScrollMouseWheelTool);
cornerstoneTools.addTool(WindowingTool);
// set tools to Active state
cornerstoneTools.setToolActive("StackScroll", { mouseButtonMask: 1 });
cornerstoneTools.setToolActive("StackScrollMouseWheel", {});
cornerstoneTools.setToolActive("Wwwc", { mouseButtonMask: 2 });
};
const displayStack = (stackMediaArray) => {
let mediaArray = [];
// 'stackMediaArray' is an array of images, each containing a buffer of the image
Promise.all(
stackMediaArray.map((mediaObject) => {
return new Promise((resolve, reject) => {
let imageBuffer = base64StringToArrayBuffer(mediaObject.buffer);
const imageId =
cornerstoneFileImageLoader.fileManager.addBuffer(imageBuffer);
mediaArray.push(imageId);
resolve(mediaObject);
}).catch(console.error);
})
);
//define the stack
const stack = {
currentImageIdIndex: 0,
imageIds: mediaArray,
};
// load images and set the stack
cornerstone.loadAndCacheImage(mediaArray[0]).then((image) => {
cornerstone.displayImage(viewerRef.current, image);
cornerstoneTools.addStackStateManager(viewerRef.current, ["stack"]);
cornerstoneTools.addToolState(viewerRef.current, "stack", stack);
});
setCornerstoneTools();
};
useEffect(() => {
if (!viewerRef.current) {
return;
}
initializeCornerstone();
displayStack(props.stackMediaArray);
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [viewerRef]);
return (
<>
<div ref={viewerRef} id="viewer" className="flex h-1/2-screen"></div>
</>
);
}
export default StackImageViewport;
This attempts to answer the question:
https://github.com/cornerstonejs/cornerstoneTools/issues/1121
however, I don't want to access DOM elements to add the event listener to.
It's also clear that events are accessible in Cornertsone.js:
https://github.com/cornerstonejs/cornerstoneTools/blob/master/src/events.js
... but I'm still not sure where to place the event listener ?
Any help would be much appreciated.
Thanks.
Listening to events in ReactJS might be confusing for someone new, even more, when using a new tool such as CornerstoneJS. However, you can create an event listener by using the window.addEventListener method, just like you would in a Vanilla JavaScript. Note that this might change from browser to mobile environments. Moreover, your cornerstoneFileImageLoader can also be a challenge.
For this purpose, you can follow the structure:
window.addEventListener('keydown', (event) => {
...
});
But now, we need to understand "where" to place it. Imagine that all of your pages are just components, as ReactJS is a component-based system. Meaning that the event listener need to happen inside the component.
For instance, you can do like that:
import React from 'react';
const App = (props) => {
window.addEventListener('keydown', (event) => {
...
});
return (
<div className='container'>
<h1>Welcome to the Keydown Listening Component</h1>
</div>
);
};

Youtube iFrame API - React, Player is Undefined

I am currently using the iFrame API from my project and I am trying to have custom volume controls so that the user is able to change the volume outside of the iframe video. I have set up the player, but it is always undefined when it's being called outside an eventHandler.
const { youtubeDetails, volume } = useContext(GlobalContext);
useYoutube(loadVideo);
let player: any;
function loadVideo() {
(window as any).YT.ready(function () {
player = new window.YT.Player("player", {
events: {
onStateChange: onStateChange
}
});
});
}
useEffect(() => {
console.log(player) // never defined
changeVolume()
}, [volume]);
function onStateChange() {
console.log(player) // always defined
}
function changeVolume() {
player.setVolume(volume * 100);
}
This is because loadVideo() is never called again after the very first rerender. Is there a work around this structure so that the goal functionality is achieved?
The custom useYoutube Hooks is as follows:
import React from "react";
export const useYoutube = (callback: any) => {
React.useEffect(() => {
if (!(window as any).YT) {
var tag = document.createElement("script");
tag.src = "https://www.youtube.com/iframe_api";
var firstScriptTag = document.getElementsByTagName("script")[0];
firstScriptTag?.parentNode?.insertBefore(tag, firstScriptTag);
tag.onload = callback;
} else {
callback();
}
}, []);
};
I know this is an old post but I ran in to the same problem and the problem seems to be that YT is loaded first and then with a delay the properties 'ready' and 'Player' is loaded.
So I had to wrap my callback in a setTimeout with aproximatly 200ms delay to be able to call the methods 'ready' and 'Player'.

Re-rendering of Google Maps Elevations API responses within React functional component

I am writing an app for vehicle tracking. With the help of Google Maps API, I am able to get directions and extract all the required info. The problem appeared with Elevations API responses. From DirectionRender class I am sending path and distance as props. GM Elevations request is done via elevator.getElevationAlongPath(option,PlotElevation). PlotElevation (elevations,status) is a callback function. However, no matter how I try to receive just one response from it (using useMemo, useEffect, I think I tried everything), still, there are problems with the re-rendering of responses. OVER_QUERY_LIMIT or endless re-render. Could someone help with that?
Thanks
const Elevation = React.memo(props =>{
const [path, setPath] = useState({...props.path})
const [distance, setDistance]=useState({...props.distance})
const [elevationArray, setElevationArray] = useState(null)
const [stop, setStop] = useState(false)
let pathElev = JSON.parse(JSON.stringify(path))
React.useEffect(() => {
setPath(props.path)
}, [props.path])
React.useEffect(() => {
setDistance(props.distance)
}, [props.distance])
let elevator = new window.google.maps.ElevationService;
let numberSamples = parseInt( distance/40)
let options = {
'path':path,
'samples':numberSamples
}
//The problem starts here
const PlotElevation = (elevations, status) => {
if (stop === false){
console.log('status',status)
console.log(JSON.parse(JSON.stringify(elevations)))
setElevationArray(elevations)
//setStop(true)
console.log(elevations[19].elevation)
return
}
}
const Memo = React.useMemo(
()=>{
elevator.getElevationAlongPath(
options,PlotElevation
)
},[elevator.getElevationAlongPath(
options,PlotElevation
)])
// elevator.getElevationAlongPath(
// {
// path: path,
// samples: 100
// }, elevations =>{
// setElevationArray({
// // We’ll probably want to massage the data shape later:
// // elevationArray: elevations
// })
// }
// )
return (
<div>
{Memo}
{console.log('path is received ', pathElev)}
{console.log('number of samples', numberSamples)}
{console.log('elevation check ',elevationArray)}
{/* {elevator.getElevationAlongPath(
options,PlotElevation)} */}
</div>
)
})
export default Elevation

Resources