In my <App> Context, I have a canvas element (#offScreen) that is already hooked in the requestAnimationFrame loop and appropriately drawing to that canvas, verified by .captureStream to a <video> element.
In my <Canvas> react component, I have the following code (which works, but seems clunky/not the best way to copy an offscreen canvas to the DOM):
NOTE: master is the data object for the <App> Context.
function Canvas({ master, ...rest } = {}) {
const canvasRef = useRef(master.canvas);
const draw = ctx => {
ctx.drawImage(master.canvas, 0, 0);
};
useEffect(() => {
const canvas = canvasRef.current;
const ctx = canvas.getContext("2d");
let animationFrameId;
const render = () => {
draw(ctx)
animationFrameId = window.requestAnimationFrame(render)
}
render();
return () => {
window.cancelAnimationFrame(animationFrameId);
}
}, [ draw ]);
return (
<canvas
ref={ canvasRef }
onMouseDown={ e => console.log(master, e) }
/>
);
};
Edited for clarity based on comments
In my attempts to render the master.canvas directly (e.g. return master.canvas; in <Canvas>), I get some variation of the error "Objects cannot be React children" or I get [object HTMLCanvasElement] verbatim on the screen.
It feels redundant to take the #offScreen canvas and repaint it each frame. Is there, instead, a way to insert or append #offScreen into <Canvas>, so that react is just directly utilizing #offScreen without having to repaint it into the react component canvas via the ref?
Specific Issue: Functionally, I'm rendering a canvas twice--once off screen and once in the react component. How do I (replace/append?) the component's <canvas> element with the offscreen canvas (#offScreen), instead of repainting it like I'm doing now?
For anyone interested, this was actually fairly straightforward, as I overcomplicated it substantially.
export function Canvas({ canvas, ...rest }) {
const container = useRef(null);
useEffect(() => {
container.current.innerHTML = "";
container.current.append(canvas);
}, [ container, canvas ]);
return (
<div ref={ container } />
)
}
Related
I would like to take image data (RGBA) from react-three-fiber
like this code on HTML
var myCanvas = document.createElement("canvas");
myCanvas.width = texture.image.width;
myCanvas.height = texture.image.height;
var myCanvasContext = myCanvas.getContext("2d"); // Get canvas 2d context
myCanvasContext.drawImage(texture.image, 0, 0); // Draw the texture
var texels = myCanvasContext.getImageData(0,0, width, height); // Read the texels/pixels back
However, I cannot access Canvas to drawImage or getImageData.
const ref = React.useRef();
React.useEffect(() => {
if (ref) {
//error
ref.current.getContext("2d");
}
}, []);
<Canvas
dpr={[1, 1.5]} //pixelRatio
ref={ref}
>
<Particles />
</Canvas>
and I tried to use useThree. there is no method to get imageData
const {
gl,
} = useThree();
lastly, I tired to get an image using useLoader as a texture
const map = useLoader(TextureLoader, "/scroll_images/img1.jpg");
so, how can I get the imageData?
In your first example, try useLayoutEffect instead of useEffect. Refs are not actually populated until the first DOM render happened. Unlike useEffect, useLayoutEffect waits until this occurred.
React.useLayoutEffect(() => {
ref.current.getContext("2d"); // <-- hopefully doesnt error
}, []);
In your second example I think its because you need to pass gl={{ preserveDrawingBuffer: true }} prop to your Canvas. Then you can use gl.domElement.toDataURL('image/png') or similar.
I have an animated background using canvas and requestAnimationFrame in my React app and I am trying to have its moving particles interact with the mouse pointer, but every solution I try ranges from significantly slowing down the animation the moment I start moving the mouse to pretty much crashing the browser.
The structure of the animated background component goes something like this:
<BackgroundParentComponent> // Gets mounted only once
<Canvas> // Reutilizable canvas, updates every frame.
// v - dozens of moving particles (just canvas drawing logic, no JSX return).
// Each particle calculates its next frame updated state every frame.
{particlesArray.map(particle => <Particle/>}
<Canvas/>
<BackgroundParentComponent />
I have tried moving the event listeners to every level of the component structure, calling them with a custom hook with an useRef to hold the value without rerendering, throttling the mouse event listener so that it does not fire that often... nothing seems to help. This is my custom hook right now:
const useMousePosition = () => {
const mousePosition = useRef({ x: null, y: null });
useEffect(() => {
window.addEventListener('mousemove', throttle(200, (event) => {
mousePosition.current = { x: event.x, y: event.y };
}))
});
useEffect(() => {
window.addEventListener('mouseout', throttle(500, () => {
mousePosition.current = { x: null, y: null };
}));
});
return mousePosition.current;
}
const throttle = (delay: number, fn: (...args: any[]) => void) => {
let shouldWait = false;
return (...args: any[]) => {
if (shouldWait) return;
fn(...args);
shouldWait = true;
setTimeout(() => shouldWait = false, delay);
return;
// return fn(...args);
}
}
For reference, my canvas component responsible of the animation looks roughly like this:
const AnimatedCanvas = ({ children, dimensions }) => {
const canvasRef = useRef(null);
const [renderingContext, setRenderingContext] = useState(null);
const [frameCount, setFrameCount] = useState(0);
// Initialize Canvas
useEffect(() => {
if (!canvasRef.current) return;
const canvas = canvasRef.current;
canvas.width = dimensions.width;
canvas.height = dimensions.height;
const canvas2DContext = canvas.getContext('2d');
setRenderingContext(canvas2DContext);
}, [dimensions]);
// make component re-render every frame
useEffect(() => {
const frameId = requestAnimationFrame(() => {
setFrameCount(frameCount + 1);
});
return () => {cancelAnimationFrame(frameId)};
}, [frameCount, setFrameCount]);
// clear canvas with each render to erase previous frame
if (renderingContext !== null) {
renderingContext.clearRect(0, 0, dimensions.width, dimensions.height);
}
return (
<Canvas2dContext.Provider value={renderingContext}>
<FrameContext.Provider value={frameCount}>
<canvas ref={canvasRef}>
{children}
</canvas>
</FrameContext.Provider>
</Canvas2dContext.Provider>
);
};
The mapped <Particle/> components are are fed to the above canvas component as children:
const Particle = (props) => {
const canvas = useContext(Canvas2dContext);
useContext(FrameContext); // only here to force the force that the particle re-render each frame after the canvas is cleared.
// lots of state calculating logic here
// This is where I need to know mouse position every (or every few) frames in order to modify each particle's behaviour when near the pointer.
canvas.beginPath();
// canvas drawing logic
return null;
}
Just to clarify, the animation is always moving regardless of the mouse being idle, I've seen other solutions that only work for animations triggered exclusively by mouse movement.
Is there any performant way of accessing the mouse position each frame in the Particle mapped components without choking the browser? Is there a better way of handling this type of interactive animation with React?
I am using react three with drei orbitControls and try to detect wether the camera is above or below zero, while moving the camera around. Currently I'm using onEnd listener to trigger my code in my ParentComponent. If I change a state of my parent component like so everything works as expected. Unfortunately if I start another change of orbit controls before ending another and change my parents state, orbit controls will break. E.g. I rotate the camera and hold down my left mouse button, than get below zero with my camera and the scroll with my mouse wheel at the same time, Orbit controls no longer works. This is especially painful when using touch. As soon as a second finger touches and I cross the ground orbit control breaks.
This is my Controls Component:
const Controls = forwardRef(({ handleCamera }, ref) => {
const { gl, camera } = useThree();
const controls = useRef(null);
useImperativeHandle(ref, () => ({
getCamera() {
return camera
},
}))
const handleChange = (e) => {
handleCamera(camera);
}
return (
<OrbitControls ref={controls}
onEnd={(e) => handleChange(e)}
/>
);
});
export default Controls;
And this is my parent:
function App() {
const handleCamera = (camera) => {
setCameraBelowSurface(camera.position.y <= 0 ? true : false);
};
return
<Canvas >
<Suspense>
<Controls
ref={controlsRef}
handleCamera={handleCamera}
/>
</Suspense>
</Canvas>
}
export default App;
I use ReactJS to display a live stream (from my webcam) using the HTML5 video element. The OpenVidu media server handles the backend.
I would like to use the canvas element to draw the video live stream onto the canvas using the drawImage() method.
I have seen other examples, but in them the video element always has a source. Mine does not have a source - when I inspect the video element all I see is:
<video autoplay id="remote-video-zfrstyztfhbojsoc_CAMERA_ZCBRG"/>
This is what I have tried, however the canvas does not work.
export default function Video({ streamManager }) {
const videoRef = createRef()
const canvasRef = createRef()
useEffect(() => {
if (streamManager && !!videoRef) {
//OpenVidu media server attaches the live stream to the video element
streamManager.addVideoElement(videoRef.current)
if (canvasRef.current) {
let ctx = canvasRef.current.getContext('2d')
ctx.drawImage(videoRef.current, 0, 0)
}
}
})
return (
<>
//video element displays the live stream
<video autoPlay={true} ref={videoRef} />
// canvas element NOT working, nothing can be seen on screen
<canvas autoplay={true} ref={canvasRef} width="250" height="250" />
</>
)
}
UPDATE: after further investigation I realised I needed to use the setInterval() function and therefore provided the solution below.
The solution is to extract the canvas logic in a self contained component, and use a setInterval() so that it will draw the video element on the canvas, every 100 ms (or as needed).
Video Component
import React, { useEffect, createRef } from 'react'
import Canvas from './Canvas'
export default function Video({ streamManager }) {
const videoRef = createRef()
useEffect(() => {
if (streamManager && !!videoRef) {
//OpenVidu media server attaches the live stream to the video element
streamManager.addVideoElement(videoRef.current)
}
})
return (
<>
//video element displays the live stream
<video autoPlay={true} ref={videoRef} />
//extract canvas logic in new component
<Canvas videoRef={videoRef} />
</>
)
}
Canvas Component
import React, { createRef, useEffect } from 'react'
export default function Canvas({ videoRef }) {
const canvasRef = createRef(null)
useEffect(() => {
if (canvasRef.current && videoRef.current) {
const interval = setInterval(() => {
const ctx = canvasRef.current.getContext('2d')
ctx.drawImage(videoRef.current, 0, 0, 250, 188)
}, 100)
return () => clearInterval(interval)
}
})
return (
<canvas ref={canvasRef} width="250" height="188" />
)
}
You are using useEffect without parameter which mean your useEffect will call in every render
useEffect(() => {
// every render
})
If you want to run your drawImage at only mount time then use useEffect with []
useEffect(() => {
// run at component mount
},[])
Or if you want to run drawImage when any parameter change then pass your parameter to it
useEffect(() => {
// run when your streamManager change
},[streamManager])
use it as per your requirement
I'm not sure how to formulate the question less vaguely, but it's about pass-by-value and pass-by-reference cases in react. And Hooks.
I am using gsap to animate a div slide-in and out is the context for this, but I'm going to guess that what the ref is used for shouldn't matter.
So, this works fine, even though this is a more class-component-typical way of passing a ref as i understand it:
const RootNavigation = () => {
var navbar = useRef();
const myTween = new TimelineLite({ paused: true });
const animate = () => {
myTween.to(navbar, 0.07, { x: "100" }).play();
};
return(
<div className="nav-main" ref={div => (navbar = div)}> // <<<<<<<<<< pass as a callback
...
</div>
)}
And this elicits a "TypeError: Cannot add property _gsap, object is not extensible" error, even though this is how the React Hooks guide would have me do it:
const RootNavigation = () => {
var navbar = useRef();
const myTween = new TimelineLite({ paused: true });
const animate = () => {
myTween.to(navbar, 0.07, { x: "100" }).play();
};
return(
<div className="nav-main" ref={navbar}> //<<<<<<<<<<<<< not passing a callback
...
</div>
)}
Could somebody explain to me what's going on here or even toss a boy a link to where it's already been explained? I'm sure some sort of Dan character has written about it somewhere, i'm just not sure what to google. Thank you!
In the first example you aren't using a ref, you are reassigning navbar through the ref callback so navbar is the DOM element.
It's the same as
let navbar = null;
return <div ref={node => (navbar = node)} />
In the second example you are using the ref object which is an object with a current property that holds the DOM element
const navbar = useRef(null)
return <div ref={navbar} />
navbar is now
{ current: the DOM element }
So you are passing the object into myTween.to() instead of the DOM element inside navbar.current
Now in the second example gsap is trying to extend the ref object itself and not the DOM element.
Why do we get the TypeError: Cannot add property _gsap, object is not extensible`?
If you look at the source code of useRef you will see on line 891
if (__DEV__) {
Object.seal(ref);
}
that React is sealing the ref object and JavaScript will throw an error when we try to extend it using Object.defineProperty() which is probably what gsap is doing.
The solution for using a ref will be to pass ref.current into tween.to()
const RootNavigation = () => {
const navbar = useRef()
const myTween = new TimelineLite({ paused: true });
const animate = () => {
myTween.to(navbar.current, 0.07, { x: "100" }).play()
}
return (
<div className="nav-main" ref={navbar}>
...
</div>
)
}