I would like to take image data (RGBA) from react-three-fiber
like this code on HTML
var myCanvas = document.createElement("canvas");
myCanvas.width = texture.image.width;
myCanvas.height = texture.image.height;
var myCanvasContext = myCanvas.getContext("2d"); // Get canvas 2d context
myCanvasContext.drawImage(texture.image, 0, 0); // Draw the texture
var texels = myCanvasContext.getImageData(0,0, width, height); // Read the texels/pixels back
However, I cannot access Canvas to drawImage or getImageData.
const ref = React.useRef();
React.useEffect(() => {
if (ref) {
//error
ref.current.getContext("2d");
}
}, []);
<Canvas
dpr={[1, 1.5]} //pixelRatio
ref={ref}
>
<Particles />
</Canvas>
and I tried to use useThree. there is no method to get imageData
const {
gl,
} = useThree();
lastly, I tired to get an image using useLoader as a texture
const map = useLoader(TextureLoader, "/scroll_images/img1.jpg");
so, how can I get the imageData?
In your first example, try useLayoutEffect instead of useEffect. Refs are not actually populated until the first DOM render happened. Unlike useEffect, useLayoutEffect waits until this occurred.
React.useLayoutEffect(() => {
ref.current.getContext("2d"); // <-- hopefully doesnt error
}, []);
In your second example I think its because you need to pass gl={{ preserveDrawingBuffer: true }} prop to your Canvas. Then you can use gl.domElement.toDataURL('image/png') or similar.
Related
I've got a Uint8ClampedArray that comes from my backend representing the pixels array of an image and I'm trying to display it in React + Typescript by using a canvas element.
That's the code:
const imgData = new ImageData(
dataArray, //Uint8ClampedArray
width,
height
);
return <Canvas
imgData={imgData}
canvasHeight={ref.current.offsetHeight} //max height available in the screen
canvasWidth={ref.current.offsetHeight * ratio}
imgHeight={height}
imgWidth={width}
/>
---
export function Canvas(props: Props) {
const canvasRef = useRef(null);
useEffect(() => {
const canvasRefCurrent: HTMLCanvasElement = canvasRef.current!;
const ctx = canvasRefCurrent.getContext("2d", { alpha: false })!;
ctx.putImageData(props.data, 0, 0);
}, [props.data]);
return ( <canvas
ref={canvasRef}
width={props.imgWidth}
height={props.imgHeight}
/> );
However, now I'd like to apply zoom/pan to the canvas and to retrieve the coordinates of a clicked point in the image. Does anybody know about any library that can do these actions?
Basically, I have some code that I wrote and it takes an image and draws it on the canvas however when I use an image link instead, the app breaks down and turns white with an error in the console saying "Process is not defined".
This is the code with the image directly imported
import image from '../images/leaf-spot-disease.jpg'
const Testing = () => {
const [imageLoaded, setImageLoaded] = useState(false);
const handleImageLoad = useCallback(() => {
setImageLoaded(true);
}, []);
const canvasRef = useRef(null)
const imageRef = useRef(null)
useEffect(() => {
if(imageLoaded) {
const image = imageRef.current
const canvas = canvasRef.current
canvas.width = 100
canvas.height = 100
const context = canvas.getContext('2d')
context.drawImage(image , 0 , 0 , canvas.width , canvas.height)
console.log(context.getImageData(0, 0, canvas.width , canvas.height).data.length)
}
}, [imageLoaded])
return (
<>
<img src={image} alt="" className={styles.theImage} ref={imageRef} onLoad={handleImageLoad}/>
<canvas className={styles.canvasCont} ref={canvasRef}>
</canvas>
</>
)
}
With the above way it works perfectly but the moment I use a link instead of importing an image like the example below the app breaks
const image = "https://storage.googleapis.com/object-cut-images/54062ac7-59dd-4684-baf4-840e6c5a2c3c.png"
When image is imported it works perfectly and when I use the above code it does not work again and nothing gets drawn on the canvas and also the page turns full white with that process is not defined error.
I have an animated background using canvas and requestAnimationFrame in my React app and I am trying to have its moving particles interact with the mouse pointer, but every solution I try ranges from significantly slowing down the animation the moment I start moving the mouse to pretty much crashing the browser.
The structure of the animated background component goes something like this:
<BackgroundParentComponent> // Gets mounted only once
<Canvas> // Reutilizable canvas, updates every frame.
// v - dozens of moving particles (just canvas drawing logic, no JSX return).
// Each particle calculates its next frame updated state every frame.
{particlesArray.map(particle => <Particle/>}
<Canvas/>
<BackgroundParentComponent />
I have tried moving the event listeners to every level of the component structure, calling them with a custom hook with an useRef to hold the value without rerendering, throttling the mouse event listener so that it does not fire that often... nothing seems to help. This is my custom hook right now:
const useMousePosition = () => {
const mousePosition = useRef({ x: null, y: null });
useEffect(() => {
window.addEventListener('mousemove', throttle(200, (event) => {
mousePosition.current = { x: event.x, y: event.y };
}))
});
useEffect(() => {
window.addEventListener('mouseout', throttle(500, () => {
mousePosition.current = { x: null, y: null };
}));
});
return mousePosition.current;
}
const throttle = (delay: number, fn: (...args: any[]) => void) => {
let shouldWait = false;
return (...args: any[]) => {
if (shouldWait) return;
fn(...args);
shouldWait = true;
setTimeout(() => shouldWait = false, delay);
return;
// return fn(...args);
}
}
For reference, my canvas component responsible of the animation looks roughly like this:
const AnimatedCanvas = ({ children, dimensions }) => {
const canvasRef = useRef(null);
const [renderingContext, setRenderingContext] = useState(null);
const [frameCount, setFrameCount] = useState(0);
// Initialize Canvas
useEffect(() => {
if (!canvasRef.current) return;
const canvas = canvasRef.current;
canvas.width = dimensions.width;
canvas.height = dimensions.height;
const canvas2DContext = canvas.getContext('2d');
setRenderingContext(canvas2DContext);
}, [dimensions]);
// make component re-render every frame
useEffect(() => {
const frameId = requestAnimationFrame(() => {
setFrameCount(frameCount + 1);
});
return () => {cancelAnimationFrame(frameId)};
}, [frameCount, setFrameCount]);
// clear canvas with each render to erase previous frame
if (renderingContext !== null) {
renderingContext.clearRect(0, 0, dimensions.width, dimensions.height);
}
return (
<Canvas2dContext.Provider value={renderingContext}>
<FrameContext.Provider value={frameCount}>
<canvas ref={canvasRef}>
{children}
</canvas>
</FrameContext.Provider>
</Canvas2dContext.Provider>
);
};
The mapped <Particle/> components are are fed to the above canvas component as children:
const Particle = (props) => {
const canvas = useContext(Canvas2dContext);
useContext(FrameContext); // only here to force the force that the particle re-render each frame after the canvas is cleared.
// lots of state calculating logic here
// This is where I need to know mouse position every (or every few) frames in order to modify each particle's behaviour when near the pointer.
canvas.beginPath();
// canvas drawing logic
return null;
}
Just to clarify, the animation is always moving regardless of the mouse being idle, I've seen other solutions that only work for animations triggered exclusively by mouse movement.
Is there any performant way of accessing the mouse position each frame in the Particle mapped components without choking the browser? Is there a better way of handling this type of interactive animation with React?
In my <App> Context, I have a canvas element (#offScreen) that is already hooked in the requestAnimationFrame loop and appropriately drawing to that canvas, verified by .captureStream to a <video> element.
In my <Canvas> react component, I have the following code (which works, but seems clunky/not the best way to copy an offscreen canvas to the DOM):
NOTE: master is the data object for the <App> Context.
function Canvas({ master, ...rest } = {}) {
const canvasRef = useRef(master.canvas);
const draw = ctx => {
ctx.drawImage(master.canvas, 0, 0);
};
useEffect(() => {
const canvas = canvasRef.current;
const ctx = canvas.getContext("2d");
let animationFrameId;
const render = () => {
draw(ctx)
animationFrameId = window.requestAnimationFrame(render)
}
render();
return () => {
window.cancelAnimationFrame(animationFrameId);
}
}, [ draw ]);
return (
<canvas
ref={ canvasRef }
onMouseDown={ e => console.log(master, e) }
/>
);
};
Edited for clarity based on comments
In my attempts to render the master.canvas directly (e.g. return master.canvas; in <Canvas>), I get some variation of the error "Objects cannot be React children" or I get [object HTMLCanvasElement] verbatim on the screen.
It feels redundant to take the #offScreen canvas and repaint it each frame. Is there, instead, a way to insert or append #offScreen into <Canvas>, so that react is just directly utilizing #offScreen without having to repaint it into the react component canvas via the ref?
Specific Issue: Functionally, I'm rendering a canvas twice--once off screen and once in the react component. How do I (replace/append?) the component's <canvas> element with the offscreen canvas (#offScreen), instead of repainting it like I'm doing now?
For anyone interested, this was actually fairly straightforward, as I overcomplicated it substantially.
export function Canvas({ canvas, ...rest }) {
const container = useRef(null);
useEffect(() => {
container.current.innerHTML = "";
container.current.append(canvas);
}, [ container, canvas ]);
return (
<div ref={ container } />
)
}
I use ReactJS to display a live stream (from my webcam) using the HTML5 video element. The OpenVidu media server handles the backend.
I would like to use the canvas element to draw the video live stream onto the canvas using the drawImage() method.
I have seen other examples, but in them the video element always has a source. Mine does not have a source - when I inspect the video element all I see is:
<video autoplay id="remote-video-zfrstyztfhbojsoc_CAMERA_ZCBRG"/>
This is what I have tried, however the canvas does not work.
export default function Video({ streamManager }) {
const videoRef = createRef()
const canvasRef = createRef()
useEffect(() => {
if (streamManager && !!videoRef) {
//OpenVidu media server attaches the live stream to the video element
streamManager.addVideoElement(videoRef.current)
if (canvasRef.current) {
let ctx = canvasRef.current.getContext('2d')
ctx.drawImage(videoRef.current, 0, 0)
}
}
})
return (
<>
//video element displays the live stream
<video autoPlay={true} ref={videoRef} />
// canvas element NOT working, nothing can be seen on screen
<canvas autoplay={true} ref={canvasRef} width="250" height="250" />
</>
)
}
UPDATE: after further investigation I realised I needed to use the setInterval() function and therefore provided the solution below.
The solution is to extract the canvas logic in a self contained component, and use a setInterval() so that it will draw the video element on the canvas, every 100 ms (or as needed).
Video Component
import React, { useEffect, createRef } from 'react'
import Canvas from './Canvas'
export default function Video({ streamManager }) {
const videoRef = createRef()
useEffect(() => {
if (streamManager && !!videoRef) {
//OpenVidu media server attaches the live stream to the video element
streamManager.addVideoElement(videoRef.current)
}
})
return (
<>
//video element displays the live stream
<video autoPlay={true} ref={videoRef} />
//extract canvas logic in new component
<Canvas videoRef={videoRef} />
</>
)
}
Canvas Component
import React, { createRef, useEffect } from 'react'
export default function Canvas({ videoRef }) {
const canvasRef = createRef(null)
useEffect(() => {
if (canvasRef.current && videoRef.current) {
const interval = setInterval(() => {
const ctx = canvasRef.current.getContext('2d')
ctx.drawImage(videoRef.current, 0, 0, 250, 188)
}, 100)
return () => clearInterval(interval)
}
})
return (
<canvas ref={canvasRef} width="250" height="188" />
)
}
You are using useEffect without parameter which mean your useEffect will call in every render
useEffect(() => {
// every render
})
If you want to run your drawImage at only mount time then use useEffect with []
useEffect(() => {
// run at component mount
},[])
Or if you want to run drawImage when any parameter change then pass your parameter to it
useEffect(() => {
// run when your streamManager change
},[streamManager])
use it as per your requirement