React three fiber add texture to obj model - reactjs

In my app I want to add texture to the loaded .obj model. I have the same situation with my .fbx loaded models. Below is my example code, but this works only with something like sphereGeometry not with a loaded model.
Thanks in Advance!
import { OBJLoader } from 'three/examples/jsm/loaders/OBJLoader'
import { useTexture } from '#react-three/drei'
const OBJModel = ({ file }: { file: string }) => {
const obj = useLoader(OBJLoader, file)
const texture = useTexture(textureFile)
return (
<mesh>
<primitive object={obj} />
<meshStandardMaterial map={texture} attach="material" />
</mesh>
)
}

primitive is not a subset of mesh. it can be a children of group.
primitive requires both geometry and material as props. Mesh requires both geometry and material as props. it's pretty obvious both cannot be used as subset of each other.
to implement your idea, you need to use only one Mesh or primitive. I'd suggest using Mesh which has abundant documentations. primitive is not documented enough.
the OBJ acquired through useLoader may have complex group inside it. Models usually contain larger sets such as group or scene. Group and Scenes can't have textures. Mesh can.
OBJ(result of useLoader) = scene => group => mesh => geometry, texture
traversing is required to acquire geometry from mesh.
// I've implemented this with Typescript,
// but it is not necessary to use 'as' for type conversion.
const obj = useLoader(OBJLoader, "/rock.obj");
const texture = useTexture("/guide.png");
const geometry = useMemo(() => {
let g;
obj.traverse((c) => {
if (c.type === "Mesh") {
const _c = c as Mesh;
g = _c.geometry;
}
});
return g;
}, [obj]);
// I've used meshPhysicalMaterial because the texture needs lights to be seen properly.
return (
<mesh geometry={geometry} scale={0.04}>
<meshPhysicalMaterial map={texture} />
</mesh>
);
I've implemented it in codesandbox. here's the working code:
https://codesandbox.io/s/crazy-dawn-i6vzb?file=/src/components/Rock.tsx:257-550

i believe obj loader returns a group, or a mesh, it wouldn't make sense to put that into a top-level mesh, and giving the top level a texture wont change the loaded obj mesh.
there are three possible solutions:
use obj.traverse(...) and change the model by mutation, this is what people do in vanilla three and it is problematic because mutation is bad, you are destroying the source data and you won't be able to re-use the model
i would suggest to convert your model to a gltf, then you can use gltfjsx https://github.com/pmndrs/gltfjsx which can lay out the full model declaratively. click the video, this is exactly what you want
if you must use obj files you can create the declarative graph by hand. there is a hook that gives you { nodes, materials } https://docs.pmnd.rs/react-three-fiber/API/hooks#use-graph

Related

react-three-fiber: How to capture events ANYWHERE in <Canvas />?

I'm trying to create a 3D experience using react-three-fiber where the user can interact with any part of the 3D canvas, including points on the canvas that both (a) intersect an object, and (b) where there is nothing intersecting the ray at all.
This means I can't use the pointer events abstractions built into <mesh /> etc., as those pointer events ONLY capture points that intersect objects.
I tried attaching handlers to the <Canvas onMouseDown={...} />, however I'm unable to access the THREE internals from there as the event passed by onMouseDown does not contain them, and the useThree hook must be deeper in the tree to access the THREE react context.
So I also tried creating a component nested inside <Canvas /> (see below) where I'd be able to use the useThree hook:
<Canvas>
<MouseHandler>
...
<mesh />
...
</MouseHandler>
</Canvas>
export function MouseHandler({ children }) {
const { Camera } = useThree()
return <Html><div onMouseDown={...}>{children}</div></Html>
}
...but then react-three-fiber complains that I have THREE objects inside HTML objects.
Anyone have any suggestions on how else I might be able to solve this?
Doesn't feel like a common enough use case that react-three-fiber would ever support, so what I ended up doing was hack it like this, using React Refs to yank a "pointer" to the three objects I needed from deeper into the tree.
const threeRef = useRef()
<MouseHandler threeRef={threeRef}>
<Canvas>
<ThreeConnect threeRef={threeRef}>
...
<mesh />
...
</ThreeConnect>
</Canvas>
</MouseHandler>
...
export function MouseHandler({ threeRef, children }) {
const { Camera } = threeRef.current
return <Html><div onMouseDown={...}>{children}</div></Html>
}
...
export default function ThreeConnect({ children, threeRef }) {
const { scene, camera } = useThree()
useEffect(() => {
threeRef.current = {
...threeRef.current,
camera,
scene,
}
}, [scene, camera])
return <>{children}</>
}
You need add onMouseDown in <mesh> for capture points that intersect objects.
And add onPointerMissed in <Canvas> for capture nothing intersecting.

useConvexPolyhedron collision on react-three/cannon clipping through object

I am having issues trying to make collision work properly with an imported glb file only used for collision.
There are two parts to this question:
I am currently getting a whole lot of faceNormal errors and vertices warnings in the console.
.faceNormals[767] = Vec3(0.999994684003945,-4.252105447140875e-10,0.003260669233505899) looks like it points into the shape? The vertices follow. Make sure they are ordered CCW around the normal, using the right hand rule.
.vertices[555] = Vec3(-18.135730743408203,9.071623802185059,-13.433568000793457)
I am not sure if this error has something to do with how I'm merging the geometry of the object or how the object is built inside blender.
When my sphere collides with the glb model it will clip through it sometimes with enough force. (holding down the W key to move the sphere forward). It clips through especially on the edges of the environment, against a flat wall it gives some resistance but still manages to clip through with enough time holding down a movement key.
Here is the code I'm using to import my glb model:
import React, { useMemo } from "react";
import { useConvexPolyhedron } from "#react-three/cannon";
import { Geometry } from "three-stdlib/deprecated/Geometry";
import {useGLTF} from "#react-three/drei";
import collision from "./Collision6.glb";
function toConvexProps(bufferGeometry) {
const geo = new Geometry().fromBufferGeometry(bufferGeometry);
geo.mergeVertices();
return [geo.vertices.map((v) => [v.x, v.y, v.z]), geo.faces.map((f) => [f.a, f.b, f.c]), []];
}
export default function Collision(props) {
const { nodes } = useGLTF(collision);
const geo = useMemo(() => toConvexProps(nodes.Collision001.geometry), [nodes]);
const [ref] = useConvexPolyhedron(() => ({ type: 'Static', mass: 100 ,position:[0,0,0], args: geo
}));
return (
<mesh
ref={ref}
geometry={nodes.Collision001.geometry}
>
<meshStandardMaterial wireframe color="#ff0000" opacity={0} transparent />
</mesh>
);
}

Updating OrbitControls rotation when camera is rotated

I'm using #react-three/fiber and I'm implementing first person controls (WASD & cursor keys) with addition of OrbitControls to navigate a scene. I'm pretty much cloning PointerLockControls and I've got it working for the WASD controls as such (updating the target vector of the OrbitControls, passed as ref):
const moveForward = (distance) => {
vec.setFromMatrixColumn(camera.matrix, 0)
vec.crossVectors(camera.up, vec)
camera.position.addScaledVector(vec, distance)
orbitControls.current.target.addScaledVector(vec, distance)
}
const moveRight = (distance) => {
vec.setFromMatrixColumn(camera.matrix, 0)
camera.position.addScaledVector(vec, distance)
orbitControls.current.target.addScaledVector(vec, distance)
}
However, I'm not quite sure how to go about updating the target when the camera is rotated. Here's how I'm rotating the camera and its working just fine without OrbitControls:
const euler = new THREE.Euler(0, 0, 0, 'YXZ' );
euler.setFromQuaternion(camera.quaternion)
euler.y -= 0.25 * radians;
camera.quaternion.setFromEuler(euler)
Preview here: https://codesandbox.io/s/wasd-with-orbit-9edup7
OK, so you can see the working version here: https://yvod70.csb.app/
The idea is quite simple, attach the camera to your player/box and then move said object instead of the camera. The camera being a child of the player will be translated and rotated relative to the player.
To do this the first step is to get a reference to the mesh:
const scene = () => {
const mesh = useRef();
return (
<Canvas>
<mesh
ref={mesh}
>
<boxBufferGeometry args={[1, 1, 1]} />
<meshBasicMaterial wireframe color={"green"} />
</mesh>
<Controls mesh={mesh} />
</Canvas>
);
}
After setting this up, we just pass the mesh ref to whatever React component we want and use it however. In our case it's to replace the camera for movement and attach the camera to the box. Which can be done like so in your Controls component:
const Controls = ({ orbitControls, mesh }) => {
/** #type {THREE.Mesh} */
const box = mesh.current;
const { camera } = useThree();
const code = useCodes();
// ...your movement code here, remember to replace camera with box.
// FIXME: Move the Controls component lower down the component tree so that we don't end up with these race conditions.
if (!!box) {
box.add(camera);
}
// ...animation code
}
These were the steps I took to attach the orbit controls to the player

Loading mesh(OBJ+MTL+JPG) with react-three-fiber, Texture is not working

In my react app, I am trying to make a view page which shows 3d-mesh exported from pix4d. This mesh consists of three types of files, (.obj, .mtl, .jpg) pix4d.com.
I am new, with react-three-fiber, which I suppose is best way to achieve my solution to my problem.
Below is whole code of react component used to load and render 3D-Model.
code
Thanks in Advance!
I want to understand how to attach texture & material to my obj rendered model.
I was looking for this answer for a couple of weeks, finally I've found a way to make it work.
Material loader is not resolving by itself the import of remote files (it does on web, not in mobile, maybe in the future it will). So, I'm creating material and assigning it images by hand.
Something like this:
import { TextureLoader } from 'expo-three';
import { OBJLoader } from 'three/examples/jsm/loaders/OBJLoader';
import { MTLLoader } from 'three/examples/jsm/loaders/MTLLoader';
// useThree is used to get scene reference
import { useThree } from 'react-three-fiber';
const textureLoader = new TextureLoader();
const mapImage = textureLoader.load(require('path/to/image1.png'))
const normalMapImage = textureLoader.load(require('path/to/image2.png'))
Note that TextureLoader from expo-three can handle the file resource returned from require()
const loaderObj = new OBJLoader();
const loaderMtl = new MTLLoader();
export default props => {
const { scene } = useThree();
loaderMtl.load(
'https://url_to/material.mtl',
mtl => {
mtl.preload();
loaderObj.setMaterials(mtl);
loaderObj.load(
'https://url_to/model.obj',
obj => {
// simple logic for an obj with single child
obj.children[0].material.map = mapImage;
obj.children[0].material.normalMap = normalMapImage;
scene.add(obj)
}
)
}
)
return null;
}
This is my first successful attempt to render an obj with mtl including a map and a normal map, so since it works, we can keep updating the code for improvements.
Another way to load model with texture is by specifying the path where your texture has been stored. In one case, mtl + obj + texture files are stored in your react project's directory 'public/models/'. So you can specify the path by calling setPath() function prior loading your material or object file and it should load your texture on the material. You may also want to make sure that in ./mtl file the texture name is correct. It should be called following by map_Kd in .mtl file.
const Model = () => {
const materialLoader = new MTLLoader().setPath('./model/').load(MaterialFile);
const objLoader = new OBJLoader().setMaterials(materialLoader).load(OojectFile);
return <primitive object={objLoader} />;
};

Creating a custom Leaflet layer control in React

I'm attempting to completely recreate or reorganize the functionality of the LayersControl component in its own separate panel using react-leaflet.
I have several filtered into their own and it works fine, but I'd like to customize the look and location of the Control element.
I've hosted the current version of my Leaflet app on github pages here. You can see the control on the right, which is the basic Leaflet control, but I'd like to the Icon on the left (the layers icon) to accomplish the same thing instead with custom react components.
Just wondering if anyone can point me in the right direction to beginning to accomplish this!
This is my current render for my react-leaflet map:
render() {
const types = [...new Set(data.map(loc => loc.type))];
const group = types.map(type =>
data.filter(loc => loc.type === type)
.map(({id, lat, lng, name}) =>
<LayersControl.Overlay name={startCase(toLower(type))}>
<LayerGroup>
<Marker key={id} position={[lat, lng]} icon=
{locationIcon}>
<Tooltip permanent direction="bottom" opacity={.6}>
{name}
</Tooltip>
</Marker>
</LayerGroup>
</LayersControl.Overlay>
));
return (
<>
<ControlPanel />
<Map
zoomControl={false}
center={this.state.center}
zoom={this.state.zoom}
maxBounds={this.state.maxBounds}
maxZoom={10}
>
<LayersControl>
<TileLayer
url='https://cartocdn-gusc.global.ssl.fastly.net//ramirocartodb/api/v1/map/named/tpl_756aec63_3adb_48b6_9d14_331c6cbc47cf/all/{z}/{x}/{y}.png'
/>
<ZoomControl position="topright" />
{group}
</LayersControl>
</Map>
</>
);
}
So theres still a few bugs in this but i've managed get most of the way (self taught react) using material UI as an example, can be seen in this sandbox link:
https://codesandbox.io/embed/competent-edison-wt5pl?fontsize=14
The general bassis is that we extend MapControl which means we have to define createLeafletElement, this has to return a generic leaflet (not react) control from the original javascript leaflet package. Essentially making a div with the domutil provided by leaflet and then portaling our react components through that div with react portals.
Again with another class extension we extend some of the classes provided by react-leaflet for layers, i pulled it out and just made a generic layer that you could define a group for, that way you could render any layer (polygon, baselayer etc) and specify the group to tell it where to go in the layer control i.e no need for specific components or overlays. As we are extending the class we need implement and pass down the methods we want to use, like addLayer, remove layer etc. During these implementations i've just added them to state to track what layers are active and such.
Not sure if there are better practices throughout everything i've implemented but this is definitely a start, hopefully in the right direction.
Bugs - The first layer in each group won't turn on correctly without the 2nd item ticked, something to do with state i think but didn't have the time to track it down
Thanks Dylan and Peter for this nice React Leaflet custom control approach. I assumed there was still a bug in the toggleLayer function. It's checked multiple checkboxes and the layers won't change properly. So I restructered a little bit and now it should work fine.
toggleLayer = layerInput => {
const { name, group } = layerInput;
let layers = { ...this.state.layers };
layers[group] = layers[group].map(l => {
l.checked = false;
this.removeLayer(l.layer);
if (l.name === name) {
l.checked = !l.checked;
this.props.leaflet.map.addLayer(l.layer);
}
return l;
});
this.setState({
layers
});
};
Just to elaborate on the bug that is mentioned in Dylans answer...
If you have more then one ControlledLayerItem, none items are added to the map until the very last item is checked. To fix this, the toggleLayer method in ControlLayer2.js has to be slightly modified:
toggleLayer = layerInput => {
const { layer, name, checked, group } = layerInput;
let layers = { ...this.state.layers };
layers[group] = layers[group].map(l => {
if (l.name === name) {
l.checked = !l.checked;
l.checked
? this.props.leaflet.map.addLayer(layer)
: this.removeLayer(layer);
}
return l;
});
this.setState({
layers
});
};
Thanks Dylan for the code, it was really helpfull.

Resources