Three.JS

Three.js Integration

This page details how Three.js is integrated into the application for rendering 3D avatars, handling animations, and creating an interactive 3D environment.

Table of Contents

• Overview
• Scene Setup
• Avatar Rendering
• Lighting
• Animation Handling
• Camera Controls
• React Integration
• Performance Considerations

Overview

We use Three.js as the core 3D rendering engine, integrated with React for building user interfaces. We use it to render avatars, handle animations, and create an interactive 3D environment with proper lighting and camera controls.

Scene Setup

The Three.js scene is set up in components like Avatar.tsx and AnimationPreview.tsx. Here's a basic outline of the scene setup:

import * as THREE from 'three';

// Scene
const scene = new THREE.Scene();

// Camera
const camera = new THREE.PerspectiveCamera(75, window.innerWidth / window.innerHeight, 0.1, 1000);
camera.position.set(0, 1.6, 3);

// Renderer
const renderer = new THREE.WebGLRenderer({ antialias: true });
renderer.setSize(window.innerWidth, window.innerHeight);
renderer.setPixelRatio(window.devicePixelRatio);
renderer.outputColorSpace = THREE.SRGBColorSpace;
renderer.toneMapping = THREE.ACESFilmicToneMapping;
renderer.toneMappingExposure = 1;
renderer.shadowMap.enabled = true;
renderer.shadowMap.type = THREE.PCFSoftShadowMap;

Avatar Rendering

Avatars are loaded using the GLTFLoader and added to the scene:

import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader';

const loader = new GLTFLoader();
loader.load(avatarUrl, (gltf) => {
  const model = gltf.scene;
  scene.add(model);
  // Additional setup for the model...
});

Lighting

We use a combination of different light types to create depth and enhance the visual quality of the scene:

// Ambient light
const ambientLight = new THREE.AmbientLight(0xffffff, 0.5);
scene.add(ambientLight);

// Directional light
const directionalLight = new THREE.DirectionalLight(0xffffff, 1);
directionalLight.position.set(5, 10, 7.5);
directionalLight.castShadow = true;
scene.add(directionalLight);

// Hemisphere light
const hemisphereLight = new THREE.HemisphereLight(0xffffff, 0x8d6e63, 0.6);
scene.add(hemisphereLight);

// Point lights for added depth
const redLight = new THREE.PointLight(0xff0000, 0.5, 10);
redLight.position.set(5, 2, 0);
scene.add(redLight);

const blueLight = new THREE.PointLight(0x0000ff, 0.5, 10);
blueLight.position.set(-5, 2, 0);
scene.add(blueLight);

This lighting setup creates a visually appealing environment that enhances the appearance of the avatar, without it the Avatar would look very washed and lacking any depth.

Animation Handling

Animations are handled using Three.js' AnimationMixer:

const mixer = new THREE.AnimationMixer(model);
const action = mixer.clipAction(animationClip);
action.play();

// Update in animation loop
function animate() {
  requestAnimationFrame(animate);
  const delta = clock.getDelta();
  mixer.update(delta);
  renderer.render(scene, camera);
}
animate();

Camera Controls

We use OrbitControls to allow user interaction with the camera:

import { OrbitControls } from 'three/examples/jsm/controls/OrbitControls';

const controls = new OrbitControls(camera, renderer.domElement);
controls.target.set(0, 1.6, 0);
controls.update();

React Integration

We integrate Three.js with React using the @react-three/fiber library. This allows us to use Three.js within React components:

import { Canvas, useFrame } from '@react-three/fiber';

function Scene() {
  useFrame((state, delta) => {
    // Update logic here
  });

  return (
    <mesh>
      {/* Mesh components */}
    </mesh>
  );
}

function App() {
  return (
    <Canvas>
      <Scene />
    </Canvas>
  );
}

Performance Considerations

To optimize performance:

1. Use useFrame for efficient updates in the render loop.
2. Implement proper cleanup in useEffect hooks to prevent memory leaks.
3. Use useMemo and useCallback to prevent unnecessary re-renders and recreations of Three.js objects.

Example:

const Avatar = React.memo(({ avatarUrl }) => {
  const mesh = useMemo(() => new THREE.Mesh(), []);
  
  useEffect(() => {
    // Load avatar
    return () => {
      // Cleanup
    };
  }, [avatarUrl]);

  return <primitive object={mesh} />;
});