Skip to content

Commit

Permalink
revisions from merve review
Browse files Browse the repository at this point in the history
  • Loading branch information
dylanebert committed Jul 16, 2024
1 parent 46aaf14 commit 8bee571
Show file tree
Hide file tree
Showing 4 changed files with 22 additions and 26 deletions.
16 changes: 9 additions & 7 deletions packages/tasks/src/tasks/image-to-3d/about.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,13 @@

Image-to-3D models can be used in a wide variety applications that require 3D, such as games, animation, design, architecture, engineering, marketing, and more.

### Meshes
### Generating Meshes

Meshes are the standard representation of 3D in industry.

Meshes can be in `.obj`, `.glb`, `.stl`, or `.gltf` format. Other formats are allowed, but won't be rendered in the gradio [Model3D](https://www.gradio.app/docs/gradio/model3d) component.

### Splats
### Generating Gaussian Splats

[Gaussian Splatting](https://huggingface.co/blog/gaussian-splatting) is a differentiable rasterization technique that represents 3D as fuzzy points.

Splats can be in `.ply` or `.splat` format. They can be rendered in the gradio [Model3D](https://www.gradio.app/docs/gradio/model3d) component using the [gsplat.js](https://github.com/huggingface/gsplat.js) library.
[Gaussian Splatting](https://huggingface.co/blog/gaussian-splatting) is a rendering technique that represents scenes as fuzzy points.

### Inference

Expand Down Expand Up @@ -44,6 +40,12 @@ pipeline.save_ply(result, result_path)
print(f"Output saved to {result_path}")
```

### Output Formats

Meshes can be in `.obj`, `.glb`, `.stl`, or `.gltf` format. Other formats are allowed, but won't be rendered in the gradio [Model3D](https://www.gradio.app/docs/gradio/model3d) component.

Splats can be in `.ply` or `.splat` format. They can be rendered in the gradio [Model3D](https://www.gradio.app/docs/gradio/model3d) component using the [gsplat.js](https://github.com/huggingface/gsplat.js) library.

## Useful Resources

- [ML for 3D Course](https://huggingface.co/learn/ml-for-3d-course)
Expand Down
11 changes: 5 additions & 6 deletions packages/tasks/src/tasks/image-to-3d/data.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,24 +51,23 @@ const taskData: TaskDataCustom = {
id: "dylanebert/3d-arena",
},
{
description: "InstantMesh-based image-to-3D demo with mesh outputs.",
description: "Image-to-3D demo with mesh outputs.",
id: "TencentARC/InstantMesh",
},
{
description: "TripoSR-based image-to-3D demo with mesh outputs.",
description: "Image-to-3D demo with mesh outputs.",
id: "stabilityai/TripoSR",
},
{
description: "Real3D-based image-to-3D demo with mesh outputs.",
description: "Image-to-3D demo with mesh outputs.",
id: "hwjiang/Real3D",
},
{
description: "LGM-based image-to-3D demo with splat outputs.",
description: "Image-to-3D demo with splat outputs.",
id: "dylanebert/LGM-mini",
},
],
summary:
"Image-to-3D models take in image input and produce 3D output. The 3D output may be a mesh (`.glb`, `.obj`) or splat (`.ply`, `.splat`).",
summary: "Image-to-3D models take in image input and produce 3D output.",
widgetModels: [],
youtubeId: "",
};
Expand Down
10 changes: 3 additions & 7 deletions packages/tasks/src/tasks/text-to-3d/about.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,13 @@ Text-to-3D models can be used in a wide variety applications that require 3D, su

This task is similar to the [image-to-3d](https://huggingface.co/tasks/image-to-3d) task, but takes text input instead of image input. In practice, this is often equivalent to a combination of [text-to-image](https://huggingface.co/tasks/text-to-image) and [image-to-3d](https://huggingface.co/tasks/image-to-3d).

### Meshes
### Generating Meshes

Meshes are the standard representation of 3D in industry.

Meshes can be in `.obj`, `.glb`, `.stl`, or `.gltf` format. Other formats are allowed, but won't be rendered in the gradio [Model3D](https://www.gradio.app/docs/gradio/model3d) component.
### Generating Gaussian Splats

### Splats

[Gaussian Splatting](https://huggingface.co/blog/gaussian-splatting) is a differentiable rasterization technique that represents 3D as fuzzy points.

Splats can be in `.ply` or `.splat` format. They can be rendered in the gradio [Model3D](https://www.gradio.app/docs/gradio/model3d) component using the [gsplat.js](https://github.com/huggingface/gsplat.js) library.
[Gaussian Splatting](https://huggingface.co/blog/gaussian-splatting) is a rendering technique that represents scenes as fuzzy points.

## Useful Resources

Expand Down
11 changes: 5 additions & 6 deletions packages/tasks/src/tasks/text-to-3d/data.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,14 @@ const taskData: TaskDataCustom = {
inputs: [
{
label: "Prompt",
content: "Enter a prompt to generate a 3D model.",
content: "a cat statue",
type: "text",
},
],
outputs: [
{
label: "Result",
content: "image-to-3d-3d-output-filename.glb",
content: "text-to-3d-3d-output-filename.glb",
type: "text",
},
],
Expand All @@ -40,16 +40,15 @@ const taskData: TaskDataCustom = {
],
spaces: [
{
description: "Shap-E based text-to-3D demo with mesh outputs.",
description: "Text-to-3D demo with mesh outputs.",
id: "hysts/Shap-E",
},
{
description: "LGM based text/image-to-3D demo with splat outputs.",
description: "Text/image-to-3D demo with splat outputs.",
id: "ashawkey/LGM",
},
],
summary:
"Text-to-3D models take in text input and produce 3D output. The 3D output may be a mesh (`.glb`, `.obj`) or splat (`.ply`, `.splat`).",
summary: "Text-to-3D models take in text input and produce 3D output.",
widgetModels: [],
youtubeId: "",
};
Expand Down

0 comments on commit 8bee571

Please sign in to comment.