🔍 Read the full analysis: Getting Started With Fine-Tuning 3D Generation Models on ThorstenMeyerAI.com
Get business pricing on monitors, keyboards and dev gear
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
Hugging Face has published scripts and containers for fine-tuning its TRELLIS.2 3D-generation model on custom asset datasets, locally or through Hugging Face Jobs. The example uses LoRA adapters and 188 footwear objects to investigate inaccurate shoe soles, but the source does not report completed results or measured improvements.
Hugging Face has published scripts and containers for fine-tuning its TRELLIS.2 3D-generation model on custom assets, with an example that trains LoRA adapters on 188 footwear objects to investigate inaccurate shoe soles. The workflow can run on a user’s machine or through Hugging Face Jobs, but the original fine-tuning guide describes the setup rather than reporting whether fine-tuning improved the generated shoes.
The example targets a specific shortcoming observed in initial tests: TRELLIS.2 could generate shoes that looked plausible overall, while producing inaccurate soles. The authors describe using a subset of 188 shoes from Google Scanned Objects (GSO) to test whether category-specific training could improve footwear geometry. The supplied account does not include a final comparison, benchmark, or before-and-after results.
The scripts use low-rank adaptation (LoRA), a method that adds comparatively small trainable adapter matrices while keeping the model’s original weights frozen. The post says this reduces the number of parameters being trained and the memory used for gradients and optimizer state. Its implementation uses the PEFT library and adds adapters to TRELLIS.2’s sparse-structure and shape-generation components.
Developers can run preprocessing and training using local Docker commands and their own GPU, or submit containerized workloads through Hugging Face Jobs. The post says users can adapt the scripts after uploading a mesh dataset to the Hugging Face Hub. The workflow requires a Hugging Face token and access to gated DINOv3, which the pipeline uses to encode input images.
Fine-tuning TRELLIS.2 for custom 3D assets
Hugging Face released scripts and containers to adapt TRELLIS.2 to specific object categories. The footwear example outlines an experiment, but reports no completed results.
What the workflow targets
The example addresses shoes that look plausible but have inaccurate soles.
Coarse shape
LoRA adapters target sparse structure to adapt the model to footwear.
Detailed shape
The shape stage generates geometry; the example aims to improve shoe details such as soles.
Stage left unchanged
The dataset lacks separate material maps, and the authors say base colors and textures are satisfactory.
How TRELLIS.2 builds a mesh
The described pipeline turns an input image into a textured mesh.
BiRefNet removes the background, then DINOv3-L encodes the image.
Sparse structure guides the model’s detailed geometry generation.
A separate texture stage generates materials before the outputs are combined into a GLB mesh.
Ways to run the experiment
Teams with suitable assets can choose a local or cloud workflow.
- Use the provided Docker commands with your own GPU.
- Run preprocessing and training scripts on your machine.
- Run containerized workloads on cloud hardware.
- Upload the mesh dataset to the Hugging Face Hub.
What teams need
Access and evaluation matter alongside the training setup.
- A Hugging Face token and access to gated DINOv3.
- A comparison of pretrained and fine-tuned outputs, including held-out footwear.
- Clear geometry criteria and reported training settings, hardware, and costs.
A Route to Category-Specific 3D
The release gives developers a practical starting point for testing whether a general-purpose 3D model can be adapted to a narrower production need. In robotics, simulation, and media production, a mesh that looks convincing may still be unsuitable if its shape or dimensions are wrong. Training on a focused collection of assets could help address such gaps, though the footwear example has not yet established that it does.
Packaging the preparation and training steps for both local hardware and cloud jobs may make experimentation more accessible to teams with relevant 3D data. The real value will depend on factors not settled by the supplied account: data preparation, hardware demands, training cost, and performance on unseen examples. The published workflow makes experimentation possible; it is not evidence of a measured quality gain.
The component choice also matters. TRELLIS.2 separates structure, detailed geometry, and materials, and this experiment focuses on structure and geometry while leaving texture generation unchanged. The authors say their footwear data has diffuse color maps but lacks the separate material maps needed to train the texture stage, and that the base model already produces satisfactory colors and textures for this task. That explains the experimental design, but does not show that it outperforms other training choices.
Popular right now
De'Longhi Classic Signature Espresso Machine with Adjustable Milk Frother
Roborock Robot Vacuum and Mop with LiDAR Navigation and Self-Empty Dock
Samsung 65-Inch Class M70H Series 4K Mini LED Smart TV (2026)
As an affiliate, we earn on qualifying purchases.
How TRELLIS.2 Generates Meshes
The post describes TRELLIS.2 as a successor to TRELLIS, which uses a unified latent representation intended to support multiple 3D output formats through decoders. The stated goal is to work across representations rather than rely on a separate generation model for each output type. TRELLIS.2’s pipeline, as described in the source, turns an input image into a textured mesh that can be exported as a GLB file.
The process begins with background removal using BiRefNet, followed by image encoding with DINOv3-L. A sparse-structure flow model predicts a coarse form; a shape flow model then produces detailed geometry, while a separate texture flow model generates surface materials. The experiment adds LoRA adapters to the sparse-structure and shape flow diffusion transformers, leaving the texture flow stage unchanged.
The example’s choice reflects the reported problem: the authors wanted to test shoe shape and sole geometry, not retrain materials. Their account presents this as a targeted plan, not a finding that other parts of the pipeline should never be fine-tuned. The supplied material also does not specify a publication date, a hardware configuration, or a completed evaluation protocol.
Results and Resource Needs Remain Unknown
The supplied source does not say whether the shoe experiment has finished or whether the adapters improved sole geometry. It provides no quantitative benchmark, before-and-after examples, or independent evaluation, and does not specify how accuracy was judged. The reported flaw comes from the authors’ initial tests; the material does not identify the test set or scoring criteria.
Practical requirements are also unclear. Although the workflow supports local GPUs and Hugging Face Jobs, the source gives no hardware specification, training time, cloud cost, or reproducibility results. It does not report whether a model trained on the 188-object subset performs well on footwear outside that collection. Results could depend on the quality and preparation of users’ datasets as well as on the chosen training settings.
Run the Workflow, Then Test It
Developers can use the published repository to prepare a mesh dataset on the Hugging Face Hub and run preprocessing and training locally or through Hugging Face Jobs. They will need a Hugging Face token and access to gated DINOv3. The source also points to a walkthrough, but the supplied material does not provide further details about its contents.
The next meaningful evidence would be a completed evaluation: examples and measurements comparing the base model with the fine-tuned version, along with the test data, hardware, training time, and costs. Testing on shoes outside the training subset would help show whether any improvement carries over to new inputs. Until those details are published, the scripts establish an available workflow, while the effect on output quality remains an open question.
Key Questions
What did Hugging Face publish?
It published scripts and containers for fine-tuning TRELLIS.2 on custom 3D assets, with options to run locally or through Hugging Face Jobs.
What is the footwear example trying to improve?
The example investigates whether fine-tuning can improve shoe geometry, including inaccurate soles reported in initial tests. It uses 188 footwear objects from Google Scanned Objects.
Does the source show that fine-tuning worked?
No. The supplied account describes the training plan but reports no completed comparison, benchmark, or measured improvement.
What do users need to run the workflow?
Users need a prepared mesh dataset, a Hugging Face token, and access to gated DINOv3. They can use local Docker commands with a GPU or submit the containerized workload through Hugging Face Jobs.
Which parts of TRELLIS.2 does the example fine-tune?
It adds LoRA adapters to the sparse-structure and shape stages and leaves texture generation unchanged. The post says the dataset lacks the separate material maps needed to train that stage.
Primary source: Hugging Face · via ThorstenMeyerAI.com
