File size: 3,399 Bytes
2a5582b 7f11111 2a5582b 7f11111 5473044 2a5582b 5473044 2a5582b 5473044 2a5582b 5473044 2a5582b 5473044 2a5582b 5473044 2a5582b 5473044 2a5582b 5473044 2a5582b 5473044 2a5582b 5473044 2a5582b 7f11111 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 | ---
library_name: renderformer-studio
license: mit
pipeline_tag: other
tags:
- neural-rendering
- rendering
- transformers
- materials
---
# RenderFormer V2
This repository bundles the two RenderFormer V2 renderer checkpoints and the
material-latent models used by the data pipeline. Source code, installation
instructions, scene conversion, and inference examples live in
[RenderFormer Studio](https://github.com/iamNCJ/RenderFormer-Studio).
The model was presented in the paper [RenderFormer-V2: Neural Rendering with Heterogeneous Scene Primitives](https://huggingface.co/papers/2609.05738). See the [project page](https://renderformer.github.io/v2/) for additional results and details.
If this repository is access-protected during release staging, authenticate
with Hugging Face before loading it. One repository revision selects every
component below as an atomic, compatible set.
## Components
| Path | Class | Resolution | Purpose |
| --- | --- | ---: | --- |
| `transformer_512` | `RenderFormerModel` | 512 | Native-512 V2 renderer |
| `transformer_2048` | `RenderFormerModel` | 2048 | Native-2048 V2 renderer |
| `material_autoencoder` | `MaterialAutoencoder` | 256 | Material image to/from 9-D latent |
| `diffspec_mapper` | `DiffuseSpecularToLatent` | — | Diffuse/specular BRDF to 9-D latent |
| `metallic_mapper` | `PrincipledBRDFToLatent` | — | Metallic/roughness BRDF to 9-D latent |
| `metallic_transmission_mapper` | `PrincipledBRDFToLatentWithTransmission` | — | Metallic/transmission BRDF to 9-D latent |
The renderer pipeline loads the Qwen-Image VAE configured by each transformer
for environment and raw-material encoding. Those auxiliary weights are an
external dependency and are not duplicated in this bundle.
## Load a renderer
`resolution` automatically selects `transformer_512` or `transformer_2048`:
```python
from renderformer import RenderFormerV2Pipeline
pipeline = RenderFormerV2Pipeline.from_pretrained(
"RenderFormer/renderformer-v2",
resolution=512,
device="cuda",
)
result = pipeline("scene.h5", resolution=512, precision="fp16")
```
Advanced callers may select a component explicitly with
`subfolder="transformer_512"`. The CLI and pipeline documentation in
RenderFormer Studio cover H5 preparation, precision, view batching, and tone
mapping.
## Load material components
```python
from renderformer.models.material import (
DiffuseSpecularToLatent,
MaterialAutoencoder,
PrincipledBRDFToLatent,
PrincipledBRDFToLatentWithTransmission,
)
repo = "RenderFormer/renderformer-v2"
material_autoencoder = MaterialAutoencoder.from_pretrained(
repo, subfolder="material_autoencoder", strict=True
).eval()
diffspec_mapper = DiffuseSpecularToLatent.from_pretrained(
repo, subfolder="diffspec_mapper", strict=True
).eval()
metallic_mapper = PrincipledBRDFToLatent.from_pretrained(
repo, subfolder="metallic_mapper", strict=True
).eval()
transmission_mapper = PrincipledBRDFToLatentWithTransmission.from_pretrained(
repo, subfolder="metallic_transmission_mapper", strict=True
).eval()
```
The autoencoder's `preprocessor_config.json` records the physical-input
contract. Prefer `encode_hdr` and `decode_hdr` for finite, nonnegative,
alpha-premultiplied linear RGB.
## Citation
Please cite both RenderFormer papers when using this bundle. Full BibTeX
entries are provided in the RenderFormer Studio README. |