---
library_name: diffusers
license: creativeml-openrail-m
base_model: stable-diffusion-v1-5/stable-diffusion-v1-5
pipeline_tag: text-to-image
tags:
- stable-diffusion
- text-to-image
- procreate
datasets:
- Jacklu0831/few-shot-creative-generation-8
---

# ProCreate checkpoint: Amedeo Modigliani

A Stable Diffusion 1.5 UNet fine-tuned for 2,000 steps on the **Amedeo Modigliani** category of [FSCG-8](https://huggingface.co/datasets/Jacklu0831/few-shot-creative-generation-8) (50 caption-image pairs of Amedeo Modigliani paintings). It is one of eight checkpoints released with **ProCreate, Don't Reproduce! Propulsive Energy Diffusion for Creative Generation** (ECCV 2024).

ProCreate is a sampling method that pushes generated images away from the reference images, so a model fine-tuned on a few examples produces novel samples in the same style instead of replicating its training data.

[Paper](https://arxiv.org/abs/2408.02226) · [Project page](https://agenticlearning.ai/procreate/) · [Code](https://github.com/agentic-learning-ai-lab/procreate-diffusion) · [Dataset](https://huggingface.co/datasets/Jacklu0831/few-shot-creative-generation-8)

## What this repository contains

Only the fine-tuned UNet (`UNet2DConditionModel`, diffusers format). Load it into a Stable Diffusion 1.5 pipeline:

```python
import torch
from diffusers import StableDiffusionPipeline, UNet2DConditionModel

pipe = StableDiffusionPipeline.from_pretrained(
    "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16
).to("cuda")
pipe.unet = UNet2DConditionModel.from_pretrained(
    "Jacklu0831/procreate-diffusion-amedeo-modigliani", torch_dtype=torch.float16
).to("cuda")
image = pipe("a Amedeo Modigliani painting of a boy in a suit and hat").images[0]
```

This samples from the fine-tuned model directly. To sample with ProCreate, use `src/inference.py` in the [code repository](https://github.com/agentic-learning-ai-lab/procreate-diffusion), which downloads this checkpoint automatically:

```bash
python src/inference.py \
    --dataset_dir few-shot-creative-generation-8/amedeo_modigliani \
    --unet_ckpt_dir Jacklu0831/procreate-diffusion-amedeo-modigliani \
    --prompt "a Amedeo Modigliani painting of a boy in a suit and hat"
```

## License

The weights are a derivative of Stable Diffusion 1.5 and are released under the [CreativeML Open RAIL-M license](https://huggingface.co/spaces/CompVis/stable-diffusion-license).

## Citation

```bibtex
@inproceedings{lu2024procreate,
  title     = {ProCreate, Don't Reproduce! Propulsive Energy Diffusion for Creative Generation},
  author    = {Lu, Jack and Teehan, Ryan and Ren, Mengye},
  booktitle = {European Conference on Computer Vision (ECCV)},
  year      = {2024}
}
```