lambdahouse/palimpsest-lora-adapters

Palimpsest — Literary Style LoRA Adapters

Mix multiple authors' writing styles in real-time using lightweight LoRA adapters on a single base model.

Each adapter captures one author's distinctive voice (~435MB). At inference time, adapters are weighted and blended per request — 70% Bulgakov + 30% Poe produces lyrical Gothic satire. No model reload, no merge, instant style change.

Available Adapters (Public Domain Authors)

Adapter	Author	Language	Corpus	Training pairs
bulgakov.lora.gguf	Mikhail Bulgakov	Russian	635K words, 20 works	1,748
lovecraft_en.lora.gguf	H.P. Lovecraft	English	262K words, 21 stories	524
doyle_en.lora.gguf	Arthur Conan Doyle	English	733K words, 10 works	1,796
poe_en.lora.gguf	Edgar Allan Poe	English	515K words, 6 volumes	756
wilde_en.lora.gguf	Oscar Wilde	English	308K words, 11 works	730
london_en.lora.gguf	Jack London	English	747K words, 10 novels	1,620

Base Models

• Russian adapters: IlyaGusev/saiga_nemo_12b — Mistral Nemo 12B, Russian-tuned, abliterated
• English adapters: natong19/Mistral-Nemo-Instruct-2407-abliterated

Both base models must be converted to GGUF Q6_K format for use with llama.cpp.

How to Use

With llama.cpp (recommended)

Russian and English adapters use different base models and must run on separate servers:

# Russian server (Saiga Nemo 12B base)
llama-server \
  -m saiga-nemo12b-base-q6k.gguf \
  --lora bulgakov.lora.gguf \
  --lora-init-without-apply --no-jinja \
  --port 8080 --ctx-size 4096

# English server (Mistral Nemo abliterated base)
llama-server \
  -m nemo-abliterated-base-q6k.gguf \
  --lora lovecraft_en.lora.gguf \
  --lora doyle_en.lora.gguf \
  --lora poe_en.lora.gguf \
  --lora wilde_en.lora.gguf \
  --lora london_en.lora.gguf \
  --lora-init-without-apply --no-jinja \
  --port 8081 --ctx-size 4096

# Per-request mixing (within same language)
curl http://localhost:8081/v1/chat/completions \
  -d '{"messages": [{"role": "user", "content": "Write a scene..."}],
       "lora": [{"id": 0, "scale": 0.7}, {"id": 2, "scale": 0.3}]}'

Note: Mixing is only supported within the same language/base model. You can mix Lovecraft + Poe (both English) or Bulgakov alone (Russian), but not Bulgakov + Poe (different base models).

With the Palimpsest web app

See github.com/lambda-house/palimpsest for a full chat interface with mixer sliders.

Training Details

• Method: QLoRA (4-bit quantized base + LoRA adapters)
• Framework: Unsloth + TRL (SFTTrainer)
• LoRA config: r=32, alpha=16, dropout=0.05
• Training: 3 epochs, lr=1e-4, cosine schedule, adamw_8bit, effective batch=8
• GPU: NVIDIA L40S 48GB
• Instructions: Generated via Gemini 2.0 Flash (unique per passage, >99% diversity)

How LoRA Mixing Works

Each adapter is a low-rank correction (~0.4GB) to the base model weights. At inference:

output = input × W_base + scale₁ × (A₁ × B₁) + scale₂ × (A₂ × B₂) + ...

Adapters are additive and commutative — order doesn't matter, mixing is instant, base model stays unchanged.

Limitations

• Style transfer works best with distinctive authors. Generic prose produces weak adapters.
• 12B model has limited capacity for sustained plot coherence over very long texts.
• Russian adapters should be used with the Saiga base; English with the Nemo abliterated base.
• Cross-language mixing (Russian adapter + English adapter) is not supported (different base models).

Citation

@misc{palimpsest2026,
  title={Palimpsest: Literary Style Mixing with LoRA Adapters},
  author={Lambda House},
  year={2026},
  url={https://github.com/lambda-house/palimpsest}
}

License

Adapters: Apache 2.0 (all trained on public domain texts) Base models: See respective model cards Training code: github.com/lambda-house/palimpsest