README.md

---
library_name: transformers
license: cc-by-nc-sa-4.0
pipeline_tag: text-ranking
tags:
  - reranker
  - sequence-classification
  - qwen3
  - multilingual
  - bfloat16
  - 32k
base_model: ContextualAI/ctxl-rerank-v2-instruct-multilingual-1b
model_type: qwen3
---

# Contextual AI Reranker v2 1B — **SequenceClassification (single-logit) Converted Model**

This repository contains a **drop-in SequenceClassification** version of the original **ContextualAI/ctxl-rerank-v2-instruct-multilingual-1b**.
It exposes a **single logit per input** (one score) that is **numerically equivalent** to the original model’s last-token **`vocab_id=0`** logit (`next_logits[:, 0]`). That means you can use standard **text-classification/CrossEncoder** tooling for fast, simple reranking—without custom logits processors—while preserving the original scores and ranking order.

> **What changed?** We copy the LM head’s **row 0** vector into a 1-logit classification head (`score.weight ← lm_head.weight[0]`), set bias to 0 (or the matching bias row if present), and keep tokenizer/padding behavior aligned with the original. Result: `SequenceClassification` output ≡ original `next_logits[:, 0]`.

---

## Highlights

* **Parity with the original**: The score from this model equals the original **ID=0** logit at the very last token position (use the same prompt template and left-padding).
* **Frictionless integration**: Works out-of-the-box with **Sentence-Transformers CrossEncoder** and standard **Transformers** classification interfaces.
* **Fast & memory-light**: Computes a single logit (`hidden_size × 1`) instead of a full vocabulary projection.
* **Multilingual** and long-context (inherits capabilities from the base reranker).

---

## Model Overview

* **Type**: Text Reranking (single-logit SequenceClassification)
* **Base**: `ContextualAI/ctxl-rerank-v2-instruct-multilingual-1b` (Qwen3 CausalLM)
* **Languages**: 100+ (inherited)
* **Params**: \~1B (inherited)
* **Context Length**: up to 32K (inherited)
* **Scoring definition**: single logit ≡ original `next_logits[:, 0]`

---

## Input Formatting (keep this template)

```text
Check whether a given document contains information helpful to answer the query.
<Document> {document}
<Query> {query}{optional_instruction} ??
```

* Use **left padding** so the **last token** aligns across a batch.
* If the tokenizer has no `pad_token`, set `pad_token = eos_token`.

---

## Updated Usage

Below are **drop-in** examples for the converted model. These mirror the original card’s behavior but through **SequenceClassification**.

### Updated Sentence Transformers Usage (CrossEncoder)

```python
from sentence_transformers import CrossEncoder

MODEL_ID = "sigridjineth/ctxl-rerank-v2-1b-seq-cls"  # or local folder

def format_prompts(query: str, instruction: str, docs: list[str]) -> list[str]:
    inst = f" {instruction}" if instruction else ""
    return [
        "Check whether a given document contains information helpful to answer the query.\n"
        f"<Document> {d}\n"
        f"<Query> {query}{inst} ??"
        for d in docs
    ]

query = "Which is a domestic animal?"
docs = ["Cats are pets.", "The moon is made of cheese.", "Dogs are loyal companions."]

ce = CrossEncoder(MODEL_ID, max_length=8192)

# Ensure original padding behavior
if ce.tokenizer.pad_token is None:
    ce.tokenizer.pad_token = ce.tokenizer.eos_token
ce.tokenizer.padding_side = "left"

prompts = format_prompts(query, "", docs)
scores = ce.predict(prompts)  # one logit per doc (higher = more relevant)

ranked = sorted(zip(scores, docs), key=lambda x: x[0], reverse=True)
for s, d in ranked:
    print(f"{s:.4f} | {d}")
```

### Updated Transformers Usage (SequenceClassification)

```python
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

MODEL_ID = "sigridjineth/ctxl-rerank-v2-1b-seq-cls"  # or local folder
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if torch.cuda.is_available() else torch.float32

def format_prompts(query: str, instruction: str, docs: list[str]) -> list[str]:
    inst = f" {instruction}" if instruction else ""
    return [
        "Check whether a given document contains information helpful to answer the query.\n"
        f"<Document> {d}\n"
        f"<Query> {query}{inst} ??"
        for d in docs
    ]

tok = AutoTokenizer.from_pretrained(MODEL_ID, use_fast=True)
if tok.pad_token is None:
    tok.pad_token = tok.eos_token
tok.padding_side = "left"

model = AutoModelForSequenceClassification.from_pretrained(MODEL_ID, torch_dtype=dtype).to(device).eval()

query = "Which is a domestic animal?"
docs = ["Cats are pets.", "The moon is made of cheese."]
prompts = format_prompts(query, "", docs)

enc = tok(prompts, return_tensors="pt", padding=True, truncation=True).to(device)
with torch.no_grad():
    logits = model(**enc).logits.squeeze(-1)     # [batch]
    # Optional: exact parity rounding with original BF16 readout
    scores = logits.to(torch.bfloat16).float().cpu().tolist()

ranked = sorted(zip(scores, docs), key=lambda x: x[0], reverse=True)
for s, d in ranked:
    print(f"{s:.4f} | {d}")
```

> **Note on parity**: Casting the output logit to **bf16 then back to float** matches the original card’s BF16 rounding step.

---

## (Reference) Original Transformers Usage (CausalLM)

If you prefer to call the original model directly, compute `next_logits[:, -1, 0]` as specified in the base card.

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

BASE_ID = "ContextualAI/ctxl-rerank-v2-instruct-multilingual-1b"
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if torch.cuda.is_available() else torch.float32

def format_prompts(q: str, inst: str, docs: list[str]) -> list[str]:
    inst = f" {inst}" if inst else ""
    return [
        "Check whether a given document contains information helpful to answer the query.\n"
        f"<Document> {d}\n"
        f"<Query> {q}{inst} ??"
        for d in docs
    ]

tok = AutoTokenizer.from_pretrained(BASE_ID, use_fast=True)
if tok.pad_token is None:
    tok.pad_token = tok.eos_token
tok.padding_side = "left"

lm = AutoModelForCausalLM.from_pretrained(BASE_ID, torch_dtype=dtype).to(device).eval()

docs = ["Cats are pets.", "The moon is made of cheese."]
prompts = format_prompts("Which is a domestic animal?", "", docs)
enc = tok(prompts, return_tensors="pt", padding=True, truncation=True).to(device)

with torch.no_grad():
    out = lm(**enc).logits[:, -1, :]        # [batch, vocab]
    scores = out[:, 0].to(torch.bfloat16).float().cpu().tolist()

for s, d in sorted(zip(scores, docs), key=lambda x: x[0], reverse=True):
    print(f"{s:.4f} | {d}")
```

---

## Conversion Details

* **Architecture**: `Qwen3ForSequenceClassification(num_labels=1)`
* **Head initialization**:

  * `score.weight ← lm_head.weight[0]` (row for `vocab_id=0`)
  * `score.bias ← 0` (or the corresponding bias term if present in LM head)
* **Tokenizer/Config**:

  * Ensure `pad_token` exists (`pad_token = eos_token` if missing)
  * Set `padding_side="left"`
  * Propagate `pad/eos/bos` IDs into the model `config` for correct batching
* **Parity check**:

  * Verified that `SequenceClassification` logit ≡ original `next_logits[:, 0]`
  * Optional BF16 round-trip on the score for exact rounding parity

---

## Intended Use & Limitations

* **Use**: Document reranking for search/QA/multilingual scenarios; batch scoring of `(query, document)` prompts.
* **Not for**: Open-ended generation; the model emits a **single score** per input.
* **License constraints**: Non-commercial & Share-Alike. If you redistribute derivatives, include attribution and the same license.
* **Bias & safety**: Inherits all limitations and potential biases of the base model; evaluate before deployment.

---

## Requirements

* **Transformers** ≥ 4.51.0
* **PyTorch** with BF16 support recommended on GPU
* Long inputs: set `max_length` accordingly (up to the inherited context window)

---

## Citation

If you use this converted model, please cite the original work:

```bibtex
@misc{ctxl_rerank_v2_instruct_multilingual,
  title = {Contextual AI Reranker v2},
  author = {George Halal and Sheshansh Agrawal and Bo Han and Arnav Palkhiwala},
  year = {2025},
  url = {https://contextual.ai/blog/rerank-v2}
}
```

---

## License

This repository follows the original **Creative Commons Attribution Non Commercial Share Alike 4.0 (CC-BY-NC-SA-4.0)** license.
You **must** provide attribution, **may not** use it commercially, and **must** distribute derivatives under the same license.

---

## Acknowledgements

All modeling, training, and evaluation credit goes to **Contextual AI** for the original `ctxl-rerank-v2` family.
This repository provides a **compatibility conversion** to a single-logit `SequenceClassification` interface for easier integration and deployment.