Nadav-marketaem commited on
Commit
aca8e34
Β·
verified Β·
1 Parent(s): b3a38d0

Address README feedback: drop unverified Qwen-Marketing claim, drop 'real/not illustrative' meta-commentary, clarify fixed eval split, cleaner pipeline example (named document var, int output)

Browse files
Files changed (1) hide show
  1. README.md +11 -8
README.md CHANGED
@@ -42,7 +42,7 @@ A high-performance regression model for assessing marketing content quality on a
42
  This model is built for **large-scale content quality filtering**, not single-document review. Typical applications:
43
 
44
  - **Pretraining corpus curation** β€” score every document in a FineWeb-scale crawl and keep only the top percentile of marketing content, the same way FineWeb-Edu uses an educational-quality classifier to filter Common Crawl.
45
- - **Dataset construction for domain-specific LLMs** β€” this classifier is what produces the filtered corpus behind [marketeam/Qwen-Marketing](https://huggingface.co/marketeam/Qwen-Marketing)'s marketing-domain pretraining/fine-tuning data.
46
  - **Marketing content quality scoring** β€” rank or threshold marketing web pages, blog posts, or landing pages by practitioner-quality writing, as opposed to spam/SEO filler.
47
 
48
  It is **not** intended as a fine-grained per-document editorial tool β€” see [Intended Use](#intended-use) for scope and [Safety & Bias](#safety--bias) for known limitations.
@@ -67,7 +67,7 @@ This model predicts marketing content quality by scoring web pages against a fiv
67
  | **Output** | Scalar regression score, nominally 0–5 (not clipped by the model itself β€” see note below) |
68
  | **Max token length** | 2048 |
69
 
70
- The head is not bounded to [0, 5]; scores for exceptionally strong documents can land slightly above 5 (see the real example in [How to Use](#how-to-use)). Downstream consumers that need a bounded integer score should clip and round, as `infer.py` does (`int_score = round(clip(score, 0, 5))`).
71
 
72
  ### Model Variants
73
 
@@ -104,7 +104,7 @@ Only BMse is recommended for production. Others are retained for reproducibility
104
  | **Optimizer** | Adam, lr=3e-4 (no weight decay) |
105
  | **Batch size** | 32 |
106
  | **Train set** | 445,116 docs (natural distribution) |
107
- | **Eval set** | 50,000 docs (held out, fixed split) |
108
  | **Epochs** | 5 (early stopping patience=3 on Spearman) |
109
  | **Best checkpoint** | Epoch 2 |
110
  | **Seed** | 42 |
@@ -138,7 +138,7 @@ Only BMse is recommended for production. Others are retained for reproducibility
138
 
139
  **Split:**
140
  - Training: 445,116 docs
141
- - Evaluation: 50,000 docs (held out, fixed split)
142
 
143
  **Distribution:** Heavily left-skewed (mean 1.52/5), typical of web crawl data. Top ~20% score β‰₯3; top ~6% score β‰₯4.
144
 
@@ -154,17 +154,20 @@ pipe = pipeline(
154
  model="marketeam/Fineweb-Classifier-Marketing",
155
  trust_remote_code=True,
156
  )
157
- pipe(
 
158
  "Best practices for B2B SaaS pricing: a tactical breakdown of tiered vs. "
159
  "usage-based models, with real conversion benchmarks from 40+ companies "
160
  "we've worked with this year."
161
  )
 
 
162
  ```
163
 
164
- Output (real, captured from this checkpoint β€” not illustrative):
165
 
166
- ```python
167
- {'label': 'quality_score', 'score': 5.024752616882324}
168
  ```
169
 
170
  `trust_remote_code=True` is required because this repo ships a custom `PreTrainedModel`/pipeline (frozen encoder + regression head is not a stock `transformers` architecture). Review `modeling_marketing_classifier.py`, `configuration_marketing_classifier.py`, and `pipeline_marketing_classifier.py` in this repo before enabling it, per standard `trust_remote_code` practice.
 
42
  This model is built for **large-scale content quality filtering**, not single-document review. Typical applications:
43
 
44
  - **Pretraining corpus curation** β€” score every document in a FineWeb-scale crawl and keep only the top percentile of marketing content, the same way FineWeb-Edu uses an educational-quality classifier to filter Common Crawl.
45
+ - **Dataset construction for domain-specific LLMs** β€” build filtered marketing-domain pretraining or fine-tuning corpora.
46
  - **Marketing content quality scoring** β€” rank or threshold marketing web pages, blog posts, or landing pages by practitioner-quality writing, as opposed to spam/SEO filler.
47
 
48
  It is **not** intended as a fine-grained per-document editorial tool β€” see [Intended Use](#intended-use) for scope and [Safety & Bias](#safety--bias) for known limitations.
 
67
  | **Output** | Scalar regression score, nominally 0–5 (not clipped by the model itself β€” see note below) |
68
  | **Max token length** | 2048 |
69
 
70
+ The head is not bounded to [0, 5]; scores for exceptionally strong documents can land slightly above 5 (see [How to Use](#how-to-use)). Downstream consumers that need a bounded integer score should clip and round, as `infer.py` does (`int_score = round(clip(score, 0, 5))`).
71
 
72
  ### Model Variants
73
 
 
104
  | **Optimizer** | Adam, lr=3e-4 (no weight decay) |
105
  | **Batch size** | 32 |
106
  | **Train set** | 445,116 docs (natural distribution) |
107
+ | **Eval set** | 50,000 docs, held out from training and reused as the same fixed split across all four head variants (BMse, Head A, Head B, LDS) for direct comparison |
108
  | **Epochs** | 5 (early stopping patience=3 on Spearman) |
109
  | **Best checkpoint** | Epoch 2 |
110
  | **Seed** | 42 |
 
138
 
139
  **Split:**
140
  - Training: 445,116 docs
141
+ - Evaluation: 50,000 docs β€” held out from training and reused as the same fixed split across BMse, Head A, Head B, and LDS, so their metrics are directly comparable
142
 
143
  **Distribution:** Heavily left-skewed (mean 1.52/5), typical of web crawl data. Top ~20% score β‰₯3; top ~6% score β‰₯4.
144
 
 
154
  model="marketeam/Fineweb-Classifier-Marketing",
155
  trust_remote_code=True,
156
  )
157
+
158
+ document = (
159
  "Best practices for B2B SaaS pricing: a tactical breakdown of tiered vs. "
160
  "usage-based models, with real conversion benchmarks from 40+ companies "
161
  "we've worked with this year."
162
  )
163
+ result = pipe(document)
164
+ print(round(result["score"]))
165
  ```
166
 
167
+ Output:
168
 
169
+ ```
170
+ 5
171
  ```
172
 
173
  `trust_remote_code=True` is required because this repo ships a custom `PreTrainedModel`/pipeline (frozen encoder + regression head is not a stock `transformers` architecture). Review `modeling_marketing_classifier.py`, `configuration_marketing_classifier.py`, and `pipeline_marketing_classifier.py` in this repo before enabling it, per standard `trust_remote_code` practice.