Apex-1-DPO
Where our open lineage began: a 1.1B dense model, trained from scratch and aligned with DPO.
- Repository
- YOON1v/Apex-1-DPO
- Parameters
- 1.12B, dense
- Architecture
- 24 layers · GQA 16Q/4KV · QK-Norm
- Training
- ~20B pretraining tokens · SFT · DPO
- Context
- 4,096 tokens (trained at 2,048)
- License
- Apache 2.0
Apex-1-DPO is our first model. It is a 1.1B dense language model, pretrained from scratch on about 20B English tokens, then fine-tuned and aligned with direct preference optimization.
Every weight was learned here, with no external pretrained checkpoint. The architecture maps one-to-one onto Qwen3, so the model loads directly with transformers; the conversion was checked for logit parity.
Here is what Apex-1-DPO shows:
Token efficiency. On 20B tokens, Apex-1 matches Pythia-1.0B, trained on 15 times more data, on the commonsense average and beats it on ARC. Its HumanEval score of 8.5 is 4.7 times Pythia’s and close to TinyLlama-1.1B’s 9.2, which saw 3T tokens.
Reading. BoolQ 62.20 ranks first among TinyLlama-1.1B, Pythia-1.0B and OPT-1.3B.
Alignment. We tried reinforcement learning with verifiable rewards first, but with GSM8K accuracy near zero the reward groups had no variance and gave no learning signal. DPO on 60K preference pairs targeted what actually needed fixing, verbosity and non-compliance, and matched or beat SFT on every summary metric.
Honest limits. Against 2024–2025 small models trained on 450 to 900 times more data, Apex-1 trails. That gap is data scale, not architecture, and it is what Apex-2 set out to close.
Scores track training data closely. Read Apex-1-DPO against its token budget: 0.02T, against 0.18T to 18T for the models beside it.
| Apex-1-DPO1.1B | Pythia-1.0B1.0B | OPT-1.3B1.3B | TinyLlama-1.1B1.1B | Llama-3.2-1B1.2B | Qwen2.5-1.5B1.5B | |
|---|---|---|---|---|---|---|
| Pretraining dataTokens | 0.02T | 0.3T | 0.18T | 3T | 9T | 18T |
| CommonsenseHellaSwag | 46.9 | 47.2 | 53.7 | 59.2 | 61.2 | 66.4 |
| ScienceARC, average | 41.6 | 38.0 | 40.1 | 42.7 | 49.2 | 58.5 |
| Physical commonsensePIQA | 68.6 | 69.2 | 72.4 | 73.3 | 74.8 | 76.1 |
| MathGSM8K | 1.9 | — | — | — | 7.6 | 61.7 |
| CodeHumanEval | 8.5 | 1.8 | — | 9.2 | 18.9 | 37.2 |
Apex-1-DPO was measured by us with lm-evaluation-harness 0.4.12 in bf16, using the protocol of the TinyLlama paper (commonsense 0-shot, GSM8K 5-shot strict, HumanEval pass@1). Other numbers come from the models’ papers and reports; expect about ±0.5 points of noise from harness versions. GSM8K and MBPP train splits partly overlap our SFT data (we evaluated on the test splits); HumanEval is fully uncontaminated. Sources: TinyLlama, Pythia, OPT, Llama 3.2, Qwen2.5.
All results, with the SFT checkpoint
| Apex-1-DPOreleased | Apex-1 SFTbefore DPO | TinyLlama-1.1B3T tokens | Pythia-1.0B300B tokens | OPT-1.3B180B tokens | |
|---|---|---|---|---|---|
| CommonsenseHellaSwag | 46.91 | 46.69 | 59.20 | 47.16 | 53.65 |
| CommonsenseARC-Easy | 52.57 | 52.65 | 55.25 | 48.99 | 50.80 |
| CommonsenseARC-Challenge | 30.72 | 29.86 | 30.10 | 27.05 | 29.44 |
| CommonsensePIQA | 68.55 | 68.23 | 73.29 | 69.21 | 72.36 |
| CommonsenseWinoGrande | 52.80 | 53.59 | 59.12 | 53.43 | 59.59 |
| ReadingBoolQ | 62.20 | 61.99 | 57.83 | 57.83 | 60.83 |
| CommonsenseOpenBookQA | 33.80 | 33.80 | 36.00 | 31.40 | 33.40 |
| CommonsenseAverage of 7 | 49.65 | 49.54 | 52.99 | 48.30 | 51.44 |
| MathGSM8K strict (5-shot) | 1.90 | 1.44 | — | — | — |
| MathGSM8K flexible (5-shot) | 2.27 | 1.97 | — | — | — |
| CodeHumanEval pass@1 | 8.54 | 8.54 | 9.15 | 1.83 | — |
| CodeMBPP pass@1 | 5.20 | 4.80 | — | — | — |
Quick start
from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained("YOON1v/Apex-1-DPO", torch_dtype="bfloat16") tokenizer = AutoTokenizer.from_pretrained("YOON1v/Apex-1-DPO") inputs = tokenizer("Write a Python function that checks if a number is prime.", return_tensors="pt") out = model.generate(**inputs, max_new_tokens=200) print(tokenizer.decode(out[0], skip_special_tokens=True))
Before you download
Apex-1-DPO or Apex-2?
Apex-2 is stronger at code, math and instruction following. Apex-1-DPO is the smaller dense model: a compact starting point for experiments and a reference for how far 20B tokens can go.
What are its limits?
It is English-only. Multi-step arithmetic is very weak (GSM8K about 2%), knowledge is limited, and it is not tuned for long multi-turn conversations beyond what the SFT and DPO data cover.
Is it free to use?
Yes. Apex-1-DPO is released under Apache 2.0.