Approach
1. Base model: compare two Apache-2.0 candidates, then pick by measurement
| Model | Licence | Why |
|---|---|---|
| Qwen3-8B | Apache-2.0 | Native tool-calling chat template and strong multilingual reasoning. Norwegian quality is unknown and has to be measured, especially Nynorsk. |
| NorMistral-7B-warm (UiO LTG) | Apache-2.0 | Continued pre-training on Norwegian, so Norwegian quality should be strong. No tool-calling training, so that has to be added. |
Llama 3.1 8B was ruled out as the main base because of its custom licence and gated access. Apache-2.0 lets us release the adapters and merged weights with no restrictions. In the pilot, both candidates are trained the same way. The one with the best combined score on NorEval and our Norwegian tool-calling test goes to the scaled run.
2. Training
- Norwegian adaptation (only if needed): a light continued pre-training with LoRA on a filtered, permissively licensed subset of the Norwegian Colossal Corpus. Mainly for Qwen3, if the pilot shows weak Nynorsk.
- Supervised fine-tuning: Norwegian instruction data plus function-calling data. The English function-calling sets are translated into Bokmål and Nynorsk, then validated: JSON schema checks, argument round-trip checks, and a human spot-check of a sample.
- Robustness: examples where the right answer is not to call a tool, or to ask a clarifying question. This targets made-up calls and made-up arguments.
3. Why 16 GB
A 7–8B model in 4-bit (NF4/GGUF Q4–Q5) needs roughly 5–6 GB for the weights. That leaves room for the KV cache and context on a 16 GB card. QLoRA training of 7–8B models on 16 GB is an established method (Dettmers et al., 2023). The pilot measures what is actually possible, including sequence length and throughput, on our card.
Tooling
Hugging Face transformers, PEFT, TRL, bitsandbytes; vLLM/llama.cpp for inference; lm-evaluation-harness (used by NorEval) and EuroEval for evaluation.