Configuration

Soup uses a single YAML config file for all settings. Run soup init to generate one.

Reproducibility and full fine-tuning (v0.73.0)

Two training keys landed in v0.73.0. Both are config keys, not CLI flags.

training:
  seed: 1234           # weight init of new params, data order, dropout
  data_seed: 7         # optional: vary data order while holding init fixed
  quantization: none   # required with r: 0, because the default (4bit) is refused
  lora:
    r: 0               # 0 means no adapter: full fine-tuning, dense checkpoint

seed and data_seed are the first seed knobs in the project's history: before this every run trained at seed 42 with no way to change it, so "run this twice with a different seed" was impossible. Both default to unset rather than to 42, deliberately, because an unset seed has to keep reproducing two different historical defaults (the trainer's 42 and the multipack sampler's 0) and a plain 42 would have silently re-ordered every existing multipack run. A boolean is refused by name, since seed: true would otherwise become seed 1.

The honest scope, and it widened in v0.73.1: the seed reaches the trainer arguments in every wrapper now, applied before the model loads, so setting it on a DPO run is real. It reached one wrapper of eighteen when it shipped in v0.73.0, which is what the original note here described. On a streamed run it additionally seeds the adapter initialisation for all five streaming tasks, which is why a streamed run is bit-reproducible from a seed while a resident 4-bit one still is not.

lora.r: 0 selects full fine-tuning on task: sft and, since v0.75.0, task: embedding: no adapter is built, and the run writes a dense checkpoint rather than a LoRA. On any other task the gate does not apply at all, and that is worth knowing before you rely on it: classifier, reranker, cross_encoder and asr gate LoRA behind their own flag and ignore the rank entirely, while the remaining tasks hand the 0 straight to PEFT, which rejects it with its own message about a positive integer rather than Soup's named one. Rank 0 was chosen because three parts of the codebase already read it as "no adapter" and because r: 0 used to crash, so no config that worked before changes meaning. It is refused with a named message alongside a non-transformers backend, a non-text modality, any quantisation (the default counts: quantization is 4bit unless you set it, which is why the block above sets none), Spectrum, LISA, layer streaming, or any LoRA-shaped option, and it refuses rather than running a no-op if your freeze settings leave nothing trainable.

Config Structure

base: meta-llama/Llama-3.1-8B-Instruct   # HuggingFace model ID (required)
task: sft                                   # Training task
# backend: unsloth                          # 2-5x faster (pip install "soup-cli[fast]")
# modality: text                            # text, vision, or audio

data:
  train: ./data/train.jsonl                 # Path to training data
  format: alpaca                            # Data format (auto-detected if omitted)
  val_split: 0.1                            # Validation split ratio
  max_length: 2048                          # Max sequence length (64-1048576)
  # image_dir: ./data/images               # For vision modality
  # audio_dir: ./data/audio                # For audio modality

training:
  epochs: 3
  lr: 2e-5
  batch_size: auto                          # auto or integer
  quantization: 4bit                        # the default; also none, 8bit, gptq, awq, hqq:Nbit, ...
  # weight_decay: 0.01                      # default 0.01
  # max_grad_norm: 1.0                      # gradient clipping, default 1.0
  # logging_steps: 10                       # log metrics every N steps
  # save_steps: 100                         # checkpoint every N steps
  # warmup_ratio: 0.03                      # 0.0-0.5, default 0.03
  # scheduler: cosine                       # LR scheduler
  # quantization_aware: false              # Enable QAT
  # optimizer: adamw_8bit
  # gradient_checkpointing: true
  # stream_layers: false                   # Train or align a model bigger than VRAM (v0.72, BETA)
  # stream_source: auto                    # auto | ram | disk
  # stream_buffers: 2                      # 2-8
  # stream_pin: false                      # v0.74.0: force the pageable store
  lora:
    r: 64
    alpha: 16
    dropout: 0.05
    # target_modules: auto                 # Auto-detected per model
    # use_dora: false                      # Weight-decomposed LoRA

output: ./output

Defaults worth knowing

These are the values a key takes when you leave it out. The quantization default is the one that surprises people: quantization is 4bit unless you set it, so a config that does not mention it trains a QLoRA.

What batch_size: auto does

auto starts from a static estimate built from the model size, sequence length, quantization and LoRA rank. On a CUDA card the SFT trainer alone then refines it with a real probe, one forward and backward at each candidate size, starting at the estimate, halving when it runs out of memory and doubling while it fits, never above four times the estimate (training.auto_batch_size_strategy chooses auto, the default, probe or static; see OOM-probe auto batch size). The answer is cached per base model, sequence length, quantization, LoRA rank and GPU. Every other task, every CPU run and static use the static estimate unchanged.

It errs high when the whole model trains. The estimate charges trainable-parameter memory as a small share of the model, at most 5% and derived from the LoRA rank, and charges none for lora.r: 0. The probe never takes an optimizer step, so it does not see the optimizer state either, which for AdamW is two extra values for every trainable parameter. With lora.r: 0 or with unfrozen_parameters (Spectrum), set batch_size explicitly, or start low and raise it. LISA (lisa_enabled) is a separate case: the probe also runs while every layer is still trainable, so its gradient figure is not LISA's either, and which way it errs was not measured. Set batch_size explicitly there as well.

Layer streaming keys

These turn on layer streaming, which trains a model larger than your VRAM by keeping the frozen base out of the card entirely. They are config keys, not CLI flags: there is deliberately no --stream-layers.

Setting any of these while stream_layers is false is refused, because the knobs would silently do nothing.

A key the schema does not declare is no longer silently dropped (v0.74.0). Until then none of the config models overrode the library default of ignoring undeclared keys, so a typo validated clean and was discarded: training.quantizaton: none trained 4-bit, training.gradient_checkpoint: true did no checkpointing, data.max_len: 512 truncated at 2048, and soup train --dry-run printed "Config valid. Ready to train!" Loading a config now walks the whole model tree and reports every key it cannot place in one report, naming the field you probably meant. Since v0.75.0 the load is refused. soup train exits 1 before the training stack is imported, and the API and Web UI loader raises ValueError with the same text. Nothing is defaulted and nothing is substituted; the message says Refused. rather than Not applied., which would have read as if the run went ahead without the key, and the scan stops at 100 findings and says so. Two things are worth knowing. A config that must stay loadable on v0.74 as well needs the key removed, not renamed, because v0.74 warns and ignores it while v0.75 refuses it, and neither applies it. And soup plan and soup apply read the YAML as a plain mapping, so they do not run this check at all. soup sweep is stricter, because a swept parameter that matches nothing produces a grid of identical arms and a meaningless winner: it exits 1 before the first arm.

That refusal covers keys the schema does not know. It does not cover a key the schema declares and no code reads. Such a key validates and does not change training (it still takes part in the config hash that soup ship --config compares, so editing one makes earlier --evidence read as stale), and soup doctor --config lists unread settings only for task: sft on backend: mlx (that table); for every other task and backend the table is empty, so it prints Every setting this config writes is read on this backend. whether or not that is true. Read that line as "not reviewed", never as an all-clear. The pages that document such a key say so, usually as "accepted by the schema, but nothing in v0.75.2 reads it", so check the key's page before assuming that setting it took effect.

stream_vram_probe and stream_vram_override are not interchangeable, and the difference is the point. stream_vram_probe is a measurement: it runs the step and reads the peak. stream_vram_override is an assertion you are making about how much VRAM is really available, which is what you need when the pre-flight cannot see the truth, as under a per-process memory cap in a hosted notebook.

Streaming also constrains keys you already set: quantization must be none or 4bit, batch_size must be a concrete number rather than auto, and task must be one of sft, dpo, orpo, simpo and kto (kto additionally needs batch_size of 2 or more). Scaling a streaming run covers sizing, the VRAM pre-flight and the disk tier; Preference losses over streaming covers the four alignment tasks.

Templates

Soup includes 21 built-in templates:

soup init --template chat          # Conversational fine-tune
soup init --template code          # Code generation
soup init --template medical       # Domain expert
soup init --template reasoning     # GRPO reasoning (DeepSeek-R1 style)
soup init --template vision        # Vision/multimodal fine-tune
soup init --template audio         # Audio/speech fine-tune
soup init --template kto           # KTO unpaired preference
soup init --template orpo          # ORPO (no reference model)
soup init --template simpo         # SimPO length-normalized preference
soup init --template ipo           # IPO regularized preference
soup init --template bco           # BCO binary classifier preference (v0.40)
soup init --template rlhf          # Full RLHF pipeline (SFT -> RM -> PPO)
soup init --template pretrain      # Continued pre-training on raw text
soup init --template moe           # MoE fine-tuning (ScatterMoE LoRA)
soup init --template longcontext   # 128k+ context fine-tuning
soup init --template embedding     # Sentence embedding fine-tuning
soup init --template tool-calling  # Function / tool-calling fine-tune (v0.25)

Four more are regulation-shaped (compliance pack, v0.71.35). Each is a valid training config on a license-clean Apache-2.0 base plus header comments naming that regime's exact commands, because Soup's compliance controls are CLI flags and commands rather than config keys:

soup init --template hipaa         # Protected Health Information
soup init --template soc2          # SOC 2 Trust Services Criteria
soup init --template eu-ai-act     # EU AI Act Annex XI/XII
soup init --template sr-11-7       # SR 11-7 Model Risk Management

Task-Specific Config Keys

A key listed for a task it does not apply to still loads: the config check does not refuse it, and the trainer for that task never reads it, so it does not change the run. A few of them (use_liger, use_flash_attn, use_ring_attention, use_galore) still go through their pre-flight check first, on any task. That check can refuse a machine that lacks the package or, for use_galore, a config with quantization: 4bit or 8bit (the default is 4bit) or backend: unsloth, so use_galore: true on a DPO run with the default quantization stops the run before it starts. rope_scaling_type can likewise print long-context warnings on any task. The curriculum_metric and curriculum_buckets rows are the other exception: with curriculum_dynamic: true (which requires curriculum: true) the dynamic-curriculum callback reads them on the 13 tasks that allow it (sft, pretrain, dpo, grpo, kto, orpo, simpo, ipo, bco, reward_model, embedding, ppo, preference). Check this column before assuming a key took effect on a DPO or GRPO run.

More keys, and where each is documented

These keys are all read by the code, and the defaults and refusals in this table are what the config loader does, checked against the schema at v0.75.2.

Two declared settings are the opposite case: accepted by the schema, but nothing in v0.75.2 reads them. They are data.remove_unused_columns (default true; the trainers listed on Data pipeline pro pass remove_unused_columns=False as a constant and the others never pass it) and the advise: block, whose three keys goal, probe and record are used by nothing (soup advise takes them as flags only, see soup advise).