Skip to main content
ZICQ

Wiki Concepts

Fine-tuning

Concepts
Aliases: fine-tuning fine-tune FT PEFT ·2026-10-10

Fine-tuning

Fine-tuning is a transfer learning process that continues updating the neural network weights of a pre-trained foundation model (Base Model) using curated datasets targeted at specific downstream tasks, proprietary domain expertise, or strict behavioral schemas.

It serves as the essential bridge translating raw world knowledge encoded during pre-training into an aligned Large Language Model (LLM) capable of rigorous instruction following, specialized industry vernacular, consistent structured output formats, and deterministic enterprise personas.


Technical Taxonomy and GPU Memory Calculations

Based on the proportion of parameters updated during training, modern fine-tuning splits into two primary regimes:

                            Fine-Tuning Spectrum for LLMs
                                         │
            ┌────────────────────────────┴────────────────────────────┐
   Full Fine-Tuning (FFT)                                    Parameter-Efficient (PEFT)
            │                                                         │
   Updates 100% of parameters                                 Updates <1% of parameters
   Huge VRAM overhead (16~20x model size)                     Minimal VRAM overhead (Consumer GPUs)
   High ceiling, prone to catastrophic forgetting             Preserves generalization, zero-latency merge
            │                                                         │
   Multi-node A100/H100 clusters                              ┌───────┴───────┐
                                                             LoRA           QLoRA
                                                          (Low-Rank)     (4-bit Base)

1. Full Fine-Tuning (FFT)

Full Fine-Tuning updates every weight tensor in the network. While it offers the highest expressive ceiling, its GPU memory footprint is massive.

When training in standard 16-bit precision (FP16/BF16) with the AdamW optimizer, the required resident VRAM per parameter is modeled by:

$$\text{Memory}{\text{FFT}} \approx \underbrace{2\Phi}{\text{Model Weights}} + \underbrace{2\Phi}{\text{Gradients}} + \underbrace{12\Phi}{\text{AdamW Momentum & Variance}} + \underbrace{\text{Activation}}_{\text{Forward Activations}} \approx 16 \sim 20 \times \Phi\text{ bytes}$$

Where $\Phi$ represents parameter count. For a standard 70B parameter model, storing weights, gradients, and optimizer states alone demands at least $1.12\text{ TB}$ of VRAM, requiring multiple 8-way H100 servers orchestrated via Tensor Parallelism and ZeRO-3 sharding.

2. Parameter-Efficient Fine-Tuning (PEFT)

To overcome compute barriers, PEFT freezes the foundation weights and introduces compact trainable modules:

Methodology Mechanism Trainable Parameters VRAM Budget (70B Example) Core Advantage
Full Fine-Tuning (FFT) Full backpropagation across all layers $100%$ $> 1,200\text{ GB}$ (Multi-node clusters) Highest task ceiling; risk of catastrophic forgetting
LoRA Freezes base weights; injects rank decomposition matrices $BA$ $0.05% \sim 0.5%$ $\sim 180\text{ GB}$ (4-GPU node) Zero inference latency overhead via offline merging
QLoRA Freezes base in 4-bit NormalFloat (NF4) + Double Quant + Paged Optimizers $0.05% \sim 0.5%$ $\sim 48\text{ GB}$ (Dual 24GB consumer GPUs) Democratized training; retains 99%+ of 16-bit LoRA accuracy
DoRA Decomposes weights into magnitude and direction components $0.1% \sim 0.8%$ $\sim 190\text{ GB}$ Bridges the accuracy gap between LoRA and Full FT

Supervised Loss Formulation and Prompt Masking

In Supervised Fine-Tuning (SFT), input sequences concatenate user prompts $x$ with target assistant responses $y$.

The standard cross-entropy loss function is formalized as:

$$\mathcal{L}{\text{SFT}}(\theta) = -\sum{t=1}^{T} m_t \log P_\theta(x_t \mid x_{<t})$$

The Prompt Loss Masking Golden Rule ($m_t$)

The training loop must enforce a binary loss mask $m_t \in {0, 1}$:

$$m_t = \begin{cases} 0, & \text{if } x_t \in \text{User Prompt or System Instruction} \ 1, & \text{if } x_t \in \text{Assistant Target Response} \end{cases}$$

Sequence:  [User: Write a quicksort implementation.] [Assistant: Here is the Python code...]
Mask m_t:   0   0   0   0   0   0   0   0   0             1    1    1    1   1   1
                ▲                                         ▲
                │                                         └─► Computes Cross-Entropy loss & gradients
                └────────────────────────────────────────────► Masked out (Loss = 0, zero gradient)

Engineering Significance: If prompt tokens are not masked out, the model expends vast capacity learning how humans phrase questions. This degrades output precision and frequently causes the model to mimic user phrasing or repeat queries during production inference.


Production Best Practices and Traps

  1. The LIMA Hypothesis (Quality Trumps Quantity): Meta's LIMA study demonstrated that 1,000 expertly curated, noise-free instruction pairs outperform 100,000 noisy scraped samples. SFT primarily teaches models interaction protocols and response formatting, rather than injecting basic factual understanding.
  2. Mitigating Catastrophic Forgetting via General Data Replay: Training exclusively on specialized domain text (such as clinical medical notes) rapidly degrades baseline coding and reasoning abilities. Production pipelines mitigate this by blending $5% \sim 15%$ general instruction data (e.g., ShareGPT, Alpaca) into domain training sets as a regularization anchor.
  3. Learning Rate Dynamics: Fine-tuning learning rates must be far lower than pre-training rates (typically $1\times 10^{-5} \sim 2\times 10^{-4}$), using cosine decay with warmup. Overly aggressive learning rates destabilize pre-trained representations in the very first epoch.

Strategic Decision Matrix: Fine-Tuning vs. RAG vs. Prompting

Dimension Prompt Engineering Retrieval-Augmented Generation (RAG) Fine-Tuning (SFT / PEFT)
Knowledge Recency Real-time (within single context window) Dynamic (near-instant index synchronization) Static (mandates training runs and deployment)
Factuality & Attribution Constrained by base model knowledge Highest (grounded by verifiable citations) Weak; cannot eliminate probabilistic hallucinations
Style & Formatting Adherence Dependent on few-shot examples Weak (retrieval fragments only) Strongest (hardens syntax and tone into weights)
Inference Token Cost High (incurs token overhead per call) Moderate to High (context passage injection) Lowest (short prompts save recurring token costs)

Sources and Key Literature