> For the complete documentation index, see [llms.txt](https://yeasy.gitbook.io/llm_internals/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian.md).

# 第二部分：训练篇

- [第五章：预训练：为什么“预测下一个词”能学到知识](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/05_pretraining.md)
- [5.1 自回归语言模型：从左到右的世界观](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/05_pretraining/5.1_autoregressive.md)
- [5.2 掩码语言模型：完形填空的智慧](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/05_pretraining/5.2_masked_lm.md)
- [5.3 编码器-解码器预训练：两种范式的统一](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/05_pretraining/5.3_encoder_decoder.md)
- [5.4 预训练数据：规模定律与数据质量的博弈](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/05_pretraining/5.4_data_scaling.md)
- [本章小结](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/05_pretraining/summary.md)
- [第六章：训练技术的底层逻辑](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/06_training_techniques.md)
- [6.1 损失函数与优化器：为什么选择 Adam](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/06_training_techniques/6.1_loss_optimizer.md)
- [6.2 学习率调度：为什么需要先预热再衰减](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/06_training_techniques/6.2_lr_schedule.md)
- [6.3 正则化策略：防止过拟合的多重手段](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/06_training_techniques/6.3_regularization.md)
- [6.4 批次与序列长度：效率与质量的平衡](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/06_training_techniques/6.4_batch_sequence.md)
- [本章小结](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/06_training_techniques/summary.md)
- [第七章：大规模分布式训练](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/07_distributed_training.md)
- [7.1 数据并行：为什么简单复制就能加速](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/07_distributed_training/7.1_data_parallel.md)
- [7.2 ZeRO 优化：如何突破单卡显存限制](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/07_distributed_training/7.2_zero.md)
- [7.3 模型并行与张量并行：拆分权重的艺术](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/07_distributed_training/7.3_model_tensor_parallel.md)
- [7.4 流水线并行与混合并行策略](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/07_distributed_training/7.4_pipeline_hybrid.md)
- [7.5 激活重计算：用时间换空间的艺术](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/07_distributed_training/7.5_activation_checkpointing.md)
- [7.6 混合精度训练：精度与速度的权衡](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/07_distributed_training/7.6_mixed_precision.md)
- [7.7 检查点管理与容错](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/07_distributed_training/7.7_checkpoint.md)
- [本章小结](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/07_distributed_training/summary.md)
- [第八章：从预训练到对齐：让模型有用且安全](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/08_alignment.md)
- [8.1 监督微调：教模型“怎么回答”](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/08_alignment/8.1_sft.md)
- [8.2 RLHF：为什么需要人类反馈参与训练](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/08_alignment/8.2_rlhf.md)
- [8.3 DPO 与新型对齐：从复杂到简洁的演化](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/08_alignment/8.3_dpo.md)
- [8.4 参数高效微调：为什么不必更新所有参数](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/08_alignment/8.4_peft.md)
- [8.5 微调会怎么坏：四种失败模式与一条共同判据](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/08_alignment/8.5_practice.md)
- [本章小结](https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/08_alignment/summary.md)