LLM Knowledge Base
Large Language Model knowledge: papers, interview questions, and core topics.
Interview Questions
- 2026-09-02 Interview: Embedding模型和Reranker模型在RAG中各自的角色是什么？能不能只用一个？
- 2026-09-01 Interview: 向量检索用余弦相似度和内积有什么本质区别？什么时候用哪个？
- 2026-08-31 Interview: RAG系统中，chunk大小的选择背后有什么理论依据？不同场景最优chunk差异多大？
- 2026-08-30 Interview: 如何设计一个能处理异构请求（不同长度、不同模型）的LLM推理集群？
- 2026-08-29 Interview: Pipeline Parallelism的bubble比例如何计算？有什么方法降低bubble？
- 2026-08-28 Interview: Tensor Parallelism中AllReduce的通信发生在哪些位置？对延迟的影响有多大？
- 2026-08-27 Interview: 如何在不重新训练的情况下将一个4K上下文模型扩展到32K？YaRN和NTK-aware的区别？
- 2026-08-26 Interview: 长上下文模型(100K+)的KV Cache显存如何管理？有哪些压缩技术？
- 2026-08-25 Interview: 结构化输出(JSON Mode)的实现原理是什么？Constrained Decoding的计算开销多大？
- 2026-08-24 Interview: Top-k、Top-p和Temperature三个采样参数之间如何交互？设置不当会怎样？
- 2026-08-23 Interview: 模型服务的First Token Latency和Throughput之间有什么trade-off？如何平衡？
- 2026-08-22 Interview: 一个7B模型FP16推理的理论吞吐量瓶颈在哪？是计算还是内存带宽？
- 2026-08-21 Interview: Prefix Caching在多轮对话场景能节省多少计算？有什么前提条件？
- 2026-08-20 Interview: Continuous Batching相比Static Batching的优势在哪？实现时有什么挑战？
- 2026-08-19 Interview: 投机解码(Speculative Decoding)的正确性是如何数学保证的？什么时候效果最好？
- 2026-08-18 Interview: vLLM的PagedAttention解决了什么问题？与传统静态内存分配相比效率提升多少？
- 2026-08-17 Interview: GPTQ和AWQ的核心区别是什么？AWQ为什么号称"activation-aware"？
- 2026-08-16 Interview: 量化中INT4和FP8各适合什么场景？为什么有些层不能量化？
- 2026-08-15 Interview: MT-Bench与AlpacaEval的评测盲区深度分析
- 2026-08-14 Interview: 知识蒸馏中Teacher-Student能力差距过大的问题与解决方案
- 2026-08-13 Interview: 模型合并(Model Merging)的工作原理与TIES-Merging、DARE的核心思想
- 2026-08-12 Interview: 过度对齐(Over-alignment)的表现形式与检测方法
- 2026-08-11 Interview: 拒绝采样与Best-of-N在对齐中的优劣势对比
- 2026-08-10 Interview: 多轮对话训练中loss计算策略的深度分析
- 2026-08-09 Interview: Chat Template设计的重要性与模板不兼容导致的问题
- 2026-08-08 Interview: 对齐税(Alignment Tax)的本质、量化与缓解策略
- 2026-08-07 Interview: Constitutional AI的自我批评机制原理与局限性
- 2026-08-06 Interview: DPO与PPO的真正优劣势对比及DeepSeek-R1回归PPO的原因
- 2026-08-05 Interview: PPO在LLM对齐中的训练难度分析与KL散度惩罚的核心作用
- 2026-08-04 Interview: RLHF中Reward Model的训练数据构造与标注一致性处理
- 2026-08-03 Interview: LoRA作为正则化手段的理论分析及与Dropout的对比
- 2026-08-02 Interview: QLoRA的NF4量化原理及双重量化(Double Quantization)详解
- 2026-08-01 Interview: LoRA的rank选择理论指导与不同任务上rank敏感性分析
- 2026-07-31 Interview: 指令数据的多样性与质量权衡及数据质量量化方法
- 2026-07-30 Interview: LIMA论文"1000条数据足够SFT"的结论适用边界分析
- 2026-07-29 Interview: SFT学习率为何远低于预训练及SFT常见问题分析
- 2026-07-28 Interview: Tensor Parallelism与Pipeline Parallelism的适用场景深度对比
- 2026-07-27 Interview: Gradient Checkpointing的时空权衡比例与checkpoint层选择策略
- 2026-07-26 Interview: ZeRO Stage 1/2/3的分片内容与Stage 3通信量分析
- 2026-07-25 Interview: 分布式训练中AllReduce通信量计算与Ring vs Tree AllReduce对比
- 2026-07-24 Interview: 为特定领域扩充Tokenizer词表的完整流程与注意事项
- 2026-07-23 Interview: 灾难性遗忘的本质原因与经典方法在LLM场景的局限
- 2026-07-22 Interview: 持续预训练(CPT)时领域数据与通用数据的配比策略
- 2026-07-21 Interview: 数据去重为什么如此重要？完全不去重和过度去重分别会导致什么问题？
- 2026-07-20 Interview: 预训练数据中代码数据的占比对模型推理能力有什么影响？有什么实验证据？
- 2026-07-19 Interview: Chinchilla定律在工业界为什么经常被违反？over-training的合理性在哪？
- 2026-07-18 Interview: Scaling Laws说loss随计算量呈幂律下降，但这个规律有没有失效的时候？
- 2026-07-17 Interview: 预训练loss突然出现spike，你的排查思路和应对策略是什么？
- 2026-07-16 Interview: 混合精度训练中BF16比FP16更适合LLM训练的根本原因是什么？
- 2026-07-15 Interview: Adam和AdamW的区别不只是名字——解释weight decay在这两个优化器中的数学差异。
- 2026-07-14 Interview: 预训练时Batch Size和Learning Rate如何协同调整？Linear Scaling Rule的局限性是什么？
- 2026-07-13 Interview: 交叉熵损失函数的label smoothing在LLM中有什么作用？什么时候该用什么时候不该用？
- 2026-07-12 Interview: BPE分词算法的合并策略对模型性能有什么影响？中文场景的分词有什么特殊考虑？
- 2026-07-11 Interview: 如果让你从零设计一个7B参数的LLM架构，你会如何分配层数、隐藏维度和头数？
- 2026-07-10 Interview: Transformer的计算瓶颈在Attention还是FFN？训练和推理时有什么不同？
- 2026-07-09 Interview: RMSNorm相比LayerNorm去掉了什么？为什么去掉均值中心化反而更好？
- 2026-07-08 Interview: SwiGLU比ReLU/GELU好在哪里？为什么现代LLM几乎都切换到了SwiGLU？
- 2026-07-07 Interview: MoE架构的Router负载均衡为什么是一个难题？DeepSeek-V2是怎么解决的？
- 2026-07-06 Interview: KV Cache在推理时如何工作？它的显存占用公式是什么？什么因素影响最大？
- 2026-07-05 Interview: GQA和MQA相比标准MHA牺牲了什么换取了什么？为什么LLaMA-2 70B选择GQA？
- 2026-07-04 Interview: Flash Attention没有改变数学计算结果，为什么能加速2-4倍？瓶颈到底在哪？
- 2026-07-03 Interview: 为什么Decoder-only架构在大规模预训练中胜出？Encoder-Decoder架构真的不行吗？
- 2026-07-02 Interview: Transformer中FFN的作用到底是什么？有研究认为FFN是知识存储的主要载体，你怎么看？
- 2026-07-01 Interview: RoPE相比绝对位置编码和ALiBi各有什么优劣？RoPE为什么能支持长度外推？
- 2026-06-30 Interview: 为什么现代LLM都用Pre-Norm而不是Post-Norm？Post-Norm有没有优势？
- 2026-06-29 Interview: Multi-Head Attention的参数量和Single-Head完全相同，那多头的优势本质上来自哪里？
- 2026-06-28 Interview: Self-Attention为什么需要Q、K、V三个矩阵？用同一个矩阵行不行？
- 2026-06-27 Interview: 为什么Transformer使用缩放点积注意力而不是加性注意力？缩放因子1/√d_k的数学直觉是什么？
- 2026-06-26 Interview: LLM技术未来三年的突破方向与核心瓶颈
- 2026-06-25 Interview: 三个月内训练超越GPT-4模型的技术策略
- 2026-06-24 Interview: LLM代码生成的能力边界与程序员工作替代分析
- 2026-06-23 Interview: 支持流式输出+多轮对话+函数调用的LLM API Gateway设计
- 2026-06-22 Interview: 判断任务是否需要大模型及传统ML的适用场景
- 2026-06-21 Interview: 大模型时代推荐系统架构的演进方向
- 2026-06-20 Interview: 训练数据的版权与隐私合规处理方案
- 2026-06-19 Interview: 多租户LLM推理平台的资源隔离与调度设计
- 2026-06-18 Interview: LLM知识更新：RAG、CPT与知识编辑的适用场景
- 2026-06-17 Interview: Fine-tuned模型训练与生产效果差异的排查思路
- 2026-06-16 Interview: 多模态模型训练中数据配比与训练顺序的影响
- 2026-06-15 Interview: 防御Prompt Injection和Jailbreak的系统方案
- 2026-06-14 Interview: 构建可靠的LLM评估体系
- 2026-06-13 Interview: 3B模型端侧手机部署优化方案
- 2026-06-12 Interview: 低延迟高吞吐向量检索系统设计与索引选型
- 2026-06-11 Interview: 开源模型与闭源API的企业落地技术选型
- 2026-06-10 Interview: 从训练目标角度分析LLM幻觉的根本原因
- 2026-06-09 Interview: 评估LLM推理能力与模式匹配的实验设计
- 2026-06-08 Interview: 从零训练行业垂直大模型的技术路线图
- 2026-06-07 Interview: 设计日均百万请求的LLM推理服务架构
- 2026-06-06 Interview: Synthetic Data训练推理能力：STAR和ReST的核心思想
- 2026-06-05 Interview: 端到端训练Agent vs 基于Prompt的Agent：优劣与趋势
- 2026-06-04 Interview: Agent可解释性：用户如何理解和信任决策过程？
- 2026-06-03 Interview: AutoGen等Multi-Agent框架的设计模式与本质优势
- 2026-06-02 Interview: Agent在真实环境中的观察空间过大问题与信息提取策略
- 2026-06-01 Interview: Agent场景中Prompt Engineering vs Fine-tuning的边界
- 2026-05-31 Interview: LLM-as-Judge的可靠性与已知偏见
- 2026-05-30 Interview: Agent安全边界设计：防止执行危险操作
- 2026-05-29 Interview: 设计一个自主软件开发Agent的完整架构
- 2026-05-28 Interview: Reward Hacking在RLHF中的表现、检测与防范
- 2026-05-27 Interview: LLM的推理是真正的逻辑推理还是高级模式匹配？
- 2026-05-26 Interview: DeepSeek-R1的核心技术贡献与RL的可能性
- 2026-05-25 Interview: Process Reward Model vs Outcome Reward Model
- 2026-05-24 Interview: o1式推理模型的训练方法论：为什么RL优于SFT？
- 2026-05-23 Interview: Test-Time Compute Scaling的本质与参数Scaling的关系
- 2026-05-22 Interview: 代码生成Agent与通用Agent的架构本质区别
- 2026-05-21 Interview: LLM的Planning能力：涌现还是学到的？
- 2026-05-20 Interview: Agent多步任务的错误恢复与Reflexion机制
- 2026-05-19 Interview: Function Calling/Tool Use的实现原理
- 2026-05-18 Interview: Agent系统可靠性评估：成功率之外的关键指标
- 2026-05-17 Interview: Multi-Agent通信协议设计：避免信息冗余和死循环
- 2026-05-16 Interview: Agent长期记忆系统的设计：纯向量存储够不够？
- 2026-05-15 Interview: ReAct框架的失败模式与fallback设计
- 2026-05-14 Interview: CoT、Tree-of-Thoughts和Self-Consistency之间的关系与区别
- 2026-05-13 Interview: Chain-of-Thought推理在什么情况下反而会降低准确率？
- 2026-05-12 Interview: RAG vs Fine-tuning：给你一个具体场景（如企业知识库问答），你如何做技术选型？
- 2026-05-11 Interview: 多路召回+融合排序的架构中，不同召回源的分数如何归一化和融合？
- 2026-05-10 Interview: RAG系统的评估为什么比端到端模型评估更难？RAGAS框架的指标设计思路是什么？
- 2026-05-09 Interview: Self-RAG的自适应检索判断机制是怎么训练的？它解决了RAG的什么痛点？
- 2026-05-08 Interview: GraphRAG相比传统向量RAG的核心优势在哪？什么场景下差异最大？
- 2026-05-07 Interview: HyDE(假设文档嵌入)为什么有时候比直接查询更有效？它的失败场景是什么？
- 2026-05-06 Interview: 当检索到的内容与模型参数化知识矛盾时，LLM如何决策？你怎么解决这个问题？
- 2026-05-05 Interview: Embedding模型和Reranker模型在RAG中各自的角色是什么？能不能只用一个？
- 2026-05-04 Interview: 向量检索用余弦相似度和内积有什么本质区别？什么时候用哪个？
- 2026-04-25 Interview: RAG系统中，chunk大小的选择背后有什么理论依据？不同场景最优chunk差异多大？
- 2026-04-24 Interview: 如何设计一个能处理异构请求（不同长度、不同模型）的LLM推理集群？
- 2026-04-23 Interview: Pipeline Parallelism的bubble比例如何计算？有什么方法降低bubble？
- 2026-04-22 Interview: Tensor Parallelism中AllReduce的通信发生在哪些位置？对延迟的影响有多大？
- 2026-04-21 Interview: 如何在不重新训练的情况下将一个4K上下文模型扩展到32K？YaRN和NTK-aware的区别？
- 2026-04-20 Interview: 长上下文模型(100K+)的KV Cache显存如何管理？有哪些压缩技术？
- 2026-04-19 Interview: 结构化输出(JSON Mode)的实现原理是什么？Constrained Decoding的计算开销多大？
- 2026-04-18 Interview: Top-k、Top-p和Temperature三个采样参数之间如何交互？设置不当会怎样？
- 2026-04-17 Interview: 模型服务的First Token Latency和Throughput之间有什么trade-off？如何平衡？
- 2026-04-16 Interview: 一个7B模型FP16推理的理论吞吐量瓶颈在哪？是计算还是内存带宽？
- 2026-04-15 Interview: Prefix Caching在多轮对话场景能节省多少计算？有什么前提条件？
- 2026-04-14 Interview: Continuous Batching相比Static Batching的优势在哪？实现时有什么挑战？
- 2026-04-13 Interview: 投机解码(Speculative Decoding)的正确性是如何数学保证的？什么时候效果最好？
- 2026-04-12 Interview: vLLM的PagedAttention解决了什么问题？与传统静态内存分配相比效率提升多少？
- 2026-04-11 Interview: GPTQ和AWQ的核心区别是什么？AWQ为什么号称"activation-aware"？
- 2026-04-10 Interview: 量化中INT4和FP8各适合什么场景？为什么有些层不能量化？
- 2026-04-09 Interview: MT-Bench与AlpacaEval的评测盲区深度分析
- 2026-04-08 Interview: 知识蒸馏中Teacher-Student能力差距过大的问题与解决方案
- 2026-04-07 Interview: 模型合并(Model Merging)的工作原理与TIES-Merging、DARE的核心思想
- 2026-04-06 Interview: 过度对齐(Over-alignment)的表现形式与检测方法
- 2026-04-05 Interview: 拒绝采样与Best-of-N在对齐中的优劣势对比
- 2026-04-04 Interview: 多轮对话训练中loss计算策略的深度分析
- 2026-04-03 Interview: Chat Template设计的重要性与模板不兼容导致的问题
- 2026-04-02 Interview: 对齐税(Alignment Tax)的本质、量化与缓解策略
- 2026-04-01 Interview: Constitutional AI的自我批评机制原理与局限性
- 2026-03-31 Interview: DPO与PPO的真正优劣势对比及DeepSeek-R1回归PPO的原因
- 2026-03-31 Interview: PPO在LLM对齐中的训练难度分析与KL散度惩罚的核心作用
- 2026-03-28 Interview: LoRA的rank选择理论指导与不同任务上rank敏感性分析
- 2026-03-27 Interview: 指令数据的多样性与质量权衡及数据质量量化方法
- 2026-03-26 Interview: LIMA论文"1000条数据足够SFT"的结论适用边界分析
- 2026-03-25 Interview: SFT学习率为何远低于预训练及SFT常见问题分析
- 2026-03-24 Interview: Tensor Parallelism与Pipeline Parallelism的适用场景深度对比
- 2026-03-23 Interview: Gradient Checkpointing的时空权衡比例与checkpoint层选择策略
- 2026-03-22 Interview: ZeRO Stage 1/2/3的分片内容与Stage 3通信量分析
- 2026-03-21 Interview: 分布式训练中AllReduce通信量计算与Ring vs Tree AllReduce对比
- 2026-03-20 Interview: 为特定领域扩充Tokenizer词表的完整流程与注意事项
- 2026-03-19 Interview: 灾难性遗忘的本质原因与经典方法在LLM场景的局限
- 2026-03-18 Interview: 持续预训练(CPT)时领域数据与通用数据的配比策略
- 2026-03-17 Interview: 数据去重为什么如此重要？完全不去重和过度去重分别会导致什么问题？
- 2026-03-16 Interview: 预训练数据中代码数据的占比对模型推理能力有什么影响？有什么实验证据？
- 2026-03-15 Interview: Chinchilla定律在工业界为什么经常被违反？over-training的合理性在哪？
- 2026-03-14 Interview: Scaling Laws说loss随计算量呈幂律下降，但这个规律有没有失效的时候？
- 2026-03-13 Interview: 预训练loss突然出现spike，你的排查思路和应对策略是什么？
- 2026-03-12 Interview: 混合精度训练中BF16比FP16更适合LLM训练的根本原因是什么？
- 2026-03-11 Interview: Adam和AdamW的区别不只是名字——解释weight decay在这两个优化器中的数学差异。
- 2026-03-10 Interview: 预训练时Batch Size和Learning Rate如何协同调整？Linear Scaling Rule的局限性是什么？
- 2026-03-09 Interview: 交叉熵损失函数的label smoothing在LLM中有什么作用？什么时候该用什么时候不该用？
- 2026-03-08 Interview: BPE分词算法的合并策略对模型性能有什么影响？中文场景的分词有什么特殊考虑？
- 2026-03-07 Interview: 如果让你从零设计一个7B参数的LLM架构，你会如何分配层数、隐藏维度和头数？
- 2026-03-06 Interview: Transformer的计算瓶颈在Attention还是FFN？训练和推理时有什么不同？
- 2026-03-05 Interview: RMSNorm相比LayerNorm去掉了什么？为什么去掉均值中心化反而更好？
- 2026-03-04 Interview: SwiGLU比ReLU/GELU好在哪里？为什么现代LLM几乎都切换到了SwiGLU？
- 2026-03-03 Interview: MoE架构的Router负载均衡为什么是一个难题？DeepSeek-V2是怎么解决的？
- 2026-03-02 Interview: KV Cache在推理时如何工作？它的显存占用公式是什么？什么因素影响最大？
- 2026-03-01 Interview: GQA和MQA相比标准MHA牺牲了什么换取了什么？为什么LLaMA-2 70B选择GQA？
- 2026-02-28 Interview: Flash Attention没有改变数学计算结果，为什么能加速2-4倍？瓶颈到底在哪？
- 2026-02-27 Interview: 为什么Decoder-only架构在大规模预训练中胜出？Encoder-Decoder架构真的不行吗？
- 2026-02-26 Interview: Transformer中FFN的作用到底是什么？有研究认为FFN是知识存储的主要载体，你怎么看？
- 2026-02-25 Interview: RoPE相比绝对位置编码和ALiBi各有什么优劣？RoPE为什么能支持长度外推？
- 2026-02-24 Interview: 为什么现代LLM都用Pre-Norm而不是Post-Norm？Post-Norm有没有优势？
- 2026-02-23 Interview: Multi-Head Attention的参数量和Single-Head完全相同，那多头的优势本质上来自哪里？
- 2026-02-23 Interview: Self-Attention为什么需要Q、K、V三个矩阵？用同一个矩阵行不行？
- 2026-02-23 Interview: 为什么Transformer使用缩放点积注意力而不是加性注意力？缩放因子1/√d_k的数学直觉是什么？
Topics
- 2026-09-02 模型合并(Model Merging)技术
- 2026-09-01 过度对齐(Over-alignment)问题
- 2026-08-31 人类评估vs自动评估
- 2026-08-30 MT-Bench/AlpacaEval评估方法
- 2026-08-29 安全对齐与Red Teaming
- 2026-08-28 SPIN自我博弈微调
- 2026-08-27 Self-Play自博弈训练
- 2026-08-26 拒绝采样(Rejection Sampling)
- 2026-08-25 多轮对话训练技巧
- 2026-08-24 System Prompt工程与最佳实践
- 2026-08-23 Chat Template与对话格式
- 2026-08-22 对齐税(Alignment Tax)
- 2026-08-21 数据质量vs数据数量的权衡
- 2026-08-20 Constitutional AI：AI自我约束
- 2026-08-19 ORPO/SimPO/KTO新型对齐算法
- 2026-08-18 DPO直接偏好优化
- 2026-08-17 PPO算法在LLM中的应用
- 2026-08-16 Reward Model训练详解
- 2026-08-15 RLHF概述：从人类反馈中学习
- 2026-08-14 Full Fine-tuning vs LoRA对比
- 2026-08-13 QLoRA：4bit量化微调
- 2026-08-12 LoRA低秩适配原理
- 2026-08-11 指令数据构造方法论
- 2026-08-10 SFT监督微调详解
- 2026-08-09 预训练Loss曲线分析与诊断
- 2026-08-08 训练稳定性技巧(梯度裁剪/Warmup)
- 2026-08-07 MoE混合专家架构
- 2026-08-06 SwiGLU激活函数
- 2026-08-05 RMSNorm：更高效的归一化
- 2026-08-04 KV Cache缓存机制
- 2026-08-03 Flash Attention原理与实现
- 2026-08-02 GQA/MQA注意力优化
- 2026-08-01 RoPE旋转位置编码
- 2026-07-31 LLaMA架构深度解析
- 2026-07-30 GPT系列架构演进(GPT-1到GPT-4)
- 2026-07-29 Chinchilla定律：最优训练配置
- 2026-07-28 Scaling Laws：模型规模的科学
- 2026-07-27 Tokenizer训练：构建你的词表
- 2026-07-26 数据去重与质量过滤技术
- 2026-07-25 预训练数据清洗与质量控制
- 2026-07-24 持续预训练(Continual Pre-training)
- 2026-07-23 掩码语言模型(Masked LM)
- 2026-07-22 自回归语言模型(Causal LM)
- 2026-07-21 预训练(Pre-training)概述
- 2026-07-20 模型参数量与计算量(FLOPs)估算
- 2026-07-19 GPU显存占用分析与计算
- 2026-07-18 分布式训练基础(DP/DDP)
- 2026-07-17 混合精度训练(FP16/BF16)
- 2026-07-16 过拟合与正则化策略
- 2026-07-15 梯度下降与优化器(Adam/AdamW)
- 2026-07-14 Batch Size与Learning Rate的关系
- 2026-07-13 困惑度(Perplexity)：衡量语言模型好坏
- 2026-07-12 交叉熵损失函数在LLM中的应用
- 2026-07-11 BPE/WordPiece/SentencePiece分词算法
- 2026-07-10 Softmax函数与温度参数
- 2026-07-09 Layer Normalization与残差连接
- 2026-07-08 Encoder与Decoder的区别与联系
- 2026-07-07 Transformer整体架构解析
- 2026-07-06 位置编码(Positional Encoding)
- 2026-07-05 Multi-Head Attention多头注意力
- 2026-07-04 Self-Attention自注意力详解
- 2026-07-03 注意力机制(Attention)的本质
- 2026-07-02 Embedding向量表示：从离散到连续
- 2026-07-01 Token与分词：LLM如何理解文字
- 2026-06-30 LLM技术发展趋势展望
- 2026-06-29 开源vs闭源LLM生态
- 2026-06-28 LLM综合评估体系
- 2026-06-27 Hallucination幻觉问题与缓解
- 2026-06-26 安全与可控文本生成
- 2026-06-25 数学推理能力提升
- 2026-06-24 代码生成模型专题
- 2026-06-23 长思考链与推理扩展
- 2026-06-22 端侧部署与模型压缩
- 2026-06-21 小模型vs大模型：效率之争
- 2026-06-20 Scaling的未来与瓶颈
- 2026-06-19 Synthetic Data合成数据
- 2026-06-18 Reward Hacking奖励黑客问题
- 2026-06-17 World Model世界模型
- 2026-06-16 o1式推理模型：深度思考
- 2026-06-15 Test-Time Compute测试时计算
- 2026-06-14 视频理解与生成模型
- 2026-06-13 语音模型Whisper架构解析
- 2026-06-12 Vision Transformer(ViT)详解
- 2026-06-11 多模态模型架构设计
- 2026-06-10 Chain-of-Thought推理
- 2026-06-09 Prompt Engineering进阶技巧
- 2026-06-08 Agent评估与调试方法
- 2026-06-07 Memory机制：长短期记忆管理
- 2026-06-06 Multi-Agent多智能体系统
- 2026-06-05 Planning与任务分解
- 2026-06-04 ReAct框架详解
- 2026-06-03 Agent架构设计模式
- 2026-06-02 Tool Use/Function Calling
- 2026-06-01 RAG vs Fine-tuning选择指南
- 2026-05-31 Agentic RAG智能体检索
- 2026-05-30 GraphRAG图谱增强检索
- 2026-05-29 RAG评估框架（Ragas等）
- 2026-05-28 多路召回与融合策略
- 2026-05-27 HyDE假设文档嵌入
- 2026-05-26 Reranking重排序技术
- 2026-05-25 文档分块（Chunking）策略
- 2026-05-24 Embedding模型选型与评估
- 2026-05-23 向量数据库选型指南
- 2026-05-22 RAG检索增强生成概述
- 2026-05-21 模型服务化部署最佳实践
- 2026-05-20 Pipeline Parallelism流水线并行
- 2026-05-19 Tensor Parallelism张量并行
- 2026-05-18 模型并行vs数据并行
- 2026-05-17 Continuous Batching连续批处理
- 2026-05-16 Prefix Caching前缀缓存
- 2026-05-15 Context Window扩展技术
- 2026-05-14 长上下文处理挑战
- 2026-05-13 结构化输出（JSON Mode/Grammar）
- 2026-05-12 Beam Search vs Sampling对比
- 2026-05-11 采样策略：Top-k/Top-p/Temperature
- 2026-05-10 TensorRT-LLM推理加速
- 2026-05-09 vLLM与PagedAttention
- 2026-05-08 GGUF格式与llama.cpp生态
- 2026-05-07 AWQ自适应权重量化
- 2026-05-06 GPTQ量化算法
- 2026-05-05 量化基础：INT8/INT4/FP8
- 2026-05-04 投机解码(Speculative Decoding)
- 2026-04-25 KV Cache深入：显存与速度的平衡
- 2026-04-24 推理优化全景图
- 2026-04-23 知识蒸馏(Distillation)
- 2026-04-22 模型合并(Model Merging)技术
- 2026-04-21 过度对齐(Over-alignment)问题
- 2026-04-20 人类评估vs自动评估
- 2026-04-19 MT-Bench/AlpacaEval评估方法
- 2026-04-18 安全对齐与Red Teaming
- 2026-04-17 SPIN自我博弈微调
- 2026-04-16 Self-Play自博弈训练
- 2026-04-15 拒绝采样(Rejection Sampling)
- 2026-04-14 多轮对话训练技巧
- 2026-04-13 System Prompt工程与最佳实践
- 2026-04-12 Chat Template与对话格式
- 2026-04-11 对齐税(Alignment Tax)
- 2026-04-10 数据质量vs数据数量的权衡
- 2026-04-09 Constitutional AI：AI自我约束
- 2026-04-08 ORPO/SimPO/KTO新型对齐算法
- 2026-04-07 DPO直接偏好优化
- 2026-04-06 PPO算法在LLM中的应用
- 2026-04-05 Reward Model训练详解
- 2026-04-04 RLHF概述：从人类反馈中学习
- 2026-04-03 Full Fine-tuning vs LoRA对比
- 2026-04-02 QLoRA：4bit量化微调
- 2026-04-01 LoRA低秩适配原理
- 2026-03-31 指令数据构造方法论
- 2026-03-31 SFT监督微调详解
- 2026-03-28 SwiGLU激活函数
- 2026-03-28 RMSNorm：更高效的归一化
- 2026-03-28 KV Cache缓存机制
- 2026-03-27 Flash Attention原理与实现
- 2026-03-26 GQA/MQA注意力优化
- 2026-03-25 RoPE旋转位置编码
- 2026-03-24 LLaMA架构深度解析
- 2026-03-23 GPT系列架构演进(GPT-1到GPT-4)
- 2026-03-22 Chinchilla定律：最优训练配置
- 2026-03-21 Scaling Laws：模型规模的科学
- 2026-03-20 Tokenizer训练：构建你的词表
- 2026-03-19 数据去重与质量过滤技术
- 2026-03-18 预训练数据清洗与质量控制
- 2026-03-17 持续预训练(Continual Pre-training)
- 2026-03-16 掩码语言模型(Masked LM)
- 2026-03-15 自回归语言模型(Causal LM)
- 2026-03-14 预训练(Pre-training)概述
- 2026-03-13 模型参数量与计算量(FLOPs)估算
- 2026-03-12 GPU显存占用分析与计算
- 2026-03-11 分布式训练基础(DP/DDP)
- 2026-03-10 混合精度训练(FP16/BF16)
- 2026-03-09 过拟合与正则化策略
- 2026-03-08 梯度下降与优化器(Adam/AdamW)
- 2026-03-07 Batch Size与Learning Rate的关系
- 2026-03-06 困惑度(Perplexity)：衡量语言模型好坏
- 2026-03-05 交叉熵损失函数在LLM中的应用
- 2026-03-04 BPE/WordPiece/SentencePiece分词算法
- 2026-03-03 Softmax函数与温度参数
- 2026-03-02 Layer Normalization与残差连接
- 2026-03-01 Encoder与Decoder的区别与联系
- 2026-02-28 Transformer整体架构解析
- 2026-02-27 位置编码(Positional Encoding)
- 2026-02-26 Multi-Head Attention多头注意力
- 2026-02-25 Self-Attention自注意力详解
- 2026-02-24 注意力机制(Attention)的本质
- 2026-02-23 Embedding向量表示：从离散到连续
- 2026-02-23 Token与分词：LLM如何理解文字
Papers
- 2026-09-02 Paper: Lost in the Middle: How Language Models Use Long Contexts
- 2026-09-01 Paper: Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
- 2026-08-31 Paper: C-Pack: Packaged Resources To Advance General Chinese Embedding (BGE)
- 2026-08-30 Paper: Text Embeddings by Weakly-Supervised Contrastive Pre-training (E5)
- 2026-08-29 Paper: Gorilla: Large Language Model Connected with Massive APIs
- 2026-08-28 Paper: Toolformer: Language Models Can Teach Themselves to Use Tools
- 2026-08-27 Paper: Active Retrieval Augmented Generation (FLARE)
- 2026-08-26 Paper: Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity
- 2026-08-25 Paper: GraphRAG: Unlocking LLM Discovery on Narrative Private Data
- 2026-08-24 Paper: Precise Zero-Shot Dense Retrieval without Relevance Labels (HyDE)
- 2026-08-23 Paper: Corrective Retrieval Augmented Generation (CRAG)
- 2026-08-22 Paper: Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection
- 2026-08-21 Paper: REALM: Retrieval-Augmented Language Model Pre-Training
- 2026-08-20 Paper: ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction
- 2026-08-19 Paper: Dense Passage Retrieval for Open-Domain Question Answering
- 2026-08-18 Paper: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- 2026-08-17 Paper: RETRO: Improving Language Models by Retrieving from Trillions of Tokens
- 2026-08-16 Paper: Model Soups: Averaging Weights of Multiple Fine-tuned Models
- 2026-08-15 Paper: NEFTune: Noisy Embeddings Improve Instruction Finetuning
- 2026-08-14 Paper: DoRA: Weight-Decomposed Low-Rank Adaptation
- 2026-08-13 Paper: QLoRA: Efficient Finetuning of Quantized Language Models
- 2026-08-12 Paper: LoRA: Low-Rank Adaptation of Large Language Models
- 2026-08-11 Paper: Scaling Data-Constrained Language Models
- 2026-08-10 Paper: Curriculum Learning for LLMs
- 2026-08-09 Paper: Deduplication与数据质量
- 2026-08-08 Paper: Textbooks Are All You Need II: phi-1.5
- 2026-08-07 Paper: Code Llama: Open Foundation Models for Code
- 2026-08-06 Paper: DeepSeek-Coder: When the Large Language Model Meets Programming
- 2026-08-05 Paper: StarCoder: May the Source Be with You
- 2026-08-04 Paper: FineWeb: Decanting the Web for the Finest Text Data
- 2026-08-03 Paper: RedPajama: An Open Dataset for Training LLMs
- 2026-08-02 Paper: The Pile: An 800GB Dataset of Diverse Text
- 2026-08-01 Paper: Rejection Sampling与Best-of-N在对齐中的应用
- 2026-07-31 Paper: UltraFeedback: Boosting Language Models with High-quality Feedback
- 2026-07-30 Paper: Zephyr: Direct Distillation of LM Alignment
- 2026-07-29 Paper: Orca: Progressive Learning from Complex Explanation Traces
- 2026-07-28 Paper: WizardLM: Empowering LLMs to Follow Complex Instructions (Evol-Instruct)
- 2026-07-27 Paper: SPIN: Self-Play Fine-Tuning
- 2026-07-26 Paper: Proximal Policy Optimization Algorithms (PPO)
- 2026-07-25 Paper: KTO: Model Alignment as Prospect Theoretic Optimization
- 2026-07-24 Paper: ORPO: Monolithic Preference Optimization without Reference Model
- 2026-07-23 Paper: Direct Preference Optimization (DPO)
- 2026-07-22 Paper: Constitutional AI: Harmlessness from AI Feedback
- 2026-07-21 Paper: LIMA: Less Is More for Alignment
- 2026-07-20 Paper: Stanford Alpaca: An Instruction-following LLaMA Model
- 2026-07-19 Paper: Self-Instruct: Aligning Language Models with Self-Generated Instructions
- 2026-07-18 Paper: Training language models to follow instructions with human feedback
- 2026-07-17 Paper: Scaling Laws for Neural Language Models
- 2026-07-16 Paper: RoFormer: Enhanced Transformer with Rotary Position Embedding
- 2026-07-15 Paper: GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
- 2026-07-14 Paper: FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
- 2026-07-13 Paper: PaLM: Scaling Language Modeling with Pathways
- 2026-07-12 Paper: DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
- 2026-07-11 Paper: Textbooks Are All You Need
- 2026-07-10 Paper: RWKV: Reinventing RNNs for the Transformer Era
- 2026-07-09 Paper: Mamba: Linear-Time Sequence Modeling with Selective State Spaces
- 2026-07-08 Paper: Mistral 7B
- 2026-07-07 Paper: LLaMA: Open and Efficient Foundation Language Models
- 2026-07-06 Paper: Training Compute-Optimal Large Language Models
- 2026-07-05 Paper: Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
- 2026-07-04 Paper: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- 2026-07-03 Paper: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
- 2026-07-02 Paper: Language Models are Few-Shot Learners
- 2026-07-01 Paper: Language Models are Unsupervised Multitask Learners
- 2026-06-30 Paper: Improving Language Understanding by Generative Pre-Training
- 2026-06-29 Paper: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
- 2026-06-28 Paper: Attention Is All You Need
- 2026-06-27 Paper: LLM Powered Autonomous Agents
- 2026-06-26 Paper: The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
- 2026-06-25 Paper: Can Large Language Models Generate Novel Research Ideas?
- 2026-06-24 Paper: Chatbot Arena: An Open Platform for Evaluating LLMs through Human Preference
- 2026-06-23 Paper: A Survey on Evaluation of Large Language Models
- 2026-06-22 Paper: Red Teaming Language Models to Reduce Harms
- 2026-06-21 Paper: A Survey on Hallucination in Large Language Models
- 2026-06-20 Paper: World Models综述: 从Ha & Schmidhuber到Genie
- 2026-06-19 Paper: Video Generation Models as World Simulators (Sora技术报告)
- 2026-06-18 Paper: Robust Speech Recognition via Large-Scale Weak Supervision (Whisper)
- 2026-06-17 Paper: Visual Instruction Tuning (LLaVA)
- 2026-06-16 Paper: Model Merging综述: TIES-Merging, DARE, SLERP
- 2026-06-15 Paper: 从DistilBERT到DeepSeek-R1-Distill: 知识蒸馏的演进
- 2026-06-14 Paper: AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
- 2026-06-13 Paper: GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- 2026-06-12 Paper: A Survey of Quantization Methods for Efficient Neural Network Inference
- 2026-06-11 Paper: SpecInfer/Medusa: Speculative Decoding加速推理
- 2026-06-10 Paper: Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression
- 2026-06-09 Paper: Efficient Transformers: A Survey
- 2026-06-08 Paper: A Survey on Mixture of Experts
- 2026-06-07 Paper: DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- 2026-06-06 Paper: Learning to Reason with LLMs (o1 System Card Analysis)
- 2026-06-05 Paper: Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
- 2026-06-04 Paper: OpenHands/OpenDevin: An Open Platform for AI Software Developers
- 2026-06-03 Paper: Large Language Models as Tool Makers (LATM)
- 2026-06-02 Paper: Cognitive Architectures for Language Agents (CoALA)
- 2026-06-01 Paper: The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Use
- 2026-05-31 Paper: AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation
- 2026-05-30 Paper: WebArena: A Realistic Web Environment for Building Autonomous Agents
- 2026-05-29 Paper: AgentBench: Evaluating LLMs as Agents
- 2026-05-28 Paper: HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face
- 2026-05-27 Paper: Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models
- 2026-05-26 Paper: Automatic Chain of Thought Prompting in Large Language Models
- 2026-05-25 Paper: STaR: Self-Taught Reasoner - Bootstrapping Reasoning With Reasoning
- 2026-05-24 Paper: Let's Verify Step by Step
- 2026-05-23 Paper: SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering
- 2026-05-22 Paper: MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework
- 2026-05-21 Paper: CAMEL: Communicative Agents for Mind Exploration of Large Language Model Society
- 2026-05-20 Paper: Voyager: An Open-Ended Embodied Agent with Large Language Models
- 2026-05-19 Paper: Generative Agents: Interactive Simulacra of Human Behavior
- 2026-05-18 Paper: Reflexion: Language Agents with Verbal Reinforcement Learning
- 2026-05-17 Paper: Self-Consistency Improves Chain of Thought Reasoning in Language Models
- 2026-05-16 Paper: Tree of Thoughts: Deliberate Problem Solving with Large Language Models
- 2026-05-15 Paper: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- 2026-05-14 Paper: ReAct: Synergizing Reasoning and Acting in Language Models
- 2026-05-13 Paper: Multi-hop Question Answering via Reasoning Chains with Retrieval-Augmented Generation
- 2026-05-12 Paper: Benchmarking Large Language Models in Retrieval-Augmented Generation (RAGAS)
- 2026-05-11 Paper: Seven Failure Points When Engineering a Retrieval Augmented Generation System
- 2026-05-10 Paper: Modular RAG: Transforming RAG Systems into LEGO-like Reconfigurable Frameworks
- 2026-05-09 Paper: Retrieval-Augmented Generation for Large Language Models: A Survey
- 2026-05-08 Paper: LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models
- 2026-05-07 Paper: YaRN: Efficient Context Window Extension of Large Language Models
- 2026-05-06 Paper: Extending Context Window of Large Language Models via Positional Interpolation
- 2026-05-05 Paper: Lost in the Middle: How Language Models Use Long Contexts
- 2026-05-04 Paper: Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
- 2026-04-25 Paper: C-Pack: Packaged Resources To Advance General Chinese Embedding (BGE)
- 2026-04-24 Paper: Text Embeddings by Weakly-Supervised Contrastive Pre-training (E5)
- 2026-04-23 Paper: Gorilla: Large Language Model Connected with Massive APIs
- 2026-04-22 Paper: Toolformer: Language Models Can Teach Themselves to Use Tools
- 2026-04-21 Paper: Active Retrieval Augmented Generation (FLARE)
- 2026-04-20 Paper: Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity
- 2026-04-19 Paper: GraphRAG: Unlocking LLM Discovery on Narrative Private Data
- 2026-04-18 Paper: Precise Zero-Shot Dense Retrieval without Relevance Labels (HyDE)
- 2026-04-17 Paper: Corrective Retrieval Augmented Generation (CRAG)
- 2026-04-16 Paper: Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection
- 2026-04-15 Paper: REALM: Retrieval-Augmented Language Model Pre-Training
- 2026-04-14 Paper: ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction
- 2026-04-13 Paper: Dense Passage Retrieval for Open-Domain Question Answering
- 2026-04-12 Paper: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- 2026-04-11 Paper: RETRO: Improving Language Models by Retrieving from Trillions of Tokens
- 2026-04-10 Paper: Model Soups: Averaging Weights of Multiple Fine-tuned Models
- 2026-04-09 Paper: NEFTune: Noisy Embeddings Improve Instruction Finetuning
- 2026-04-08 Paper: DoRA: Weight-Decomposed Low-Rank Adaptation
- 2026-04-07 Paper: QLoRA: Efficient Finetuning of Quantized Language Models
- 2026-04-06 Paper: LoRA: Low-Rank Adaptation of Large Language Models
- 2026-04-05 Paper: Scaling Data-Constrained Language Models
- 2026-04-04 Paper: Curriculum Learning for LLMs
- 2026-04-03 Paper: Deduplication与数据质量
- 2026-04-02 Paper: Textbooks Are All You Need II: phi-1.5
- 2026-04-01 Paper: Code Llama: Open Foundation Models for Code
- 2026-03-31 Paper: DeepSeek-Coder: When the Large Language Model Meets Programming
- 2026-03-31 Paper: StarCoder: May the Source Be with You
- 2026-03-28 Paper: Rejection Sampling与Best-of-N在对齐中的应用
- 2026-03-27 Paper: UltraFeedback: Boosting Language Models with High-quality Feedback
- 2026-03-26 Paper: Zephyr: Direct Distillation of LM Alignment
- 2026-03-25 Paper: Orca: Progressive Learning from Complex Explanation Traces
- 2026-03-24 Paper: WizardLM: Empowering LLMs to Follow Complex Instructions (Evol-Instruct)
- 2026-03-23 Paper: SPIN: Self-Play Fine-Tuning
- 2026-03-22 Paper: Proximal Policy Optimization Algorithms (PPO)
- 2026-03-21 Paper: KTO: Model Alignment as Prospect Theoretic Optimization
- 2026-03-20 Paper: ORPO: Monolithic Preference Optimization without Reference Model
- 2026-03-19 Paper: Direct Preference Optimization (DPO)
- 2026-03-18 Paper: Constitutional AI: Harmlessness from AI Feedback
- 2026-03-17 Paper: LIMA: Less Is More for Alignment
- 2026-03-16 Paper: Stanford Alpaca: An Instruction-following LLaMA Model
- 2026-03-15 Paper: Self-Instruct: Aligning Language Models with Self-Generated Instructions
- 2026-03-14 Paper: Training language models to follow instructions with human feedback
- 2026-03-13 Paper: Scaling Laws for Neural Language Models
- 2026-03-12 Paper: RoFormer: Enhanced Transformer with Rotary Position Embedding
- 2026-03-11 Paper: GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
- 2026-03-10 Paper: FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
- 2026-03-09 Paper: PaLM: Scaling Language Modeling with Pathways
- 2026-03-08 Paper: DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
- 2026-03-07 Paper: Textbooks Are All You Need
- 2026-03-06 Paper: RWKV: Reinventing RNNs for the Transformer Era
- 2026-03-05 Paper: Mamba: Linear-Time Sequence Modeling with Selective State Spaces
- 2026-03-04 Paper: Mistral 7B
- 2026-03-03 Paper: LLaMA: Open and Efficient Foundation Language Models
- 2026-03-02 Paper: Training Compute-Optimal Large Language Models
- 2026-03-01 Paper: Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
- 2026-02-28 Paper: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- 2026-02-27 Paper: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
- 2026-02-26 Paper: Language Models are Few-Shot Learners
- 2026-02-25 Paper: Language Models are Unsupervised Multitask Learners
- 2026-02-24 Paper: Improving Language Understanding by Generative Pre-Training
- 2026-02-23 Paper: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
- 2026-02-23 Paper: Attention Is All You Need