References

Askell, Amanda, Yuntao Bai, Anna Chen, et al. 2021. “A General Language Assistant as a Laboratory for Alignment.” arXiv Preprint arXiv:2112.00861. https://arxiv.org/abs/2112.00861.

Bach, Stephen H., Victor Sanh, Zheng-Xin Yong, et al. 2022. “PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts.” Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: System Demonstrations. https://arxiv.org/abs/2202.01279.

Bahdanau, Dzmitry, Kyunghyun Cho, and Yoshua Bengio. 2015. “Neural Machine Translation by Jointly Learning to Align and Translate.” International Conference on Learning Representations. https://arxiv.org/abs/1409.0473.

Bai, Yuntao, Andy Jones, Kamal Ndousse, et al. 2022. “Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback.” arXiv Preprint arXiv:2204.05862. https://arxiv.org/abs/2204.05862.

Bai, Yuntao, Saurav Kadavath, Sandipan Kundu, et al. 2022. “Constitutional AI: Harmlessness from AI Feedback.” arXiv Preprint arXiv:2212.08073. https://arxiv.org/abs/2212.08073.

Ben Zaken, Elad, Shauli Ravfogel, and Yoav Goldberg. 2022. “BitFit: Simple Parameter-Efficient Fine-Tuning for Transformer-Based Masked Language-Models.” Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Short Papers). https://arxiv.org/abs/2106.10199.

Bengio, Yoshua, Patrice Simard, and Paolo Frasconi. 1994. “Learning Long-Term Dependencies with Gradient Descent Is Difficult.” IEEE Transactions on Neural Networks 5 (2): 157–66.

BigScience Workshop. 2022. “BLOOM: A 176B-Parameter Open-Access Multilingual Language Model.” arXiv Preprint arXiv:2211.05100. https://arxiv.org/abs/2211.05100.

Bradley, Ralph Allan, and Milton E. Terry. 1952. “Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons.” Biometrika 39 (3/4): 324–45.

Brown, Tom B., Benjamin Mann, Nick Ryder, et al. 2020. “Language Models Are Few-Shot Learners.” Advances in Neural Information Processing Systems. https://arxiv.org/abs/2005.14165.

Chen, Tianqi, Bing Xu, Chiyuan Zhang, and Carlos Guestrin. 2016. “Training Deep Nets with Sublinear Memory Cost.” arXiv Preprint arXiv:1604.06174. https://arxiv.org/abs/1604.06174.

Chen, Yulong, Yang Liu, Liang Chen, and Yue Zhang. 2021. “DialogSum: A Real-Life Scenario Dialogue Summarization Dataset.” Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. https://arxiv.org/abs/2105.06762.

Christiano, Paul F., Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. “Deep Reinforcement Learning from Human Preferences.” Advances in Neural Information Processing Systems. https://arxiv.org/abs/1706.03741.

Chung, Hyung Won, Le Hou, Shayne Longpre, et al. 2022. “Scaling Instruction-Finetuned Language Models.” arXiv Preprint arXiv:2210.11416. https://arxiv.org/abs/2210.11416.

DeepSeek-AI. 2024. “DeepSeek-V3 Technical Report.” arXiv Preprint arXiv:2412.19437. https://arxiv.org/abs/2412.19437.

DeepSeek-AI. 2025. “DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning.” arXiv Preprint arXiv:2501.12948. https://arxiv.org/abs/2501.12948.

Dettmers, Tim, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022. “LLM.int8(): 8-Bit Matrix Multiplication for Transformers at Scale.” Advances in Neural Information Processing Systems. https://arxiv.org/abs/2208.07339.

Dettmers, Tim, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. “QLoRA: Efficient Finetuning of Quantized LLMs.” Advances in Neural Information Processing Systems. https://arxiv.org/abs/2305.14314.

Devlin, Jacob, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. “BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding.” Proceedings of NAACL-HLT 2019. https://arxiv.org/abs/1810.04805.

Frantar, Elias, and Dan Alistarh. 2023. “SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot.” Proceedings of the 40th International Conference on Machine Learning. https://arxiv.org/abs/2301.00774.

Frantar, Elias, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2023. “GPTQ: Accurate Post-Training Quantization for Generative Pre-Trained Transformers.” International Conference on Learning Representations. https://arxiv.org/abs/2210.17323.

Gao, Leo, John Schulman, and Jacob Hilton. 2023. “Scaling Laws for Reward Model Overoptimization.” Proceedings of the 40th International Conference on Machine Learning. https://arxiv.org/abs/2210.10760.

Gao, Luyu, Aman Madaan, Shuyan Zhou, et al. 2023. “PAL: Program-Aided Language Models.” Proceedings of the 40th International Conference on Machine Learning. https://arxiv.org/abs/2211.10435.

Gemma Team. 2024. “Gemma 2: Improving Open Language Models at a Practical Size.” arXiv Preprint arXiv:2408.00118. https://arxiv.org/abs/2408.00118.

Gliwa, Bogdan, Iwona Mochol, Maciej Biesek, and Aleksander Wawer. 2019. “SAMSum Corpus: A Human-Annotated Dialogue Dataset for Abstractive Summarization.” Proceedings of the 2nd Workshop on New Frontiers in Summarization. https://arxiv.org/abs/1911.12237.

Greshake, Kai, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. 2023. “Not What You’ve Signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection.” Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security. https://arxiv.org/abs/2302.12173.

Gururangan, Suchin, Ana Marasović, Swabha Swayamdipta, et al. 2020. “Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks.” Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. https://arxiv.org/abs/2004.10964.

Hendrycks, Dan, Collin Burns, Steven Basart, et al. 2021. “Measuring Massive Multitask Language Understanding.” International Conference on Learning Representations. https://arxiv.org/abs/2009.03300.

Hinton, Geoffrey, Oriol Vinyals, and Jeff Dean. 2015. “Distilling the Knowledge in a Neural Network.” arXiv Preprint arXiv:1503.02531. https://arxiv.org/abs/1503.02531.

Hochreiter, Sepp, and Jürgen Schmidhuber. 1997. “Long Short-Term Memory.” Neural Computation 9 (8): 1735–80.

Hoffmann, Jordan, Sebastian Borgeaud, Arthur Mensch, et al. 2022. “Training Compute-Optimal Large Language Models.” Advances in Neural Information Processing Systems. https://arxiv.org/abs/2203.15556.

Holtzman, Ari, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. “The Curious Case of Neural Text Degeneration.” International Conference on Learning Representations. https://arxiv.org/abs/1904.09751.

Houlsby, Neil, Andrei Giurgiu, Stanislaw Jastrzebski, et al. 2019. “Parameter-Efficient Transfer Learning for NLP.” Proceedings of the 36th International Conference on Machine Learning. https://arxiv.org/abs/1902.00751.

Hu, Edward J., Yelong Shen, Phillip Wallis, et al. 2022. “LoRA: Low-Rank Adaptation of Large Language Models.” International Conference on Learning Representations. https://arxiv.org/abs/2106.09685.

Huang, Yanping, Youlong Cheng, Ankur Bapna, et al. 2019. “GPipe: Efficient Training of Giant Neural Networks Using Pipeline Parallelism.” Advances in Neural Information Processing Systems. https://arxiv.org/abs/1811.06965.

Ji, Ziwei, Nayeon Lee, Rita Frieske, et al. 2023. “Survey of Hallucination in Natural Language Generation.” ACM Computing Surveys 55 (12). https://arxiv.org/abs/2202.03629.

Kaplan, Jared, Sam McCandlish, Tom Henighan, et al. 2020. “Scaling Laws for Neural Language Models.” arXiv Preprint arXiv:2001.08361. https://arxiv.org/abs/2001.08361.

Karpukhin, Vladimir, Barlas Oğuz, Sewon Min, et al. 2020. “Dense Passage Retrieval for Open-Domain Question Answering.” Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. https://arxiv.org/abs/2004.04906.

Kim, Yoon, and Alexander M. Rush. 2016. “Sequence-Level Knowledge Distillation.” Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. https://arxiv.org/abs/1606.07947.

Kirkpatrick, James, Razvan Pascanu, Neil Rabinowitz, et al. 2017. “Overcoming Catastrophic Forgetting in Neural Networks.” Proceedings of the National Academy of Sciences 114 (13): 3521–26. https://arxiv.org/abs/1612.00796.

Kojima, Takeshi, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. “Large Language Models Are Zero-Shot Reasoners.” Advances in Neural Information Processing Systems. https://arxiv.org/abs/2205.11916.

Kwon, Woosuk, Zhuohan Li, Siyuan Zhuang, et al. 2023. “Efficient Memory Management for Large Language Model Serving with PagedAttention.” Proceedings of the 29th Symposium on Operating Systems Principles. https://arxiv.org/abs/2309.06180.

Lambert, Nathan, Jacob Morrison, Valentina Pyatkin, et al. 2024. “Tülu 3: Pushing Frontiers in Open Language Model Post-Training.” arXiv Preprint arXiv:2411.15124. https://arxiv.org/abs/2411.15124.

Lester, Brian, Rami Al-Rfou, and Noah Constant. 2021. “The Power of Scale for Parameter-Efficient Prompt Tuning.” Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. https://arxiv.org/abs/2104.08691.

Leviathan, Yaniv, Matan Kalman, and Yossi Matias. 2023. “Fast Inference from Transformers via Speculative Decoding.” Proceedings of the 40th International Conference on Machine Learning. https://arxiv.org/abs/2211.17192.

Lewis, Patrick, Ethan Perez, Aleksandra Piktus, et al. 2020. “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.” Advances in Neural Information Processing Systems. https://arxiv.org/abs/2005.11401.

Li, Xiang Lisa, and Percy Liang. 2021. “Prefix-Tuning: Optimizing Continuous Prompts for Generation.” Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics. https://arxiv.org/abs/2101.00190.

Lialin, Vladislav, Vijeta Deshpande, and Anna Rumshisky. 2023. “Scaling down to Scale up: A Guide to Parameter-Efficient Fine-Tuning.” arXiv Preprint arXiv:2303.15647. https://arxiv.org/abs/2303.15647.

Liang, Percy, Rishi Bommasani, Tony Lee, et al. 2023. “Holistic Evaluation of Language Models.” Transactions on Machine Learning Research. https://arxiv.org/abs/2211.09110.

Lin, Chin-Yew. 2004. “ROUGE: A Package for Automatic Evaluation of Summaries.” Text Summarization Branches Out, 74–81. https://aclanthology.org/W04-1013/.

Liu, Nelson F., Kevin Lin, John Hewitt, et al. 2024. “Lost in the Middle: How Language Models Use Long Contexts.” Transactions of the Association for Computational Linguistics 12. https://arxiv.org/abs/2307.03172.

Liu, Yinhan, Myle Ott, Naman Goyal, et al. 2019. “RoBERTa: A Robustly Optimized BERT Pretraining Approach.” arXiv Preprint arXiv:1907.11692. https://arxiv.org/abs/1907.11692.

Llama Team, AI @ Meta. 2024. “The Llama 3 Herd of Models.” arXiv Preprint arXiv:2407.21783. https://arxiv.org/abs/2407.21783.

McCloskey, Michael, and Neal J. Cohen. 1989. “Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem.” In Psychology of Learning and Motivation, vol. 24. Academic Press.

Micikevicius, Paulius, Sharan Narang, Jonah Alben, et al. 2018. “Mixed Precision Training.” International Conference on Learning Representations. https://arxiv.org/abs/1710.03740.

Mitchell, Margaret, Simone Wu, Andrew Zaldivar, et al. 2019. “Model Cards for Model Reporting.” Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT*). https://arxiv.org/abs/1810.03993.

National Institute of Standards and Technology. 2023. Artificial Intelligence Risk Management Framework (AI RMF 1.0). NIST AI 100-1. NIST. https://doi.org/10.6028/NIST.AI.100-1.

National Institute of Standards and Technology. 2024. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile. NIST AI 600-1. NIST. https://doi.org/10.6028/NIST.AI.600-1.

Ouyang, Long, Jeff Wu, Xu Jiang, et al. 2022. “Training Language Models to Follow Instructions with Human Feedback.” Advances in Neural Information Processing Systems. https://arxiv.org/abs/2203.02155.

Papineni, Kishore, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. “BLEU: A Method for Automatic Evaluation of Machine Translation.” Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, 311–18. https://aclanthology.org/P02-1040/.

Rafailov, Rafael, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. 2023. “Direct Preference Optimization: Your Language Model Is Secretly a Reward Model.” Advances in Neural Information Processing Systems. https://arxiv.org/abs/2305.18290.

Raffel, Colin, Noam Shazeer, Adam Roberts, et al. 2020. “Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer.” Journal of Machine Learning Research 21 (140): 1–67. https://arxiv.org/abs/1910.10683.

Rajbhandari, Samyam, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. 2020. “ZeRO: Memory Optimizations Toward Training Trillion Parameter Models.” SC20: International Conference for High Performance Computing, Networking, Storage and Analysis. https://arxiv.org/abs/1910.02054.

Rein, David, Betty Li Hou, Asa Cooper Stickland, et al. 2023. “GPQA: A Graduate-Level Google-Proof Q&A Benchmark.” arXiv Preprint arXiv:2311.12022. https://arxiv.org/abs/2311.12022.

Sanh, Victor, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. “DistilBERT, a Distilled Version of BERT: Smaller, Faster, Cheaper and Lighter.” arXiv Preprint arXiv:1910.01108. https://arxiv.org/abs/1910.01108.

Sardana, Nikhil, Jacob Portes, Sasha Doubov, and Jonathan Frankle. 2024. “Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws.” Proceedings of the 41st International Conference on Machine Learning. https://arxiv.org/abs/2401.00448.

Schulman, John, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel. 2016. “High-Dimensional Continuous Control Using Generalized Advantage Estimation.” International Conference on Learning Representations. https://arxiv.org/abs/1506.02438.

Schulman, John, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. “Proximal Policy Optimization Algorithms.” arXiv Preprint arXiv:1707.06347. https://arxiv.org/abs/1707.06347.

Sennrich, Rico, Barry Haddow, and Alexandra Birch. 2016. “Neural Machine Translation of Rare Words with Subword Units.” Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. https://arxiv.org/abs/1508.07909.

Shao, Zhihong, Peiyi Wang, Qihao Zhu, et al. 2024. “DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models.” arXiv Preprint arXiv:2402.03300. https://arxiv.org/abs/2402.03300.

Sharma, Mrinank, Meg Tong, Tomasz Korbak, et al. 2024. “Towards Understanding Sycophancy in Language Models.” International Conference on Learning Representations. https://arxiv.org/abs/2310.13548.

Shoeybi, Mohammad, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. 2019. “Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism.” arXiv Preprint arXiv:1909.08053. https://arxiv.org/abs/1909.08053.

Srivastava, Aarohi, Abhinav Rastogi, Abhishek Rao, et al. 2023. “Beyond the Imitation Game: Quantifying and Extrapolating the Capabilities of Language Models.” Transactions on Machine Learning Research. https://arxiv.org/abs/2206.04615.

Stiennon, Nisan, Long Ouyang, Jeff Wu, et al. 2020. “Learning to Summarize from Human Feedback.” Advances in Neural Information Processing Systems. https://arxiv.org/abs/2009.01325.

Su, Jianlin, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. 2024. “RoFormer: Enhanced Transformer with Rotary Position Embedding.” Neurocomputing 568. https://arxiv.org/abs/2104.09864.

Sun, Mingjie, Zhuang Liu, Anna Bair, and J. Zico Kolter. 2024. “A Simple and Effective Pruning Approach for Large Language Models.” International Conference on Learning Representations. https://arxiv.org/abs/2306.11695.

Touvron, Hugo, Thibaut Lavril, Gautier Izacard, et al. 2023. “LLaMA: Open and Efficient Foundation Language Models.” arXiv Preprint arXiv:2302.13971. https://arxiv.org/abs/2302.13971.

Turpin, Miles, Julian Michael, Ethan Perez, and Samuel R. Bowman. 2023. “Language Models Don’t Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting.” Advances in Neural Information Processing Systems. https://arxiv.org/abs/2305.04388.

Vaswani, Ashish, Noam Shazeer, Niki Parmar, et al. 2017. “Attention Is All You Need.” Advances in Neural Information Processing Systems. https://arxiv.org/abs/1706.03762.

Wang, Alex, Yada Pruksachatkun, Nikita Nangia, et al. 2019. “SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems.” Advances in Neural Information Processing Systems. https://arxiv.org/abs/1905.00537.

Wang, Alex, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019. “GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding.” International Conference on Learning Representations. https://arxiv.org/abs/1804.07461.

Wang, Xuezhi, Jason Wei, Dale Schuurmans, et al. 2023. “Self-Consistency Improves Chain of Thought Reasoning in Language Models.” International Conference on Learning Representations. https://arxiv.org/abs/2203.11171.

Wang, Yubo, Xueguang Ma, Ge Zhang, et al. 2024. “MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark.” Advances in Neural Information Processing Systems (Datasets and Benchmarks Track). https://arxiv.org/abs/2406.01574.

Wei, Jason, Maarten Bosma, Vincent Y. Zhao, et al. 2022. “Finetuned Language Models Are Zero-Shot Learners.” International Conference on Learning Representations. https://arxiv.org/abs/2109.01652.

Wei, Jason, Xuezhi Wang, Dale Schuurmans, et al. 2022. “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models.” Advances in Neural Information Processing Systems. https://arxiv.org/abs/2201.11903.

Wu, Shijie, Ozan Irsoy, Steven Lu, et al. 2023. “BloombergGPT: A Large Language Model for Finance.” arXiv Preprint arXiv:2303.17564. https://arxiv.org/abs/2303.17564.

Yao, Shunyu, Jeffrey Zhao, Dian Yu, et al. 2023. “ReAct: Synergizing Reasoning and Acting in Language Models.” International Conference on Learning Representations. https://arxiv.org/abs/2210.03629.

Zhao, Yanli, Andrew Gu, Rohan Varma, et al. 2023. “PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel.” Proceedings of the VLDB Endowment 16 (12). https://arxiv.org/abs/2304.11277.

Zheng, Lianmin, Wei-Lin Chiang, Ying Sheng, et al. 2023. “Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena.” Advances in Neural Information Processing Systems (Datasets and Benchmarks Track). https://arxiv.org/abs/2306.05685.

Zhou, Chunting, Pengfei Liu, Puxin Xu, et al. 2023. “LIMA: Less Is More for Alignment.” Advances in Neural Information Processing Systems. https://arxiv.org/abs/2305.11206.