Awesome Learning to Reason with LLMs: From Prompt Engineering to OpenAI o1 A Survey

This repository explores recent advancements in reasoning with LLMs, starting from conventional methods like Chain-of-Thought (CoT) prompting to the latest innovations, such as OpenAI's learning-based CoT reasoning in the o1 model series. It provides a comprehensive overview of how reasoning with LLMs has evolved, inlcuding recent advancements, key research papers, and discussions from blogs, social media, and expert talks.

---

Table of Contents

• Official OpenAI Documents
• News about LLM Reasoning OpenAI o1
  • Blogs
  • Talks
  • Social Media Posts
• Awesome Papers
  • How to Contribute
  • Learning-based Reasoning
  • Prompt Engineering-based Reasoning

---

Official OpenAI Documents

• OpenAI o1 Hub
  • Main page introducing the o1 series, including details about its capabilities, reasoning tasks, and usage limits.
• OpenAI o1 System Card
  • A document outlining the safety evaluations, potential risks, and mitigations applied to the o1 model​.
• OpenAI Reasoning Guide
  • A guide focusing on how OpenAI models can enhance reasoning tasks, explaining methods like chain-of-thought reasoning and how models handle complex problem-solving.

News about LLM Reasoning OpenAI o1

Blogs

• [OpenAI] Learning to Reason with LLMs
  • Discuss advancements in improving reasoning with LLMs, focusing on how structured thinking processes, like chain-of-thought reasoning, can significantly enhance complex problem-solving.
• [OpenAI] OpenAI o1 Mini: Advancing Cost-Efficient Reasoning
  • Introduce the OpenAI o1-mini model, a cost-effective solution optimized for coding and reasoning, offering a balance between performance and affordability.
• [Nathan Lambert] OpenAI’s Strawberry, LM self-talk, inference scaling laws, and spending more on inference
• [Nathan Lambert] Reverse engineering OpenAI’s o1
• [Nouha Dziri] Have o1 Models Cracked Human Reasoning?

Talks

• [Noam Brown (OpenAI)] Parables on the Power of Planning in AI: From Poker to Diplomacy
• [Noam Brown, Ilge Akkaya and Hunter Lightman (OpenAI)] Teaching LLMs to Reason Better

Social Media Posts

Twitter

• [OpenAI Developers] Ask Me Anything - [Summary]
• [Jason Wei] Super excited to finally share what I have been working on at OpenAI!

Reddit

• OpenAI's new model o1 really can reason
• OpenAI releases o1, its first model with ‘reasoning’ abilities
• OpenAI announces o1

Awesome Papers

How to Contribute

• We welcome every researcher who contributes to this repository. Please feel free to pull requests or contact us if you find any related papers that are not included here.

The process to submit a pull request is as follows:

• a. Fork the project into your own repository.
• b. Add the Title, Paper link, Conference, Project/Code link in README.md using the following format:

  |[Title](Paper Link)|Venue|[Code](Code link)|

• c. Submit the pull request to this branch.

Learning-based Reasoning

Title	Venue	Code
STaR: Self-Taught Reasoner Bootstrapping Reasoning With Reasoning	NeurIPS 2022	Code
Large Language Models Can Self-Improve	EMNLP 2023	-
Training Chain-of-Thought via Latent-Variable Inference	NeurIPS 2023	-
V-STaR: Training Verifiers for Self-Taught Reasoners	COLM 2024	-
Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking	arXiv 2024	-
MindStar: Enhancing Math Reasoning in Pre-trained LLMs at Inference Time	arXiv 2024	-
Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planning	arXiv 2024

Prompt Engineering-based Reasoning

Title	Venue	Code
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models	NeurIPS 2022	-
Large Language Models are Zero-Shot Reasoners	NeurIPS 2022	-
Automatic Chain of Thought Prompting in Large Language Models	arXiv 2022	Code
Self-Consistency Improves Chain of Thought Reasoning in Language Models	ICLR 2023	-
Tree of Thoughts: Deliberate Problem Solving with Large Language Models	NeurIPS 2023	Code
Large Language Model Guided Tree-of-Thought	arXiv 2024	Code
ART: Automatic multi-step reasoning and tool-use for large language models	arXiv 2023	-
Active Prompting with Chain-of-Thought for Large Language Models	ACL 2024	Code