🧑‍💻 What is “AI Engineering?”

🧑‍🎓 Ideal Student

🤔  Prerequisites

🏆 Grading and Certification

📅 Detailed Course Schedule & Curriculum

🏗️ Prototyping (Build)

Week 1: Introduction, Vibe Check, and Context Engineering

• Understand course structure
• How to succeed as a certified AI Engineer on Demo Day!
• Meet your cohort, peer supporters, and journey group!
• LLM prototyping best practices, how to spin up quick end-to-end prototypes and vibe check them! | Vibe Checking The AIE Challenge

🚧 Advanced Build: Make improvements to improve the vibes, reevaluate | LLM: OpenAI GPT models UI: Vibe Coded w/ React Deployment: Vercel

Relevant papers

• Chain-of-Thought
• Principled Instructions
• LMs are few-shot learners | | **Live Session 2: 🗃️ Context Engineering I: Retrieval Augmented Generation (RAG)

-** Context Engineering, Level 0 (Prompt Engineering) best practices and the LLM Application Stack

• Understand embedding models and similarity search
• Understand Context Engineering, Level 1 (Retrieval Augmented Generation = Dense Vector Retrieval + In-Context Learning)
• Understand the limitations of Context Engineering Levels 0 and 1
• Build a Python RAG app from scratch | Building a Pythonic RAG App from Scratch

🚧 Advanced Build: Add one or more optional ”extras” to the RAG pipeline | LLM: OpenAI GPT models Embedding Model: OpenAI embeddings Orchestration: OpenAI Python SDK

Relevant papers/blogs

• The LLM Application Stack
• RAG
• Limitations of Embedding-Based Retrieval (Aug 2025)
• The rise of “context engineering” (June 2025)
• Context Engineering, the O’G’ (posted in July 2025 on YouTube) |

Week 2: Building Production-Grade AI Applications

-** Understand the state of production LLM application use cases in industry

• Ideate with peers & peer supporters
• Build an end-to-end RAG application using everything we’ve learned so far
• Introduce primary cohort use case
• Understand how to use and remotely host leading open-source models | Deploy an E2E Pythonic RAG Application

🚧 Advanced Build: Determine a specific use-case for RAG, and adapt your challenge to that new use case. | LLM: OpenAI GPT models, Anthropic Claude Embedding Model: OpenAI embeddings Orchestration: OpenAI Python SDK User Interface: Vibe Coded w/ React Deployment: Vercel Inference & Serving: Together AI

Relevant papers/blogs

• Identifying and Scaling Use Cases (April 2025) - Check out Cohort 7’s Industry Use Case Update (June 2025) | | Live Session 4: ⛓️ Production-Grade RAG with LangGraph with Open-Source LLMs

• Understand how to use and locally host leading open-source models

• Why LangChain, OpenAI, QDrant, LangSmith

• Understand LangChain & LangGraph core constructs

• Build a RAG system with LangChain and Qdrant

• Intro to LangSmith for evaluation and monitoring | Build a production-grade RAG application with LangGraph and locally-hosted open-source models

| LLM: OpenAI GPT models, The dopest new OSS drop Embedding Model: OpenAI embeddings, EmbeddingGemma Orchestration: LangChain & LangGraph Vector Database: QDrant Evaluation & Monitoring: LangSmith Inference & Serving: ollama

Relevant papers/blog

• Is LangGraph Used in Production? (February 2025)
• LangChain & LangGraph 1.0 alpha releases (Sep 2025)
• Welcome EmbeddingGemma (Sept 2025) |

Week 3: Context Engineering II: Agents and Multi-Agent Systems

-** Answer the question: “What is an agent?”

• Understand how to build production-grade agent applications using LangGraph
• Understand Context Engineering Level 2 from first principles
• Understand Deep Agents and how they differ from shallow agents
• How to use LangSmith to evaluate agentic RAG applications | 1. Build your first Agent (e.g., agentic RAG) application!

2. Evaluate your Agentic RAG application using off-the-shelf and custom evaluators in LangSmith | LLM: OpenAI GPT models Embedding Model: OpenAI embeddings Orchestration: LangChain & LangGraph Vector Database: QDrant Function Calling: OpenAI Tools Evaluation & Monitoring: LangSmith

Relevant papers

• ReAct (Oct, 2022)

• How to think about agent frameworks (April, 2025)

• Context Engineering (for agents) (July 2025)

• Deep Agents (July 2025) | | Live Session 6: 🔄 Multi-Agent Applications

• Understand what multi-agent systems are and how they operate.

• Extend the primary cohort use case to a multi-agent solution ****- Build a production-grade multi-agent applications using LangGraph | Building a multi-agentic LangGraph application that allows us to separate search and retrieval (Research Team) from generating the final output (Document Writing Team)

🚧 Advanced Build: Build a graph to produce a social media post about a given Machine Learning Paper.2. Evaluate your RAG application using off-the-shelf and custom evaluators in LangSmith | LLM: OpenAI GPT models Embedding Model: OpenAI embeddings Orchestration: LangChain & LangGraph Vector Database: QDrant Function Calling: OpenAI Tools

Relevant papers/blogs

• Don’t Build Multi-Agents (June 2025)
• How we built our multi-agent research system (June 2025)
• Zero to One: Learning Agentic Patterns (May 2025) |

Week 4: Evaluating AI Applications with Synthetic Data

-** An overview of Synthetic Data Generation (SDG)

• How to use SDG for Evaluation
• Generating high-quality synthetic test data sets for RAG applications in general and for our primary cohort use case specifically
• How to use LangSmith to baseline performance, make improvements, and then compare | Create synthetic test data generation for RAG evaluation, load into LangSmith, and evaluate RAG against synthetic test data

🚧 Advanced Build: Reproduce the RAGAS Synthetic Data Generation Steps - but utilize a LangGraph Agent Graph, instead of the Knowledge Graph approach. | LLM: OpenAI GPT models Embedding Model: OpenAI embeddings Orchestration: LangChain & LangGraph Vector Database: QDrant Evaluation: RAG ASessment, LangSmith

Relevant papers/blogs - All about synthetic data generation (Nov 2024)

• Mastering LLM Techniques: Evaluation (Jan 2025) | | Live Session 8: 📊 RAG and Agent Evaluation

• Build RAG and Agent applications with LangGraph

• Evaluate RAG and Agent applications quantitatively with the RAG ASsessment (RAGAS) framework

• Use metrics-driven development to improve agentic applications, measurably, with RAGAS | 1. Build and evaluate RAG application with RAGAS RAG metrics

2. Build and evaluate ReAct agent application with RAGAS agent metrics | LLM: OpenAI GPT models Embedding Model: OpenAI embeddings Orchestration: LangChain & LangGraph Vector Database: QDrant Function Calling: OpenAI Tools Evaluation: RAGAS

Relevant papers

• RAGAS (2023)
• Hard-learned lessons from 2 years of Improving AI Applications (May 2025)
• Evaluating the Evaluators (Aug 2025)
• In Defense of Evals (Sep 2025) |

Week 5: Advanced Retrievers & Agentic Reasoning

• Understand how advanced retrieval and chunking techniques can enhance RAG
• Compare the performance of retrieval algorithms for RAG
• Understand the fine lines between chunking, retrieval, and ranking for our primary cohort use case
• Learn best practices for retrieval pipelines | Build a RAG application and evaluate different retrieval strategies

🚧 Advanced Build: Implement RAG-Fusion using the LangChain ecosystem. | Our Standard RAG Stack for Building and Evaluating Apps 👆

Relevant papers BM25 Reciprocal Rank Fusion | | Live Session 10: 🧠 **Advanced Agentic Reasoning

-** Discuss best-practice use of reasoning models

• Understand planning and reflection agents
• Build an Open-Source Deep Research agent application using LangGraph
• Investigate evaluating complex agent applications with the latest tools | Build and evaluate an unrolled version of Open Deep Research from LangGraph | Our Standard Agent Stack for Building and Evaluating Apps👆

Relevant Papers CoT Prompting Self-Refine Reflexion Scaling Test-Time Compute |

Week 6: Certification Challenge and OpenAI Agents SDK

-** Introduce the Certification Challenge

• Pitch your problem, audience, and solution
• Network with people outside of your group | The Certification Challenge

1. Define Problem & Audience
2. Propose Solution
3. Deal with Data
4. Build E2E Agentic RAG Prototype
5. Create Golden Test Data Set
6. Assess Performance | Up to you! | Breakout Room: Demo Day Project Pitches, within and across journey groups. | | Live Session 12: 🤖 OpenAI Agents SDK

• Understand the suite of tools for building agents with OpenAI and the evolution of their tooling
• Core constructs of the Agents SDK and comparison to other agent frameworks
• How to use monitoring and observability tools on the OpenAI platform | [Extra Credit] Building and evaluate an a multi-agent Research Bot with a Planner Agent, Search Agent, and Writer Agent | LLM: OpenAI GPT models Embedding Model: OpenAI embeddings Orchestration: Agents SDK Function Calling: OpenAI Tools Evaluation & Monitoring: OpenAI Platform

Relevant blog New Tools for Building Agents | |

🚢 Production (Ship)