🧑‍💻 What is “AI Engineering?”

🧑‍🎓 Ideal Student

📜 AI Engineering Learning Targets

🏆 Grading and Certification

🤔  Prerequisites

📅 Detailed Course Schedule & Curriculum

🧑‍💻 Prerequisites

The AI Engineering Bootcamp Challenge

*Note: These challenges will required you to:

1. Set up your computer for app dev, whether Mac, Windows, or Linux
2. Set up API Keys, including for OpenAI GPT models and Claude
3. Set up Google Colab Pro account to access GPUs
4. Set up Hugging Face Account (billing not yet required)

You will learn a heck of a lot just from completing these challenges!

Don’t forget to using the #ask-aim channel if you get stuck!* | Part 0: Dev Env Setup (with Cursor)

Part I: Beyond ChatGPT: Build and Deploy Your First LLM Application

Part II: Fine-Tune Llama 3.1-8B-Instruct | Interactive AI App Dev Env Setup Version Control: GitHub CLI: Shell for Unix-like OS (WSL) Package & Env Management: uv Python Notebooks: Jupyter / Colab Code Editor: Cursor / Claude Web App Framework: FastAPI Containers: ****Docker

LLM App Stack Tooling (for Part I) LLM: OpenAI GPT models UI: Vibe Coded w/ React Deployment: Vercel

Fine-Tuning Tooling (for Part II) LLM: Llama 3.1 8B Instruct Quantization: ‣ Fine-Tuning: Hugging Face ‣ library, LoRA

Relevant papers LLaMA Low-Rank Adaptation QLoRA |

🏗️ Prototyping (Build)

Week 1: Introduction, Vibe Check, and RAG

• Understand course structure
• How to succeed as a certified AI Engineer on Demo Day!
• Meet your cohort, peer supporters, and journey group!
• LLM prototyping best practices, from scoping to prompting to vibe checking | Vibe Checking The AIE Challenge

🚧 Advanced Build: Make improvements to improve the vibes, reevaluate | LLM: OpenAI GPT models UI: Vibe Coded w/ React Deployment: Vercel

Relevant papers

• Chain-of-Thought
• Principled Instructions
• LMs are few-shot learners | | **Session 2: Embeddings and Retrieval Augmented Generation (RAG)

-** Prompt Engineering best practices

• Overview of the LLM App Stack
• Understand embedding models and similarity search
• Understand Retrieval Augmented Generation = Dense Vector Retrieval + In-Context Learning
• Build a Python RAG app from scratch | Building a Pythonic RAG App from Scratch

🚧 Advanced Build: Add one or more optional ”extras” to the RAG pipeline | LLM: OpenAI GPT models Embedding Model: OpenAI embeddings Orchestration: OpenAI Python SDK

Relevant papers

• Attention is all you need
• RAG |

Week 2: Production-Grade RAG Apps

• T**he state of production LLM application use cases in industry
• Understand Demo Day expectations
• Ideate with peers & peer supporters
• Build an end-to-end RAG application using everything we’ve learned so far | Deploy Pythonic RAG Application

🚧 Advanced Build: Create a FastAPI backend | LLM: OpenAI GPT models, Anthropic Claude Embedding Model: OpenAI embeddings Orchestration: OpenAI Python SDK User Interface: Vibe Coded w/ React Deployment: Vercel | | Session 4: Production-Grade RAG with LangGraph

• Why LangChain, OpenAI, QDrant, LangSmith
• Understand LangChain & LangGraph core constructs
• Understand (enough) LangGraph and LangSmith
• Build a RAG system with LangChain and Qdrant
• How to use LangSmith for evaluation and monitoring | Build and evaluate a production-grade RAG application with LangGraph

🚧 Advanced Build: Extending the Graph with Complex Flows | LLM: OpenAI GPT models Embedding Model: OpenAI embeddings Orchestration: LangChain & LangGraph Vector Database: QDrant Evaluation & Monitoring: LangSmith |

Week 3: Agents and Multi-Agent Systems

-** Answer the question: “What is an agent?”

• Understand how to build production-grade agent applications using LangGraph
• How to use LangSmith to evaluate more complex agentic RAG applications | Build and evaluate a production-grade agentic RAG application

🚧 Advanced Build: Create an agent with 3 toosl that can research a specific topic of your choice. Deploy with a Chainlit (or custom) front end | LLM: OpenAI GPT models Embedding Model: OpenAI embeddings Orchestration: LangChain & LangGraph Vector Database: QDrant Function Calling: OpenAI Tools Evaluation & Monitoring: LangSmith

Relevant papers

• ReAct | | Session 6: Multi-Agent Applications

• Understand what multi-agent systems are and how they operate.

• Build a production-grade multi-agent applications using LangGraph | Building a multi-agent LangGraph application for writing, editing, and planning a LinkedIn post

🚧 Advanced Build: Build a graph to produce a social media post about a given Machine Learning paper that employs an additional team to verify correctness, theme, and style | LLM: OpenAI GPT models Embedding Model: OpenAI embeddings Orchestration: LangChain & LangGraph Vector Database: QDrant Function Calling: OpenAI Tools |

Week 4: RAG & Agent Evaluation with Synthetic Data

-** An overview of Synthetic Data Generation (SDG)

• How to use SDG for Evaluation
• Generating high-quality synthetic test data sets for RAG applications
• How to use LangSmith to baseline performance, make improvements, and then compare | Create synthetic test data generation for RAG evaluation, load into LangSmith, and evaluate RAG against synthetic test data

🚧 Advanced Build: Reproduce the RAGAS Synthetic Data Generation Steps - but utilize a LangGraph Agent Graph, instead of the Knowledge Graph approach. | LLM: OpenAI GPT models Embedding Model: OpenAI embeddings Orchestration: LangChain & LangGraph Vector Database: QDrant Evaluation: RAG ASessment, LangSmith

Relevant papers WizardLM | | Session 8: RAG and Agent Evaluation

• Build RAG and Agent applications with LangGraph
• Evaluate RAG and Agent applications quantitatively with the RAG ASsessment (RAGAS) framework
• Use metrics-driven development to improve agentic applications, measurably, with RAGAS | 1. Build and evaluate RAG application with RAGAS RAG metrics

2. Build and evaluate ReAct agent application with RAGAS agent metrics | LLM: OpenAI GPT models Embedding Model: OpenAI embeddings Orchestration: LangChain & LangGraph Vector Database: QDrant Function Calling: OpenAI Tools Evaluation: RAGAS

Relevant papers RAGAS |

Week 5: Advanced RAG + Agents I

• Understand how advanced retrieval and chunking techniques can enhance RAG
• Compare the performance of retrieval algorithms for RAG
• Understand the fine lines between chunking, retrieval, and ranking
• Learn best practices for retrieval pipelines | Build a RAG application and evaluate different retrieval strategies

🚧 Advanced Build: Implement RAG-Fusion using the LangChain ecosystem. | Our Standard RAG Stack for Building and Evaluating Apps 👆

Relevant papers BM25 Reciprocal Rank Fusion Contextual Retrieval | | **Session 10: Advanced Agentic Reasoning

-** Discuss best-practice use of reasoning models

• Understand planning and reflection agents
• Build an Open-Source Deep Research agent application using LangGraph
• Investigate evaluating complex agent applications with the latest tools | Build and evaluate an unrolled version of Open Deep Research from LangGraph | Our Standard Agent Stack for Building and Evaluating Apps👆

Relevant Papers CoT Prompting Self-Refine Reflexion Scaling Test-Time Compute |