🧑‍💻 What is “AI Engineering?”

🧑‍🎓 Ideal Student

📜 AI Engineering Learning Targets

🤔 Prerequisites

🏆 Grading and Certification

📅 Course Schedule

• By Session!

****- Understand course structure

• Meet your cohort!
• Overview of Prototyping LLM applications via Prompt Engineering, RAG, and Fine-Tuning
• Overview of Generative Pre-Trained Transformer (GPT) | 🧑‍💻 Building and 🚀 Sharing Your First GPT in OpenAI’s GPT Store | **GPT Store **OpenAI GPTs

| | | Session 2: First LLM Application Thursday, Apr 4: 4:00 to 6:00 PM PT

****- Understand components required to set up your LLM application development environment including Git, Unix Command Line, Conda, Jupyter Notebook, and VS Code

• Set up LLM application development environment
• Understand the components required to build your first LLM application including the OpenAI API, Chainlit, Docker, and Hugging Face
• Build your first LLM application! | 🧑‍💻 Building, Containerizing, Deploying, and 🚀 Sharing Your First LLM Application

Public GitHub Repo: ‣

Public GitHub Repo: ‣ | LLM: OpenAI GPT models User Interface: Chainlit Deployment: Docker, Hugging Face | | | Session 3: First RAG Application Tuesday, Apr 9: 4:00 to 6:00 PM PT

****- Understand Retrieval Augmented Generation = Dense Vector Retrieval + In-Context Learning

• RAG ~ Question Answering (QA) over documents
• Build a Python RAQA system from scratch!
• Set up visibility and monitoring tooling with WandB | 🧑‍💻 Building Your First Retrieval Augmented QA Application in Python with Visibility Tooling

Public GitHub Repo: ‣ | LLMOps Visibility: Weights and Biases LLM: OpenAI GPT models Embedding Model: OpenAI embeddings User Interface: Chainlit Deployment: Docker, Hugging Face | | | Session 4: First Agent Application Thursday, Apr 11 4:00 to 6:00 PM PT

****- A background of agents in AI, including the Reasoning-Action (ReAct) framework

• The intersection of agents and retrieval (e.g., RAG)
• Project discussions!
• How the OpenAI Assistants API can be used to build agentic applications | Building and Sharing Your First Agentic RAG Application | LLMOps Visibility: Weights and Biases LLM: OpenAI GPT models, OpenAI Assistants API, and Function Calling Embedding Model: OpenAI embeddings User Interface: Chainlit Deployment: Docker, Hugging Face | | | Session 5: First Fine-Tuning Tuesday, Apr 16: 4:00 to 6:00 PM PT

****- Understand Parameter Efficient Fine-Tuning (PEFT), Low-Rank Adaptation (LoRA), Quantization, and QLoRA

• The primary aims of fine-tuning when building LLM applications
• How to fine-tune open-source LLMs using a parameter-efficient approach with Hugging Face! | 🧑‍💻 Building and 🚀 Sharing Your First Fine-Tuned version of Mistral-7B with PEFT-QLoRA | LLM: Mistral 7B Quantization: ‣ Fine-Tuning: Hugging Face ‣ library, LoRA | | | Session 6: LangChain RAG Thursday, Apr 18: 4:00 to 6:00 PM PT

****- Understand LangChain v0.1 core constructs required to build RAG chains with LangChain Expression Language (LCEL)

• Build a RAG system with LangChain and FAISS
• Prompt and Embedding Caches | 🧑‍💻 Building and 🚀 Sharing your First RAG QA Application with LangChain | LLMOps Visibility: Weights and Biases LLMOps Infrastructure: LangChain Vector Database: ‣ LLM: OpenAI GPT models Embedding Model: OpenAI embeddings | | | Session 7: RAG Evaluation Tuesday, Apr 23: 4:00 to 6:00 PM PT

****- Evaluate RAG quantitatively with the RAG ASessment (RAGAS) Framework

• Build a simple retrieval system, baseline performance
• Implement retrieval improvements, measure with RAGAS | 🧑‍💻 Building and 🚀 Sharing an Improved LangChain RAG Application | LLMOps Visibility: Weights and Biases LLMOps Infrastructure: LangChain LLMOps Evaluation: RAGAS Vector Database: ‣ LLM: OpenAI GPT models Embedding Model: OpenAI embeddings User Interface: Chainlit Deployment: Docker, Hugging Face | | | Session 8: LlamaIndex RAG Thursday, Apr 25: 4:00 to 6:00 PM PT

****- Understand the LlamaIndex data framework, including core constructs like nodes and query engines

• Versioning indexes
• Build an agentic RAG system with both qualitative (i.e., Semantic RAG) and quantitative (e.g., tabular-data RAG) pipelines | 🧑‍💻 Building and 🚀 Sharing a LlamaIndex RAG Pipeline with NL2SQL and Metadata Filtering | LLMOps Infrastructure: LlamaIndex Vector Database: VectorStoreIndex LLM: OpenAI GPT models Embedding Model: OpenAI embeddings User Interface: Chainlit Deployment: Docker, Hugging Face | | | Session 9: Industry Use Cases Tuesday, Apr 30: 4:00 to 6:00 PM PT

****- Understand some of the most common industry use cases for production LLM applications

• Ideate with an AI coach and peers

• See a complete end-to-end Demo Day example of a production application with RAG, Fine-Tuning, and Agent patterns | 🧑‍💻 Building and 🚀 Sharing a Demo Day project idea using AIM’s Build🏗-Ship🚢-Share 🚀 GPT | Project Ideation: Build🏗-Ship🚢-Share 🚀 GPT | | | Session 10: Open-Source Production RAG Thursday, May 2: 4:00 to 6:00 PM PT

• Understand how to deploy open-source models (LLMs and embedding models) to scalable endpoints for production LLM and RAG applications

• Hugging Face Leaderboards (Open LLM, Massive Text Embedding Benchmark)

• How to use LangSmith as an evaluation and monitoring tool for your production applications, including setting up Custom Evaluators | 🧑‍💻 Building and 🚀 Sharing an Open-Source LLM-powered LangChain RAG Application, Evaluating and Benchmarking in LangSmith | LLMOps Visibility: LangSmith LLMOps Evaluation: LangSmith LLMOps Infrastructure: LangChain LLM: Llama 2 Embedding Model: From MTEB Leaderboard LLM Serving: Hugging Face Inference Endpoints | | | Session 11: Agentic RAG in Production Tuesday, May 7: 4:00 to 6:00 PM PT

****- Understand how to build production-grade agentic RAG applications using LangChain and LangGraph

• Understand Agent Types in LangChain and how LangGraph is used to add Cycles to LLM applications
• How to use LangSmith to evaluate and improve more complex RAG applications with agentic workflows | 🧑‍💻 Building and 🚀 Sharing an Agentic LangChain RAG Application, Improving Performance in LangSmith | LLMOps Visibility: LangSmith LLMOps Evaluation: LangSmith LLMOps Infrastructure: LangChain Agents: LangGraph LLM: OpenAI GPT models Embedding Model: OpenAI embeddings | | | Session 12: Domain-Adapted RAG with Fine-Tuning ****Thursday, May 9: 4:00 to 6:00 PM PT

****- Domain Adaptation of LLM Applications through Fine-Tuning

• How to do End-to-End Fine-Tuning of RAG Applications | 🧑‍💻 Building and 🚀 Sharing a Production E2E RAG Application | LLMOps Visibility: LangSmith LLMOps Evaluation: LangSmith LLMOps Infrastructure: LangChain LLM: Mistral 7B Embedding Model: From MTEB Leaderboard Quantization: ‣ Fine-Tuning: Hugging Face ‣ library, LoRA E2E Fine-Tuning: ‣ | | | Session 13: Demo Day Dress Rehearsal Tuesday, May 14: 4:00 to 6:00 PM PT

****- Final Demo Day Example by Dr. Greg and the Wiz

• Overview of LLM Ops Ecosystem
• Finalize Run of Show for Demo Day | Code Freeze: Finish your LLM Application Prototype using RAG + Fine-Tuning | Varies | | | Session 14: AI Makerspace Demo Day & Graduation! Thursday, May 16: 4:00 to 6:00 PM PT

****- All are welcome! | TBD | Varies | |