🧑‍💻 What is “AI Engineering?”

🧑‍🎓 Ideal Student

📜 AI Engineering Learning Targets

🤔 Prerequisites

🏆 Grading and Certification

📅 Course Schedule

• By Session!

****- Understand course structure

• Meet your cohort!
• Overview of Prototyping LLM applications including the patterns of Prompt Engineering, RAG, Fine-Tuning, and Agents
• A very brief history of the Generative Pre-Trained Transformer (GPT) | 🧑‍💻 Building and 🚀 Sharing Your First GPT in OpenAI’s GPT Store | **GPT Store **OpenAI GPTs

| | Session 2: First LLM Application Thursday, May 30th: 4:00 to 6:00 PM PT

- Prompt Engineering best-practices

• ****Understand the basics of prompting through the OpenAI GPT-4o (Omni) API
• Understand the components required to build your first LLM application including the OpenAI GPT-4 Omni API, Chainlit, Docker, and Hugging Face
• Build, ship, and share your first LLM application! | 🧑‍💻 Building, Containerizing, Deploying, and 🚀 Sharing Your First LLM Application

Public GitHub Repo: ‣

Public GitHub Repo: ‣ | LLM: OpenAI GPT models User Interface: Chainlit Deployment: Docker, Hugging Face | | Session 3: First RAG Application Tuesday, June 4th: 4:00 to 6:00 PM PT

****- Understand Retrieval Augmented Generation = Dense Vector Retrieval + In-Context Learning

• RAG ~ Question Answering (QA) over documents
• Build a Python RAQA system from scratch!
• Set up visibility and monitoring tooling with WandB | 🧑‍💻 Building Your First Retrieval Augmented QA Application in Python with Visibility Tooling

Public GitHub Repo: ‣ | LLMOps Visibility: Weights and Biases LLM: OpenAI GPT models Embedding Model: OpenAI embeddings User Interface: Chainlit Deployment: Docker, Hugging Face | | Session 4: First Agent Application Thursday, June 6th: 4:00 to 6:00 PM PT

****- A background of agents in AI, including the Reasoning-Action (ReAct) framework

• The intersection of agents and retrieval (e.g., RAG)
• Project discussions!
• How the OpenAI Assistants API can be used to build agentic applications | Building and Sharing Your First Agentic RAG Application | LLMOps Visibility: Weights and Biases LLM: OpenAI GPT models, OpenAI Assistants API, and Function Calling Embedding Model: OpenAI embeddings User Interface: Chainlit Deployment: Docker, Hugging Face | | Session 5: First Fine-Tuning (Embedding Model) Tuesday, June 11th: 4:00 to 6:00 PM PT

****- Understand the basics of embedding models and what they’re used for

• Learn to leverage quantization via the bitsandbytes library for loading models onto your local machine
• Fine-tune embeddings for domain adaptation | 🧑‍💻 Building Your First Fine-Tuned Embedding model using LlamaIndex & HF Sentence Transformers | **LLM: Snowflake arctic-embed-m** Quantization: ‣ Fine-Tuning: Hugging Face Sentence Transformers, LlamaIndex Synthetic Data Generation & Fine-Tuning | | Session 6: LangChain RAG Thursday, June 13th: 4:00 to 6:00 PM PT

****- Understand LangChain v0.1 core constructs required to build RAG chains with LangChain Expression Language (LCEL)

• Build a RAG system with LangChain and FAISS
• Creating prompt and Embedding Caches | 🧑‍💻 Building your First RAG QA Application with LangChain | LLMOps Visibility: Weights and Biases LLMOps Infrastructure: LangChain Vector Database: ‣ LLM: OpenAI GPT models Embedding Model: OpenAI embeddings | | Session 7: Prototyping End-to-End RAG Applications Tuesday, June 18th: 4:00 to 6:00 PM PT

• Build a RAG system with LCEL
• How to build front-end UIs for RAG applications
• How to collect and leverage user feedback to improve performance | 🧑‍💻 Building and 🚀 Sharing your First RAG QA Application with LangChain | LLMOps Visibility: Weights and Biases LLMOps Infrastructure: LangChain Vector Database: ‣ LLM: OpenAI GPT models Embedding Model: OpenAI embeddings User Interface: Chainlit Deployment: Docker, Hugging Face | | Session 8: RAG Evaluation Thursday, June 20th: 4:00 to 6:00 PM PT

****- Evaluate RAG quantitatively with the RAG ASessment (RAGAS) Framework

• Build a simple retrieval system using LangChain LCEL, baseline performance
• Implement retrieval improvements, measure with RAGAS | 🧑‍💻 Building, improving, and 🚀 Sharing an Improved LangChain RAG Application with Advanced Retrieval | LLMOps Visibility: Weights and Biases LLMOps Infrastructure: LangChain LLMOps Evaluation: RAGAS Vector Database: ‣ LLM: OpenAI GPT models Embedding Model: OpenAI embeddings User Interface: Chainlit Deployment: Docker, Hugging Face | | Session 9: LlamaIndex RAG Tuesday, June 25th: 4:00 to 6:00 PM PT

****- Understand the LlamaIndex data framework, including core constructs like nodes and query engines

• Versioning indexes

• Build an agentic RAG system with both qualitative (i.e., Semantic RAG) and quantitative (e.g., tabular-data RAG) pipelines | 🧑‍💻 Building and 🚀 Sharing a LlamaIndex RAG Pipeline with NL2SQL and Metadata Filtering | LLMOps Infrastructure: LlamaIndex Vector Database: VectorStoreIndex LLM: OpenAI GPT models Embedding Model: OpenAI embeddings User Interface: Chainlit Deployment: Docker, Hugging Face | | Session 10: Open-Source Production RAG Thursday, June 27th: 4:00 to 6:00 PM PT

• Understand how to deploy open-source models (LLMs and embedding models) to scalable endpoints for production LLM and RAG applications

• Hugging Face Leaderboards (Open LLM, Massive Text Embedding Benchmark)

• How to use LangSmith as an evaluation and monitoring tool for your production applications, including setting up Custom Evaluators | 🧑‍💻 Building and 🚀 Sharing an Open-Source LLM-powered LangChain RAG Application, Evaluating and Benchmarking in LangSmith | LLMOps Visibility: LangSmith LLMOps Evaluation: LangSmith LLMOps Infrastructure: LangChain, Hugging Face x LangChain **LLM: **Llama 3 Embedding Model: From MTEB Leaderboard LLM Serving: Hugging Face Inference Endpoints | | Session 11: Fine-Tuning Open-Source LLMs Tuesday, July 2nd: 4:00 to 6:00 PM PT

****- Understand Parameter Efficient Fine-Tuning (PEFT), Low-Rank Adaptation (LoRA), Quantization, and QLoRA

• The primary aims of fine-tuning when building LLM applications
• How to fine-tune open-source LLMs using a parameter-efficient approach with Hugging Face! | 🧑‍💻 Building and 🚀 Sharing Your First Fine-Tuned version of Mistral-7B with PEFT-QLoRA | LLM: Mistral 7B Quantization: ‣ Fine-Tuning: Hugging Face ‣ library, LoRA | | Session 12: Agentic RAG in Production Tuesday, July 9th: 4:00 to 6:00 PM PT

****- Understand how to build production-grade agentic RAG applications using LangChain and LangGraph

• Understand Agent Types in LangChain and how LangGraph is used to add Cycles to LLM applications
• How to use LangSmith to evaluate and improve more complex RAG applications with agentic workflows | 🧑‍💻 Building and 🚀 Sharing an Agentic LangChain RAG Application, Improving Performance in LangSmith | LLMOps Visibility: LangSmith LLMOps Evaluation: LangSmith LLMOps Infrastructure: LangChain Agents: LangGraph LLM: OpenAI GPT models Embedding Model: OpenAI embeddings | | Session 13: Industry Use Cases Thursday, July 18th: 4:00 to 6:00 PM PT

****- Understand some of the most common industry use cases for production LLM applications

• Ideate with an AI coach and peers
• See a complete end-to-end Demo Day example of a production application with RAG, Fine-Tuning, and Agent patterns
• Overview of LLM Ops Ecosystem | 🧑‍💻 Building and 🚀 Sharing a Demo Day project idea using AIM’s Build🏗-Ship🚢-Share 🚀 GPT | Project Ideation: Build🏗-Ship🚢-Share 🚀 GPT | | Session 14: Demo Day Dress Rehearsal Tuesday, July 23rd: 4:00 to 6:00 PM PT

****- Final Demo Day Example by Dr. Greg and the Wiz

• Finalize Run of Show for Demo Day | 🥶 Code Freeze: Finish your LLM Application Prototype using RAG + Fine-Tuning | Varies | | Session 15: AI Makerspace Demo Day & Graduation! Thursday, July 25th: 4:00 to 6:00 PM PT

****- All are welcome! This event is open to the public! | AIE1 Demo Day Run of Show

AIE2 Demo Day Run of Show | AIE1 Cohort 1 Demo Day Presentations |