Electives
Complete exactly two electives. Choose the skills you want to develop, check their prerequisites, and complete the work in both chosen sections.
LLM Applications and RAG
Build an LLM feature that retrieves relevant evidence, returns validated results, and shows where its answers succeed or fail.
Before you start: Software Engineering for AI, LLM Fundamentals and Agent Workflows, and basic API skills from APIs, Containers & Deployment.
Learn
- Make model requests, manage messages and context, and handle timeouts, unavailable models, and invalid responses.
- Frame tasks clearly, generate structured outputs, and validate the response schema separately from factual correctness.
- Prepare documents with source metadata; use chunking, embeddings, similarity search, and a lexical-search baseline.
- Supply retrieved context, preserve source references, and handle questions with insufficient supporting evidence.
- Evaluate retrieval separately from answer correctness, grounding, and citation support.
- Compare one meaningful change and record quality, latency, and available usage or cost information.
Resources
Claude Academy: Building with the Claude API
Required selected sections. Complete the introductory API material through structured responses, the prompting and evaluation sections, and the retrieval lessons through embedding search and BM25. The remaining course sections are not required for this elective.
Claude Academy: Implementing the RAG flow
Required within this elective. Work through the preceding chunking and embedding lessons, implement the retrieval example, and adapt it to your own documents. Extend the returned passages into a sourced answer.
Hugging Face: RAG Evaluation
Required within this elective. Use separate retrieval and answer checks; review a sample manually. An automated model judge is not required.
Practice
- Implement a small model-call feature with a documented response schema and failure handling.
- Build a knowledge base from public or approved documents and preserve source identifiers.
- Create evaluation questions with expected supporting passages, including questions the collection cannot answer.
- Compare a baseline with one change to chunking, retrieval depth, or search strategy; inspect retrieval and answer failures separately.
- Expose the feature through your application with source references and a clear insufficient-evidence response.
Expected output
A working LLM/retrieval feature, validated response handling, evaluation questions, separate retrieval and answer results, and a short comparison explaining the chosen change and a meaningful failure.
Workload
Workload estimate pending pilot. Include practice, debugging, review, and revision in your study time.
Submission evidence and completion checks
Keep the expected output, runnable instructions, test/evaluation results, and your explanation of a meaningful failure. Submit through review and record the resulting revision. Explain your approach and diagnose a relevant failure during review. Revise incomplete evidence after feedback; resource completion alone does not establish competence.
Agent Development and Tool Integration
Build a bounded agent or tool-based workflow that completes a defined task with understandable execution and verifiable results.
Before you start: Software Engineering for AI, LLM Fundamentals and Agent Workflows, and basic API skills from APIs, Containers & Deployment. The introductory model-call material below provides the LLM preparation; completing the LLM Applications and RAG elective is not a prerequisite.
Learn
- Make a basic model request and distinguish a fixed workflow from an agent that selects its next action.
- Define tools with clear descriptions, argument schemas, validation, results, and failure behavior.
- Execute tool requests through a controlled loop; manage task state, stopping limits, and recovery.
- Use scoped access and explicit approval points for actions that need them.
- Understand MCP clients, servers, and tools; inspect or implement a small integration when it fits your work.
- Evaluate task outcomes and execution traces, then compare with a simpler workflow using the same underlying tools.
Resources
Claude Academy: Building with the Claude API
Required selected sections. Complete the basic request material if not already covered, then the introductory tool-use sequence through multi-turn execution. Apply tool definitions and result handling to one bounded task.
Claude Academy: Workflows vs agents
Required within this elective. Study the surrounding agent/workflow lessons and compare fixed control flow with model-directed tool use. Advanced orchestration patterns are further reading.
Claude Academy: Introduction to Model Context Protocol
Required selected lessons. Study the architecture, clients, tool definitions, and inspector. Adapt the practical example when an MCP connection fits your work; resources, prompts, and advanced protocol features are further reading.
Anthropic: Demystifying evals for AI agents
Required within this elective. Use task outcomes, tool-call records, and repeated trials to inspect behavior.
Practice
- Choose a bounded task, such as finding relevant records and producing a validated report. Define success before implementation.
- Implement tools and a workflow or agent; log calls, validate arguments, and enforce stopping limits.
- Test unavailable tools, invalid arguments, repeated actions, and incomplete tasks. Add a small read-only MCP integration if it suits the task.
- Compare against a simpler baseline and explain whether agentic control improves the result.
Expected output
A working tool-based application, validated tools, an evaluation set, execution traces, and a design explanation. Show a successful task and a controlled failure. An MCP implementation can deepen the elective; a framework certificate alone does not complete it.
Workload
Workload estimate pending pilot. Include practice, debugging, review, and revision in your study time.
Submission evidence and completion checks
Keep the expected output, runnable instructions, test/evaluation results, and your explanation of a meaningful failure. Submit through review and record the resulting revision. Explain your approach and diagnose a relevant failure during review. Revise incomplete evidence after feedback; resource completion alone does not establish competence.
Model Adaptation and Experimentation
Determine whether adapting a pretrained model improves a defined task.
Before you start: applied ML and deep learning; use LLM Fundamentals and Agent Workflows for LLM-oriented projects.
Learn
- Transfer learning and fine-tuning; select a model and dataset appropriate to the task and available compute.
- Prepare data, preserve an untouched test set, and establish the pretrained baseline.
- Track configurations, metrics, artifacts, and reproducible run information.
- Compare quality and resource use, and analyze regressions.
Resources
Hugging Face LLM Course: Fine-tuning a pretrained model
Required within this elective. Follow the chapter's hands-on sequence to adapt a small model. A modest classification task is sufficient.
MLflow: Experiment Tracking
Required within this elective. Record parameters, metrics, and artifacts locally.
UvA Tutorial 2: Introduction to PyTorch
Required within this elective. Revisit model structure, autograd, and training/evaluation behavior as needed.
Practice
- Evaluate the unchanged pretrained model on a fixed task.
- Adapt it using a manageable training run and record at least two useful configuration comparisons.
- Compare on the same evaluation data, inspect improvements and regressions, and preserve the resulting model artifact.
- Reproduce inference from the saved artifact without rerunning training.
Expected output
A baseline comparison, tracked experiments, a model artifact, and a short adaptation report. An improvement is not guaranteed: a well-supported explanation of a failed adaptation demonstrates useful engineering judgment.
Workload
Workload estimate pending pilot. Include practice, debugging, review, and revision in your study time.
Submission evidence and completion checks
Keep the expected output, runnable instructions, test/evaluation results where applicable, and your explanation of a meaningful failure. Submit through review and record the resulting revision. Explain your approach and diagnose a relevant failure during review. Revise incomplete evidence after feedback; resource completion alone does not establish competence.
Backend Architecture and Data Systems
Extend an AI service with reliable persistence and background work.
Before you start: Software Engineering for AI, Systems, Networking & Data, APIs, Containers & Deployment, and Engineering Fundamentals SQL.
Learn
- Relational schema design, constraints, transactions, indexes, and query inspection.
- Separate request handling from a background task; track job state and handle retries.
- Use caching deliberately and explain invalidation or expiry.
- Define clear responsibilities between API, inference, database, and worker components.
Resources
PostgreSQL: Tutorial
Required within this elective. Focus on tables, queries, joins, updates, foreign keys, and transactions. Extend into indexes for your own queries.
Redis: Quick Starts
Required within this elective. Use Python access and an expiring cache for one suitable operation.
FastAPI: Tutorial — User Guide
Required within this elective. Use current reference sections for the API and dependencies around your data layer.
Practice
- Persist requests, results, or jobs in PostgreSQL using constraints suited to the data.
- Move a slow operation to a worker and expose a way to inspect its status.
- Make retry behavior safe for your chosen operation and test a failed job.
- Add one justified cache or index and compare behavior before and after.
Expected output
A service with a database and worker, a system diagram, tests for a meaningful failure path, and an explanation of the chosen schema and performance change.
Workload
Workload estimate pending pilot. Include practice, debugging, review, and revision in your study time.
Submission evidence and completion checks
Keep the expected output, runnable instructions, test/evaluation results where applicable, and your explanation of a meaningful failure. Submit through review and record the resulting revision. Explain your approach and diagnose a relevant failure during review. Revise incomplete evidence after feedback; resource completion alone does not establish competence.
Delivery, Monitoring, and MLOps
Make changes observable and releases recoverable.
Before you start: Software Engineering for AI, Systems, Networking & Data, APIs, Containers & Deployment; ML evaluation or the LLM evaluation in LLM Applications and RAG provides the evaluation for your AI component.
Learn
- Extend automated tests into a release workflow.
- Version application releases and model or prompt artifacts.
- Observe request latency, errors, and AI behavior using logs, metrics, and traces.
- Keep software regression tests and model/LLM evaluation visible.
- Deploy a new release, detect a failure, and restore a known working version.
Resources
GitHub Actions: Building and testing Python
Required within this elective. Extend the APIs, Containers & Deployment module's basic workflow into build/release checks appropriate to your environment.
Docker 101 Tutorial
Required within this elective. Reuse image-building skills and apply explicit release tags.
MLflow: Experiment Tracking
Required within this elective. Connect model artifacts and evaluation results to a versioned run. LLM-based work can instead version their prompts/configuration and evaluation results.
Practice
- Automate tests and build an identifiable release artifact.
- Record latency and error behavior for a normal request and an induced failure.
- Deploy an updated release, verify its behavior, and demonstrate restoration of the previous working version.
- Rerun the same AI evaluation after a model or prompt change and report regressions.
Expected output
A release workflow, an observable service, a versioned evaluation record, and a demonstrated recovery procedure. Use one environment; the learning outcome is a reliable delivery process.
Workload
Workload estimate pending pilot. Include practice, debugging, review, and revision in your study time.
Submission evidence and completion checks
Keep the expected output, runnable instructions, test/evaluation results where applicable, and your explanation of a meaningful failure. Submit through review and record the resulting revision. Explain your approach and diagnose a relevant failure during review. Revise incomplete evidence after feedback; resource completion alone does not establish competence.
Finish this module
Complete two sections and submit the evidence required by each. Keep their results and explanations separate, even when both use the same repository. Revise incomplete work after review.





