Open LLM Systems Engineering courseReviewed Aug 23, 2026

YOUR WORK · IN YOUR BROWSER

Technical portfolio and evidence passport

A progress bar says what you opened. A portfolio proves what you designed, measured, and operated. Use this space to turn every competency into a reviewable artifact.

0Resources completed/ 143
0Tools run/ 12
0Mock examslatest 30
0Competencies verified/ 11

Expected evidence

Evidence map

Status is a local self-assessment. “Verified” means you retain an artifact, a metric, and a review; it is not an official credential.

01

Foundations and models

Explain transformers, tokenization, inference, and model selection through explicit trade-offs.

Expected evidence

An executable comparison and a model decision backed by evaluation.

02

Prompt systems

Treat prompts, schemas, and tools as versioned, testable contracts.

Expected evidence

A regression dataset and A/B pipeline with explicit criteria.

03

RAG and retrieval

Design ingestion, chunking, search, reranking, grounding, and RAG evaluation.

Expected evidence

A measured RAG system with answerable and unanswerable cases, citations, and retrieval metrics.

04

Agent interfaces

Design tools, MCP, memory, and authority boundaries without delegating control to the model.

Expected evidence

An agent with three typed tools, explicit permissions, and negative-path tests.

05

Harness Engineering

Build the environment that supplies context, capabilities, isolation, signals, and evaluations to an agent.

Expected evidence

A reproducible harness with a capability manifest, sandbox, traces, and isolated evals.

06

Loop Engineering

Control iterations, budgets, progress, retries, concurrency, stopping, and resumption.

Expected evidence

A bounded, resumable loop that proves termination, idempotency, and failure recovery.

07

Graph Engineering

Model execution, knowledge, and provenance graphs with explicit invariants and measurable queries.

Expected evidence

A hybrid system that connects state, evidence, and decisions without losing provenance.

08

Evaluation

Separate model, retrieval, and application quality through reproducible evaluations.

Expected evidence

A regression suite with a baseline, rubric, failed samples, and acceptance gates.

09

LLMOps

Observe latency, cost, quality, safety, and distribution shifts in production.

Expected evidence

An operational dashboard, alerts, and an incident runbook.

10

Safety and governance

Model threats, constrain tools, and apply privacy, guardrails, and auditability.

Expected evidence

A threat model, prompt-injection test, and verifiable controls.

11

Product architecture

Make build-vs-buy, cost, reliability, and user-experience decisions.

Expected evidence

ADRs, cost analysis, and a technical field-project review.

Data portability

Download your progress and pseudonymous history in one file. Import only replaces the current state after validation.