[ BESMALA ]

Portfolio / AI Learning Lab / Batch 06

Research & Web · 2 study sheets

Batch 06

Build a repeatable method for reading AI papers, then review the web concepts used by AI engineers.

Three passes over a paper: pass one ten minutes triage; pass two one hour careful read with evidence audit; pass three hours-long virtual reimplementation

Research Papers and Experiments · sheet 1 of 2 in this chapter

Reading AI Research Papers

Research Papers and Experiments Beginner → Intermediate structured sheet · Batch 6 companion to the LLM Research Roadmap (your original file)

0This sheet and your LLM Research Roadmap

Your LLM Research Roadmap (your original document, kept as a standalone file — llm-research-roadmap.html) answers “what should I read, in what order?” — the curated sequence of papers and topics. This sheet answers the other half: “how do I actually read one?” — the method, the templates, and the habit of turning reading into runnable experiments. They compose: pick the next paper from the roadmap, apply this sheet to it, log the result in §10. Neither replaces the other.

1Paper reading is a trainable skill

The question: why does reading a paper take a beginner 8 hours and an expert 30 minutes — and what does the expert know?

Intuition

Beginners read papers like novels: linearly, every word, hoping comprehension arrives at the end. Experts read them like hunting: they arrive with questions (“what’s new? does the evidence hold? is it usable?”) and extract only what answers them. The skill isn’t speed-reading — it’s knowing where information lives and what to ignore on each visit.

Technical — the three failure modes this sheet fixes

  • The linear slog — reading front-to-back, drowning in section 3’s math before knowing why it matters. Fix: passes (§2), anatomy (§3).
  • The abstract believer — accepting claims without auditing evidence. Fix: baselines/ ablations (§6), reproducibility check (§7).
  • The vanishing read — papers read, nothing retained, nothing built. Fix: template (§9), experiment log (§10), portfolio path (§11).
Knowledge check

What question should you be able to answer before deciding to do a deep read of a paper? Answer: pass 1’s triage: what category is it, what does it claim to contribute, and do I care for my current goals? If the contribution sentence (§4) doesn’t surface anything you need, stop at pass 1 — that’s the system working, not failing.

2The three-pass method

The question: how do experts get 80% of a paper in 10 minutes and 95% in an hour?
fig 1 · architecture table 1 · results the paper PASS 1 · ~10 min title · abstract · headings · figures · conclusion output: triage — category, claim, “do I care?” → maybe STOP here PASS 2 · ~1 h careful read · audit evidence: tables, baselines, ablations output: can I summarize the method and trust the numbers? PASS 3 · hours virtual reimplementation each pass is narrower and deeper. Most papers deserve pass 1 only — a handful a year deserve pass 3. method: Keshav (2010) — “How to Read a Paper” (§14). pass 3 = mentally rebuild the work: challenge assumptions, spot the hidden choices the text skips.
The three passes (adapted from Keshav’s method). The decision points between passes are the whole trick: most papers should die at pass 1 by design.

Technical — what each pass produces

  • Pass 1 — triage (5 questions): Category (method/analysis/survey/benchmark)? Context (what line of work)? Claims? Contributions (§4)? Clarity (is the writing itself a red flag)? Output: a one-sentence answer to “what does this paper claim to add?”
  • Pass 2 — evidence audit: read the method with care but skip long proofs; study every figure and table (§5); check baselines and ablations (§6); note unread references. Output: the §9 template mostly filled.
  • Pass 3 — rebuild: for the few papers that anchor your area: assume you must reimplement it (§8) and identify every step the paper glosses over. Output: implementation notes — the hidden decisions that never make it into the text.

Common misconception

“Deep read = reading every word in order.” Depth comes from rebuilding, not from linearity. Experts skim ruthlessly and grind only where the contribution lives.

Knowledge check

You did pass 2 on a paper and can’t state its contribution in one sentence. What happened, and what’s the correction? Answer: the passes ran out of order — §4’s contribution hunt (usually end of the introduction) is a pass-1 deliverable. Go back to pass 1 with that specific question; if the intro still doesn’t yield one, the paper may genuinely be a “we tried X” report — note that and move on.

3Paper anatomy — where information lives

The question: for each section, what question does it answer — and what should you extract?

Technical — the section map

SectionQuestion it answersWhat you extract
AbstractWhy should I care?the claim, in the authors’ best words
IntroductionWhat problem, why now, what’s new?the contribution sentence(s) (§4) + the problem framing
Related workWhat came before?2–3 anchor papers to read later (add to the roadmap)
MethodHow does it work?the mechanism sketch + every hyperparameter you can find
ExperimentsDoes the evidence hold?baselines, datasets, ablations, the honest caveats (§6)
Conclusion / LimitationsWhat’s admitted?stated scope + limitations (§7)
AppendixCan I reproduce it?full hyperparameters, training details, extra results

Real anchors, from papers you already know: in “Attention Is All You Need” (Vaswani et al., 2017) the contribution is the architecture itself — recurrence replaced by attention, in Fig. 1 and §3; in “Language Models are Few-Shot Learners” (Brown et al., GPT-3, 2020) the contribution is an empirical claim about scale — carried almost entirely by tables and figures, not equations. Same anatomy, opposite payload — the section map tells you where to hunt in each.

Common misconception

“The method section is the heart of every paper.” Only for method papers. Analysis papers live in experiments; benchmark papers live in the dataset/spec sections. Ask pass 1’s category question first, then aim your effort at that section.

Knowledge check

Where do you look for the exact learning rate a paper used — and where do you look when it isn’t there? Answer: appendix / implementation details first; then the released code (config files — the ground truth); then footnote/baseline references. If it’s nowhere: log it as a reproducibility gap (§7) — that’s data, not failure.

4Finding the main contribution

The question: what sentence, in which paragraph, tells you what the paper adds?

Technical — the hunt

  • Where: the last one or two paragraphs of the introduction — usually opened by “In this paper, we…” / “Our contributions are…”. Secondary sources: the abstract’s third sentence, the first paragraph of the method.
  • Classify it: new architecture (Transformer) · new training recipe/objective (Chinchilla’s compute-optimal scaling, Hoffmann et al. 2022) · new capability claim (GPT-3 few-shot) · new analysis/benchmark · a survey. The class determines which section carries the burden of proof.
  • The one-sentence test: “Compared to [prior approach], this paper adds [mechanism/claim], shown by [evidence type].” If you can’t fill the blanks, you don’t understand the paper yet — and now you know precisely what to hunt in pass 2.

Common misconception

“Everything in the paper is the contribution.” A paper usually has one load-bearing idea; the rest is scaffolding, engineering, and incremental padding. Name the one idea before reading anything else twice.

Knowledge check

Fill the one-sentence test for the Transformer paper. Answer: “Compared to recurrent/convolutional sequence models, this paper replaces recurrence with attention entirely, shown by matching state-of-the-art translation quality with significantly less training time (parallelizable attention).” Any close variant is fine — the blanks force mechanism + evidence, which is the point.

5Reading equations, tables, figures

The question: the payload is in the math and the plots — how do you read them without drowning?

Technical — the extraction rules

  • Equations — identify before deriving: (1) what are the inputs and outputs, with shapes? (2) which symbols are learned parameters vs hyperparameters vs data? (3) what does each term contribute intuitively? The scaled dot-product attention formula is the example from your Transformers sheet: Q·Kᵀ scores compatibility, √dk stabilizes, softmax normalizes, ×V delivers content — four operations, no derivation needed to use it.
  • Tables — read headers first, then hunt deltas: what are rows (methods?) and columns (datasets/metrics?)? Which cell is “best” and what comparison supports the bolding? Compute the delta yourself: relative gain = (new − baseline) / baseline
  • Figures — ask what claim each one carries: architecture figures answer “how does it work”; loss/scaling curves answer “does the trend hold”; qualitative examples answer “what does it look like.” If a figure answers none of these for you, note it and move on.
  • Cross-check figures vs text: the text says “significant improvement” — the y-axis says +0.3. Papers sell; axes testify. Always read the axes.

Worked — a 60-second table read (illustrative numbers)

A paper reports accuracy 84.6 vs a baseline of 82.1. Two computations you should do on sight:

absolute gain = 84.6 − 82.1 = +2.5 points
relative gain = 2.5 / 82.1 ≈ +3.0%

Then the credibility questions: over how many runs (variance)? on which split? is 82.1 the tuned baseline or the authors’ re-run of someone else’s method? (§6.) These numbers are illustrative — the habit is what transfers.

Knowledge check

A scaling-law figure shows loss falling linearly with compute on a log-log plot. What single sentence should you write in your template? Answer: “Loss scales as a power law in compute (linear on log-log), across the measured range” — plus the honest boundary: within the measured range; extrapolation beyond the plotted region is the authors’ (and your) assumption, not data.

6Baselines, ablations, fair comparison — worked

The question: the main table looks great — how do you audit whether the comparison means anything?

Technical — the two audit tools

  • Baseline = “what existed before / what’s trivial.” You need both kinds: prior state-of-the-art (is this progress?) and trivial (majority class, a simple heuristic — is the task even hard?).
  • Ablation = remove one component at a time and re-measure. This is how a paper proves each piece matters, rather than asserting it. A model with five claimed innovations and no ablation table is asking for faith, not granting evidence.

The fair-comparison checklist: same data and split? same evaluation protocol? same compute/tuning budget? was the baseline tuned, or used out-of-the-box? multiple seeds with variance reported, or one lucky run?

ablation table, as bars (illustrative numbers) accuracy full model · 82.1 − component A · 78.4 − component B · 81.9 nothing removed Δ = −3.7 — A matters Δ = −0.2 — B is decoration? constructed teaching example — the reading habit is what transfers
The audit question the figure answers: which claimed components actually carry the result? Here A earns its keep; B (−0.2, within plausible run-to-run noise) does not — unless the paper shows variance bars proving otherwise.

Common misconception

“Ablation deltas are exact.” Every Δ has run-to-run variance of its own; −0.2 without error bars is indistinguishable from zero. When a paper reports single-seed numbers, mentally add “±unknown” to every delta — and treat tiny deltas as unproven.

Knowledge check

A paper beats a baseline by +2.5 points but used 4× the compute and a private training set. Which §6 checklist items does this trip, and what would a fair test look like? Answer: compute budget and data access are unequal — the comparison confounds method with resources. Fair test: baseline trained on the same data with comparable (or reported) compute, or the new method evaluated under the baseline’s budget.

7Limitations & the reproducibility check

The question: before you invest a weekend implementing this — will it survive contact with reality?

Technical — the pre-implementation audit

  • Read the limitations section first (top venues now encourage requiring one). Authors telling you where their method breaks is a credibility signal, not weakness — no limitations section on a methods paper is itself a finding.
  • The reproducibility scan (five checks): code released? data available (or at least the split documented)? hyperparameters complete (appendix/configs)? compute reported (how many GPUs for how long)? multiple seeds? Each missing item is a tax you’ll pay if you build on it.
  • Scope honesty: “improves on benchmark X with setting Y” ≠ “works everywhere.” Write the scope sentence into your template (§9) verbatim — future-you will otherwise over-generalize.
  • Replication signal: has anyone else reproduced/built on it? (Citations that use it, open issues on the repo, follow-up papers.) A flashy result with zero reuse after a year deserves suspicion.
Knowledge check

Paper A: strong results, no code, no seeds, no limitations section. Paper B: slightly weaker results, code, three seeds, candid limitations. Which do you build a month of work on, and why? Answer: B. Build-on-ability is a property of verification and iteration, not peak numbers — A’s numbers are unverifiable at your budget, and its limits are unknown unknowns. (Report both fairly in your notes; choose B for engineering.)

8From paper to code

The question: you understood it — what is the minimal first artifact to build?

Technical — the mapping and the ladder

The paper→repo mapping you already know from our GPT project:

Paper artifactBecomesIn our repo
architecture figuremodel code, layer by layermodel.py (attention → blocks → head)
hyperparameter tableconfig dataclass / CLI flagsGPTConfig, train.py args
training details (§appendix)optimizer/schedule settingsAdamW, warmup+cosine, clipping
main results tableeval script + your own honest tableloss curves, samples, Stage-1 metrics

The implementation ladder (climb one rung at a time):

  1. Core mechanism at toy scale — smallest runnable version (nanoGPT is Karpathy’s pass-3 of GPT-2; our repo is yours of the same idea). Goal: the mechanism works at all.
  2. Overfit a tiny batch — the sanity gate from our training notes: if you can’t drive loss ≈ 0 on 64 samples, nothing else is debugging.
  3. Match a published number you can afford — small dataset, small config; expect proximity, not equality (different hardware/library versions — Evaluation sheet §9’s limits).
  4. Extend — your own ablation (§6!) on the mechanism: change one thing, measure.

Common misconception

“Reproducing means matching the paper’s headline number.” Full reproduction often costs more compute than you have — and isn’t the learning goal. Matching the mechanism’s behavior at a scale you can afford, with honest notes on what differs, is the standard working definition.

Knowledge check

Your toy reimplementation trains fine but can’t match the paper’s small-dataset number. Two likely causes to check first? Answer: (1) config mismatch — a hidden hyperparameter in the paper’s appendix/code (schedule, warmup, batch) differing from yours; (2) preprocessing/ tokenization mismatch — a different data pipeline, not a different model. Both are §3-appendix and released-code questions before they’re modeling questions.

9The paper-reading template

The question: what does a finished read look like as an artifact you can revisit?

Paper reading note — [title] · [authors, year] · [arXiv id / venue]

  1. One-sentence contribution — “Compared to ___, adds ___, shown by ___.” (§4)
  2. Category — architecture / training recipe / capability claim / analysis / survey.
  3. Pass level reached — 1 / 2 / 3 (be honest; note the date).
  4. Mechanism in 3 steps — the method, your words, no quoted sentences.
  5. Evidence audit — baselines used (tuned?) · ablations present? · seeds/variance? · the one table you trust most and why. (§6)
  6. Key numbers extracted — deltas you computed, with absolute and relative. (§5)
  7. Reproducibility scan — code Y/N · data Y/N · hparams complete Y/N · compute reported Y/N · multi-seed Y/N. (§7)
  8. Scope & limitations — the authors’ stated limits + your own suspicion.
  9. Anchor papers — 2–3 references worth adding to the roadmap.
  10. Verdict & next action — “read more later” / “implement at toy scale (see log #___)” / “cite but don’t build on it.” Every read ends in a decision.

Common misconception

“Templates kill flow.” They store flow — the template is the difference between “I read that paper once” and a searchable, comparable card collection that compounds across a degree.

Knowledge check

Which template field catches the “vanishing read” failure all by itself? Answer: the verdict & next action (item 10) — a read that ends without a decision is indistinguishable, six weeks later, from a read that never happened.

10The experiment log

The question: when you do implement — how do you record settings and results so they’re comparable?

Technical — one line per run, forever

# experiments.jsonl — one JSON object per run (append-only, greppable)
{"run_id": "2025-01-15_a1b2c3d4",
 "paper": "attention-is-all-you-need",
 "hypothesis": "removing the scaling factor destabilizes softmax at d=512",
 "code_sha": "a1b2c3d4",
 "config_hash": "9f8e7d...",
 "seed": 1337,
 "metrics": {"val_loss": 2.31, "steps_to_diverge": 140},
 "verdict": "hypothesis supported — divergence without scaling",
 "next_action": "repeat with 2 more seeds before claiming"}
  • Why JSONL: append-only (history is never edited), one run per line (diff-able, script-readable), greppable with the tools you already have.
  • The five pins ride along (Evaluation sheet §9): code SHA, config hash, seed, plus the hypothesis written before the result — pre-registration at desk scale (Evaluation §11’s principle, one run at a time).
  • The habit that matters: hypothesis and verdict as separate fields — you wrote what you expected first, and the log preserves whether reality agreed. That’s how “I tried some things” becomes evidence.
Knowledge check

Your log shows three runs of the same config with val_loss 2.31, 2.28, 2.35. What does the existence of the three runs buy you that one run doesn’t? Answer: a variance estimate — ±0.035 of run-to-run noise — which is exactly the instrument needed to judge whether a future “improvement” to 2.25 means anything (compare against §6’s same lesson at paper scale).

11From reading to portfolio

The question: how does a paper become something a recruiter can see?

Technical — the five contribution levels

  1. Reading notes — filled §9 templates, published (repo/blog). Demonstrates judgment.
  2. Annotated walkthrough — a paper explained with your diagrams (the sketch sheets in this Lab are exactly this for the Transformer and RAG literature).
  3. Toy reimplementation — runnable code + honest comparison notes (§8’s ladder).
  4. Extension/ablation — your own experiment on the mechanism, logged (§10). Demonstrates research behavior, not just consumption.
  5. Application — the idea transferred to a domain you own (e.g., retrieval ideas applied to a RAG system — Chapter 4’s territory).

Honest framing (per this Lab’s rules): label each artifact by level — “toy-scale reimplementation of X’s mechanism” is a strong, true portfolio line; “reproduced X” is not, unless you matched published numbers at published scale. The Lab’s provenance footers model the wording.

Knowledge check

You have one weekend and want a portfolio artifact from a paper. Which level maximizes demonstrable skill per hour? Answer: usually level 2–3: an annotated walkthrough with a working toy mechanism beats a half-finished level-4 study — and level 1 notes alone are the floor, not the ceiling. Choose by what you can finish and defend.

12Quiz

Attempt each before expanding.

Q1 — What are the outputs of passes 1, 2, 3 — and at which decision points can you stop?

Pass 1 → triage sentence (stop if irrelevant). Pass 2 → filled template + evidence verdict (stop unless it anchors your area). Pass 3 → implementation notes. Each handoff is a deliberate continue/stop decision. (§2)

Q2 — Where do you hunt the contribution sentence, and what test verifies you found it?

Last paragraphs of the introduction (“In this paper, we…”). Test: fill “compared to ___, adds ___, shown by ___.” (§4)

Q3 — Full model 82.1; minus component B: 81.9. What do you conclude — and what extra evidence would you demand?

Tentatively: B contributes ≈ nothing (Δ −0.2, plausibly noise). Demand: multi-seed error bars on the ablation before declaring B decorative. (§6)

Q4 — +2.5 points on an 82.1 baseline: relative gain?

2.5/82.1 ≈ +3.0%. Always compute both — absolute points and relative % — before judging significance. (§5)

Q5 — Name three of the five reproducibility scan checks.

Any three of: code released, data/split available, hyperparameters complete, compute reported, multiple seeds. (§7)

Q6 — Why are hypothesis and verdict separate fields in the experiment log?

Pre-registration at run scale: the log preserves what you expected before seeing the result, preventing after-the-fact rationalization. (§10; Evaluation sheet §11)

13Exercise — first real pass + the log stamper

Implementation challenge

Two deliverables. A: run a real pass-1/pass-2 on the next paper in your roadmap (time-boxed: 10 min + 45 min), filling the §9 template. B: build the experiment-log stamper below — the tool that makes every future implementation run reproducible by default (Evaluation §9’s five pins, automated).

# start_experiment.py — pure Python; stamps a log entry with the repro pins
import json, hashlib, subprocess, datetime, pathlib, uuid

LOG = pathlib.Path("experiments.jsonl")
REQUIRED = ["run_id", "date", "paper", "hypothesis", "code_sha",
            "config_hash", "seed", "metrics", "verdict", "next_action"]

def git_sha():
    """Pin 1: code version. Degrades honestly outside a repo."""
    try:
        return subprocess.check_output(
            ["git", "rev-parse", "HEAD"], text=True, stderr=subprocess.DEVNULL).strip()[:8]
    except Exception:
        return "no-git"

def file_hash(path):
    """Pin 2: data/config version — sha256 of any file (or a marker)."""
    p = pathlib.Path(path)
    if not p.exists():
        p.write_text("# config placeholder")   # demo: create then hash
    return hashlib.sha256(p.read_bytes()).hexdigest()[:16]

def stamp(paper, hypothesis, seed=1337, metrics=None,
         verdict="pending", next_action="run + fill metrics"):
    entry = {
        "run_id": str(uuid.uuid4())[:8],
        "date": datetime.datetime.now().isoformat(timespec="seconds"),
        "paper": paper,
        "hypothesis": hypothesis,          # written BEFORE the result — §10
        "code_sha": git_sha(),              # pin 1
        "config_hash": file_hash("config.json"),   # pin 2
        "seed": seed,                        # pin 4
        "metrics": metrics or {},
        "verdict": verdict,
        "next_action": next_action,
    }
    missing = [k for k in REQUIRED if k not in entry]
    assert not missing, f"entry missing pins: {missing}"
    with LOG.open("a") as f:
        f.write(json.dumps(entry) + "\n")
    return entry

e1 = stamp("attention-is-all-you-need",
           "dropping the sqrt(d_k) scale makes softmax saturate at d=512")
e2 = stamp("attention-is-all-you-need",
           "same config, new seed → val_loss within ±0.05", seed=42)

# --- asserts: the log is a real, parseable ledger ------------------------------
lines = LOG.read_text().strip().splitlines()
assert len(lines) >= 2
loaded = [json.loads(l) for l in lines]
assert all(set(REQUIRED) <= set(e) for e in loaded)      # every run carries every pin
assert loaded[0]["seed"] != loaded[-1]["seed"]              # two seeds = variance plan
assert e1["config_hash"] == e2["config_hash"]               # same config → same hash
print(f"log OK — {len(loaded)} runs, pins present, hypotheses pre-registered")

Deliverables: (1) the script runs and all asserts pass; (2) your filled §9 template for the roadmap paper, saved next to the log; (3) one paragraph: which assert encodes the “two seeds = variance plan” habit, and what Evaluation §9 says about the honest limits of that plan? Stretch: add a --link field that references a prior run_id, so an ablation chain (full → −A → −B) is navigable in the log — the §6 figure, made queryable.

14References

Primary sources, verified at writing time. Paper characterizations summarize the papers’ own framing; no benchmark scores are quoted. The §5/§6 tables use constructed illustrative numbers, labeled as such.

Progress checklist

Ticks persist in this browser only (localStorage).

Provenance: structured sheet from Batch 6 of the AI Learning Lab project — a personal study resource, not a claim of expertise. Companion: your original LLM Research Roadmap (standalone file, preserved as-is). Verify specifics against the references before use in graded work.

Last reviewed: not yet — set this date after your first full pass.

Web Development Fundamentals · sheet 2 of 2 in this chapter

Web Concepts for AI Engineers

Web Development Fundamentals Beginner → Intermediate structured sheet · Batch 6 prerequisite: AI Engineering (Stage 5, recommended)

1The request lifecycle

The question: you type a URL and press Enter — what happens, in order, before pixels appear?
browserclient / frontend 1 · DNS DNSname → IP 2 · TLS load balancerTLS ends here (§2) 3 · HTTP application server validate · business logic maybe the AI call (§12) Stage 5’s FastAPI lives here cachefast answers for repeat asks (§8) databasedurable state (§8) private network 4 · response (status + headers + JSON/HTML) 5 · browser renders — or your frontend code parses the JSON 6 · cache check next time: 304 Not Modified can skip the body (§8)
The full journey of one request. Every section of this sheet attaches to a numbered step — the lifecycle is the sheet’s spine.

Intuition

Every web interaction is the same shape: a request (method + URL + headers + optional body) traveling to a server, and a response (status code + headers + optional body) traveling back. Master this shape and every protocol discussion becomes vocabulary.

Knowledge check

Which lifecycle steps happen before your server code runs at all? Answer: DNS resolution, the TCP connection, and the TLS handshake (plus the load balancer’s health-routing decision). These explain why “empty” requests still cost milliseconds — and why connection reuse matters.

2HTTP & HTTPS

The question: what exactly does the “S” add — and what does it not fix?

Technical

  • HTTP — a text-based request/response protocol: METHOD /path HTTP/1.1, headers (key: value), a blank line, an optional body. Stateless by design: each request carries everything needed — which is why state must live somewhere (headers, cookies, tokens, a database).
  • HTTPS = HTTP inside TLS: the client and server negotiate encryption before any application data flows; the server presents a certificate (signed by a certificate authority) proving it owns the domain. Guarantees: encryption (nobody on the wire reads it), integrity (nobody modifies it undetected), authenticity (the server is who the certificate says).
  • What TLS does not fix: a malicious server, bad application logic, leaked secrets, XSS in your frontend. HTTPS protects the road, not the destination.
  • Where TLS ends: at the load balancer in Stage 5 §9’s architecture — internal traffic then rides the private VPC. That’s the deliberate design: one encrypted public door, private interior.

Common misconception

“HTTPS means the site is safe/trustworthy.” It means the connection is encrypted and the domain owns a certificate. Scam sites use HTTPS too. Security is layers; TLS is one.

Knowledge check

Why is sending an API key over plain HTTP burned even if “it’s just a test”? Answer: HTTP is plaintext — any network hop (coffee-shop Wi-Fi, ISP, proxy) can read the header. Keys are credentials regardless of environment; and the habit, not the endpoint, is what leaks in production.

3Methods — and idempotency

The question: GET vs POST everyone knows — what’s the property that actually matters at 3 a.m.?

Technical — the methods and their safety contracts

MethodIntentSafe?Idempotent?
GETread — never changes server stateyesyes
POSTcreate / process / triggernono
PUTreplace a resource entirelynoyes
PATCHpartial updatenonot guaranteed
DELETEremovenoyes (deleting twice = same state)
  • Safe = read-only (GET/HEAD/OPTIONS). Idempotent = doing it N times leaves the same state as once. POST is the odd one out — which is exactly why the Agents sheet’s retry rules (§8) and payment idempotency keys exist.
  • Consequences in code: GETs can be cached, pre-fetched, retried blindly. POSTs need dedupe keys or careful retry policy. Routers map methods to handlers; using POST for reads (or GET for writes) breaks caches and expectations.
Knowledge check

A client retries a timed-out PUT /v1/prefs/42. Harmless. It retries a timed-out POST /v1/orders. Harmful. Explain via the table. Answer: PUT is idempotent — the retry re-writes the same state. POST is not — each retry can create another order. Fix at the API design level: idempotency keys for POSTs that must be retry-safe (Agents sheet §8).

4Status codes — who fixes what

The question: the API returned 401 — is that your bug or the caller’s?

Technical — the working set (real codes, official semantics)

CodeMeaningWho fixes it
200 / 201 / 204OK / created / success-no-body—
301 / 302moved / redirectserver owner (links)
304not modified — cache is fresh (§8)— (it’s a success!)
400 / 422bad request / unprocessable (schema)caller
401unauthenticated — “who are you?”caller (credentials)
403unauthorized — “you can’t, I know who you are”permissions design
404not foundcaller (URL) or server (routing)
429too many requests — slow downcaller (backoff)
500 / 502 / 503 / 504server error / bad gateway / unavailable / gateway timeoutserver owner

The triage rule: 4xx = the request was wrong (fix the client or the docs); 5xx = the server broke its promise (fix the server, page someone). Stage 5 §5 adds the discipline: never 200-with-error-body — monitoring reads statuses, not vibes.

Knowledge check

401 vs 403 in one pair of sentences — and why do login flows return 401 (not 403) for a bad password? Answer: 401 = unauthenticated (identity unknown/invalid; “try authenticating”); 403 = authenticated but not permitted (identity known; “don’t ask again”). A bad password means identity was never established → 401.

5JSON & REST — a worked exchange

The question: what does “REST API” actually constrain, request by request?

Technical

  • JSON — the exchange format: objects, arrays, strings, numbers, booleans, null. One gotcha that reaches AI work: no native float32, no dates — NumPy types must become native before serialization (Stage 5 §5’s check), and timestamps are strings (ISO 8601 by convention).
  • REST conventions: URLs name resources (nouns): /v1/predictions, /v1/models/7; methods express the verb; status codes express the outcome; requests are stateless. The API contract = methods × routes × schemas × statuses.

Worked — one full exchange against an inference endpoint

# request
POST /v1/predict HTTP/1.1
Host: api.example.com
Content-Type: application/json
Content-Length: 34

{"text": "The model works great"}

# response
HTTP/1.1 200 OK
Content-Type: application/json
X-Request-Id: 7f3a-42

{"label": "positive", "confidence": 0.94}

Read it like a contract: the method says “process”; the body carries validated JSON; the status carries the verdict; the headers carry the metadata (the request ID that stitches logs — Stage 5 §12). Every AI API you’ll ever build is this exchange with a different body schema.

Knowledge check

Why does Content-Type: application/json matter on the request when the server “could figure it out”? Answer: the server parses based on it — missing/wrong content type means the body arrives as opaque bytes (422 or 400 from the parser). It also signals intent for CORS (§10: JSON POSTs are exactly what triggers a preflight).

6Frontend vs backend

The question: where does each responsibility live — and what’s the only thing connecting them?
FRONTEND runs in the user’s browser UI · forms · charts · state of the view React/Next.js — or plain HTML code the user can read & modify BACKEND runs on your servers validation · auth · business rules AI inference · DB access (Chapters 4–5) the only place secrets may live THE CONTRACT HTTP + JSON + status codes everything crossing this line is visible, forgeable, and must be re-validated server-side — always
Two territories, one contract. The two red lines are the exam answers: frontend code is public; the network boundary is hostile.

Technical — the two laws

  1. Never trust the client. Frontend validation is UX; server validation is security (Stage 5 §5). Anything arriving over HTTP is attacker-controlled until validated.
  2. Secrets never cross the line. API keys live in backend env vars (§9). Anything in frontend code/bundles is public the moment you deploy.
Knowledge check

A teammate hides an OpenAI key in a React component “because it’s just the demo.” Why does the demo key leak, precisely? Answer: frontend code ships to every visitor — the key is in the JS bundle, readable via view-source/DevTools/network tab, and scraped by bots that crawl bundles for key patterns. The pattern fix: the frontend calls your backend, the backend holds the key and proxies the call.

7Authentication vs authorization — JWT worked

The question: how does a stateless server know who you are on request #1,000?

Technical — the vocabulary and the token

  • Authentication (AuthN): who are you? — password check, OAuth login, API key.
  • Authorization (AuthZ): what may you do? — roles, scopes, per-resource rules (the RAG sheet’s ACL-at-retrieval is AuthZ).
  • Sessions vs tokens: classic sessions store state server-side (cookie → session store). JWT (JSON Web Token, RFC 7519) carries identity inside the token — stateless verification, ideal for APIs.

Worked — the three parts of a JWT

eyJhbGciOiJIUzI1NiJ9 . eyJzdWIiOiJ1c2VyLTQyIn0 . 3Kt9x…
      ↑                        ↑                        ↑
   header (base64url)    payload (base64url)      signature (HMAC)

header  → {"alg": "HS256", "typ": "JWT"}
payload → {"sub": "user-42", "role": "demo", "exp": 1735689600}
signature → HMAC-SHA256(header + "." + payload, server_secret)

The two properties that matter:

  1. Signed, not encrypted. Anyone can decode the payload (it’s base64url — an encoding, not crypto). Never put secrets in a JWT payload. What the signature buys: the server can verify the token was issued by it and untouched since.
  2. Self-contained & expiring. The server checks the signature and exp — no database hit. The trade: you can’t instantly revoke a valid JWT; you keep expiry short and refresh, or keep a denylist.

Ties together: Stage 5 §5 (401 vs 403 now mechanical: missing/bad token → 401; valid token, wrong scope → 403) and §8 (the HMAC secret lives only in the backend).

Common misconception

“JWTs are encrypted.” The most common token misunderstanding in interviews. Base64url decodes with atob() in a browser console — paste one and read it. The signature prevents forgery, not reading.

Knowledge check

A user edits their JWT payload to "role": "admin" and calls your API. What happens, mechanically? Answer: the signature check fails — the payload no longer matches the HMAC the server issued. The server rejects with 401. Change the payload without the secret → invalid signature; every time.

8Databases & caching — the 304 flow

The question: the same data gets requested 10,000 times an hour — what remembers, and what asks again?

Technical — durable state vs fast answers

  • Database — the durable source of truth (Stage 5 §10’s RDS/pgvector). Every write lands here; consistency rules apply.
  • Cache layers, fastest first: browser cache → CDN (edge) → application cache (Redis-style, keyed lookups) → DB. Each layer answers “do I already have this?” so the expensive layer (DB, or a model call!) is spared.
  • HTTP caching — the two headers that run the web: Cache-Control: max-age=3600 (“this response is fresh for an hour”) and validators: the server stamps ETag: "v7"; the client replays it with If-None-Match: "v7"; unchanged content → 304 Not Modified — no body, just the cheap answer.

Worked — the 304 round trip

# first request
GET /v1/models/7  →  200 OK
ETag: "v7"
(body: 2 KB of model metadata)

# same client, later — sends its copy's stamp
GET /v1/models/7
If-None-Match: "v7"
  →  304 Not Modified      # zero-body response: ~2 KB saved, DB untouched

# after the model was updated server-side (ETag now "v8")
GET /v1/models/7 + If-None-Match: "v7"  →  200 OK, ETag: "v8", fresh body

The cache mindset for AI systems: the same logic applies above your model — identical prompt + same model version ⇒ identical answer is cacheable (app-level, keyed by input hash). That’s why deterministic temperature-0 endpoints are cheap to scale and sampled ones are not.

Common misconception

“Caching is an optimization bolted on later.” It’s a correctness question first: what may go stale, for how long, per resource? A wrongly cached personalized response (one user’s data served to another) is a security bug, not a speed bump — cache keys must include identity where it matters.

Knowledge check

Your API serves user-specific dashboards and you add a CDN with default settings. What can go wrong, and what’s the minimal correct configuration? Answer: the CDN may serve user A’s authenticated response to user B (shared cache + personalized body). Minimal fix: mark those routes Cache-Control: private, no-store (or key the cache on identity) — caching policy is a per-resource decision, never a global default.

9Environment variables & secrets

The question: the same code runs in dev, staging, and prod — what’s allowed to differ, and where does it live?

Technical — compressed from Stage 5 §8

  • Config via environment: DB URLs, model URIs, log levels — read from env vars (os.environ / settings.py), one image, many environments.
  • Secrets hierarchy: local .env (git-ignored) → platform secret store → never in code, never in the repo, never in frontend bundles (§6’s law).
  • Fail fast: required vars crash at boot, not on request 4,000 (Stage 5 §8’s os.environ[...] vs .get() distinction).
The full treatment — rotation, least-privilege pairing with IAM, the burned-key-in-git-history scenario — lives in Stage 5 §8. This sheet’s unique addition: the frontend rule — anything that reaches the browser bundle is public (§6), which is why AI keys always sit behind your own backend proxy.
Knowledge check

Your static site calls an LLM API directly from the browser with a “restricted” key. Name two independent failures. Answer: (1) the key is public — bundle inspection reveals it regardless of “restrictions”; (2) you can’t enforce per-user authZ or rate limits — every visitor spends your quota directly. Fix: thin backend proxy holding the key.

10CORS — the preflight, worked

The question: your frontend at localhost:3000 calls your API at localhost:8000 — the request works in curl but the browser blocks it. Why?

Technical — same-origin and its escape hatch

  • The browser’s same-origin policy: scripts may read responses only from the same origin (scheme + host + port all equal). localhost:3000 vs localhost:8000 = different origins, full stop.
  • CORS (Cross-Origin Resource Sharing) is the server-side opt-out: the API declares which origins may read its responses via Access-Control-Allow-Origin. It’s enforced by the browser; curl doesn’t care — which is exactly why APIs “work in Postman but fail in the app.”
  • Simple vs preflighted requests: GET/HEAD and simple POSTs (form-encoded) go straight through; a JSON POST (Content-Type: application/json) is not simple — the browser first sends an OPTIONS preflight and demands permission.
browserlocalhost:3000 APIlocalhost:8000 · FastAPI 1 · OPTIONS preflight · Origin: localhost:3000 · Access-Control-Request-Method: POST 2 · Access-Control-Allow-Origin: http://localhost:3000 3 · permission granted → the REAL request flies: POST /v1/predict (JSON) 4 · response readable by frontend code — contract honored if step 2 is missing/mismatched → browser blocks the RESPONSE (the server may still have executed it!)
CORS in four messages. The subtle part: a missing CORS header doesn’t stop execution — it stops readability. Your POST ran; the frontend just isn’t allowed to see the answer.

Worked — the FastAPI fix

from fastapi.middleware.cors import CORSMiddleware

app.add_middleware(
    CORSMiddleware,
    allow_origins=["http://localhost:3000"],   # exact origins — never "*"
    allow_methods=["GET", "POST"],
    allow_headers=["Content-Type", "Authorization"],
)

The fix lives on the server — the #1 beginner misdiagnosis is hunting the frontend for a bug that only the API can grant. And never ship allow_origins=["*"] with credentials: that’s “any website may call your API as its visitors,” which is how a CSRF-adjacent hole opens.

Common misconception

“CORS is a security feature that protects my API.” Inverted: CORS loosens the browser’s protection for chosen origins; it protects users from other sites reading your responses. Disabling it entirely doesn’t make your API safe — it makes every website able to read it (from browsers).

Knowledge check

Dev tools show the preflight OPTIONS returning 200 with the right ACAO header, yet the real POST still fails. First thing to inspect? Answer: the real response’s headers — ACAO must also be present on the actual response, and credentials/headers lists must cover what the request sends (e.g., Authorization must be in Access-Control-Allow-Headers). Preflight passing ≠ request passing; they’re checked separately.

11API testing

The question: the endpoint “works” — how do you prove it, cheaply and repeatedly?

Technical — the layered habit

  • Manual exploration: curl / httpie / Postman — curl -X POST localhost:8000/v1/predict -H "Content-Type: application/json" -d '{"text":"hi"}'. Great for the first look, worthless as a safety net.
  • Contract tests in CI: FastAPI’s TestClient — assert status codes and response schemas (Stage 5 §15’s four tests are the template). The contract is status + schema, so that’s what the tests pin.
  • Negative tests first: empty body, wrong types, oversize input, missing auth — the 4xx half of the contract is where APIs actually break.
  • Public playgrounds: httpbin.org echoes your requests — perfect for seeing exactly what headers/bodies you really send while learning.
Knowledge check

Your test suite only checks 200 paths. Name the bug class it will miss, with a concrete example. Answer: error-path regressions: e.g., a dependency upgrade makes validation return 500 (stack trace) instead of 422 — users see server errors, monitoring pages on 5xx, and no 200-test ever noticed. Every schema rule deserves a failing-input test.

12Exposing an AI model through an API

The question: everything in this sheet, pointed at one goal — your model, reachable, safely.

Technical — the pattern, assembled

An AI endpoint is a normal REST endpoint whose handler calls a model (Stage 5 §4’s FastAPI core). What’s special about AI endpoints:

  • Schema = the product: the request schema (what the model accepts), response schema (label + confidence + model_version — always ship the version so clients can react, Stage 5 §13).
  • Latency is part of the contract: state the budget; stream when answers are long — token-by-token via SSE (text/event-stream: chunks sent as generated, the chat-typing effect) instead of one blocked 200 (LLM Fundamentals §2’s per-token loop, exposed).
  • Statelessness & idempotency: inference is a pure function of (input, model version) — cacheable at temperature 0 (§8), retryable with idempotency keys for POSTs (§3, Agents sheet §8).
  • Security stack, all of it: auth on every endpoint (§7), rate limits (429 + backoff), input validation before any model compute (Stage 5 §5), secrets behind the backend (§9), CORS pinned to your frontend origin (§10), and never trusting client data — including text destined for a prompt (Agents sheet §9’s injection warning applies to your API’s callers too).

Worked — the whole path, two lines of shell

# the client you'll write in the exercise — everything above in one exchange
curl -X POST http://localhost:8000/v1/predict \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer <jwt>" \
     -d '{"text": "The cat is sleeping"}'

# → 200 {"label": "...", "confidence": 0.94, "model_version": "1.2.0"}
# → 422 if the body violates the schema   → 401 if the JWT is bad
# → 429 if you hammer it                  → 503 while the model loads (Stage 5 readyz)

Every status code in that comment block was earned in an earlier section — that’s the sheet closing its own loop.

Knowledge check

Why include model_version in every response? Answer: clients and logs can attribute behavior to a version — debugging “it changed!” becomes a version comparison instead of archaeology; and canary rollouts (Stage 5 §13) become observable from the client side.

13Quiz

Attempt each before expanding.

Q1 — Which lifecycle steps run before your application code, and why do they make “simple” requests cost milliseconds?

DNS, TCP connect, TLS handshake (plus LB health routing). Each is a network round-trip before any bytes of your request arrive — hence keep-alive/connection reuse. (§1–2)

Q2 — A timed-out POST /orders retried automatically double-charged a customer. Which property is missing and what’s the API-side fix?

Idempotency. POST isn’t idempotent; the fix is idempotency keys (client-generated, server deduped) or classifying the operation non-retryable. (§3)

Q3 — A valid JWT with "role":"user" is hand-edited to "admin". What exactly fails?

The HMAC signature — recomputing it requires the server secret. Server rejects with 401. Payload readable ≠ payload editable. (§7)

Q4 — ETag "v7" is served; client sends If-None-Match: "v7"; content unchanged. Status and body?

304 Not Modified, empty body — the client keeps its cached copy. Fresh content would flip it to 200 with a new ETag. (§8)

Q5 — curl succeeds, browser fetch fails on a JSON POST. Who fixes it and how?

The server (API owner): CORS is granted server-side — add the origin to allow_origins (FastAPI CORSMiddleware), covering the preflight and the real response. (§10)

Q6 — Why must an AI endpoint validate input before the model and include model_version in responses?

Validation: the model is the most expensive component — garbage must die at the cheap boundary (and unvalidated text is an injection surface, Agents sheet §9). Version: behavior attribution and canary observability. (§12)

14Exercise — client + CORS, fully offline

Implementation challenge

Extend Stage 5 §15’s mini API: add CORS middleware, then write tests that prove the preflight contract and build a tiny client with retries that respect status codes. Everything runs offline with pytest — the browser’s rules, enforced in tests.

# pip install fastapi pytest httpx      (extends the Stage-5 exercise app)
# app.py — add to the existing FastAPI app:
from fastapi.middleware.cors import CORSMiddleware

app.add_middleware(
    CORSMiddleware,
    allow_origins=["http://localhost:3000"],   # §10: exact, never "*"
    allow_methods=["GET", "POST"],
    allow_headers=["Content-Type"],
)

# test_web.py — the browser's contract, as pytest
from fastapi.testclient import TestClient
from app import app
c = TestClient(app)

FRONT = {"Origin": "http://localhost:3000",
         "Access-Control-Request-Method": "POST",
         "Access-Control-Request-Headers": "content-type"}

def test_preflight_granted_for_allowed_origin():
    r = c.options("/v1/predict", headers=FRONT)
    assert r.status_code == 200
    assert r.headers["access-control-allow-origin"] == "http://localhost:3000"

def test_preflight_denied_for_stranger():
    evil = dict(FRONT, Origin="https://evil.example")
    r = c.options("/v1/predict", headers=evil)
    assert "access-control-allow-origin" not in r.headers \
        or r.headers["access-control-allow-origin"] != "https://evil.example"

def test_real_request_carries_cors_headers():
    r = c.post("/v1/predict", json={"text": "hello"},
               headers={"Origin": "http://localhost:3000"})
    assert r.status_code == 200
    assert r.headers["access-control-allow-origin"] == "http://localhost:3000"

# client.py — a caller that respects the §4 triage rule
import httpx, time

def predict(base, text, retries=2):
    for attempt in range(retries + 1):
        r = httpx.post(f"{base}/v1/predict", json={"text": text}, timeout=5)
        if r.status_code == 200:
            return r.json()
        if r.status_code == 429:                     # §4: slow down, politely
            time.sleep(0.5 * (2 ** attempt)); continue
        if 400 <= r.status_code < 500:               # §4: caller bug — retry is pointless
            raise ValueError(f"client error {r.status_code}: {r.text}")
        if attempt == retries:
            raise RuntimeError(f"server error {r.status_code}")   # 5xx: last attempt
        time.sleep(0.5)

Deliverables: (1) all three CORS tests pass, and the stranger-origin preflight is demonstrably blocked; (2) the retry client: one test proving 429 → backoff → success (use a counter endpoint or monkeypatched responses); (3) one paragraph: why the client treats 4xx and 5xx differently — connect it to §4’s “who fixes it” column and the Agents sheet §8’s idempotency rule. Stretch: add an /v1/predict-stream SSE endpoint that yields three chunks, and read it with httpx.stream() — you’ve now served a model the way chat UIs consume one (§12).

15References

Official specs and docs, verified at writing time. The JWT example uses illustrative values; status-code semantics follow RFC 9110 / MDN exactly.

Progress checklist

Ticks persist in this browser only (localStorage).

Provenance: structured sheet from Batch 6 of the AI Learning Lab project — a personal study resource, not a claim of expertise. Companion: your original Web Concepts document (standalone file, preserved as-is). Verify specifics against the references before use in graded work.

Last reviewed: not yet — set this date after your first full pass.

The source material is preserved in full. Open the complete original HTML collection.