90-Day Plan | behavior.engineering

Guiding Principles

One domain at a time. Each week has a primary focus. Resist the urge to scatter.
Build artifacts, not just knowledge. Every phase produces something portfolio-ready.
Test against target job descriptions. Identify 3–5 target job descriptions for Model Behavior Architect / Engineer roles at AI labs and AI-product companies. Return to them regularly and audit your gaps against the specific skills they name.
Prefer primary sources. Papers, documentation, and model outputs over summaries.

Phase 1 — Foundations (Days 1–30)

Theme: Build the conceptual scaffolding. Everything in Phase 2 and 3 depends on having this solid.

Week 1: The Model Behavior Design Landscape

Goal: Develop a precise mental model of what this role actually does and doesn’t do.

Daily focus areas:

Day 1–2: Collect 4–6 current job descriptions for Model Behavior Architect / Engineer roles (Anthropic, OpenAI, Perplexity, Mistral, and Notion have all posted versions of this role). Read and annotate them with a single question in mind: What does a person in this role do on a Tuesday afternoon? Extract verbs. Group them into clusters (evaluate, prompt, pipeline, collaborate, document).
Day 3–4: Read Anthropic’s Constitutional AI paper (Bai et al., 2022) and the Collective Constitutional AI article. Map how they connect to the language in target job descriptions around “alignment finetuning,” “honesty,” and “character.”
Day 5–6: Read OpenAI’s InstructGPT paper (Ouyang et al., 2022). Understand RLHF end-to-end: SFT → reward model → PPO. Diagram it by hand.
Day 7: Write a 500-word personal definition of model behavior design that distinguishes it from: prompt engineering as a service, UX design, annotation work, and agent engineering. This becomes a portfolio artifact.

Deliverable: Written role definition document (portfolio artifact #1).

Week 2: Alignment Techniques in Depth

Goal: Move from familiarity to fluency on the core training-side techniques referenced in target job descriptions.

Daily focus areas:

Day 8–9: RLHF deep dive. Understand reward modeling, preference data collection, and where human judgment enters the pipeline. Read the Learning from Human Preferences paper (Christiano et al., 2017).
Day 10–11: Constitutional AI and RLAIF. Understand how AI feedback replaces human annotation at scale. Study the critique-revision loop. Reference the CAI paper and Anthropic’s explainer.
Day 12–13: Understand Direct Preference Optimization (DPO) as an alternative to PPO-based RLHF. Read the DPO paper (Rafailov et al., 2023). Compare tradeoffs.
Day 14: Write a comparative reference document: RLHF vs. CAI vs. DPO. When is each used? What does a model behavior designer contribute to each? This is portfolio artifact #2.

Deliverable: Alignment techniques comparison doc (portfolio artifact #2).

Week 3: Evaluation Methodology

Goal: Develop a rigorous, systematic approach to model evaluation — the single most cited skill across model behavior job descriptions.

Daily focus areas:

Day 15–16: Study the HELM benchmark paper (Liang et al., 2022) and BIG-Bench (Srivastava et al., 2022) to understand how large-scale evaluation frameworks are structured. What dimensions do they measure? What do they miss?
Day 17–18: Study preference-based evaluation. Read the Chatbot Arena paper (Chiang et al., 2024). Understand Elo/Bradley-Terry ranking. Explore the Arena leaderboard directly.
Day 19–20: Study LLM-as-judge methodology. Read “Judging LLM-as-a-Judge” (Zheng et al., 2023). Understand bias types: position bias, verbosity bias, self-enhancement bias.
Day 21: Design your first evaluation framework from scratch for a specific behavioral domain (e.g., honesty, refusal calibration, or instruction-following). Define: what does “good” look like? How do you measure it? What are the failure modes? This is portfolio artifact #3.

Deliverable: Evaluation framework design doc for one behavioral domain (portfolio artifact #3).

Week 4: Prompt Engineering and Context Engineering

Goal: Distinguish expert-level prompt engineering from basic usage, and understand “context engineering” as a system-level discipline.

Daily focus areas:

Day 22–23: Study chain-of-thought prompting (Wei et al., 2022), few-shot prompting mechanics, and role/persona prompting. Understand why these work, not just that they do.
Day 24–25: Read Anthropic’s prompt engineering documentation in full. Study their system prompt structure guidance, XML tag conventions, and the reasoning behind them.
Day 26: Understand the distinction between prompt engineering (single-turn optimization) and context engineering (system-level, multi-surface, production-grade). This distinction maps directly to roles at product-focused AI companies like Notion and Perplexity.
Day 27–28: Hands-on in Anthropic Console. Run structured experiments: vary system prompts systematically across 5–10 dimensions for a single behavioral target. Document results.
Day 29–30: Write a context engineering case study from your Console experiments. Include: the behavioral target, system prompt variants, observed outputs, and what you changed and why. Portfolio artifact #4.

Deliverable: Context engineering case study (portfolio artifact #4).

Phase 2 — Applied Skills (Days 31–60)

Theme: Move from theory to tooling. Build the pipeline fluency that model behavior roles assume.

Week 5: Annotation and Preference Data

Goal: Understand how training data for behavior fine-tuning is actually built.

Daily focus areas:

Day 31–32: Study how annotation guidelines are written for alignment tasks. Read Anthropic’s model card and any public annotation rubrics from AI labs. Understand what annotators are trained to do.
Day 33–34: Set up Label Studio locally. Create a simple annotation project for a behavioral dimension (e.g., helpfulness vs. harmlessness tradeoff). Annotate 20–30 examples yourself.
Day 35–36: Study inter-annotator agreement metrics (Cohen’s kappa, Krippendorff’s alpha). Understand why consistency matters and where it breaks down on subjective tasks.
Day 37: Write annotation guidelines for one behavioral category from scratch. Model them on published rubrics but make them your own. Portfolio artifact #5.

Deliverable: Annotation guidelines document (portfolio artifact #5).

Week 6: Evaluation Tooling

Goal: Gain hands-on proficiency with the eval tooling stack.

Daily focus areas:

Day 38–39: Set up and run Promptfoo. Build a basic eval suite for a prompt you care about. Understand how it scores outputs and what test case formats it supports.
Day 40–41: Explore ChainForge or Freeplay for comparative prompt testing. Run the same behavioral target across 3 prompt variants. Document results.
Day 42–43: Read Braintrust documentation. Understand how it structures datasets, evaluators, and experiment tracking. Compare its workflow to Promptfoo.
Day 44: Build a structured eval suite with at least 30 test cases across 3 failure mode categories for a behavioral domain of your choice. Portfolio artifact #6.

Deliverable: Eval suite with documented methodology (portfolio artifact #6).

Week 7: Edge Case and Failure Mode Analysis

Goal: Develop a systematic method for finding where models break — a core competency in every model behavior role.

Daily focus areas:

Day 45–46: Study adversarial prompting: jailbreaking methods, prompt injection, and role-play boundary exploitation. Understand these as diagnostic tools, not exploits.
Day 47–48: Build a category coverage matrix for a behavioral domain. Map the space of inputs by: user intent, surface type, edge severity, and expected failure mode.
Day 49–50: Run structured edge case testing in the Anthropic Console. Document at least 20 distinct failure modes with: the prompt, observed output, expected output, and root cause hypothesis.
Day 51: Write an edge case report for one behavioral domain. Format it as an internal analysis document — the kind you’d present to a research team. Portfolio artifact #7.

Deliverable: Edge case analysis report (portfolio artifact #7).

Week 8: Policy and Behavior Specification Writing

Goal: Learn to translate ethical and product goals into precise behavioral specifications — a differentiating skill.

Daily focus areas:

Day 52–53: Study existing public behavior specifications: Anthropic’s usage policy, OpenAI’s model spec, and Meta’s Llama usage policy. What do they specify? How are they structured? Where are they vague?
Day 54–55: Study moral philosophy frameworks relevant to AI policy: consequentialism, deontology, virtue ethics, contractualism. Understand how each maps to different design choices (rule-based vs. outcome-based behavior).
Day 56–57: Enroll in and begin Yale’s Moralities of Everyday Life (Coursera). Work through Week 1–2 content. Take notes in a format that connects to model behavior design decisions.
Day 58–60: Write a behavior specification document for one product scenario (e.g., an AI writing assistant used in education). Define: permitted behaviors, prohibited behaviors, ambiguous cases, and the ethical reasoning behind each call. Portfolio artifact #8.

Deliverable: Behavior specification document (portfolio artifact #8).

Phase 3 — Expertise and Integration (Days 61–90)

Theme: Synthesize everything. Produce capstone work. Practice articulating your expertise.

Week 9: Synthetic Data and Pipeline Design

Goal: Understand how behavior data is generated at scale — a skill explicitly called out in model behavior roles at labs focused on frontier model development.

Daily focus areas:

Day 61–62: Study synthetic data generation methods: self-play, Constitutional AI critique loops, Alpaca-style instruction generation. Understand quality tradeoffs.
Day 63–64: Design a synthetic data pipeline on paper for a specific behavioral improvement target. Map: data generation → filtering → annotation → training signal. Identify where quality can degrade at each step.
Day 65–66: Implement a minimal version of this pipeline using the Anthropic API. Generate 50+ synthetic training examples for a behavioral target. Filter them by a defined quality rubric.
Day 67: Document the pipeline as a technical writeup. Portfolio artifact #9.

Deliverable: Synthetic data pipeline writeup with generated examples (portfolio artifact #9).

Week 10: Model Strategy and Cross-Functional Thinking

Goal: Develop the strategic layer — how model behavior decisions connect to product, cost, latency, and user experience.

Daily focus areas:

Day 68–69: Study how model benchmarking differs from behavioral evaluation. Understand MMLU, HumanEval, and MT-Bench, but also understand why they’re insufficient for product behavior decisions.
Day 70–71: Study model comparison methodology as practiced in industry. Read Chatbot Arena methodology in detail. Design a model comparison protocol for a specific product decision (e.g., choosing between two models for a use case).
Day 72–73: Continue Yale Moralities of Everyday Life. Focus on the psychology of moral judgment — directly applicable to predicting how users and annotators respond to model outputs.
Day 74: Write a model strategy memo: given a product scenario, recommend a model choice with rationale grounded in behavioral evaluation data, cost, and user impact. Portfolio artifact #10.

Deliverable: Model strategy memo (portfolio artifact #10).

Week 11: Capstone Behavioral Experiment

Goal: Produce the central portfolio artifact — a full behavioral experiment with a research question, methodology, results, and implications.

Daily focus areas:

Day 75–76: Select a research question in model behavior design. Examples: How does system prompt length affect instruction-following precision? or What prompt structures minimize sycophancy in evaluation tasks?
Day 77–78: Design the experiment: define variables, control conditions, evaluation rubric, and sample size. Write the methodology section first.
Day 79–81: Run the experiment. Use the Anthropic Console + Promptfoo or ChainForge. Collect and record outputs systematically.
Day 82–83: Analyze results. Identify patterns, surprises, and limitations. Write up findings.
Day 84: Finalize the capstone as a publishable document (blog post format or internal report format). Portfolio artifact #11 — your anchor piece.

Deliverable: Full behavioral experiment write-up (portfolio artifact #11, anchor piece).

Week 12: Portfolio Assembly and Interview Preparation

Goal: Integrate everything. Present your expertise. Practice communicating it under pressure.

Daily focus areas:

Day 85–86: Assemble the portfolio. Organize artifacts #1–11 into a coherent narrative arc. Write a one-page introduction that frames your trajectory and philosophy.
Day 87: Audit the portfolio against each target job description. For each required skill listed in the job descriptions you’ve collected, identify which portfolio artifact demonstrates it. Fill any gaps.
Day 88: Practice explaining your capstone experiment out loud. Time yourself. Aim for a clear 5-minute version and a 15-minute deep dive version.
Day 89: Practice behavioral and conceptual interview questions: How would you evaluate honesty in a model? Walk me through how you’d design an evaluation for a new product feature. What’s the difference between a benchmark and a behavioral eval?
Day 90: Final review. Write a one-page reflection: what do you know now that you didn’t on Day 1? What’s your honest remaining gap? What’s the first role you’re applying to and why?

Deliverable: Assembled portfolio, interview readiness, target role list.

Portfolio Summary

#	Artifact	Phase	Role Signal
1	Role definition document	1	All
2	Alignment techniques comparison	1	Anthropic, OpenAI
3	Evaluation framework design	1	All
4	Context engineering case study	1	Perplexity, Notion
5	Annotation guidelines	2	Anthropic, Mistral
6	Eval suite with methodology	2	Perplexity, Mistral, Notion
7	Edge case analysis report	2	All
8	Behavior specification document	2	Anthropic, OpenAI
9	Synthetic data pipeline writeup	3	Mistral, OpenAI
10	Model strategy memo	3	Notion, Perplexity
11	Behavioral experiment (capstone)	3	All

Key Resources

Papers (read in order of phase)

Tooling

Anthropic Console (primary sandbox)
Promptfoo (batch eval, entry-level)
ChainForge / Freeplay (comparative prompt testing)
Braintrust (advanced eval platform, study docs)
LangSmith (advanced eval platform, study docs)
Label Studio (annotation)
Chatbot Arena Leaderboard (live model comparison)

Credentials

Yale Moralities of Everyday Life — Coursera (begin Day 56)
LSE Ethics of AI Online Masterclass — pursue after Day 90 if not already enrolled

Reference Frameworks

Weekly Check-In Prompts

Use these at the end of each week to self-assess:

What did I produce this week that I could show a hiring manager?
Which skill did I move from “aware of” to “practiced”?
What concept am I still fuzzy on — and what’s the most direct way to resolve it?
Am I going deep enough, or am I skimming for familiarity?

This plan is built from the skill signals found in Model Behavior Architect / Engineer job descriptions at leading AI labs and AI-product companies. Revisit your target job descriptions at the start of each phase to stay calibrated.