Optimize and fine-tune AI agents for production: Review

The 'Prepare the Training Data' unit hit me like a wall of text. It talks about cleaning data, formatting it into user/assistant turn pairs, and making sure the tone is consistent. Fine, but as a PM, my question is: whose job is this? Is this a data scientist spending three weeks labeling conversations? An offshore team? Or something the Foundry tool automates? The module never answers that.

It's a tour of concepts, not a tour of a workflow. It introduced me to the alphabet soup—SFT, RFT, DPO—and I can now nod along in meetings when an engineer mentions them. But if they asked me, 'Should we use SFT or invest in the data for DPO?' this course leaves me with zero framework for making that call. What's the cost difference? How much more performance do we get? How long does each take? Is one better for fixing factual errors and another for fixing a sassy tone?

That's the missing piece. The content lives in this weird space between a high-level whitepaper and a technical tutorial, succeeding at neither. The exercise at the end wasn't even a sandbox; it was a multiple-choice quiz about the definitions I had just read. I get the concepts now, but I have no intuition for them. This won't help me write a better PRD or challenge an engineering estimate.