Custom reward functions for multi-turn reinforcement learning with Amazon Nova Forge
Amazon Web Services Maria Masood
AWS showed how to build custom reward functions for multi-turn RL in Nova Forge. The trap: one wrong signal can teach the model the wrong habit while training still looks fine.
Based on reporting by Amazon Web Services, Maria Masood — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
AWS is pushing a very specific message here: in multi-turn reinforcement learning, the reward function is the real product. If you get it slightly wrong, the model can learn the wrong behavior and still look healthy on the charts. That matters even more for agentic tasks, where the model acts over several steps instead of firing off one neat answer.
The post centers on Amazon Nova Forge’s Bring Your Own Orchestration path, which lets the reward logic run in your own environment. Nova Forge handles the rollouts, message passing, and conversation state across turns; your container handles the task logic, from user simulation to code execution to verification. There is also a serverless multi-turn RL option, now generally available, for teams that don’t want to manage that setup, but this walkthrough uses BYOO.
The learning setup is reinforcement fine-tuning with GRPO, not supervised fine-tuning. Instead of training on curated examples with annotated reasoning, Nova Forge samples completions from the current model and scores them with code you write. For multi-turn tasks, the reward is an aggregate over the whole trajectory, not a verdict on one response. That’s the key difference. And it’s why the shape of the reward matters so much: the model optimizes whatever you actually score, not whatever you meant.
AWS’s worked example is a collaborative-coding task built over 500 unique programming tasks, using Amazon Nova Lite 2.0, LoRA, Amazon SageMaker HyperPod, and the Nova Forge BYOO path. The setup is deliberately tricky. The model gets an under-specified coding request, a simulator holds back the full spec, and the model must ask before coding if it wants the missing detail. The reward combines four pieces: correctness, an “asked before coding” signal, a penalty for guessing immediately, and a loop penalty for repetitive behavior.
That mix is the point. A single terminal reward is often too sparse. A pure behavior reward can be gamed. So AWS argues for a composite reward that makes the desired move directly rewardable and the failure mode explicitly worse. The post also spends time on safe code execution inside the reward, using isolated execution, resource limits, no credentials, no network, and a per-run sentinel so the model can’t fake the result. It closes by warning about reward collapse, where one component quietly stops contributing even though the overall training curves still look fine.
My take — AI-written commentary, not fact-checked reporting
This is the part of RL people keep trying to skip: the reward function is not a footnote, it’s the whole game. Multi-turn agents make that painfully obvious, because a lazy proxy can look productive right up until it teaches the model to spin in circles for a living. AWS is right to put the boring plumbing and the safety checks front and center; the industry’s habit of treating them as optional is how you get expensive nonsense with a dashboard.
Read more about this at: Amazon Web Services