Multi-Turn Reinforcement Learning on Amazon SageMaker AI: A hands-on guide
Amazon Web Services Daniel Quang
AWS showed how to train a model on multi-step tasks, not just single prompts. It used Wordle to prove the point and got a big jump in solve rates for about $339 in tokens.
Based on reporting by Amazon Web Services, Daniel Quang — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Most fine-tuning setups are built for one prompt and one answer. That works fine until the job looks more like an agent: file the expense report, triage the ticket, take the next step based on what just happened. Then the useful signal is the result of the whole sequence, not a gold label for each move. AWS is leaning into that with Multi-Turn Reinforcement Learning on Amazon SageMaker AI.
The setup is split cleanly in two. SageMaker AI runs the training job and updates the policy model. Amazon Bedrock AgentCore hosts the agent, the Wordle environment, and the reward function. For each rollout, SageMaker sends a task to the agent, the agent calls the policy model, plays the game across as many turns as needed, and returns one reward at the end. The training uses Group Relative Policy Optimization to update a LoRA adapter from the spread of rewards across rollouts of the same word.
The example task is Wordle, which is a neat fit because the rules are simple and the score is objective. But AWS is pretty explicit that this is only a teaching case. The real target is a small, lower-cost model learning a multi-step workflow that public models were never trained on. In the run described here, GPT-OSS-20B went from a 61 percent solve rate to 84 percent on held-out words at default reasoning effort, and from 15 percent to 69 percent at the low effort setting it trained at.
The bill is token-shaped, not instance-shaped. AWS says the 100-step run used 491.6 million prefill tokens, 159.7 million sample tokens, and 643.3 million train tokens across 105,486 rollouts in 12 hours 40 minutes. At us-east-1 list prices for GPT-OSS-20B, that came out to about $339 in training tokens, plus about $79 for the AgentCore runtime. The train side dominated, which is exactly what you’d expect when every extra turn keeps dragging the whole history back through the model.
The walkthrough also shows how much of this lives and dies on plumbing. The agent and reward sit in one Python file. The dataset contains only the secret word. The runtime has to start the HTTP server in the right order or it times out after 30 seconds. And the IAM trust principal has to be job.sagemaker.amazonaws.com, not the more obvious-looking one, or CreateJob fails after retries. The whole thing is less glamorous than the marketing pitch, which is usually a sign it might actually work.
My take — AI-written commentary, not fact-checked reporting
This is the kind of AWS feature that matters because it is boring in the right places. Not a grand new model, just a way to train models on messy, multi-step work without pretending every task is a neat prompt-and-answer pair. The industry keeps selling magic; the real progress is usually a pile of permissions, tokens, and one very stubborn reward signal.
Read more about this at: Amazon Web Services