Quick Answer: Amazon engineers replaced Claude Sonnet 4.6 with Strands Decider, a decision model, to label internal tickets. The swap cut token costs about 98% at published list prices and dropped average call time from 4,215 ms to 897 ms. It worked because the job was a closed choice from a fixed catalog of 20 labels at high volume.
Here's a number that should make any team running an LLM classifier pause: roughly $22,200 per million tickets for Claude Sonnet 4.6, versus about $500 for a small specialized model. Strands Agents reported that two Amazon engineers made that swap on one of the company's largest internal ticketing systems. The model they moved to performed just as well on their tests, and the team says the change will save millions of dollars in token costs each year.
Why Sonnet 4.6 Was the Wrong Tool for the Job
Sonnet 4.6 was a sensible pick at first. Its balance of intelligence and cost suited a workload that ran on every ticket, from customer escalations to security findings. The nine-month-old model handled the work well.
The problem was the shape of the task. Each ticket came with a long system prompt, and the agent generated JSON and free-text reasoning to choose one of 20 category labels. Most of that output was never used. The team measured 6,554 input tokens and 170 output tokens per ticket with Sonnet.
Here's the uncomfortable part: a general-purpose generator was doing a classifier's job. Reasoning text costs output tokens at $15.00 per million, and every ticket paid for a fresh pass through the full prompt. When the answer is one label out of a known set, most of that spend buys nothing.
What a Decision Model Actually Does
A decision model can't generate text. It receives a state and a question with a set of options, then returns a probability for each option plus a confidence score. For ticket labeling, the mapping is direct: the ticket is the state, and the label catalog is the set of options.
This is where most teams misjudge the tradeoff. The model isn't writing a justification. It's scoring options. That narrows what you need to get right, and it changes what the prompt must contain.
The Strands Decider had a hard constraint: a 4,096-token context window. Amazon's production label descriptions alone were longer than that, so the team rebuilt the catalog. Here's how they adapted it:
- Rewrote each of the 20 labels as a one- or two-sentence criterion.
- Sent those criteria as the options of a single choice question.
- Added a "none" option so the model could decline to pick a label.
- Passed the same ticket data Sonnet received, except that on long tickets the decider drops the oldest comments to fit.
The team started with the original 2B Strands Decider, then switched to an E4B build based on Gemma 4 after it scored clearly better on a test set. They then tuned the confidence threshold a label needed before it was applied.
The Three-Way Switch That Made the Swap Safe
The part of this story most teams will copy, whatever model they choose, is the rollout mechanism. The labeling Lambda reads a three-way setting from AWS AppConfig on every message:
- BEDROCK: Sonnet labels every ticket, and Strands Decider isn't called.
- DUAL: Both models run in parallel. Sonnet's results are applied, and the team compares them against the Strands Decider output.
- DECIDER: Strands Decider labels every ticket, and Bedrock isn't called.
Offline evals told the team less than live disagreements did. Running both models on real traffic let them judge every mismatch in context. It also made rollback cheap: going back took a 45-second config change instead of a redeploy.
That said, DUAL mode doubles inference for the duration of the test. Budget for that overlap before you start, and make sure someone is actually reviewing the disagreements. A dashboard nobody reads doesn't validate anything.
The Numbers: Latency, Tokens, and Cost
The team's measurements on the same tickets tell a clear story:
| Metric | Claude Sonnet 4.6 | Strands Decider E4B | Change |
|---|---|---|---|
| Average time per call | 4,215 ms | 897 ms | 78% faster |
| Input tokens per ticket | 6,554 | 2,009 | About 69% fewer |
| Output tokens per ticket | 170 | 1 | About 99% fewer |
| Total tokens per ticket | 6,724 | 2,010 | 70% fewer |
| Cost per ticket (list price) | $0.0222 | About $0.0005 | About 98% lower |
The token drop had two sources. The decider produces a single choice instead of JSON, and its prompt is much shorter. Those two facts alone explain most of the savings.
The cost gap is wider than the token gap, though, and that's the detail worth understanding. Sonnet is billed per token, so every token you cut saves money in a linear way. The decider runs on a dedicated SageMaker instance, an ml.g5.xlarge with one A10G GPU and 24 GB of memory, at about $1.408 per instance-hour. The model used about 19 GiB, and one request took about 640 ms. At that speed, one instance can handle roughly 5,600 tickets an hour, which works out to about $250 per million tickets at full utilization. Amazon budgets about $500 per million to leave headroom for bursts. That's roughly 44 times cheaper than Sonnet.
Here's where the math gets tricky. Your real cost depends on utilization. An instance sitting idle at 10% load costs ten times more per ticket than one running hot. If your traffic is spiky or your volume is small, that math can flip, and Sonnet's pay-per-token model may be the cheaper option.
Where the Cost Math Gets Honest
Three caveats matter before you copy this.
First, the 98% figure uses published us-east-1 on-demand list prices. It leaves out the Lambda that runs the pipeline, which is itself cheaper now because each ticket takes less time. The team reported a 74% drop in average Lambda duration on the day it turned off Sonnet usage. It leaves out the SQS queue, CloudWatch, and engineering time. Your negotiated pricing, reserved capacity, or savings plans will move the numbers.
Second, the source comes from the Strands team, which sells the model and the serving notes. The measurements may be accurate, but the framing comes from the vendor. Re-run the comparison on your own tickets before you trust it.
Third, the most surprising finding was about JSON key order. Sending the same content with keys in a different order moved the output probabilities by up to 0.19. That's enough to flip a label near a confidence threshold. The team now sends every request with sorted keys. If you serve a decision model behind an API, check this before you compare any two runs, because a "regression" might just be serialization noise.
When a Decision Model Won't Replace Your LLM
The source is explicit about why this worked: the task is a closed choice from a known catalog, the volume is high, and the latency of the old path mattered. Those conditions are the whole argument.
If your labels need explanation text for an auditor, a decision model can't produce it. If the catalog changes every week, you'll be rewriting criteria constantly, and the rewrite is the expensive part. If the context that matters doesn't fit in a few thousand tokens, the decider's window will force you to drop information the model needs. The common advice that "smaller models are cheaper" skips these constraints. A smaller model that can't see the relevant context will be cheap and wrong.
The infrastructure is also more than a model swap. Amazon's SageMaker endpoint runs in an isolated network with no outbound access. SageMaker copies the model weights from S3 before start-up. Ticket events arrive in an SQS queue, which triggers a Lambda that calls the endpoint. The whole stack is defined in CDK with Beta, Gamma, and Prod stages. Most of that is standard AWS plumbing, and you can build it with the same services.
Frequently Asked Questions
What's the difference between Strands Decider vs Sonnet for ticket labeling?
Sonnet 4.6 is a general-purpose language model that generates JSON and reasoning text for each ticket. Strands Decider is a decision model that scores a fixed set of options and returns one choice with a confidence score. In Amazon's test, the decider used about 2,010 tokens per ticket against Sonnet's 6,724, and it ran about 78% faster per call.
What is a decision model?
A decision model takes a state, such as a ticket, and a question with a fixed list of options. It returns a probability for each option plus a confidence score. It doesn't generate free text, which makes it fast and cheap for classification. It's a poor fit when you need explanations or when the option set changes often.
How do I safely switch from an LLM to a decision model?
Run both models in parallel on real traffic before you cut over. Use a runtime switch, such as AWS AppConfig, with modes for LLM-only, both, and decider-only. Review every disagreement, and keep the rollback path short enough to use under pressure. Amazon's rollback took a 45-second config change.
Why does JSON key order change the model's output?
Decision models are sensitive to the exact input they receive, and key order changes the token sequence. Amazon measured probability shifts of up to 0.19 from reordering keys alone. Sort your keys consistently before inference, or you'll mistake formatting changes for model regressions.
When is a decision model not worth it?
Skip it when labels need written explanations, when the catalog changes weekly, or when the useful context won't fit in a few thousand tokens. It's also a poor fit when your traffic is too low to keep a dedicated instance busy, because idle GPU time erases the per-ticket savings.
Keep Your Labels, Lose the Overhead
The lesson isn't that Strands Decider beats Sonnet. It's that a closed-choice task doesn't need a model that writes paragraphs. Build your own comparison this week: pull 200 recently labeled tickets, run them through both models, and log every disagreement with the reason. Pass this to anyone on your team who's paying per token for a classification job they could run on a single GPU.