With the recent release of the new Anthropic, expectations in our community have been huge. That’s why, as soon as Amazon made them available through Bedrock, we got to work integrating them into our Framework to explore their full potential.
Model Types and Use Cases
To understand the significance of this update, it’s important to briefly explain the different model types and how we use them:
1. Fast and cost-effective models, with limited reasoning capabilities.
2. Slower and more expensive models, but with much more advanced reasoning capabilities, costing 5 to 10 times more.
In tasks like retrieval-augmented generation (RAG), simpler models are usually sufficient, as long as we handle the indexing process in the Vector Store well. However, when dealing with more complex architectures, such as agents that require advanced reasoning or SQL query generation, these models often fall short.
For example, when creating SQL agents, the fast and cost-effective models can fail—not necessarily because they can’t reach the answer, but because they require multiple iterations of trial and error, which can lead to suboptimal queries for medium to high complexity cases. Until recently, this meant we had to rely on more powerful models to ensure reliable results.
A Game Changer with Haiku 3.5
In recent months, we’ve seen significant improvements in the reasoning capabilities of the more affordable models. A good example is gpt-4o-mini, which, despite being classified as a “simple” model, has proven capable of handling a wide range of tasks, including SQL agents.
However, with the previous Haiku model on Bedrock, we faced limitations. While it was fast and cost-effective, its performance in SQL queries or reasoning tasks wasn’t optimal, which led us to choose the Sonnet model in those cases.
With the release of Haiku 3.5, the game has changed. This model not only generates exceptional SQL queries but also performs very well with moderately complex agents. For more complex processes, like multi-agent systems with numerous tools, we still use Sonnet, but Haiku 3.5 has become the preferred option for most use cases.
What Results Have We Achieved?
Although we haven’t conducted exhaustive comparisons, our empirical tests with Haiku 3.5 have left us very impressed. Here are our preliminary conclusions:
- Versatility in SQL dialects: Haiku 3.5 generates clear and well-structured SQL queries in various dialects, such as Oracle, SQL Server, and Databricks SQL, effortlessly.
- Fewer iterations needed: While Sonnet occasionally required 2 or 3 iterations to perfect a query, Haiku 3.5 achieves accurate results on the first attempt.
- Reduced cost: Haiku 3.5 is three times cheaper than Sonnet, making it a highly efficient choice in terms of cost-performance. Although its price is higher than the previous Haiku 3, the jump in capability justifies the cost.
Cost Comparison
Referring to the prices listed on AWS Bedrock Pricing, we observe the following:
The image is sourced from https://aws.amazon.com/bedrock/pricing/.
- Haiku 3.5 is significantly cheaper than Sonnet, and while it costs more than the previous Haiku 3, the improvement in power is notable.
- Opus, a high-performance model, is still available, but we haven’t had the chance to fully evaluate it yet.
Conclusion
Haiku 3.5 has quickly risen on our list of preferred models due to its balance of cost and performance. For most use cases, this model is the ideal choice, leaving Sonnet and other higher-end models as alternatives for specific scenarios that require additional power.
We encourage you to try Haiku 3.5 and see for yourselves how it can optimise your processes. We’re confident it will be a valuable addition to your toolkit.