Amazon Downsizes Anthropic Models for Internal Use

Amazon engineers are working to convert Anthropic's large-scale AI models into smaller, lower-cost versions for internal use, according to reports. This technique is called "distillation," which involves transferring knowledge from a larger model to a smaller one. Generally, downsized models can significantly reduce processing costs.

Behind this move is a change in pricing structure. Currently, Amazon pays for Anthropic's models based on compute time, but next year it plans to shift to a pricing model based on the number of tokens (the smallest units of text) processed by the model. Token-based pricing tends to accumulate costs as usage increases, and continuing large-scale internal use could result in significantly higher expenses.

In response to this situation, Amazon is also examining alternative AI providers such as OpenAI, which has been revealed. The company appears to be seeking to reduce dependence on specific AI providers and secure flexibility on costs. It is taking a two-pronged approach: developing in-house models through distillation and comparative evaluation of external services.

Amazon made a maximum investment of $4 billion in Anthropic in 2023, and the two companies maintain a deep partnership by providing Anthropic models through AWS, Amazon's cloud service. Precisely because of this, the current initiative to reduce internal costs is noteworthy as stemming purely from the perspective of in-house cost management, separate from the investment and business partnership relationship.

Model distillation is increasingly being used as a cost management tool as more companies adopt AI at scale. Continuing to use large AI models as-is increases cost pressures alongside increased usage, making the creation of smaller models tailored to a company's own needs a rational option. For a company the size of Amazon, the impact is not insignificant.

However, models created through distillation do not perform exactly the same as the original model. There can be constraints on accuracy and capabilities, and it is necessary to determine which applications are suitable. For uses such as internal business automation or auxiliary tools, performance is often sufficient, and differentiated use according to purpose is expected to be a key focus going forward.

How Amazon optimizes AI costs ahead of next year's pricing transition is a development that other major corporations facing similar cost challenges can learn from. Corporate strategies surrounding AI utilization cost management are likely to become increasingly diverse in the future, and the tension between provider pricing and user companies' in-house development and procurement strategies is expected to continue.