Anthropic API vs BentoML

Claude API uses pay-as-you-go pricing per million tokens (MTok). Haiku 4.5 costs $1 input/$5 output per MTok; Sonnet 5 costs $2 input/$10 output; Opus 5 costs $5 input/$25 output; Fable 5 costs $10 input/$50 output per MTok.

The framework is, under Apache 2.0, and you can run it entirely on your own infrastructure. BentoCloud, the managed platform run by the company, is a paid service billed on the compute it runs for you.

Batch processing saves 50% on API costs. Prompt caching reduces token costs by up to 90% for cached reads (charged at 80% discount compared to standard rates). Fast Mode for Opus 5 costs 2x standard pricing for up to 2.5x faster response speeds.

Not for a single service, which is just a container. You need it once you want autoscaling, multiple models and rolling deployments on your own infrastructure, which is the point at which the managed option starts to look attractive.

Self-serve access uses usage-based tiers with automatic rate limit increases as volume grows. Enterprise customers receive custom rate limits, monthly invoice billing, and hands-on support at negotiated pricing.

For one model it is not very different and FastAPI is simpler. The difference is at four or ten models, where you would otherwise be maintaining ten sets of the same Dockerfile, batching logic, dependency pinning and health check code.

US-only inference costs 1.1x pricing for input and output tokens across all model tiers compared to standard multi-region pricing.

Yes, and the project publishes tooling aimed at that specifically, but the constraints are the usual ones: accelerator memory, batching strategy and the cost of holding a GPU that is idle between requests.

A directory, versioned and archivable, containing your service code, the model files it needs, the exact Python dependencies and instructions for running it. It is the unit you build into an image and deploy.