RightNow Inference (RunInfra) vs SiliconFlow

Automatically keeps repeated calls from one agent session on the replica that already holds its context, reaching reported cache-hit rates around 99% and cutting effective input costs.

One API key works with both the OpenAI and Anthropic SDKs, so existing agent code can switch providers with minimal changes.

Serves open-weight models with published pricing, tool calling, JSON mode, streaming and image-input capabilities, plus live output-speed and cache metrics.

Cache hits appear in usage and are billed at the cached rate, with a published measurement methodology and per-model effective input rates.

Access a large catalog of LLMs and multimodal models through a single endpoint, so most existing SDK code works after swapping the base URL and key.

Start with pay-per-use serverless calls, then move to dedicated endpoints, reserved H100/H200/MI300 GPUs or elastic GPU (FaaS) deployment as throughput and cost requirements grow.

Customize supported models on your own data and push the tuned variant to a hosted endpoint without building a training pipeline from scratch.

Transparent rate tables list input, cached-input and output prices per million tokens, plus per-image rates for generation models, making budget planning concrete.