LLM inference: inside ReadyServer’s shared-capacity API

LLM inference: inside ReadyServer’s shared-capacity API

Ready Server Pte Ltd, a Singapore hosting provider, has answered a set of operational and data-handling questions about its shared-capacity LLM API (LLM inference) — the detail its original announcement did not carry. The answers are unusually specific, and they make the service’s position clear: cheap, shared, best-effort inference, not a guaranteed commercial API.

The LLM inference service began, according to director Alan Woo, because customers wanted to run a local language model on an entry-level Mac mini (M4, 16GB) but their virtual private servers could not manage it. ReadyServer’s design goal is parity: performance “similar or better” than that Mac mini, delivered from the company’s own infrastructure rather than the customer’s desk.

The limits

Those limits are tight and stated plainly. Each VPS may send one API request at a time; additional requests queue, up to four; anything beyond that gets rejected; and the maximum time a request waits in the queue is 120 seconds. The service-level agreement, Woo says, is “best effort.” For anyone weighing whether to build on it, that is the crucial sentence: there is no committed throughput, no availability target, and no guarantee that a request will be served rather than queued or turned away.

How it runs

Under the hood, the service is a conventional vLLM deployment. Requests are load-balanced by vLLM-router across a group of vLLM servers powered by Nvidia L40S GPUs, all hosted in ReadyServer’s own data centre in Singapore. Fair-share scheduling uses vLLM’s priority policy: each request carries a priority based on 24-hour rolling token usage, so lighter users get served first.

That matches how vLLM works. The engine runs first-come-first-served by default but supports a priority policy in which the scheduler handles requests by a given priority — lower value first — with arrival time breaking ties, and the scheduler preempts the lowest-priority request when the key-value cache fills. Tagging priority by recent usage is a fair-share variant on the same machinery. vLLM-router, meanwhile, is the project’s own lightweight load balancer, with policies including cache-aware, power-of-two and round-robin. The mechanisms are standard; the novelty is the packaging and price point.

Data handling

On data, the answers are reassuring for a shared service. Prompts and outputs are not there in the logs; ReadyServer stresses that it is not an AI model company and does not train models; and all inference and storage sit in its own Singapore data centre. For developers who care where their prompts go, that is a clearer picture than many API vendors provide.

The model choice is narrow for now: two Qwen3 27B variants from Alibaba Cloud, both under the Apache 2.0 licence, which permits commercial hosting. Woo says other models are under licensing review, and that an embedding model (qwen3-embedding, also Apache 2.0) is in testing.

The shared-hosting model

The most striking part of the answers is the scaling posture. There is no plan to offer dedicated capacity or to charge per token; the fair-use scheduling “indirectly” limits how much any customer consumes; and the company tells customers who need guaranteed performance, directly, to look at OpenAI, Anthropic or OpenRouter. Woo compares the model to shared web hosting: a flat fee, shared server resources, and a best-effort service because many users share the same machine.

That analogy is honest, and it is also a warning. Shared hosting is cheap precisely because it oversubscribes — the provider assumes most accounts are idle most of the time — and it works until a customer’s needs grow beyond the shared pool. The same logic applies here: ReadyServer’s API is well-suitable for light, intermittent use, and structurally unsuited to workloads that need predictable latency or throughput.

What to watch

Three things will show how the service evolves. First, whether the model catalogue widens as licensing reviews conclude, since two Qwen3 variants is a narrow choice. Second, whether the embedding model ships, which would broaden the use cases. Third, whether the “no dedicated capacity” stance holds as customers ask for more, since that is the line between a hobbyist LLM inference tier and a business one.

What the answers do and don’t establish

They establish the concurrency and queue limits, the best-effort SLA, the no-logging and Singapore-data-centre position, the vLLM/L40S architecture, the current models and their licensing, and the absence of dedicated-capacity plans. They do not establish independent performance benchmarks, actual usage or reliability data, or how the service behaves under sustained heavy load.

LLM inference: inside ReadyServer's shared-capacity API

The honest summary: ReadyServer is offering a small, cheap, honestly-described slice of shared LLM inference, and the follow-up answers are more candid than most vendor material. The limits are real and disclosed; the data-handling story is favourable; and the company says serious workloads should go elsewhere. For a developer who wants to try a model without paying for a guaranteed API, that clarity is the point.

Editor’s Note: This article draws on the written answers supplied by Ready Server Pte Ltd, plus independent technical context from vLLM’s documentation and scheduling research, the vLLM-router project, and analysis of priority queuing for shared GPU inference. The service’s limits, architecture and data-handling claims are as described by the company; performance parity with a Mac mini M4 is the company’s framing and is not independently verified. No interview beyond the written Q&A through email.