A token-metered gateway to AI models: an async FastAPI backend, responses arriving as a stream, balance topped up in crypto.
Access to the strong models runs into a subscription and a foreign bank card. What was needed was a service where you pay for what you actually use rather than a month up front, and without a card from another country.
The second requirement was responsiveness. A user should not stare at an empty screen while the model thinks: the answer has to appear as it is generated, the way it does in the original interfaces.
The balance is counted in tokens and drawn down per request. You can see what each conversation cost — with no standing fee for the days you do not use it.
Streaming over SSE: text appears as it is generated rather than landing whole half a minute later. The wait stops feeling like a freeze.
Balance credited automatically, with no manual reconciliation of payments and no card on file.
FastAPI and asyncio: connections to the models stay open for a long time, and a synchronous server would simply stall on them. The async design lets one process carry dozens of open streams at once.
The stream from the model is not buffered on the server but passed through to the client as it arrives. Any intermediate accumulation would defeat the point entirely — the user would be back to waiting for the whole answer.
Access to the strong models without a subscription or a foreign card: you pay in tokens and the answer arrives as a stream.