Trying OpenRouter
Winkletter • 26 Nov 2024 •
I may have been doing chat wrong. Today I decided to finally try OpenRouter, the platform that connects to all the AI models. It can be used as an API for your own projects, a chat interface that connects to multiple models at once, and you can also generate API keys to use in other applications like my favorite web tool WebSim.
As a test I used OpenRouter to generate my writing exercises for this week. Final cost = 18.5 US cents. This process has long input and output responses, and with OpenRouter I was able to prompt both ChatGPT 4o and Claude 3.5 Sonnet simultaneously.
| Timestamp | Model | Tokens (in/out) | Cost | Speed |
|---|---|---|---|---|
| Nov 25, 11:53:57 PM | Claude 3.5 Sonnet | 6773 → 421 | 0.03$ | 59.0tps |
| Nov 25, 11:53:56 PM | GPT-4o | 8727 → 535 | 0.02$ | 86.6tps |
| Nov 25, 11:48:57 PM | Claude 3.5 Sonnet | 5659 → 682 | 0.03$ | 63.5tps |
| Nov 25, 11:46:59 PM | GPT-4o | 6497 → 1833 | 0.03$ | 105.2tps |
| Nov 25, 11:46:47 PM | Claude 3.5 Sonnet | 5357 → 298 | 0.02$ | 59.2tps |
| Nov 25, 11:09:49 PM | GPT-4o | 2451 → 1598 | 0.02$ | 92.7tps |
| Nov 25, 11:09:43 PM | Claude 3.5 Sonnet | 1987 → 610 | 0.02$ | 57.8tps |
| Nov 25, 10:48:33 PM | GPT-4o | 196 → 1148 | 0.01$ | 52.0tps |
| Nov 25, 10:48:21 PM | Claude 3.5 Sonnet | 206 → 596 | 0.01$ | 56.6tps |
At this rate I don’t see myself using more than the $10 I loaded into OpenRouter by the end of the year, even if I run both models simultaneously for every prompt. So it looks like its cheaper. And I can compare multiple models. And, I can even control some of the various parameters like Max Tokens, Temperature, and a bunch of stuff I don’t understand.
I also like how it timestamps every interaction so I can easily measure how long I took to finish a task.
Comments
When you purchase credits they take 0.6% + $0.03 which is just rounding up Stripe’s larger cut. A reply on Reddit suggests they charge the same amount as the suppliers, and make money from the savings buying at bulk.
Thanks dude! I think I’ll try it out. Opens up the possibilities on Cursor
I subscribe to Anthropic for Claude Sonnet, but is constrained to just their APIs. Been wanting to sign up OpenRouter to enjoy them all… especially the APIs to open source LLMs like llama3.1
Do OpenRouter take a % cut from the money you spent? Their monetisation model isn’t very clear