For three months ChatGPT has been something you visit: a website, a text box, a queue. From Wednesday it is something any programmer can build into anything. OpenAI has released the ChatGPT API — the same model that runs the chatbot, called gpt-3.5-turbo, offered to developers at $0.002 per 1,000 tokens, roughly 750 words. That is a tenth of what OpenAI charged for its previous best model. It released a speech-to-text API, Whisper, on the same day.
What an API is
An application programming interface is a door a company leaves open for other people’s software. Your app sends OpenAI a message; OpenAI’s model replies; your app shows the reply to your user. The user never sees OpenAI and does not need an account with it — they see your app, with a chatbot inside. Microsoft has had this access for a year, which is how Bing got its chat. Now everyone has it, on a credit card.
Who launched with it
Snapchat’s My AI, a chatbot pinned to the top of 750 million users’ friend lists; Instacart’s “Ask Instacart”, which turns “what do I need for a Ramadan iftar for six?” into a shopping list; Shopify’s shopping assistant in its Shop app; and Quizlet’s Q-Chat tutor. Each was built in weeks. OpenAI also changed its data policy: text sent through the API is no longer used to train its models unless the developer opts in, which removes the main objection businesses had.
What the price means in taka
At $0.002 per 1,000 tokens, a chatbot exchange of 300 words in and 300 out costs about a tenth of a US cent — a hundred conversations for a taka, in round numbers. A customer-service bot answering ten thousand queries a day would cost less than a single call-centre agent’s lunch. The cost of adding a language model to a product has, in one announcement, stopped being a consideration. That is the significance: not that the model is new, but that it is now cheaper than the humans it substitutes for, everywhere, at once.
Why it matters in Dhaka
Bangladesh’s software industry does not train models and will not for years; it builds products for clients. From today it can build products with ChatGPT inside — a Bangla customer-service bot for a bank, an English-writing assistant for a garment exporter’s sales team, a question-answering layer over a company’s documents — for a marginal cost near zero. The model’s Bangla is weak, and the first useful local products will be English-facing or will translate. But the constraint that kept AI out of a 20-person Dhaka software house — the cost and the access — ended on 1 March. Expect the pitch decks to follow.
What a token actually costs you
Pricing by the token confuses people because it is not pricing by the question. A token is roughly three-quarters of an English word, and both directions are charged: the prompt you send and the answer you receive. Longer conversations cost more every turn, because most applications resend the whole conversation as context each time.
That is the line that catches new builders. A chatbot that remembers a long conversation is re-paying for the entire history on every message, so cost grows with the square of the conversation length rather than in step with it. The fix is as old as the problem: summarise older turns instead of resending them.
Prices fell much further in the years after this announcement, which made the arithmetic friendlier without changing its shape.
Source: ZDNet, via Prothom Alo.
