Vibe Coder · Unit 2: How Claude and ChatGPT Work · Lesson 7 of 7
Tokens and cost
When your code calls a model you pay by the token. Input, output, the context window, and three levers that cut the bill.
Read the short notes and pictures.Do everything inside a blue frame.
In your own product, you pay by the token
The Claude and ChatGPT apps charge a flat monthly price. When your own code calls a model through the API, the bill works differently: you pay for every token you send and every token the model writes. Understanding tokens is how you predict the cost and the speed of an AI feature before you build it.
What a token is
A token is a piece of text, often a short word or part of a longer one. As a rule of thumb for English, 1,000 tokens is about 750 words. Code, numbers and other languages usually take more tokens per word. Every API response tells you exactly how many tokens the request used.
Input and output are priced separately

Input tokens
Everything you send: instructions, the conversation so far, documents and tool results.
Output tokens
Everything the model writes, including its thinking. Priced several times higher than input.
What you send
system: instructions 400 tokens history: earlier turns 2,100 tokens user: new question 60 tokens input total 2,560 tokens
What you get back
assistant: the answer 380 tokens
output total 380 tokens
cost = input x input price
+ output x output priceEvery turn sends the whole conversation again
The API keeps no memory between requests. To continue a conversation, your code sends all of it again, plus the new message. Turn 20 of a chat sends turns 1 to 19 as input. Cost per turn grows as the chat grows, and so does the time to respond. Everything must also fit in the model’s context window.
Three levers that cut the bill

Shorter context
Send only what the task needs. Summarise old turns, retrieve the relevant passages instead of whole documents, and cap the length of the answer.
A smaller model
Route simple jobs such as classifying, extracting and short replies to the small model, and keep the large one for hard requests. The price per token differs by several times between sizes.
Caching
Prompt caching stores the part of the prompt that repeats, such as long instructions or a reference document, so later requests read it at a fraction of the input price.
Engineers: cost the AI feature
Pick the one AI feature your product will have. Estimate input and output tokens per request, requests per user per day and users in the first month, then work out a monthly cost on a small and a large model using the prices on the provider’s pricing page. Add it to the brief under "Cost".
Done when
You will check this estimate against real usage in the APIs unit.
Keep these
- Through the API you pay per token: input and output, with output priced higher.
- About 1,000 tokens is 750 English words.
- Each turn sends the whole conversation again, so long chats cost more per turn.
- Cut cost with shorter context, a smaller model for simple jobs, and caching.