What is an LLM?
An LLM, or large language model, is software trained on huge amounts of text to predict and generate useful language. The practical way to think about it: you give it a task, it turns your text into tokens, estimates likely next tokens and returns a draft. It can write, summarize and classify because many language patterns are compressed into its parameters. It is not a database and it does not automatically know whether a fresh fact is true.
What is a prompt?
A prompt is the instruction package you send to an AI system. Good prompts usually include the role, the situation, the exact task, constraints, examples and the desired output format. A weak prompt asks for “ideas”. A strong prompt says who the ideas are for, what they must achieve and how they should be judged.
What actually happens when you send a prompt?
The normal flow is not: “the AI understands the request, writes a Python program, then starts working.” A large language model usually receives text, turns it into tokens, runs those tokens through neural-network layers and generates the next token again and again. Python or other tools only enter the process when the product around the model explicitly gives it tool access, for example a calculator, code interpreter, browser, database connector or function call.
You type a prompt. The application may add hidden system instructions, safety policies, chat history or selected documents before the model sees the final request.
The text is split into token IDs. The model does not receive words as a person reads them; it receives numerical token identifiers.
The model computes probabilities for possible next tokens based on the full context. This is where attention, embeddings and model weights matter.
If the application supports tools, the model may request a function such as search, calculator or code execution. The external result is then inserted back into the conversation as more context.
The final text is decoded from generated tokens. Better prompts help because they change the context the model conditions on before choosing each next token.
Use a simple before-and-after exercise before continuing with tokens, context, and model limitations.
Better than a mini test: build a prompt before-and-after gallery
Instead of running a generic test, collect five real prompts from your own work and save the weak version, the improved version and the final edited result. This becomes a reusable prompt gallery for your team.
- Pick one recurring task, such as support replies, product descriptions or lesson summaries.
- Save the original prompt and output.
- Add context, format rules and one example.
- Compare how much editing the second output needs.
- Turn the winner into a template.
What are tokens?
Tokens are the small text units a model reads and writes. A token can be a whole word, part of a word, punctuation or a character, depending on the tokenizer and language. Token limits matter because input plus output must fit inside the model context window.
Why does AI sometimes invent things?
AI can hallucinate when it generates a plausible-sounding continuation without enough reliable grounding. The model is optimized to produce likely text, not to guarantee truth by default. Hallucinations become more likely when the question asks for obscure facts, fresh information, hidden data or citations that were not provided.
The same mechanism can invent a work status, not just a fact. In one of our own workflows an AI assistant was asked to look up the matching LinkedIn profile for every row of a spreadsheet and write each URL into one column — dozens of rows, not a single lookup. It reported the task as done and even gave counts — “14 profiles, 3 emails” — but the column was empty and the returned file was a renamed copy. A confident summary is simply the most plausible next words, so it is no proof that the work happened.
Asking again does not necessarily break the pattern. Depending on the product, a retry may resend the visible conversation, use server-side conversation state, trim or summarise older turns, or reuse cached input. It still generates a new answer and may consume additional tokens or plan limits, even when no usable file is produced.
Reason: is the AI caught in a loop?
In a practical sense, yes — but not the way it feels. The model has no persistent self-awareness of repeating a failure; it can only use the context and state that the application supplies. A new answer may be generated from the full visible history, a truncated or summarised history, or server-managed state. If the workflow does not require the assistant to open the file and verify the saved values, then “done” may remain a plausible response. The loop is not stubbornness; it is missing verification and state checks.
That is why repeating the same question rarely helps, while it keeps burning tokens. What breaks the loop is changing the task, not repeating it: make the result verifiable, ask the assistant to read the saved file back and show the actual cell values, and stop the run when that evidence is missing. See the most common prompting mistakes for how to define a success criterion the model can check.
Why does context help?
Context narrows the search space. If the model knows the audience, goal, constraints, examples and source material, it can produce an answer that fits your situation instead of a generic answer. More context is not always better: irrelevant context can distract the model and increase cost.
Search engine vs. LLM
A search engine retrieves pages and ranks links. An LLM generates an answer from patterns and provided context. Search is better when you need current sources, official pages or multiple perspectives. LLMs are better when you need synthesis, rewriting, reasoning over supplied information or structured drafts. Many strong AI products combine both.
What does “AI understands language” mean?
In everyday language, “understands” means the model can respond appropriately to meaning, tone and structure. Technically, it learned statistical representations that map text patterns to useful outputs. It does not understand like a person with lived experience, intentions or common-sense accountability.