A shared vocabulary before we dive in · AI in Action
Eight terms you'll hear today, in plain language — with an honest note on what each one does and doesn't mean. The goal is to separate the promise from the reality.
Large Language Model (LLM)
The AI that predicts and generates text. It's trained on huge amounts of writing to guess the most likely next words.
Reality: it's a very good pattern-matcher, not a fact database. It can sound confident and still be wrong — so verify anything that matters.
Tokens
The chunks of text the model reads and writes — roughly a word or piece of a word. Usage and cost are measured in tokens.
Reality: "how many tokens" is really "how much text." Longer documents and longer answers cost more and take longer.
Context Window
How much text the model can "hold in mind" at once — your prompt plus everything it's working with in this conversation.
Reality: it's finite. Past the limit, earlier details drop off. That's why pointing it at a knowledge base beats pasting everything in.
Prompt Engineering
Writing clear, structured instructions so the AI gives you a useful result — the difference between "chatting" and "engineering an outcome."
Reality: not magic words. It's just being specific about Role, Context, Task, and Format (that's the Prompt Builder tab).
Memory
Whether the AI carries information from earlier — within a conversation, or saved across sessions if the tool supports it.
Reality: by default it does not remember you between sessions, and it isn't a system of record. Don't rely on it to "remember" important facts — put those in a knowledge base.
Knowledge Base
A curated set of your verified, organizational documents that you give the AI to work from — its trusted "brain" of facts.
Reality: this is how you make answers accurate and current. The AI cites your documents instead of guessing from training data.
RAG / Grounding
Retrieval-Augmented Generation: the AI first retrieves the most relevant passages from your knowledge base, then writes an answer grounded in them — with citations.
Reality: this is the whole demo today. Grounding is what turns "casual chat" into a trustworthy answer you can trace back to a source.
Agents
AI set up to take steps toward a goal — not just answer, but do a task (look something up, draft a doc, run a check) across multiple steps.
Reality: the most over-hyped term. In government use it should stay bounded and human-confirmed — it proposes and acts within guardrails; a person still decides. Not an autonomous employee.