Choose a task, not a technology
Begin with recurring work that people can describe and review: sorting enquiries, extracting fields from a known document type, finding approved internal guidance or drafting a first response from supplied facts. Write down the current steps, volume, pain points and the person responsible for the final decision. If nobody can say what a good output looks like, the pilot is not ready.
Keep the first boundary tight. Name the users, inputs, output and situations that remain outside scope. NIST’s AI Risk Management Framework organises practical risk work around Govern, Map, Measure and Manage. For a small pilot, that translates into assigning ownership, understanding the use context, testing performance and deciding how issues will be handled.
Define success before building
Create a representative test set from the real task, with sensitive details removed or handled in an approved environment. Include ordinary examples, difficult examples and cases where the system should decline or ask for help. Decide what reviewers will record: factual support, completeness, unsafe disclosure, time to correct and whether the output was useful.
A single average score can hide the failure that matters most. Set acceptance thresholds by consequence. A drafting aid may tolerate more revision than a workflow that sends information to a customer. Define which mistakes require an immediate stop, who reviews them and what evidence would justify expanding the pilot.
Inspect the full workflow
The model is only one part of the result. Check where information comes from, whether it is current, what the provider retains, who can access the tool and where outputs go next. NIST’s Generative AI Profile highlights risks that may be amplified by generative systems and recommends actions across the AI lifecycle. Your pilot plan should therefore include input controls, human review, logging and a route for users to report unexpected behaviour.
Test with the people who will use the workflow. Observe where instructions are unclear, where they over-trust an answer and where the tool creates extra work. Record versions and changes so that a later result can be compared fairly with the original test.
Finish with a decision
At the end, choose to stop, revise, continue within the same boundary or prepare a separately governed expansion. Document what was tested, the known limits, the remaining human checks, ownership and expected operating cost. A pilot earns its value by reducing uncertainty and supporting that decision, even when the answer is not to deploy.
Use a short decision record that links the evidence to the choice. Include the tested version, test-set description, results by failure type, user feedback, unresolved risks and the person accepting the next step. If the pilot continues, set a review date and conditions that would pause use. This preserves what the team learned and makes future comparisons meaningful.