“How do we choose an AI pilot worth expanding?”
Choose a recurring task with a clear boundary, review capacity and a measurable baseline. Agree on the evidence for expansion—and the conditions for revision or a stop—before the pilot starts.
Technology changes. The questions about work, trust, value and accountability return.
Our “Prediction AI in Project Controls” presentation materials use this pattern to frame today’s AI concerns. It is an interpretive analogy, not a claim that every technology follows the same history or that past adoption proves the case for an AI application.
For a project team, the concerns are concrete:
| Concern | What people need answered | What implementation must establish |
|---|---|---|
| Labor — “Will it replace me?” | How will my work, judgment and responsibilities change? | Defined tasks, training, reviewer capacity and a clear account of what remains with people. |
| Data — “Can I trust it?” | Which information supports this answer, and can I challenge it? | Permitted sources, authority, version, context and a review route. |
| Cost — “Where is the value?” | Will the benefit survive the cost of integration, checking and correction? | A baseline and measures of total effort, quality and time to an accepted response. |
| Accountability — “Who answers when it is wrong?” | Who may approve, intervene or stop the workflow? | Named decision owners, permission limits and escalation responsibilities. |
Four recurring AI concerns and what implementation must establish to answer them.
- Concern
- Labor — “Will it replace me?”
- What people need answered
- How will my work, judgment and responsibilities change?
- What implementation must establish
- Defined tasks, training, reviewer capacity and a clear account of what remains with people.
- Concern
- Data — “Can I trust it?”
- What people need answered
- Which information supports this answer, and can I challenge it?
- What implementation must establish
- Permitted sources, authority, version, context and a review route.
- Concern
- Cost — “Where is the value?”
- What people need answered
- Will the benefit survive the cost of integration, checking and correction?
- What implementation must establish
- A baseline and measures of total effort, quality and time to an accepted response.
- Concern
- Accountability — “Who answers when it is wrong?”
- What people need answered
- Who may approve, intervene or stop the workflow?
- What implementation must establish
- Named decision owners, permission limits and escalation responsibilities.
These concerns deserve operational answers. Calling them resistance misses the work needed to earn adoption. Equally, waiting for every uncertainty to disappear leaves the organization with little practical experience on which to base its next decision.
The dilemma: A team needs experience to judge where AI is useful, but it needs boundaries before gaining that experience on consequential project work. Broad deployment does not resolve that tension. A limited, measurable application can create a responsible path through it.
In an illustrative design-revision case, AI might summarize a document and suggest affected records. Accepting an impact, revising a baseline or directing work requires separate authority. The first application can start before every system is integrated, provided its sources, scope and review conditions are clear.
This is where “AI last” matters: it describes an architectural dependency, not an instruction to postpone all experimentation. Even a small pilot needs the minimum foundation for its intended use: controlled sources, appropriate context, permissions, review and a named owner. A document-summary pilot and a forecast supporting a major commitment require different levels of assurance. Prediction AI also needs a suitable model, relevant backtesting and calibration, and explicit uncertainty. AI can help prepare the foundation; it cannot replace the conditions required for consequential use.
Different AI roles need different checks
| Role | Intended work | Minimum assurance question |
|---|---|---|
| Generative AI | Summarize a controlled revision and suggest relevant records. | Can the reviewer trace material claims to permitted sources and check omissions and unsupported statements? |
| Prediction AI | Estimate plausible cost or schedule effects. | Is the model suitable for this use, tested on relevant outcomes and calibrated where applicable, with uncertainty explicit? |
| Agentic AI | Retrieve, route or monitor within approved permissions. | Which actions are permitted, logged and stoppable; which require approval; and how are errors contained or reversed where feasible? |
Minimum assurance questions by AI role.
- Role
- Generative AI
- Intended work
- Summarize a controlled revision and suggest relevant records.
- Minimum assurance question
- Can the reviewer trace material claims to permitted sources and check omissions and unsupported statements?
- Role
- Prediction AI
- Intended work
- Estimate plausible cost or schedule effects.
- Minimum assurance question
- Is the model suitable for this use, tested on relevant outcomes and calibrated where applicable, with uncertainty explicit?
- Role
- Agentic AI
- Intended work
- Retrieve, route or monitor within approved permissions.
- Minimum assurance question
- Which actions are permitted, logged and stoppable; which require approval; and how are errors contained or reversed where feasible?
A workflow may combine these roles. Passing the checks for one does not authorize the others. The NIST AI Risk Management Framework 1.0 (2023) provides broader guidance on governing, mapping, measuring and managing AI risks. The role checks and pilot agreement here are xPM's practical synthesis, not a NIST-prescribed template.
Do-how—turn the four concerns into one pilot agreement. Choose a recurring task with the people who perform and review it. Record what changes in their work, the permitted evidence, the approval boundary and the stop conditions. Establish current performance, then measure review effort, corrections and time to an accepted response during the pilot. Update the procedure before extending it to another team.
Apply this to your project: What evidence would make you expand your current pilot—and what would make you stop it?
A pilot agreement the team can use
The example below is a proposed template for a document-review pilot, not a report of measured xPM results. Complete the named roles, dates, sample and acceptance criteria before starting.
| Agreement field | Example entry / decision to complete |
|---|---|
| Task and boundary | Summarize controlled design revisions and suggest affected records for one selected work package. No automatic work instruction, purchase commitment or baseline change. |
| People and capacity | Name the process owner, technical reviewer, backup and approving manager. Allocate review time and training; record the effect on existing responsibilities. |
| Permitted inputs | Record project, document ID, governing revision, as-of date and source owner; flag conflicting or missing records for review. |
| Baseline and comparison | Record current review effort, correction effort and time to an accepted response on comparable tasks. Define the sample and review period before the pilot. |
| Acceptance criteria | Agree on quality, traceability, total-effort and turnaround criteria with the responsible team. Set use-specific limits; do not invent a universal pass score. |
| Stop / containment | Suspend the affected workflow if it exceeds permissions, discloses restricted content or repeatedly supplies unsupported material claims. Name who can stop it and the manual fallback. |
| Decision at review date | Expand only if agreed criteria are met and the next team is ready; revise if remediable gaps remain; stop if the use cannot be justified. Record the reasons. |
Proposed pilot agreement template for a document-review pilot.
- Agreement field
- Task and boundary
- Example entry / decision to complete
- Summarize controlled design revisions and suggest affected records for one selected work package. No automatic work instruction, purchase commitment or baseline change.
- Agreement field
- People and capacity
- Example entry / decision to complete
- Name the process owner, technical reviewer, backup and approving manager. Allocate review time and training; record the effect on existing responsibilities.
- Agreement field
- Permitted inputs
- Example entry / decision to complete
- Record project, document ID, governing revision, as-of date and source owner; flag conflicting or missing records for review.
- Agreement field
- Baseline and comparison
- Example entry / decision to complete
- Record current review effort, correction effort and time to an accepted response on comparable tasks. Define the sample and review period before the pilot.
- Agreement field
- Acceptance criteria
- Example entry / decision to complete
- Agree on quality, traceability, total-effort and turnaround criteria with the responsible team. Set use-specific limits; do not invent a universal pass score.
- Agreement field
- Stop / containment
- Example entry / decision to complete
- Suspend the affected workflow if it exceeds permissions, discloses restricted content or repeatedly supplies unsupported material claims. Name who can stop it and the manual fallback.
- Agreement field
- Decision at review date
- Example entry / decision to complete
- Expand only if agreed criteria are met and the next team is ready; revise if remediable gaps remain; stop if the use cannot be justified. Record the reasons.
Use one shared record throughout the pilot. A faster draft is useful only if the accepted response is better or less costly after review and correction.
Scenario combinations — AI adoption
Readiness means that the controls, people and evidence required for the stated use have been checked. It is not a guarantee of future value. Labor, data, cost and accountability are the four readiness questions behind these labels.
THE AI ADOPTION DILEMMA
Three priorities. Different implementation conditions.
Explore what changes when launch speed, rollout scope and verified readiness are prioritized together.
Trade-off conditions: a new application, uneven preparation, a short deadline and fixed implementation/review capacity.
ON = included in this scenario.
OFF = not prioritized in this scenario.
A2: Fast launch + verified readiness. Ready to start. Limited rollout.
Ready to start. Limited rollout.
- Situation and consequence
- Rollout is limited to the task and teams whose conditions can be checked within the available time.
- Practical response
- Start with a bounded use case, named reviewers and stop conditions; expand only when the next team's readiness is established.
Compare all four scenarios
A1Fast launch + enterprise-wide rollout
- Situation and consequence
- Readiness across all teams cannot be assumed. Source mapping, role definition, training and review capacity may not be established before launch. This describes a risk, not an acceptable route to consequential use.
- Practical response
- Restrict unready teams to approved learning activities; verify readiness before granting consequential permissions. Reduce scope or add preparation time and capacity.
Deployed widely. Readiness not yet verified across teams.
A2Fast launch + verified readiness
- Situation and consequence
- Rollout is limited to the task and teams whose conditions can be checked within the available time.
- Practical response
- Start with a bounded use case, named reviewers and stop conditions; expand only when the next team's readiness is established.
Ready to start. Limited rollout.
A3Enterprise-wide rollout + verified readiness
- Situation and consequence
- The fast-launch target may move because readiness must be established across different records, teams and operating conditions.
- Practical response
- Complete staged preparation, source checks, training and review arrangements before widening permissions.
Ready across teams. More preparation time required.
A4All three supported
- Situation and consequence
- Possible when the necessary foundation already exists or adequate preparation and capacity are available. There is no inherent three-way impossibility.
- Practical response
- Verify the existing foundation for this use; retain monitoring, review and outcome measurement after launch.
Prepared foundations. Fast, governed rollout.
A1Fast launch + enterprise-wide rollout
- Situation and consequence
- Readiness across all teams cannot be assumed. Source mapping, role definition, training and review capacity may not be established before launch. This describes a risk, not an acceptable route to consequential use.
- Practical response
- Restrict unready teams to approved learning activities; verify readiness before granting consequential permissions. Reduce scope or add preparation time and capacity.
Deployed widely. Readiness not yet verified across teams.
A2Fast launch + verified readiness
- Situation and consequence
- Rollout is limited to the task and teams whose conditions can be checked within the available time.
- Practical response
- Start with a bounded use case, named reviewers and stop conditions; expand only when the next team's readiness is established.
Ready to start. Limited rollout.
A3Enterprise-wide rollout + verified readiness
- Situation and consequence
- The fast-launch target may move because readiness must be established across different records, teams and operating conditions.
- Practical response
- Complete staged preparation, source checks, training and review arrangements before widening permissions.
Ready across teams. More preparation time required.
A4All three supported
- Situation and consequence
- Possible when the necessary foundation already exists or adequate preparation and capacity are available. There is no inherent three-way impossibility.
- Practical response
- Verify the existing foundation for this use; retain monitoring, review and outcome measurement after launch.
Prepared foundations. Fast, governed rollout.
Discuss a pilot worth testing
As a Project Controls and decision-support consultancy, xPM starts with the project task and the people accountable for it. If your team has an AI use case but the boundary or success criteria remain unclear, that is a useful starting point for a conversation with xPM. We can discuss one recurring task, its evidence and review requirements, and what would justify extending it. The pilot agreement above provides a practical basis for that discussion.
Continue the series
Structure First. AI Last. A Named Human Decides.
Sources and method note
- NIST — Artificial Intelligence Risk Management Framework (AI RMF 1.0), 2023: general risk-management context, not validation of this pilot or its results.
- The four concerns, pilot agreement and A1–A4 scenarios are the author's implementation synthesis. They describe conditional choices, not a quantified law or observed xPM performance.