“Is our organization ready to use the AI capability it is building?”
Readiness depends on more than the system: people need the capacity, competence and authority to use it. Assess the controls and the operating environment separately, then measure results against agreed criteria.
Technical readiness creates an opportunity. The working environment determines whether people can use it consistently.
A system may retrieve current documents, connect project records and produce a clear recommendation. Yet the reviewer may lack time, the project manager may lack authority, or the organization may discourage reporting unwelcome information.
Under those conditions, a more capable tool can deliver more recommendations into the same unresolved workflow.
This connects directly to our ongoing AI implementation design and Project Control Index development. Project Control Index is the working name for our developing assessment approach, not a validated benchmark. Our project delivery experience informs the questions we are testing. We are examining how evidence, assessment, findings, approvals and action fit together. The operational question is whether those arrangements work when a real team must use them.
Reading the ASU-led IP2M METRR research, whose team includes Professor David Grau, gave me an important shift in perspective. Its attention to both maturity and the operating environment sharpened a question for our own work: are we assessing the system closely enough while taking its operating conditions for granted?
DOE's IP2M METRR training overview provides background on this work. Integrated Project/Program Management Maturity and Environment Total Risk Rating (IP2M METRR) examines earned value management system (EVMS) maturity and environment. Its findings do not directly validate an AI implementation model or establish a causal formula for AI performance. They provide a research lens that we intend to investigate further in our own assessment and implementation work.
For us, that means examining whether people can challenge suggested evidence, express disagreement during review, secure time to assess a finding and obtain a timely decision from the authorized person. It also means asking whether lessons from one run are carried into the next.
The dilemma: Investing in technical capability can appear more tangible than improving review capacity, incentives and decision rights. But expanding the technology without those conditions can increase activity without improving the organization's ability to respond.
Our developing assessment workflow already separates process completeness, control integrity and outcomes. My reading of that research motivates us to examine the environmental dimension more deeply, alongside other established frameworks, and test how it should influence our questions and priorities. We are describing a direction of investigation, not announcing a validated new index.
Assess three dimensions in one case
The following is an illustrative review, not a scored assessment or a validated AI maturity instrument.
| Dimension | Evidence to examine | Example finding and response |
|---|---|---|
| System maturity | Traceability, source/version checks, exception handling, approval records and repeatability. | The revision is linked correctly, but an unresolved source conflict has no defined route. Assign a route and test the exception before wider use. |
| Enabling environment | Reviewer time and competence, backup coverage, decision rights, escalation and freedom to challenge. | The designated reviewer has no backup. Provide capacity and delegated coverage; do not treat a named role alone as an effective control. |
| Observed outcomes | Accepted-response time, review and correction effort, rework and relevant project results against agreed criteria. | Results are recorded but no comparable baseline exists. Report the observations; do not claim improvement or attribute it to AI. |
Three dimensions assessed in one illustrative case.
- Dimension
- System maturity
- Evidence to examine
- Traceability, source/version checks, exception handling, approval records and repeatability.
- Example finding and response
- The revision is linked correctly, but an unresolved source conflict has no defined route. Assign a route and test the exception before wider use.
- Dimension
- Enabling environment
- Evidence to examine
- Reviewer time and competence, backup coverage, decision rights, escalation and freedom to challenge.
- Example finding and response
- The designated reviewer has no backup. Provide capacity and delegated coverage; do not treat a named role alone as an effective control.
- Dimension
- Observed outcomes
- Evidence to examine
- Accepted-response time, review and correction effort, rework and relevant project results against agreed criteria.
- Example finding and response
- Results are recorded but no comparable baseline exists. Report the observations; do not claim improvement or attribute it to AI.
Keep the findings separate. A successful case may depend on exceptional effort; a well-controlled process may still produce an unfavorable project outcome. Assess repeatability and contributing factors before drawing wider conclusions.
Do-how—review the technology and its environment in the same case. Follow one AI-supported assessment through evidence review, agreement, finding, approval and action. Check both whether the controls worked and whether the team had the capacity and authority to use them. Record disagreement, delays and unresolved conditions. Assign an improvement, then use a subsequent assessment to examine what changed.
Apply this to your project: If your strongest reviewer were unavailable, would the process still work?
Scenario combinations — maturity and use
Three dimensions: System maturity · Enabling environment · Observed outcomes.
System maturity concerns dependable controls and workflows. An enabling environment concerns the capacity, incentives, behavior and authority needed to use them. Observed outcomes concern measured results after implementation and action.
Boundary condition: the evidence for each dimension must be assessed independently. This is not a choose-two trade-off: two established dimensions neither exclude nor prove the third. In the scenarios below, an unestablished dimension is unknown unless evidence demonstrates a deficiency.
The Maturity Dilemma
System maturity · Enabling environment · Observed outcomes
Explore what each combination of evidence establishes—and what it leaves unknown.
Boundary condition: the evidence for each dimension must be assessed independently. This is not a choose-two trade-off: two established dimensions neither exclude nor prove the third.
ON = evidenced in this case.
OFF = unknown unless evidence demonstrates a deficiency.
M1: System maturity + enabling environment. Strong foundations. Outcomes still need measurement.
Strong foundations. Outcomes still need measurement.
- Situation and consequence
- Improved outcomes still require measurement. A sound system and supportive conditions do not prove that benefits have occurred.
- Practical response
- Establish a baseline and follow actual use, decision latency, rework, review effort and relevant project results. Distinguish implementation activity from benefit.
Compare all four scenarios
M1System maturity + enabling environment
- Situation and consequence
- Improved outcomes still require measurement. A sound system and supportive conditions do not prove that benefits have occurred.
- Practical response
- Establish a baseline and follow actual use, decision latency, rework, review effort and relevant project results. Distinguish implementation activity from benefit.
Strong foundations. Outcomes still need measurement.
M2System maturity + observed outcomes
- Situation and consequence
- An enabling environment is not established by those two observations. Results may rely on exceptional individuals, temporary support or favorable conditions.
- Practical response
- Check workload, reviewer capacity, decision rights, challenge culture and repeatability across teams. Do not infer a sustainable environment from one successful case.
Results observed. Operating conditions still need assessment.
M3Enabling environment + observed outcomes
- Situation and consequence
- System maturity is not established. Skilled people may compensate for inconsistent controls or manual workarounds.
- Practical response
- Examine evidence handling, repeatability, exceptions and whether performance survives changes in personnel or workload.
Results observed. System maturity still needs verification.
M4All three evidenced
- Situation and consequence
- All three can be evidenced, but recorded outcomes may be favorable or unfavorable. Effective use requires examining results against agreed acceptance criteria, repeatability and other contributing factors; it cannot be inferred from measurement alone.
- Practical response
- Monitor over time, examine other contributing factors and compare against the baseline; use findings to improve both the system and its environment.
All three assessed. Judge results against agreed criteria.
M1System maturity + enabling environment
- Situation and consequence
- Improved outcomes still require measurement. A sound system and supportive conditions do not prove that benefits have occurred.
- Practical response
- Establish a baseline and follow actual use, decision latency, rework, review effort and relevant project results. Distinguish implementation activity from benefit.
Strong foundations. Outcomes still need measurement.
M2System maturity + observed outcomes
- Situation and consequence
- An enabling environment is not established by those two observations. Results may rely on exceptional individuals, temporary support or favorable conditions.
- Practical response
- Check workload, reviewer capacity, decision rights, challenge culture and repeatability across teams. Do not infer a sustainable environment from one successful case.
Results observed. Operating conditions still need assessment.
M3Enabling environment + observed outcomes
- Situation and consequence
- System maturity is not established. Skilled people may compensate for inconsistent controls or manual workarounds.
- Practical response
- Examine evidence handling, repeatability, exceptions and whether performance survives changes in personnel or workload.
Results observed. System maturity still needs verification.
M4All three evidenced
- Situation and consequence
- All three can be evidenced, but recorded outcomes may be favorable or unfavorable. Effective use requires examining results against agreed acceptance criteria, repeatability and other contributing factors; it cannot be inferred from measurement alone.
- Practical response
- Monitor over time, examine other contributing factors and compare against the baseline; use findings to improve both the system and its environment.
All three assessed. Judge results against agreed criteria.
Research reference
Discuss the conditions for effective use
For xPM, a Project Controls and decision-support consultancy, readiness includes how the team will work. If a capable system is producing recommendations that the team cannot consistently review or act on, we can discuss the gap between technical capability and operating conditions. One assessment case can give the conversation a concrete basis: who reviewed it, what delayed action and what evidence of the result exists.
Continue the series
Structure First. AI Last. A Named Human Decides.