Most AI roadmaps list new pilots, but the real work is a list of open data defects. Treat the roadmap as a remediations backlog, and burn it down before you add more pilots.
The slide and the backlog
Every vendor deck has the same slide. It shows the next model, the next agent, and a calendar full of pilots.
The slide is easy to approve. It promises progress without asking what is broken today.
Under every pilot on that slide sits a data path. Orders flow into billing. Claims flow into reserves. Customer records flow into credit decisions.
Each path has known defects. Somebody already knows about most of them. They sit in tickets, in spreadsheets, or in the head of one tired engineer.
The slide does not show those defects. It shows a clean line from today to the next model. That gap between the slide and the data is where most AI plans get stuck.
Why the pilot calendar keeps slipping
A pilot meets a defect, and the pilot stalls. The team then points at the prompt, the vendor, or the model.
The cause is usually older and more boring:
- A field is null when it should not be.
- Two systems disagree on what counts as an active customer.
- A daily load fails quietly once a week.
None of these defects are new. The pilot only made them visible.
So the honest version of the roadmap is not a list of pilots. It is a list of defects you must close before those pilots can work.
What goes in the backlog
Start with the paths that carry money and exposure. I would not try to fix all data at once. Pick the revenue and risk paths, and write down every open defect on them.
For each defect, I would record these items:
- The data path it sits on, such as quote to cash or claim to reserve.
- What is wrong, in one plain sentence.
- The decision or report it breaks.
- The owner who can fix it, by name and team.
- How you detect it today, if at all.
- When it was first seen.
- Which planned pilots depend on that path.
That last item matters most. It turns a quiet data ticket into a visible blocker on the roadmap.
Keep the backlog in one shared place that leaders can see. A defect list that lives only inside the data team will not change any roadmap decision.
How to rank the defects
A backlog without a rank becomes another place where work goes to wait. Rank each defect with a few plain questions:
- Does it touch money that customers pay or the business owes?
- Does it touch regulatory, credit, or safety risk?
- Do people make decisions on it every day, or only once a quarter?
- Does anyone detect it today, or does it fail silently?
- How many planned pilots depend on it?
- Can one owner fix it, or does it need several teams?
Silent defects on revenue and risk paths go to the top. A loud defect that someone already checks by hand can wait a little.
Do not rank by how easy the fix is. Easy fixes feel good, but they rarely unblock the pilot that leadership cares about.
Revisit the rank every month. New pilots add new dependencies. A defect that looked minor can move to the top when a new use case starts to rely on that path.
A burn-down cadence before new pilots
The backlog only works if it moves. I would run it on a fixed weekly rhythm.
Here is the cadence I would use:
- Review the top defects each week with the owners in the room.
- Close, reassign, or split any defect that has not moved in two weeks.
- Add a check for each fixed defect, so it cannot return quietly.
- Track open defects per path, and show the count next to the pilot list.
- Approve a new pilot only when its data path is below an agreed defect limit.
The fifth step is the hard one. It means a pilot waits when its data is not ready. That wait is cheaper than a pilot that fails in front of a customer.
What changes for leaders
This approach changes the questions leaders ask. Instead of asking which model to buy next, they ask which path is ready.
I expect three changes when a team works this way:
- Pilots start later but stall less often.
- Data owners get credit for fixes that were invisible before.
- Vendor demos get judged against your real defects, not their sample data.
It also changes the conversation with vendors. When a vendor shows the next model slide, ask which of your open defects it fixes. Usually the answer is none.
That is fine. The model is not the problem. But it means the vendor slide is not your roadmap.
Your roadmap is the backlog. The pilots are what you earn when the backlog gets shorter.
Ship the fixes first
Your next AI win probably will not come from a new model. It will come from closing the defects that already block the pilots you have.
Open the backlog, rank it, and burn it down every week. Then let the pilot calendar grow.