How we score a request
A panel scores every request against 8 criteria. We use the same 8 every time, so we can compare one request with another.
We score against what the team sends us, any documents they attach, and the conversation we have with them. The panel talks each one through and agrees a colour.
What the colours mean
- Green
- Strong. This part supports using AI here.
- Amber
- Not proven yet. We need to see more before we decide.
- Red
- Weak or missing. It needs a lot of work, or AI is the wrong answer.
The 8 criteria answer 4 questions.
Is the problem worth solving?
Business value
Would AI make a real difference, or would a simpler fix do just as well? We score green when a team can show in numbers what AI would save them.
Impact on people
Who is affected, and how do we know? A green score needs research or first-hand experience behind it. A claim with nothing behind it scores red.
Is it ready to build on?
Data readiness
Is the data there, and can we use it? Tidy, well understood data that we can reach scores green. Scattered across formats with no tagging, and it is a red.
Process stability
AI is hard to add to a way of working that keeps changing. A process mapped from start to finish, and holding still, is what green looks like. One being redesigned is a red.
Is AI the right answer?
How well AI fits
Language, searching and sorting are what AI is good at, and that work earns a green. A task with one right answer every time scores red, because AI would make it less reliable.
Risk
What happens if it gets something wrong? Here green means low risk, so the scale runs the opposite way to the others. We look at safety first, then money, reputation and the law.
Will it help anyone else?
Reuse by other teams
Could another team pick up what we build? If the work leaves behind something someone else could use, that is a green.
Reach across Defra
How far does it spread? Work that reaches several organisations in Defra scores green. Work that stays with the team who asked scores red.
How we read the scores
We look at all 8 scores together. The pattern matters more than the count.
Three patterns come up often:
- The idea would help a lot of people, but the team is not ready. We come back to it later.
- The work is sound, but another Defra team is better placed to build it. We pass it on.
- Only part of the request suits AI. We take that part and refer the rest.
What we usually see
Some patterns repeat across the requests we have scored so far.
The need is almost always real. Nearly every request scores green or amber for the people it would help, and most would help more than one team. Teams rarely bring us a problem that affects only them.
About half the time, a team can show in numbers what AI would save. The rest have a real problem with no figure attached yet. We ask for one before we score it green.
What holds a request back is usually the data. Data readiness scores red more often than anything else. The idea is sound, but the information it needs is spread out, patchy or hard to reach.
The process underneath is the next most common problem. It is hard to build AI into a way of working that is still being redesigned.
Risk sits in the middle almost every time. Almost nothing we see carries a safety risk. Almost everything puts money or trust at stake if it goes wrong.
Taken together, the ideas are worth doing and would help a lot of people. What usually holds them up is the data and the process underneath.
Send us a problem to look at
Tell the AI Capability and Enablement team about a problem in your area and we will score it with you.