Case Study — AI-First Engineering Maturity
From copilots to coding factories
Helping a 120-person engineering organisation track and accelerate its journey to AI-first product development.
The CTO wanted to move the organisation toward an AI maturity model but had no structured way to know where engineering teams actually stood, where they were heading, or how to set meaningful targets. Copilot usage was reportedly up — but with no measurement behind it, nobody could say whether that meant real capability or just faster keystrokes.
Snowberry designed an AI-first engineering maturity model anchored by a long-term ambition, and — because self-reported scores are only as honest as the person filling in the form — built a proprietary application that reads the signal directly from the client's GitHub organisation instead of asking teams to report on themselves.
“Code must not be written by humans. Code must not be reviewed by humans.” The North Star the model is anchored to
The model
Five levels — L0 (Artisan) through L4 (Autonomous Factory) — gave the organisation a shared language and a measurable target. At baseline, the org's centre of gravity sat at L2: agents were already opening pull requests, but the bulk of routine work hadn't yet moved to a supervised, multi-agent pipeline.
Engineers write, review, test and deploy everything by hand. AI is not present anywhere in the workflow.
Same workflow, same humans, but engineers use an AI copilot to write code faster. The process itself has not changed.
Engineers hand a ticket or bug to an AI agent. The agent independently determines what to change, writes the fix, and opens a pull request. A human reads and approves every PR before it merges.
A defined, cross-team workflow orchestrates multiple agents end-to-end — writing the fix, reviewing it, running security checks, creating and running tests — with humans setting guardrails and approving at explicit gates, not reviewing every line.
For low-risk, reversible work, the system detects issues, fixes them, verifies them and ships them before any human is notified. Humans set direction, policy and guardrails. The factory executes.
How the scoring works
Self-assessment supplements automated signals, most heavily at L3-L4 where intent and governance context matter more than what a repo scan alone can tell. Step through a level below to see its questions, its objective signals, and try scoring a team of your own.
Self-score + objective signal = validated maturity score.
L0 Artisan Manual / no AI
Engineers write, review, test and deploy everything by hand. AI is not present anywhere in the workflow.
A PR looks exactly like it would have five years ago — no AI fingerprints anywhere in the diff or the pipeline.
Self-assessment (0 = No, 3 = Yes)
- Engineers write all production code without any AI tool assistance
- All PRs are opened, reviewed and merged entirely by humans
- No AI-generated content exists in our CI/CD pipeline or test suite
- The team has not experimented with any coding agent or AI workflow tool
Your team's L0 score: 0%
Objective signals
- AI copilot acceptance rate in IDE = 0%
- 0 agent-opened PRs in the last 30 days
- No AI step in any CI/CD pipeline configuration
L0.5 Configured The often-skipped step
AI tools are installed and licensed, but the team has deliberately set them up: written team-specific instructions, defined which work types are appropriate for delegation, and established shared norms.
The repo has a copilot-instructions file and a norm for reviewing AI output — the infrastructure L1-L2 adoption actually depends on.
Self-assessment (0 = No, 3 = Yes)
- We have written team-specific instructions for our AI coding tools
- We have documented which ticket/work types are suitable for agent delegation
- There is a shared team norm for reviewing and accepting AI-generated code
Your team's L0.5 score: 0%
Objective signals
- copilot-instructions.md or equivalent in ≥50% of team repos
- AGENTS.md or a delegation-criteria doc in any repo
- AI-specific branch protection rules or PR templates
L1 Assisted Individual helper
Same workflow, same humans, but engineers use an AI copilot to write code faster. The process itself has not changed.
An engineer opens the IDE, a copilot suggests the next few lines, they accept or reject — the PR still starts and ends with a human.
Self-assessment (0 = No, 3 = Yes)
- The majority of engineers have an active AI coding tool installed and used regularly
- AI suggestions are accepted for a meaningful share of code written
- We have started using AI to assist with tests, documentation or PR descriptions
- There is a shared team norm or guidance for how AI tools should be used
Your team's L1 score: 0%
Objective signals
- ≥70% of engineers show active AI tool usage in the last 30 days
- AI code acceptance rate ≥20%
- All PRs still opened by a human — no agent-opened PRs
L2 Delegated Delivery Agent opens the PR
Engineers hand a ticket or bug to an AI agent. The agent independently determines what to change, writes the fix, and opens a pull request. A human reads and approves every PR before it merges.
A ticket gets assigned to an agent instead of a person; twenty minutes later a PR is open and waiting for human review.
Self-assessment (0 = No, 3 = Yes)
- We regularly assign tickets or bugs directly to an AI agent that produces a PR without step-by-step guidance
- We have defined and documented which work types are appropriate for agent delegation
- The agent's PRs go through the same review and approval gate as human PRs
- We track agent PR acceptance rate and use it to improve delegation criteria
Your team's L2 score: 0%
Objective signals
- ≥10% of merged PRs opened by an AI agent in the last 30 days
- Median time from ticket assignment to agent PR open under 60 minutes
- Agent PR merge rate ≥50%
L3 Supervised Factory End-to-end pipeline, gated approval
A defined, cross-team workflow orchestrates multiple agents end-to-end — writing the fix, reviewing it, running security checks, creating and running tests — with humans setting guardrails and approving at explicit gates, not reviewing every line.
A bug ticket runs through detection, fix, automated review, security scan and test generation before a human ever opens it, and only stops for approval at a defined checkpoint.
Self-assessment (0 = No, 3 = Yes)
- We have a standard multi-agent workflow that handles tickets from detection through PR, review and testing without per-step human input
- Automated security scanning is a mandatory, agent-triggered gate in the pipeline
- Automated test generation and execution runs on every agent-opened PR before a human sees it
- Human approval is required only at defined, documented checkpoints
Your team's L3 score: 0%
Objective signals
- ≥50% of routine work passes through the multi-agent pipeline
- Agent-generated PRs meet the team's automated test-coverage threshold
- Security scan pass rate on agent PRs tracked and reported quarterly
L4 Autonomous Factory Ships before anyone is notified
For low-risk, reversible work, the system detects issues, fixes them, verifies them and ships them before any human is notified. Humans set direction, policy and guardrails. The factory executes.
A dependency vulnerability is detected, patched, tested and deployed overnight — the team learns about it from the changelog, not a ticket.
Self-assessment (0 = No, 3 = Yes)
- A defined class of low-risk changes ships to production without human review or approval
- The system automatically detects, diagnoses and initiates a fix for certain issue classes
- Rollback is automated and regularly tested — we can prove it works
- Engineers spend the majority of their time on direction and exceptions, not code production
Your team's L4 score: 0%
Objective signals
- % of production deployments triggered autonomously (no human-initiated deploy step)
- Automated rollback test pass rate ≥95%
- Human code review involvement rate for the autonomous pipeline under 10%
How the application works
Scoring isn't self-reported. Snowberry's code analysis application connects to the client's GitHub organisation, and for each tech area, a defined set of repositories is analysed continuously — not just at quarterly workshop time.
- GitHub org
- Snowberry agents
- Maturity scores
- Dashboard
- PR authorship Human vs. agent-opened, read directly from GitHub PR author metadata.
- Agent PR quality Merge rate, time-to-merge, review comment volume, revert rate.
- Pipeline composition Presence of AI-driven steps, automated security scans, test generation in CI/CD.
- AI configuration files copilot-instructions.md, AGENTS.md, AI-specific PR templates and branch rules.
- Deployment patterns Human-initiated vs. autonomously triggered deploys.
- Strategy alignment Repo-level signals mapped against the client's own engineering strategy principles.
“Teams stop collecting data. They start acting on it.”
Where they are
Six product teams, tracked quarter over quarter. The model catches forward movement and regression alike — two teams pushed from L2 into L3 in a single quarter, and one slid back, exactly the kind of signal a once-a-year survey would have missed entirely.
| Team | L0 | L1 | L2 | L3 | L4 |
|---|---|---|---|---|---|
| Team A | 0% | 5% | 45% | 50% | 0% |
| Team B | 0% | 15% | 60% | 25% | 0% |
| Team C | 5% | 50% | 35% | 10% | 0% |
| Team D | 0% | 25% | 60% | 15% | 0% |
| Team E | 0% | 10% | 40% | 50% | 0% |
| Team F | 5% | 40% | 45% | 10% | 0% |
Continuous tracking
Because the application runs continuously rather than quarterly, the website's view of it is a dashboard, not a snapshot — org-level distribution, per-team velocity, and individual repos worth a closer look.
Org distribution
Updated: Q1 20262.0/4
- L0 2%
- L1 25%
- L2 59%
- L3 14%
- L4 0%
Team velocity
| Team | Level | Q1 2026 | Q4 2025 | Trend |
|---|---|---|---|---|
| Team A | L3 | 2.5 | 2.1 | ↑ up |
| Team B | L2 | 2.1 | 1.9 | ↑ up |
| Team C | L1 | 1.5 | 1.6 | ↓ down |
| Team D | L2 | 1.9 | 1.8 | ↑ up |
| Team E | L3 | 2.4 | 2.0 | ↑ up |
| Team F | L2 | 1.6 | 1.6 | → flat |
Repo spotlight
- Team A / repo-checkout L3.6 Leading the team, closest to L4
- Team C / repo-payments L0.8 Pulling behind the team average
- Team F / repo-notifications L1.2 Stuck at the same level for 2+ quarters
This engagement is ongoing. Scores are tracked quarterly, and the application keeps running between check-ins — quarterly time now goes to interpreting data the team already has, not collecting it.
Want to understand where your engineering organisation sits on the AI maturity curve? Let's map it together.
Get in touch