5 lessons implementing CD3 prioritisation at scale
Anna Miedzianowska, Product Manager, Ocado Technology
A couple of years ago, we supported Özlem Yüce’s talk at ACE Krakow. She’s an inspirational Chief Product Officer and coach, and we were so impressed by her talk about CD3 that we invited her to give presentations and training at our Polish and UK offices.
I was confident that CD3 would really improve our processes, but although good in theory, the new process quickly presented some challenges. The most burning issue was related to the single total number titled CoD (Cost of Delay), attached to the Elevator Pitch for each proposed project.
We started to see some huge numbers, and it was a mystery to me why requests with costs of delay as high as thousands of pounds a week would not be picked up the development teams instantly.
It was time to investigate the calculations behind CoDs provided, and this took me on the most interesting journey.
Not only did I learn more about various areas of the business, the process helped remove emotion, bias and personal influence from prioritisation conversations. So how did we do it?
Lesson 1: Uncover your assumptions
An interesting thing happened to our stakeholders: they became incredulous at other teams’ numbers. “No way this number can be so high, surely someone used revenue instead of profit for their calculations!”
This led to meetings where we openly reviewed the calculations and agreed on what value certain assumptions should hold. Not only did this help to align the values of all the projects, but it also helped to align all the people involved. Stakeholders became more aware of other departments’ needs and the value of their work.
Drilling down through the calculations, we uncovered some interesting assumptions. Let’s look at examples:
- ‘Saving X hours of developer support per week = £Y/week in value’
- ‘Extra X orders per week = £Y/week in value’
- ‘Increased conversion rate: 5% = £Y/week in value’
The first example is quite simple and obvious — you can use historical data to test this value.
The second example raises a question: is the value of extra orders presented as extra revenue or extra profit? It’s worth being explicit listing assumptions (it should be profit).
The third example is even more interesting. How do you know that this new feature is going to increase the conversion by exactly 5%? Maybe it will not change anything at all? A simple range would be better here: 0–5%. The problem is that we will not end up with a single number at the end. More on this later.
Lesson 2: It’s not about precision
A frequently asked question is, ‘Can you really quantify everything?’ My answer is, ‘Yes, but it’s not always trivial’.
Don’t try to get a single number answer for every project; sometimes a range will do as well. It’s all about reducing uncertainty: a wide range is better than no numbers at all. A narrow range is, of course, better than a wide range, but if it’s made up then it gives a false impression of precision and certainty. Remember why are you doing this calculation: it’s not about precision, it’s to help with understanding a relative value of all projects, so they can be prioritised against each other.
You can find more on quantifying everything topic in this book: How to measure anything by Douglas W. Hubbard.
Lesson 3: Change is the only constant
Question: what’s the difference between a Christmas tree sale in November versus December versus January?
Well, what’s the point of selling Christmas trees in January?
The value of Cost of Delay for any given project might depend on time (possibly even have ‘best before’ and ‘expiry’ dates), so understanding the urgency profile is crucial.
Saving a user two hours a day by automating a task will have the same value in December and in January, so delaying this work will only defer the benefits. Delaying a Christmas trees sale will result in reducing the total value of this work.
What all this means is that CoD should be reviewed periodically. Your updated CD3 might then change your prioritisation decision.
Lesson 4: Look beyond the numbers
CD3 works great for maximising the value of short term initiatives. The projects have (now) clearly defined benefits and effort estimates and because of that, we can compare many ‘uncomparable’ initiatives, a.k.a. apples and oranges.
But what about long-term research and innovation projects, something very important to Ocado Technology? Calculating value here can be much harder, and you have to look past the numbers, understand the real reason for that piece of work, and the potential value it can bring. Value here might be cultural, technological, corporate… you’ll need to be flexible, and the way you prioritise that value using CD3 will differ.
We use a joker card to override CD3 value if necessary.
You can find more innovation horizons in Lean Enterprise book by Jez Humble, Joanne Molesky and Barry O’Reilly.
Lesson 5: Be open to discussion
Maths clearly shows the financial value of CD3 prioritisation, but lots of value lies in the transparency and alignment this framework causes. I did not expect this side effect.
I spoke to some of my product owners about the process and what they’d observed. I think Lloyd, product owner in Payments and Routing, summed it up best:
“Adoption of CoD has made conversations with stakeholders a great deal easier. It’s a powerful tool for achieving one of our primary responsibilities — determining the correct priorities when balancing the ‘iron triangle’ of Value, Effort and Urgency for a given set of opportunities.
Firstly, it re-emphasises the importance of having good, clean data to calculate the CoD; if you don’t have this then you’ll fall at the first hurdle, and will fall back to error prone decision-making based on gut and intuition. Secondly, it’s rarely about precision when making CoD calculations; what you’re trying to prioritise are initiatives which may be value-adding by different orders of magnitude in terms of £ per week. Ultimately it’s better to have some CoD value assigned to every item — be it a wide range or at a low confidence interval — than none at all.
Finally, I don’t believe that it’s a silver bullet to ‘solve’ prioritisation, given the myriad subtleties which the real-world of product development throws up and the need for innovation, risk-taking and failure, but it does enable discussions to be data-driven.”
Originally published at https://www.ocadogroup.com.
