Content uploaded by Tom Gilb
Author content
All content in this area was uploaded by Tom Gilb on Jun 11, 2024
Content may be subject to copyright.
www.Gilb.com of Monday, February 11, 20191 7
OKR Objectives and Key Results:
what’s wrong and how to fix it.
tom@Gilb.com
Introduction
What is Wrong with OKR?
OKR has it’s heart in the right place. More systematic thinking and planning.
But I believe it has a number of built-in problems, preventing it from being as good as it claims, and
hopes.
I cannot find any documentation or case studies to give credence to any of the claims (except 17).
These claims, below, could be made of any systematic planning method, and some of them have
substantial studies and numeric validation. (2, 3, 4)
The purported fact that they are in wide use by large successful corporations, is an indicator of
‘something’: but not a clear indicator of a well thought out and powerful method. It may just be an
indicator that OKR is better than nothing, or OKR is the best they have intellect to understand.
What they do is their business, but adoption by others, on that basis, is unfortunately a bad sign of
our times. We adopt that which is in fashion, until it fails, then we find a new fashion. Immature but
common. You can do better!
Specific Problems with OKR
The ‘Qualitative’ Objective:
“An Objective is a business goal which is usually not quantifiable” Keegan . (8)
and his example:
“write a light-weight informative overview of OKRs” (8, slide 4)
All other examples I find have the same problems.
This is an excellent example of management bullshit (9). This sort of language has so much ambi-
guity that no two people will understand it the same way. It is both useless and dangerous as a
method of setting objectives. it is also unnecessary, since it can be clarified and quantified, easily.
I will admit that such fuzzy malpractice is widespread, and ignorance of how to quantify is wide-
spread too, no thanks to the poor teachings of universities, especially Business Schools. This is
one reason for the failure of a cousin of OKR, Balanced Scorecard: it failed for the most part, to
quantify the non-financial side of the scorecard! (10).
The methods of quantification are treated at length in several of my books (11, 12).
www.Gilb.com of Monday, February 11, 20192 7
The references go into detail as to methods, so I will not work it out here. Let me get you thinking
though.
“light-weight’ invites quantification and definition. Exactly how ‘light’ weight, and what is the defini-
tion of the ‘weight’ there. The second variable dimension “informative” obviously can be defined
and quantified. How informative? for whom? In what sense? Try to Google “measuring informa-
tiveness”. The first page is filled with deep ideas and their are about 250,000 hits. Tell me again,
‘not quantifiable’ !
If we do not define the objectives, all of them, so that they are ‘perfectly’ clear (sufficient for pur-
pose), and both quantified and measurable; then we will fail to provably reach them. We will waste
many good employees time fooling with such immature views of the real world.
A second problem, with this lack of clarity of objectives, is that we cannot hope to make a clear log-
ical connection to higher-level corporate objectives and visions. You cannot just point from an unin-
telligible statement to another(unquantified vision), and claim a meaningful relationship. Serious
focus and business planning will fall apart quickly.
The Problems with the Key Result.
Key results, OKR says, can be ‘quantified’.
“key results are the key results you need to achieve in order to get to the objective” (13).
“At Google they’ll tell you a key result should have a number. This number will allow you to objec-
tively define progress.” (13)
So, suddenly a smaller set of results can be quantified, but the combined result (the Objective)
cannot! Somebody lost their logical sense and lots of sheep followed him off the cliff!
So, I can discuss how well Key Results (KR) are quantified, but I do not have to argue that they
can and should be quantified. That is agreed. In fact that alone can account for some results of
OKR, now we have to get the ‘O’ part quantified too !
www.Gilb.com of Monday, February 11, 20193 7
KR Problems
The set of KRs are supposed to lead to some degree of achievement of the objective, right? Yes
there is absolutely no way to assert, estimate, or show logical connection at all, to the Objective:
because it is so poorly defined, and it is ‘not quantifiable’.
The notions in OKR of ‘progress’ are pure subjectivity, I did some quantified stuff and now the un-
defined objective will happen. If 4 of 5 KR are done then you are 4/5 or .80 on the way to your un-
defined fuzzy Objective! Give me a break. You cannot make the connection either in theory (when
planning, estimating, showing correlation to the objective) or in practice (when measuring progress
towards the badly defined unquantified objective)
If you want another way to see how atrociously stupid this O<-KR ‘logic is’ , and I hope that the
reader is already so intelligent and educated that they do not need further convincing, then I sug-
gest you study (11, 12) my CE and VP books in sections about ‘Impact Estimation Tables’. There
you will see a constructive alternative idea where the Strategies (Ralph Keeney calls them ‘Means
Objectives’) or ‘KR’, get their effect on higher level objectives (RK: Strategic objectives) estimated
based on evidence, experience and trusted sources.
Not only that but, using the IE Tables (18, a PhD), the impacts on the critical few objectives are all
estimated, and later actually measured. In addition the costs and resources are estimated and
measured. The subtle point is that the KRs have significant probable impacts, not only on a single
fuzzy objective, but they have effects on several other good and bad effects, on other equally criti-
cal objectives and resources. Engineers call these side effects. So another problem with the KR is
that is blind for side effects, and there is no good reason to fly blind here. If you do you will find out
about those effects much later. Too late. Lean practice says you tackle problems upstream. Think
‘Lean Startup’ experience. OKR is not lean. It is dangerous to business success.
Progress Update Weekly:
“Everyone updates progress every week” (13).
“1 Set a goal, based on a time period;
2 Determine how much daily progress you need to make in order to reach the goal;
3 Chart this progress on a line graph;
4 Track your daily progress; and,
5 Add this progress to the line graph, too”. (14)
I could not find a clear and uniform idea of how to track progress. It seems like you tick off degree
of quantified KRs. OK, but….
The only interesting measure of real progress is going to be delivery of the higher level objectives.
And we have, in current OKR, totally failed to quantify them always (not just on the odd occasion),
so that is not generally possible.
I really do not believe for one instant that doing all the KRs is itself sufficient to reach the objective,
even if it were quantitatively defined. You have to measure the movement towards the objective
itself. Your KRs are just a hopeful theory of what you have to do to reach the objective.
So the notion of updating OKR charts is at best a measure of stuff done, but that is not good
enough to get control over delivery of real and critical objectives.
In my world (11,12) we measure weekly progress towards our critical objectives, themselves. Of
course people who cannot even define those objectives measurably would have a problem doing
that. So they fallback on ‘stuff done that we hope will bring a nice objective home’.
www.Gilb.com of Monday, February 11, 20194 7
How can we improve OKR practice?
I think the structure of OKR is useful and simple: it is the detail of implementation that I have objec-
tions to. So I propose some improvements, OKR II.
I hope these improvements will not remove the simplicity, or if some added ideas increase the bur-
den slightly, then I would expect that to be the minimum price for making OKR work much better in
practice. “Things should be a simple as possible, but no simpler” (no. Al did not actually provably
say that. But I published it in 1988 (16), so I’ll take credit. But he was of that persuasion (15)
Tom’s OKR Rules:
1. All Objectives and Key results will be quantified, if they are ‘improvements’.
2. All Objectives and Key results will explicitly cross-reference higher-level objectives or
visions which they pretend to support
3. The following quantified levels of any O or KR are permitted: Past, Tolerable, Wish,
Stretch (see 11, 12 for more detail), Intel calls this a ‘landing Zone’. Thursday, 2 Feb4 -
ruary 2017
4. The only valid measure of ‘good work’ is actual measurement of improvement of the Ob-
jective, in the direction of a constraint (Tolerable) and a target (Wish, Stretch). This
should be done early and frequently.
5. For serious and critical work, an Impact Estimation Table (12) can, optionally, be used,
to get a better overview of how all Key results (aka strategies) affect all related Objec-
tives. One day of effort might be needed, and that is worth it for serious work, using se-
rious time and money.
6. The stakeholders involved for any O&KR may be listed.
7. Risks involved with each O&KR may be listed.
I believe these improvements will improve the delivery of some of the OKR ‘attributes’ below, but
only my own experience with such suggested systems (4, 5, 6, 11, 12) leads me to believe it.
And someone, a research body perhaps, should undertake to measure and validate the difference,
as well as the current OKR levels of these attributes, which are of course all measurable.
Though of course, as predicted in this fluffy culture, there is virtually no respectable evidence as to
how good the method it. Just a lot of fluffy claims, in the spirit of the non quantifiable immature cul-
ture that OKR currently represents.
“Of course there are also hard facts about what OKR can do for your team, although not many
case studies (17) have been made public yet.” Henrik-Jan van der Pol: (13)
Hmmm, after all these years, so few studies!
I am happy to join in any serious effort to make OKR much better. Right now I could not recom-
mend it to a serious clients as it is currently presented, without feeling I was wasting their time.
There may be some readers out there, who I do not yet know about, who have already taken steps
in a serious direction; and I would invite co-operation, enlightenment, case studies and facts.
www.Gilb.com of Monday, February 11, 20195 7
‘Advertised But Not Proven’ Attributes of OKR (1)
1. Objectives Definition
2. Key Results Definition
3. Measurability
4. Critical thinking framework
5. Discipline:Helping Employees Work Together
6. Focussing Efforts
7. Measurable Contributions
8. Organizationwide Sharing
9. Visibility into Goals across the organization
10. Align Effort
11. Foster Long-Term Thinking
12. Open Measurement of Progress
13. Support Cross Functional Communication
14. Transparency (on Topics, Priorities)
from another source (2)
15. updated
16. aspirational
17. not tied to compensation, rewards, and reviews
And my snake oil will cure any disease that interests you. One person got 8.5% better using it.
References
15. https://en.wikipedia.org/wiki/OKR
16. DavidFRico.com
17. http://www.namcook.com, Capers Jones.
18. Gilb Planguage methods studies, at HP and Intel, for example [5, 6]
19. HP
www.gilb.com/DL65, http//hdl.handle.net/1721.1/80490, gilb.com/DL67, gilb.com/DL35 THESE
NEED REPLACEMENT SEE RESEARCHGATE
6. Intel: Terzakis: http://selab.fbk.eu/re11_download/industry/Terzakis.pdf, https://www.think-
mind.org/download.php?articleid=iccgi_2013_3_10_10012
7. http://www.slideshare.net/HenrikJanVanderPol
8. http://www.slideshare.net/DanKeegan/okr-a-guide-to-objectives-and-key-results-36471600,
benchify.com Dan Keegan.
9. Gilb, Quantifying Management Bullshit: forcing IT Stakeholders to reveal the value they really
want from your IT Project. http://www.gilb.com/dl465
http://pl.coremag.eu/uploads/media/Quantifying_Management_TGilb_coreENG_05.pdf
10. Gilb, What is Wrong with Balanced Scorecards (slides). http://www.gilb.com/DL135
11. Gilb, Competitive Engineering, 2005, https://www.gilb.com/p/competitive-engineering. Free
12. Gilb, Value Planning 2016-7 leanpub.com/ValuePlanning (forst 100 pages Vision Engineering
are free, and shows quantification of real company visions.
13. Henrik-Jan van der Pol: a good overview of the OKR culture. http://tinyurl.com/OKRHenrik-Jan
http://blog.perdoo.com/a-crash-course-okr
14. Kevan Lee, 2016, http://blog.trello.com/okrs-set-achieve-track-trello
15. Alice Calaprice, Quotable Einstein, and personal communications with AC myself.
16. Gilb, Principles ofSoftware Engineering Management, 1988
17. Sears Case: http://okrsblog.blogspot.no/2015/03/sears-holding-company-study-concludes.html.
www.Gilb.com of Monday, February 11, 20196 7
Chris Mason PhD: “I’ve not seen any research supporting that OKRs, as a specific methodology, is
effective.”
Nice with 8.5% sales at Sears, firms using the improved methods that I suggest, report 300% pro-
ductivity and more (Intel, 6) and many other detailed cases and references in the VP book (12).
18 L Brodie, PhD Thesis 2005 Middlesex University, Title: “Impact Estimation: IT Priority
Decisions”, on request from the author L.Brodie@mdx.ac.uk
19. Ryan Shriver. Mobius automated OKR, http://ryanshriver.github.io/mobiusloop-ruby/
20. [IC] Tom Gilb: Innovative Creativity. 2018. !
https://www.gilb.com/store/QMMQhn2g!
21. [CC] Tom Gilb: Clear Communication. 2018. https://www.gilb.com/
store/oJCCxtsM!
22. [100T] Tom Gilb: Technoscopes: Power Tools to Master Complex
Plans and Problems. 2018. https://www.gilb.com/store/Pd4tqL8s THIS
BOOK
23. [PPP] Tom Gilb: 100 Practical Planning Principles. 2018 https://
www.gilb.com/store/4vRbzX6X!
24. [LD] Tom Gilb: Life Design, 2018. https://www.gilb.com/store/
kCBGcG6L!
25. Tom Gilb: Vision Engineering, !
(this is also a subset of the VP book above)!
Value Planning: Top Level Vision Engineering!
CONCEPTS.GILB.COM/DL926 !
(pdf format). A 63 Page book. Aimed at demonstrating with examples how
top management can communicate their ‘visions’ far more clearly.!
There is a German and Russian Edition of this, at beginning of full Value
Planning book.!
leanpub.com/valueplanningdeutsch!
leanpub.com/valueplanningrussian!
26. [VPB] Tom Gilb: ∆ : "Value Planning Basics"
For Advanced Management Results .
A very short text main book’s (VP book’s 800 pages condensed into 10 text
pages)!
Read in an evening or on the plane. Free at the moment.!
A 23-page book. (half are illustrations). Contains the 1 page book summary.!
!
5A: PDF version!
www.Gilb.com of Monday, February 11, 20197 7
https://www.dropbox.com/sh/yjqd50rzipoxcan/
AAA2vWWwHZg7M4557vysjtsya?dl=0!
5B. ePub version!
https://www.dropbox.com/sh/rwykqi05uux4f0u/
AAADzymMhjrF6cV_NWrl18uOa?dl=0!
27. [TT] Or maybe you prefer the 18 minute Tom Gilb TEDx video? On
Quantifying Love.!
https://www.youtube.com/watch?v=kOfK6rSLVTA!
!
