We’re Doing Scrum But…
I come from a big project, high ceromony background. I learned Software Engineering as contractor at IBM in the 80’s, was a Marine, and believe in the PMBoK (Deputy PM on OPM3® Second Edition). Over the last 10 years I have studied and learned Agile from books and through mentoring relationships. I have applied what I understood and have experienced improved performance. But I still thought of Agile/Scrum as a less than responsible way to do work. I have heard a lot of push back on Agile and Scrum in blogs and on Twitter lately. “Scrum doesn’t work”, they are saying. “Scrum makes the devleopers life easier but hurts the customer”, is another one. “The lack of overall budget/timeline is a killer (only look 1 sprint forward, really)”, is a charge hurled this week.
Last week, I went to the source and completed Jeff Sutherland’s Certified Scrum Master course. During the training with Jeff Sutherland last week, he presented a ScrumBut test. We are doing Scrum but… means you aren’t doing Scrum. Not doing Scrum means you likely won’t get the benefits other teams that are doing Scrum have claimed. Scrum, when done well, is a responsible way to develop software and if positioned maturly is a benefit to management. With permission from Jeff, here is the ScrumBut test. Take the test, add up your scores and divide the result by 9.
Question 1 – Iterations
- No Iterations – 0
- Iterations > 6 weeks – 1
- Variable Length < 6 weeks – 2
- Fixed iteration length 6 weeks – 3
- Fixed iteration length 5 weeks – 4
- Fixed iteration length 4 weeks or less – 10
Question 2 – Testing within the Sprint
- No dedicated QA – 0
- Unit tested – 1
- Feature tested – 5
- Feature tested as soon as completed – 7
- Software passes acceptance testing – 8
- Software is deployed – 10
Question 3 – Agile Specification
-No requirements – 0
- Big requirements documents – 1
- Poor user stories – 4
- Good requirements – 5
- Good user stories – 7
- Just enough, just in time specifications – 8
- Good user stories tied to specification as needed – 10
Question 4 – Product Owner
- No Product Owner – 0
- Product Owner who doesn’t understand Scrum – 1
- Product Owner who disrupts team – 2
- Product Owner not involved in team – 2
- Product Owner with clear product backlog estimated by team before Sprint Planning meeting (READY) – 5
- Product Owner with a release roadmap with data based on team velocity – 8
- Product Owner who motivates the team – 10
Question 5 – Product Backlog
- No Product Backlog – 0
- Multiple Product Backlogs – 1
- Single Product Backlog – 3
- Product Backlog clearly specified and prioritized by ROI (business value) before Sprint Planning (READY) – 5
- Product Owner has release plan based on Produce Backlog – 7
- Product Owner can measure ROI (business value) based on real revenue, cost per story point, or other metrics – 10
Question 6 – Estimates
- Product Backlog not estimated – 0
- Estimates not produced by team – 1
- Estimates not produced by planning poker – 5
- Estimates produced by planning poker by team – 8
- Estimate error < 10% – 10
Question 7 – Burndown Chart
- No burndown chart – 0
- Burndown chart no updated by team – 1
- Burndown chart in hours/days not accounting for work in progress (partial tasks burn down) – 2
- Burndown chart only burns down when task is done (TrackDone pattern) – 4
- Burndown only burns down when story is done – 5
- Add 3 points of team knows velocity
- Add two points if Product Owner release plan based on known velocity
Question 8 – Team Disruption
- Manager or Project Leader disrupts team – 0
- Product Owner disrupts team – 1
- Managers, Project Leaders or Team Leaders telling people what to do – 3
- Have Project Leader and Scrum roles – 5
- No one disrupting team, only Scrum roles – 10
Question 9 – Team
- Tasks assigned to individual during Sprint Planning – 0
- Team members do not have any overlap in their area of expertise – 0
- No emergent leadership – one or more team members designated as a directive authority – 1
- Team does not have the necessary competency – 2
- Team commits collectively to Sprint goal and backlog – 7
- Team members collectively fight impediments during the sprint – 9
- Team is in hyperproductive state – 10
My best recent project scored a 6.7 on this test and we didn’t call it Scrum – we just considered it responsible management of an analytics projecyt. We did perform really well in a tough situation though. If you score less than a 6 on average, you probably aren’t doing Scrum. That may be good or bad – but if you aren’t doing Scrum, don’t call it Scrum. If you aren’t performing at a hyperproductive level, look at the areas where you didn’t score as well. It is likely they will highlight the next most important impediment for you to focus on removing. Fix that and move forward.
This is a great, thoughtful summary, Dennis. Thanks for providing that quiz.
My biggest recent Agile-ish project scored around a 5.9, but we weren’t overtly doing Scrum. We were a very small team intentionally following Scrum-like-ish practices. It worked quite well! The customer has been happy. And my colleague on the project has picked up a tremendous amount of implementation knowledge. Win-Win-Win.
My current project is scoring 7.3 on this (we follow an agile methodology), BUT I have a few issues with this test – nothing on retrospectives, its far too specific on certain practices, and nothing on engineering practices.
Nothing on retrospectives: Scrum is all about inspect and adapt, hence the reason for retrospectives. But how many teams are doing it really? Not many – http://bit.ly/qJ7FH Somewhere along the way Scrum seems to have lost the inspect and adapt mindset and become a list of practices to follow (and if you dont -> its ScrumBut).
Too specific on certain practices: When did Planning Poker become a part of Scrum? What if estimation is done by the team but they use some other method? What are the differences between “just in time specifications” and user stories?
No engineering practices: Jeff has himself been quoted at various times that engineering practices are critical for success with Scrum. Why is there nothing about it in the test?
An Alternative:
IMHO, There is a better test for assessing agility. It can be adapted slightly to account for Scrum. It is based on Alistair Cockburn’s 7 principles. Jeff Patton has blogged about it here – http://www.agileproductdesign.com/consulting/agile_concepts/evaluating_agile_projects.html
Every question in Alistair’s test has a simple answer that gets to the heart of what agile is all about.
Personally, I agree with your first two items. It should include something on retrospectives. If you aren’t fundamentally inspecting and adapting you are doing Scrum.
I felt the planning poker was too specific a practice at first. Upon reflection, if Jeff wants to say you have to play planning poker to be Scrum I guess he can. You can be agile and get benefits from other planning methods. If you have tried planning poker and have found something better (through inspection and adapting) that’s great.
Scrum is not about engineering practices. It is explicity and intentionally separate. Scrum can be implemented in two days on a team – XP engineering practices take longer. What I heard Jeff say was that in his experience the teams that were hyperproductive did both.
I like Jeff’s test a lot. Alistar is a brilliant guy (from his writing – I have never met him). Thank you for the reference. I recommend people look at it. There are some exceptional practices there and Crystal has been a very successful method for many organizations. In order to effectively adapt your processes you should be familiar with how other Agile teams have achieved hyperproductivity. However, most of the people that are failing at Agile/Scrum are not scoring a 7 here and then wanting credit for additional valuable practices.
Well I certainly agree that engineering practices are not a part of Scrum, and not really required on a Scrum test, but Jeff has been re-quoted so often that Scrum alone is not enough that I was surprised it didn’t make it to his test.
Admittedly, I’m getting this information third hand, so maybe he didn’t say that exactly. But see this for example, just a few hours ago – http://twitter.com/keithb_b/status/1283652048
You are right, I heard him say it last week. The deal is that Scrum can be done immediately. And when done in areas other than Software Development, the engineering practices are different. So it’s not part of Scrum. You should be doing At Least Scrum if you want hyperproductive software teams.
I have reflected on the Crystal test and I have a critical concern that is addressed in the ScrumBut test but not on the Crystal test. There is no call for a product owner or estimated and prioritized backlog or release plan in the Crystal test. This is a big deal in my perspective. The lack of meaningful budget and schedule is one of the problems management has with many agile adoptions.
Yeah, the Crystal test is more focussed on principles than practices. So it asks if you deliver frequently (Question 1) and whether you talk to the experts (Question 6) regularly. The experts in Q6 are the equivalent of the product owner in Scrum.
The test figures that if you do these two, then you’ll be delivering the important features regularly. You can use any practice to get there – prioritized backlog is one option, but there are other equivalent practices too. An alternative to a prioritized backlog is the Product Map method – http://bit.ly/i67bm
So the test says that you can use any practice, but at the end of the day you need to deliver important features regularly. It leaves you free to use practices as you see fit, yet you can assess your agility.
A little bit of modification in terminology and addition of some Scrum specific practices and you’ve got a good Scrum test.
The ScrumBut test is a good way for a completely new Scrum team to ensure they don’t skip vital practices. But should you decide to explore outside the boundaries of Scrum, there are a lot of interesting things happening in the agile world that dont fit within the ScrumBut test.
I never got Jeff’s idea, or any others idea for that sake, about testing how good you were at scruming… why compete in who is the better at following a process? What business value does your company gain from that? And btw they are completely missing the 1st agile point: “Individuals and interactions over processes and tools” – Scrum is a process and a tool so don’t focus on it or you’ll risk doing a “CMMI” on it and thereby stopping the possibility of making any business value!
I Just ran my last team though the test and got a 9,3 – But I don’t think the team would have cared one bit if I had told them, I believe they were bright enough to figure out if they were delivering any think of business value or not.
Even thou I like Jeff a lot and he has a great knowledge about software development, one must also remind oneself of Jeff’s business is selling scrum!
Hans,
Thanks for your comment. I started to write a response to this and it became so long I turned it into a blog post for tomorrow – so check that out. Here is my short response to your comment. The Scrumbut test is for teams that think they are doing Scrum but aren’t getting the results they hoped to get. It is intended to point out gaps in the teams capabilities that are leading to their shortfall. It isn’t intended for the extremely mature team that has already figured this out.
On the other hand, I disagree with your assertion that CMMI by itself stops the possibility of making any business value. CMMI draws a reasonable roadmap to improving maturity in software development teams. I believe the real point is that when we focus on process over results, as some CMMI teams have done, we are focused on the wrong thing. The same thing can happen with any of the Agile methodologies. Your team has high process discipline or you wouldn’t be getting the results you are. Process discipline is important even on Agile teams.
Dennis
LOL –> the CMMI bit was a joke, but you must admit that the CMMI roadmap is a bit more complex than an agile process like Scrum – I should mention that I’ve seen as many, if not more, fail at a “simple” process like Scrum, as I have on CMMI – (I’m not implying that they fail for the same reason or that I’ve seen an equal amount of CMMI vs. agile implementations). And I do agree with you that you can use Jeff’s test at a mini “Gap” analysis, it’s just that I got a bit tired of “my process is better than your process”. And for some more agreement: Yes, discipline is very important no matter what processes you go for – try RUP without discipline and see if you can deliver anything of any business value?!? I’ll be on the lookout for your blog posting.
[...] a look at the Scrumbut test I learned about from Jeff Sutherland. Whether you are doing (or even like) Scrum – when [...]
[...] Were doing Scrum But Last week, I went to the source and completed Jeff Sutherland’s Certified Scrum Master course. During the training with Jeff Sutherland last week, he presented a ScrumBut test. We are doing Scrum but… means you aren’t doing Scrum. Not doing Scrum means you likely won’t get the benefits other teams that are doing Scrum have claimed. Scrum, when done well, is a responsible way to develop software and if positioned maturly is a benefit to management. With permission from Jeff, here is the ScrumBut test. Take the test, add up your scores and divide the result by 9. Quiz em Were doing Scrum But [...]