When Apple released the Macintosh it changed the face of computing. Graphical user interfaces, drag and drop icons, clickable menus. Since its release, the personal computer has never been the same.
The same thing happened with the release of XP. Like an earthquake, it shook just about everything we traditionally believed and practiced in software delivery down to its core.
And then both failed.
The Mac got displaced by the cheaper IBM PC Junior.
And XP was pushed to the side by the less threatening Scrum.
In this article I would like to explore why Scrum has become so popular, the challenges this popularity brings to Agile, and why, like the Mac, I don’t think we’ve heard the last of XP.
Maintaining the Status Quo
One reasons I believe Scrum has grown so popular, is because unlike XP, it struck the right balance between maintaining the status quo and change.
The first version of XP was threatening. If you weren’t a developer or a customer, it wasn’t really clear what your role on an XP project was. With developers and customers joined at the hip, XP teams delivered at a speed and level or quality, seldom seen outside of startups.
(Note: In reality we did many XP projects with analysts and testers but I want to make a point, so bear with me).
But this speed and efficiency came at a price. It radically changed the status quo.
- Testers felt threatened because developers were writing tests.
- Analysts questioned their role if developers were speaking directly with customers.
- And Project Managers were perhaps the most disrupted. XP trivialized their best laid plans, and had them embrace the one thing they had been trained to eliminate on all software projects – uncertainty and change.
Scrum on the other hand was different. Instead of insisting developers and customers sit together and build software, Scrum said: “Why don’t you guys form a team, and ship something of value every 30 days.” It didn’t say how to do that. Only that working software every 30 days was the goal.
This was music to the established players ears.
- Analysts could analyze.
- Testers could test.
- Project Managers could PM.
I call this Corporate Scrum. Everyone could pretty much do exactly what they did before, but now in shorter 30 day cycles. Much less radical. Way more status quo.
Where XP alienated. Scrum embraced.
The challenge with Scrum
This of course lead to some challenges. Scrum is all about planning. It doesn’t talk engineering. This lead to a lot of Scrum teams doing the easy stuff (daily standups and Sprint planning), while failing on the hard (consistently delivering high quality working software).
Planning is easy. Delivery is hard.
And I think the Scrum community itself could do more here. Not highlighting, or actively promoting the XP practices around unit testing, refactoring, TDD, and continuous integration runs the risk of seeing the term Flaccid Scrum grow in popularity.
The Tribal knowledge of XP
And I think us XP’ers can do our part by sharing our stories and wisdom around practices like:
- YAGNI
- Simplest thing that could possibly work
- Yesterday’s weather
- Test everything that could possibly break
- Production readiness
These euphemisms are too important to forget. And the spirit and technical excellence that contributed to XP’s early success will be necessary for Scrum too.
This is the beginning. Not the end
So while XP feels like the Apple Macintosh of the 90s, it would be premature to write XP off. Many of its ideas are only now becoming widely accepted. And many more are only just beginning to re-emerge.
It may never reach the heights today’s Mac, or displace the IBM PC Junior that is Scrum, but its influence and spirit are still being felt, and will be, for years to come.
The Agile Warrior 
Feb 04, 2014 @ 03:50:48
Well written and I agree 100%, keep them coming 😀
Feb 05, 2014 @ 16:14:54
Brilliant. On all point this helps provide more light, not heat
Feb 06, 2014 @ 06:53:12
Yeah I was initially more enamoured with XP. But hey I’m a developer originally. Scrum dealt with people aspects better – unavoidable part the people. But yes still believe in the core of XP which still correlates alot with scrum in agile principles. The selling bit of XP’s we got wrong. Got peoples back up. Maybe more combined with Kanban and we avoid the notorious Satir change model effect.
Feb 06, 2014 @ 09:45:24
Great article; Scrum and XP can play really well together.
Scrum and XP in small research teams: http://arxiv.org/abs/1202.0652
Feb 09, 2014 @ 12:51:15
Nice article. Even though some semplifications have been put in place, I see your point.
I would adjust one phrase and say that planning is easier than delivering, not just easy. The introduction of short releases allow people to make better estimations as long as they discover new information and gain experience on the project, then therefore adjust the plan. It’s something that was missing as well.
Ps. You might want to review the difference between “it’s” and “its” in English 😉
Feb 16, 2014 @ 17:03:17
Hi Jonathan,
Great post! It’s fairly easy for teams and organizations to “adopt” Scrum as a starting point, and I do support that approach. But, that isn’t far enough. Adopting Scrum without good engineering practices is where organizations fail mightily. Teams performing Scrum often think they’re agile; yet, they execute design sprints and development sprints and coding sprints, then pull in testers for test sprints.
Alternatively, they perform sprints in traditional fashion which, of course, leaves testing to be cut short at the end. It’s exceedingly difficult to continue waterfall practices using XP practices like TDD and continuous integration.
Bravo! Now all that’s left to do is coach them out of it.
Brian Irwin
Mar 18, 2014 @ 21:19:09
I like the attempt at explaining some agile methods in terms of history but there are a couple issues… Firstly, XP and Scrum aren’t “either or”, in fact in the “early” days of both, it was common to do “Scrum with an XP center”, which i did back in 2003 and which works well. Scrum is a wrapper, XP is development centric – they are designed for different audiences / concerns. It’s rare (and imo ill-advised) that you just do one. Secondly, i’d say it’s just as easy to keep status quo doing XP as it is Scrum. I’m sure somewhere, someone is doing YAGNI, pairing, direct customer engagement, etc inside a waterfall method. What you’re calling “corporate Scrum” is lazy Scrum and insincere Agile – and isn’t Scrum. Companies that suck at Scrum suck at every other method they try, they sucked at PMI, RUP, AUP….
Mar 19, 2014 @ 13:21:22
Hi Kurt,
Thank you for the comment. First some comments/opinions.
In the early days it wasn’t XP and Scrum – it was just XP.
You can do just one (XP) just fine without the other (Scrum). The reverse can’t be said.
Agree that companies that suck at one methodology will likely fail at others.
If you go back to the earliest XP/Agile Universe conferences (2002, 2003) it was all XP.
http://www.informatik.uni-trier.de/~ley/db/conf/xpu/xpu2002.html
http://www.informatik.uni-trier.de/~ley/db/conf/xpu/xpu2003.html
(search these links and compare how many times you see the world XP mentioned vs Scrum).
XP was the hot thing. XP was what was tipping the world on it’s head. Everyone wanted to know how user stories could possible replace traditional requirements docs, this test first programming worked, the heretical notion of programmers pairing, and how you could possibly plan projects with nothing more than index cards.
And we were introducing and doing a lot of XP at client sites. We had Kent’s White Book, Ron’s Pink Book, Martin/Kent’s Green Book for planning. And life was good.
We of course had heard of Scrum (we bumped into Ken at a mutual client). But we didn’t take Scrum seriously because we thought 30 days was a ridiculously long time for an iteration, and it didn’t offer anyone that we didn’t already have with XP (we had the planning game with XP).
Then around 2005/2006 something strange started to happen. We started to hear a lot more customers asking for Scrum.
This felt strange to us. Why was Scrum getting popular? Didn’t people know it was a poor mans Agile? No one (at least at Thoughtworks) took Scrum seriously. We thought it was a joke. All it focused on was the planning (the easy part) while punting on the hard (software engineering and delivery).
This was about the same time certification started to take off. Companies were looking for a fast way to get there people ‘Agile’ and Scrum certification offered the perfect ticket.
Not only that, while XP had slowly been figuring out how to work effectively with other groups (testers, analysts, and project managers) Scrum brilliantly worked with what was already there. It put PM’s back in control of projects, and people could pretty much continue working as there were before (I know this isn’t proper Scrum, it’s just how I see a lot of companies doing it, and I understand why – because it’s easy and it maintains the status quo).
My feelings of Scrum are mixed. I love the fact it’s popularity has given us a chance to do Agile at corporations in a way that XP never could.
What I don’t like is the fact that it leaves out the best parts and roots of Agile (the software engineering) and dumbs it down to the same level of planning a busy parent would use with faced with a long weekend.
Scrum has become so popular (and so much money has been thrown at it) that it’s shifted the focus from how to do great delivery, to something else (how to make money and get people certified).
Apologies. You are hearing the griping of an old XP’er pining for the days of old, who feels like his methodology of choice lost out to an far inferior product (hence the Mac PC reference).
But I do want to make it clear that XP is a complete Agile method in it’s own right, and that Scrum (in my opinion) is not because it punts on the very foundation upon which all Agility is based – sound software engineering.
Cheers – Jonathan
Mar 19, 2014 @ 14:16:43
Fair enough Jonathan, i guess “early days” is a bit relative. I remain a huge proponent of XP practices, but i also think Scrum (if done right) adds to a development groups capability. I don’t think Scrum is just about planning or “Sprints” – that’s like when i hear people say “oh, XP? ya, we don’t pair program here”.
I have written in the past that like alot of other things, once Scrum “crossed the chasm” it became much easier to dilute and abuse. That’s too bad, but I’ll take that over traditional predecessors any day.
I can’t help but see the irony that what you and others don’t like about Scrum is what Scrum (Ken, community, etc.) don’t like about SAFe — that it’s about certification, the “corporatization” of Agile, making money, etc. That’s not directed at you or XP, just interesting that Scrum is both target and accuser 🙂
Will close on that i agree “Scrum in not enough”, I always coach that without solid engineering and teaming practices (ala XP), Scrum becomes only an incremental gain – not sure why that’s not a “part” of Scrum literally (ala Scrum Guide) but it’s a part of my “scrum guide” :).
thanks for writing this, on a lot of points i agree, and while i say “XP practices”, i understand XP can stand alone. i “re-affiliated” with Scrum.org a year or so ago largely in part because they added training that focused on engineering (i’m a PST for PSD .net) – it’s largely XP practices that are taught.
KC
James Shore wrote about some of these same topics in 2008 http://www.jamesshore.com/Blog/The-Decline-and-Fall-of-Agile.html
Mar 19, 2014 @ 15:56:43
That’s good to hear Kurt.
I don’t begrudged anyone for being affiliated with Scrum (even if it does feel a bit icky).
We are all just trying to make the world a better place.
Scrum is one vehicle for doing that.
Cheers – Jonathan
Jul 30, 2014 @ 15:47:46
Well said Jonathan. I love the analogy and would like to extend it beyond the PC/Mac war era and into the 21st Century… when it all changes ( to paraphrase Capt’n Jack Harkman). In the current world the PC is losing ground (PC sales last qtr decreased 2%) while the Mac sales are strong and growing (15% up same time frame). And these numbers do not address the larger change in computer interfaces – the move to mobile & tablet touch interfaces (largely lead by Apple). So to extend the analogy…. might XP have a similar resurgence in the near future. Not because it reinvents the next innovation in SW dev (e.g. touch interfaces) but because it always was and still is a better underlying construct. It is the foundations of Agile. After the industry has it’s run with Scrum and sees little value in the 2 day CSM certification, and that planning along does not create satisfied customers – portions of IT will return to the roots of Agile, they will rediscover the XP practices and values.
Just rereading Beck’s white book’s table of contents the other week and it is a remarkable statement of what I believe will/should work in SW Dev.
Turning the dial to 11.
David
Oct 15, 2014 @ 14:50:58
Hi Jonathan,
I should start by saying that I don’t disagree with what you’re saying here and I very much like your posts and I’m grateful you take the time to share them.
What I imagine you would respond to this is probably what you would respond to Kurt, so I don’t expect you to have to repeat yourself 🙂
Having said that, I do have a bit of an axe to grind on what I see to be subtle, implicit, scrum bashing going on around the place (I know you didn’t explicitly do any scrum bashing). Every time I see or hear of “scrum” failures I really get the feeling that people are missing the point. It is more often than not the case that the scrum master doesn’t do his or her job effectively, people pick bits they want to do and bits they don’t want to do without really understanding the reasons why, or they’re fundamentally not talented (yet) at providing customer value through delivery of quality software. The *people* are the problem, not the process. [Disclaimer: unless the process is fundamentally not suited to the problem you’re trying to solve]
It’s language like: “One reasons I believe Scrum has grown so popular, is because unlike XP, it struck the right balance between maintaining the status quo and change.” that sentence belies the subtle yet noticeable distaste for scrum. Scrum is a process, it doesn’t think, feel, take actions on people’s behalf or have an opinion. So what *people* did was accept their version scrum because it didn’t change their status quo.
Confusing the acts of people with the process is showing a bias along the lines of “Religion doesn’t believe in equal rights” vs. the unbiased “Some people don’t believe in equal rights”. We all have biases that grow for various reasons, experiences, frustrations and old opinions etc, but they are fundamentally harmful so I wanted to point yours out! 😀
For qualification: I am a scrum master at my current place of work (Engineering application with long(ish) release cycles) and was a developer and still develop software. I believe that scrum has its place and when used effectively and for the right reasons can be an excellent guide or starting point for agile. I’m moving to a company (set up by ThoughtWorks as it happens) that uses a more mature form of agile that is better suited to its environment (web), about which I’m very excited. And to give you some faith in the efficacy of scrum, scrum is the framework that allowed me to get through to people in retrospectives that we need to actually use XP and not just pay lip service to it. We are now reducing our technical debt, delivering higher quality software more often and communicating with each other in a more effective way than ever. We have also had the hard conversations and made the changes such that there were no more project managers, we didn’t need managers in each of the disciplines (such as testing, coding etc) so the status quo was well and truly disrupted!
Maybe one day you’ll find a place that suits scrum and will give it a place in your arsenal of problem solving tools. (I notice that is how Spotify got started, absolutely loved the video on spotify you shared a link to btw, although it seems mighty complicated in places).
All the best,
Jake
Oct 16, 2014 @ 03:37:59
Good points Jake. I don’t disagree with anything you’ve said here. Agree we all have our biases. Thank you for point out mine.
Cheers – Jonathan
Iterating Toward Legacy - Scrum's Achilles Heel - SolutionsIQ
Feb 18, 2015 @ 20:54:27
Rachel Davies | Finding-Marbles
Jul 27, 2015 @ 08:25:51
What ever happened to XP? | Finding-Marbles
Jul 27, 2015 @ 17:23:25