XP Tech Practices Workshop at Spotify

Last week, a group of us did an XP Tech Practices workshop here in Stockholm. We covered Test-Driven Development, Refactoring, Unit Testing, Pair Programming, and a host of other Old school XP practices. Here are some notes and a write up about the course.

What is XP

XP (Extreme Programming) was the first really popular Agile method. Not only did it introduce Agile planning concepts such as user stories, adaptive planning, and iterative development. It helped popularize engineering practices such as unit testing, refactoring, test-driven development, and continuous integration.

It isn’t as widely practiced today as it was in the early 2000’s (Scrum has largely taken over). But it has many wonderful practices and ideas that deserve repeating and are still very much relevant for software teams today.

This workshop aimed at revitalizing some of software these practices, showing participants who they were originally meant to be performed, and then discussing how they can be introduced into their workplace.

How do you feel about your work?

We start off the course by asking participants how they feel about their work. Specifically around quality and design. We do this to start a conversation around what quality is, how we know when we have good design, and what obstacles prevent us from reaching them in our software.

Build something!

We then break the ice by asking participants to code something up (like a calculator). We do this to get the juices flowing, engage the participants, and just get them hacking. This helps us see where they are coming from, their familiarity with the programming language, and how comfortable they are writing code.

We don’t judge or do anything with the code that we produce here. We simply park it, and revisit it later. We then introduce XP.

XP Intro

XP can best be described as the software methodology that turns the knobs up to 11.

If code reviews are good, we will review code all the time (pair programming).
If testing is good, we will test continuously (acceptance tests, unit tests, TDD, demos).
If design is good, we will design continuously, every minute and every day (refactoring).
If simplicity is good, we vow to always leave the system in its simplest state.
If integration is good, we will integration continuously (continuous integration).
And if iterative development is good, we will iterate from day one, continuously getting feedback on our product, plan, and ability to please our customer.

XP Demo

Once we’ve covered the spirit and intent of XP, we demo the practices. We pair program up a sample problem, applying TDD, refactoring, unit testing, and pair programming in an extreme way, just to give the participants a sense of how all the practices work.

We then revisit the exercise they did in the ice breaker, and try it again, this time using the XP practices, specifically writing the tests first.

Build a game!

We then unleash them on their own codebase – specifically a game. They take what we’ve covered, and apply the practices in as an extreme a way as they can, and we build a fun interactive game over the course of several iterations.

XPisms

Over the span of the course we look for opportunities to introduce XPisms. Things like:

YAGNI – You aint gonna need it. A reminder to keep things simple, and not add excess functionality / code unless absolutely required (a good test is whether you can write a failing test first before adding the new code).

Production Readiness – A system spends a lot more time in production than in development. So why don’t we start treating it that way, and write code as if we were in production from day one.

Doing the simplest thing that could possibly work –

There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult. – C. A. R. Hoare

Test everything that could possibly break – We can’t test everything. But we know we can’t test nothing. So we start by testing everything we think could break. And evolving our suites from there.

Dials to 11 – XP’s take on hard problems is to take them head on. Taking this approach takes many of the traditionally hard problems on software projects, and breaks them down into much smaller, easier to digest pieces.

Courage – Have the courage to write that failing unit test. Have the courage to say no to the hack on your finger tips. Have the courage to say no to the unrealistic schedule, and help another team out when they really need it.

Why we feel this is important

It may seem that XP is all about engineering. It’s not. It’s just as much about culture. But the culture XP wants the team to operate under doesn’t happen if certain practices aren’t in place. This course is about explaining the practices, showing how they work, and then helping teams adopt them their work place.

A special thanks to Henrik Kniberg for the initial course outline and for his support in making this course happen.

Quotes

“I was supposed to have a Hackday today, but this is WAY better”

“I found the course to be a lot of fun, and learned a lot about TDD/XP in the process. The exercises, particularly the four-part exercise to build a game for a customer, were a very good balance of difficulty and fun, and allowed me to learn at a good pace without being bored by trival tasks or overwhelmed by difficult ones.

In fact, the game task actually, perhaps accidentally, was almost tricking me into learning stuff. You know you give a kid a toy that’s actually teaching them maths without them knowing because they’re having fun with the flashy lights? It was kinda like that.

★★★★★ Five stars, genuinely one of the best in-house workshops I’ve attended.”

“Got a good amount of new perspective on XP and TDD. Got into the habit of writing unit tests during instead of after development. And it was a lot of fun!”

Advertisement

The Agile Samurai Bootcamp

It is with great pleasure that I proudly pronounce the unveiling of my latest course: The Agile Samurai Bootcamp.

Based on The Agile Samurai book, this course is the perfect intro for people wanting to get into Agile, but unsure of where to start.

The course is super expensive, a whopping $25 (take that Scrum certification), and I challenge you to find a better introduction to Agile out there.

You can learn more about the course at here, but if you act now you can save 25% by clicking the link below:

• Learn Agile

I hope you enjoy the course.

Jonathan Rasmusson

You Can’t Plan Learning

A question I sometimes get from executives when I showing them where key insights, innovations, and breakthrough occurred on projects is: “That’s great. But why didn’t you just plan a little more and discover those before you started the project?”

I get where they are coming from. They want certainty in an upfront plan. They don’t want any surprises. And they want their software projects to execute just like the power plants and factories they’re use to building.

Except software doesn’t work like that.

You can’t plan the upfront learning that comes from iterating a product or service with a customer, and helping them discover what they really want.

Some customers can tell exactly what they want. But many others don’t. They know they have a need. But they don’t know what is possible, or how to get there.

It reminds me of this story about pottery making (from the book “Art and Fear”).

The ceramics teacher announced on opening day that he was dividing the class into two groups.
All those on the left side of the studio, he said, would be graded solely on the quantity of work they produced, all those on the right solely on its quality.

His procedure was simple: on the final day of class he would bring in his bathroom scales and weigh the work of the “quantity” group: 50 pounds of pots rated an “A”, 40 pounds a “B”, and so on.

Those being graded on “quality”, however, needed to produce only one pot — albeit a perfect one — to get an “A”.

Well, came grading time and a curious fact emerged: the works of highest quality were all produced by the group being graded for quantity.

It seems that while the “quantity” group was busily churning out piles of work-and learning from their mistakes — the “quality” group had sat theorizing about perfection, and in the end had little more to show for their efforts than grandiose theories and a pile of dead clay.

So by all means plan. But don’t count on planning alone to get you there. The best results come from building, iterating, and learning. And you can’t plan that.

The Agile Mindset

Agile has been around for over a decade, a lot of people are doing it, and that’s great.

But I see a lot of organizations struggling. Not so much with the tools and practices. But mostly in the mind – the head.

Here are a list of thoughts and attitudes companies need to get if they are going to truly adopt Agile as a means of delivery.

The plan is going to change

Plan the work, work the plan. That’s the mantra traditional project management has been teaching PMs for years. Except that it doesn’t work. Companies that expect software projects to be straight lines. But they look a lot more like this:

and it’s this unwillingness to change the plan that kills them – Agile or not.

Everyones has a plan until they get punched in the face. – Mike Tyson

Software projects are a bit like getting in the ring with Mike Tyson. They punch you in the face. And if you can’t take a hit, or are so tightly boxed in a corner that you have no room to maneuver, you are going to get knocked out.

Adaptive planning enables Agile teams to roll with the punches, take advantage of opportunities when things don’t go their way, and avoid much of the drama and dysfunction traditionally seen on projects that refuse to adapt.

So get used to changing the plan.

You don’t have all the requirements

Accept it. You don’t.

You might think you’ve got them all. You may read books telling you to gather them all before you start. But you can’t. Why? Because like Kent Beck once said:

The act of delivering software, changes the requirements.

Which means no matter how hard you try, you’re not going to have them all before you start. And the ones you do have are going to change. Just accept it and plan accordingly.

There is no change control board

This concept is so foreign to Agile delivery I almost cringe in bringing it up.

On an Agile project the only person who decides whether a feature gets implemented is the customer. They don’t need to seek anyones permission or approval. It’s their money. They can spend however they like.

Agile sides with the customer. If the customer wants a change, they can have it. Agilists believe this leads to better decisions, better products, and puts the responsibility for deciding how to spend the money where it belongs – in the hands of the customer, not the PM.

The cost of change isn’t high

Traditional thinking is that the cost of change on projects is high. That’s why this picture is still printed and promoted in modern project management bodies of knowledge.

This used to be true in software. But not anymore.

With the advent of the personal computer, just-in-time compilers, and phones with more processing power than older mainframes, we can make changes to software at extremely low cost.

We’ve also gotten a lot smarter. We have software engineering practices today that allow us to make changes with confidence at speed. That changes the game.

Instead of fearing and resisting change, you are now free to embrace it. And use it to your customers advantage.

There are no predefined titles or roles

Roles blur on Agile projects.

I know you have narrow, clearly defined, titles, roles, and responsibilities. Agile teams don’t care about any of that. What they do care about is doing a good job and removing anything that gets in the way.

So don’t get discouraged when your developers want to test, your testers what to be involved upfront in the analysis, and your Project Managers what to contribute and program. You are going to have happier people, most engaged teams, if you don’t limit them with a formal title or narrowly defined role.

You are going to fail

There’s a price to pay for all this progressive, initiative, risk taking way of working. You are going to periodically fail. It happens.

You can’t build innovative software or create great products without periodically going over the edge. Good companies will forgive you. The bad ones won’t. The good news is you get to choose which you would rather work for.

They are not requirements

Let’s think about this word. Requirement. It rings of something completely necessary. Something we absolutely must have.

So if we can deliver 80% of the business value from 20% of the requirements, what are the the other 80%? They’re not requirements. We didn’t need them.

Drop the word requirements from your software project vocabulary. It’s the wrong word. Instead use features, and treat them as things your software project may or may not have. Not requirements.

They are estimates

You know what an estimate is right? It’s a guess.

Guess how long it would take you to roll ones with a pair of dice. How long would it take you to find a missing card in a deck of cards.

Guess. Try it out. And then compare your actuals to your estimates. Think you are going to be within 10%. Care to make a wager on that?

The way we size software projects isn’t all that different. We estimate, or guess.

Except instead of treating these estimates as guesses, we treat them as commitments (and that’s where we get into trouble).

Agile estimates aren’t commitments. They are best guesses based on limited knowledge.

We turn them into commitments the only way we know how. By building something, seeing how long that takes, and then extrapolating that for the rest of the project (it’s not rocket science, but it works).

Working software is the definition of success

For a lot of companies getting projects in on time and on budget is the overriding definition of success. And Agile would agree that projects have to work within their means.

But rather put so much attention on the plan (which by itself does nothing) Agile takes that time and energy and puts it back in the one thing that does add value – the software itself.

That may sound obvious, but we deliver a lot of projects that are on time, and on budget, but deliver no value. It’s gotta stop.

Chaos is the norm

In 2004 Doug Decarlo wrote an article I really liked contrasting the different mindsets between the traditional and extreme (Agile) project manager.

Newtonian – Stability is the norm	Quantum – Chaos is the norm
The world is linear and predictable	Uncertainty reigns
It is controllable	Expect surprises
We can minimize change	We should welcome change
Add rigor to the process to increase the feeling of security	Relax controls to increase the feeling of security
Deliver on the planned result	Deliver to the desired result
Use the plan to drive results	Use results to drive the plan
Aim, air, fire	Fire, then redirect the bullet
Keep tight control on the process	Keep the process loose
Manage to the baseline	Manage to what’s possible
Get it right the first time	Get it right the last time

I think this hits it bang on. You either believe everything can be planned, controlled, and thought of ahead of time, or you don’t.

Agilists think Quantum. And their practices, management and expectation setting techniques reflect that.

Three simple truths

Once you accept these three simple truths, leading agile projects becomes a lot easier.

You don’t stress as much about schedules (we know we’re already late!) You stop trying to own problems that are outside your sphere of control. And you just accept that there is always going to be more to do than time and money allow.

You stop taking things personally.

And software is personal. You put a lot of yourself into a software project, and it’s easy to take feedback, criticism, and things like schedule pressure personally.

But accepting these simple truths frees you from all that. It allows you to see that which is clearly, and to not try and change something that can’t be changed.

OK. That’s it for now. I plan on referring back to this article in the future, but I wanted to get some stuff out there on paper, so when when helping companies get better at software delivery, I can direct them here for a conversation starter, and then get to work.

OK – let’s talk.

Fresh Eyes

Sometimes you get so close to a problem, or you are so intimate with it, that you can’t see how to go about it any other way.

No where is this more true than traditional project planning.

Traditionally, all that mattered to planners to was meet the plan.
On time. On budget.
Plan the work, work the plan.
That’s all that mattered.

There were very strong incentives for managers to do this.
They were paid to meet the plan.
Bonuses hinged on hitting the plan.

And this relentless drive to make the plan, is what has driven, and contributed to much of the drama and dysfunction we associated with traditional project planning. Especially in software.

That’s why change never comes from the status quo.
It takes new eyes and approaches to see problems and solve problems.

It took a group of programmers, to stare down traditional project managers and say enough is enough.

If you have too much to do and not enough time, do less.

This is the basis for Extreme Programming and Agile Planning.
This is why we flex on scope.

Traditional project management is still grappling with this profound, yet simple insight.
They want to be Agile, and be seen as flexible when it comes to adapting the plan.
But they’ve got half a century of baggage and expectation setting slowing them down.

That’s why a revolution in project planning could never come from Project Managers.
They were too busy maintaining the status quo.

It took a group of engineers, with a healthy dose of common sense, to stop the madness, and bring sense back into the process.

XP is the Mac of Agile

When Apple released the Macintosh it changed the face of computing. Graphical user interfaces, drag and drop icons, clickable menus. Since its release, the personal computer has never been the same.

The same thing happened with the release of XP. Like an earthquake, it shook just about everything we traditionally believed and practiced in software delivery down to its core.

And then both failed.

The Mac got displaced by the cheaper IBM PC Junior.
And XP was pushed to the side by the less threatening Scrum.

In this article I would like to explore why Scrum has become so popular, the challenges this popularity brings to Agile, and why, like the Mac, I don’t think we’ve heard the last of XP.

Maintaining the Status Quo

One reasons I believe Scrum has grown so popular, is because unlike XP, it struck the right balance between maintaining the status quo and change.

The first version of XP was threatening. If you weren’t a developer or a customer, it wasn’t really clear what your role on an XP project was. With developers and customers joined at the hip, XP teams delivered at a speed and level or quality, seldom seen outside of startups.

(Note: In reality we did many XP projects with analysts and testers but I want to make a point, so bear with me).

But this speed and efficiency came at a price. It radically changed the status quo.

• Testers felt threatened because developers were writing tests.
• Analysts questioned their role if developers were speaking directly with customers.
• And Project Managers were perhaps the most disrupted. XP trivialized their best laid plans, and had them embrace the one thing they had been trained to eliminate on all software projects – uncertainty and change.

Scrum on the other hand was different. Instead of insisting developers and customers sit together and build software, Scrum said: “Why don’t you guys form a team, and ship something of value every 30 days.” It didn’t say how to do that. Only that working software every 30 days was the goal.

This was music to the established players ears.

• Analysts could analyze.
• Testers could test.
• Project Managers could PM.

I call this Corporate Scrum. Everyone could pretty much do exactly what they did before, but now in shorter 30 day cycles. Much less radical. Way more status quo.

Where XP alienated. Scrum embraced.

The challenge with Scrum

This of course lead to some challenges. Scrum is all about planning. It doesn’t talk engineering. This lead to a lot of Scrum teams doing the easy stuff (daily standups and Sprint planning), while failing on the hard (consistently delivering high quality working software).

Planning is easy. Delivery is hard.

And I think the Scrum community itself could do more here. Not highlighting, or actively promoting the XP practices around unit testing, refactoring, TDD, and continuous integration runs the risk of seeing the term Flaccid Scrum grow in popularity.

The Tribal knowledge of XP

And I think us XP’ers can do our part by sharing our stories and wisdom around practices like:

• YAGNI
• Simplest thing that could possibly work
• Yesterday’s weather
• Test everything that could possibly break
• Production readiness

These euphemisms are too important to forget. And the spirit and technical excellence that contributed to XP’s early success will be necessary for Scrum too.

This is the beginning. Not the end

So while XP feels like the Apple Macintosh of the 90s, it would be premature to write XP off. Many of its ideas are only now becoming widely accepted. And many more are only just beginning to re-emerge.

It may never reach the heights today’s Mac, or displace the IBM PC Junior that is Scrum, but its influence and spirit are still being felt, and will be, for years to come.

The book that changed the game

I miss XP. It hit me hard how much when I read Uncle Bobs Ode to Kent’s White Book.

Man those were good times. We were going to change the world. It was us against them. It was the tyranny of the traditional against the brash, naivety of the new. And It was the programmers who finally stood up and said “Enough!”.

And man did XP make waves.
Test first? Ridiculous.
Recipes cards for requirements? You gotta be kidding.
Simplest thing that could possibly work? Grow up.

It’s unfortunate that people don’t talk about XP like we used to. But it’s practices are alive and well practiced everyday by thousands around the world.

Let’s take a look at why this book has so influential, the impact it’s had on our industry, and why we still aren’t all using it today.

Testing

XP tipped testing on it’s head.
No longer an after thought, left till the end, XP made testing the center of the universe on software projects
You didn’t write code until you had a failing test on XP projects
You didn’t check code in until all the tests ran
The tests needed to be continuously passing – 100%
You weren’t done, until all of your customers tests passed
Customers wrote tests. Developers wrote tests. Everyone tested on an XP project.

Design

XP challenged a lot of what we traditional thought was good (upfront) design.
Taking more of an options approach, XP recommended not adding complexity until you absolutely needed it (YAGNI).

This was XP’s way of pushing back against the over engineering going on in the late 90s. XP taught us to design continuously (not once) while introducing us to the importance of language, and the power of a good metaphor.

Requirements & Analysis

The story card was XPs greatest innovation.
Instead of trying to get everything written down and get everything right upfront, XP said jot the idea down on one of your mom’s recipe cards, and have a conversation later with your customer if it ever comes up.
You wouldn’t believe the howls of protest.
How this could possibly work!
Where’s the tracability?
Where are the requirements?
How are you going to fit all those requirements on that one little index card?

Planning

1000s of books had been written on how to manage and plan software projects. XP pretty much blew them all up.
XP trivialized what it took to plan software project.
It punted on the whole ‘plan the work, work the plan’ thing by telling the truth (we don’t know exactly when we are going to be done), and rejecting static plans. Instead it said, we are going build something, see how long that takes, and then feed that back into the plan. Then we’ll tell you how are date is looking. No wishful thinking or management by miracle here.

XP was the first to suggest that when you have too much to do, and not enough time, you should do less. Rocket science. I know.

Controversy

You can see where all this is going.
XP disrupted just about every traditional role on the software project.
It showed how dysfunctional things in our industry really were.
And not everyone appreciated that.
XP’s problem was it’s lack of inclusiveness.
It was great if you were a programmer or a customer.
But it was terrible if you did anything else.
It threatened testers (developers were now writing the tests)
It threatened analysts (developers were now talking directly to customers)
And it especially threatened project managers (small groups of professionals don’t require much in the way of management).

For these reasons many attacked XP and the Agile movement. And that might have been it if it wasn’t for the emergence and soft sell of Scrum.

Scrum to the rescue

For all of XP’s divisiveness, Scrum was the exact opposite.
Scrum was for everyone.
Small, self organizing teams made up of multi-disciplinary people delivering software every 30 days. Sounds great.
Traditional testers could test
Analysts could analyze.
Everyone had a spot at the table and was a large part why so many flocked to it.

PM’s were perhaps the most happy. Not only where the back in change. They finally had a means to ‘control’ these Agile projects, while simultaneously gaining a new title themselves – Scrum Master.

This was of course maddening to XPers and felt like a huge step backwards. Not only did Scrum not bring anything new or innovative to the party (from the XPers point-of-view). It completely left the most important parts out – the software!

But despite all these things, Scrum was hugely successful in one area where XP failed. Scrum was Agile’s beach head in enterprises and organizations.

Scrum made it safe for people to talk Agile. I am also extremely grateful to Scrum for promoting the concept of the self organizing team – no titles or roles on projects.

Ripples for years to come

Some to the largest organizations in the world are applying XP at a mass scale. XP had a huge affect on how Google tests software. And, not sure if anyone has noticed, Kent spends a lot of time these days at Facebook. I don’t know what the arrangement is, but that place reeks of XP. And for anyone who wants to see world class engineering and problem solving on a global scale check out how Facebook releases software or how they do development and deployment (hint – there is no QA department).

But love it or hate it, it’s hard to deny how much this book has affected our industry. Apparently it’s now recommending reading for management gurus. While on the other hand I know many in our industry would be quite happy to see it quietly die.

XP matters. This book matters, and many of us wouldn’t be where we are today if it weren’t for Kent and Ward. That book took courage to write. Let us hope we can show the same courage and continue talking about it in the future.

For more information checkout:
– My recommended reading list (XP Books near the top)
– The original IEEE Paper
– Good ol Wikipedia

The Quantum vs Newtonian Belief System

In 2004 Doug Decarlo wrote an article I really liked contrasting the different mindsets between the traditional and extreme (Agile) project manager. I think this hits it pretty bang on.

Newtonian – Stability is the norm	Quantum – Chaos is the norm
The world is linear and predictable	Uncertainty reigns
It is controllable	Expect surprises
We can minimize change	We should welcome change
Add rigor to the process to increase the feeling of security	Relax controls to increase the feeling of security
Deliver on the planned result	Deliver to the desired result
Use the plan to drive results	Use results to drive the plan
Aim, air, fire	Fire, then redirect the bullet
Keep tight control on the process	Keep the process loose
Manage to the baseline	Manage to what’s possible
Get it right the first time	Get it right the last time

New Agile Planning Video

Just released a new Agile In a Nutshell Planning video. You can see it and the other Agile In a Nutshell videos (like user stories and estimation) here.

Here are some slides from the deck.

Agile Planning

---

---

---

---

---

---

---

---

---

---

---

---

Estimation War Story – The Double Whammy

Everyone loves a project that comes in ahead of schedule and under budget. But if you consistently too under budget, people will accuse you of sandbagging your estimates and accuse you of being a little too cautious.

This caught me once. We sized a project using the estimates as a range technique, and convinced management to use the upper numbers.

We then estimated our team velocity and again, used a conservative number around how fast we thought we could go.

Turns out we the project was simpler than we thought, and we ended up going faster than we thought we could go, and came in way ahead of where we thought we were going to be. Great news!

Now, I would way rather be on this side of a project (ahead of schedule and under budget) than the other. But when I look back and wondered why our numbers were so big I realized were conservative in our estimates twice:

• Once for the estimates.
• Once for the team velocity.

It’s not the end of the world. This was a really scary project, and our inception deck highlighted a lot of unknowns and first times for the company. There was also some fear and pressure to get things right. But I didn’t quite appreciate at the time that we were sizing conservatively twice here.

I don’t regret sizing it this way. Hindsight is always 20/20.

The more interesting question is now that we are aware of it, what will we do next time.

Happy planning!