Delivering a new product to 60 million users

Last week was an exciting one for me and my team here at Spotify. We launched the Spotify player on the PS3 and PS4 and brought the Spotify music experience to over 60 million PlayStation users. Here are some notes on how we did it.

How it was run

This was a big project – one that spanned many tribes and cut right across the entire organization. Marketing, sales, licensing, distribution, all of tech. It was also very distributed with Sony and Spotify teams spread across San Francisco, London, Stockholm, Tokyo, and New York.

Spotify doesn’t have project managers. But for this project we did appoint someone to play the role of an overall programme manager. This was key because there were a lot of moving parts, and someone needed to help coordinate.

We did several onsite visits. Many teams came to Stockholm. We went to Tokyo. And there were a lot of early morning/late night calls (not to mention a tonne of email and Slack chats). Every Friday we also had a standing lunch where anyone interested could get an update on where things were at along with the current problems we were facing.

My team was responsible for building the app that would run the Spotify player on the PS3 & PS4. Four developers, three QA, a designer, a Product Owner, and me the coach.

Challenges we faced

Spotify doesn’t normally do projects. Instead we prefer giving teams a goal, and then letting them decide when best to release. We didn’t have that luxury. Sony’s old music application was due to shutdown at end of March, and we needed to be ready with our player to take it’s place. So in that sense, this was a very traditional project with a really hard deadline.

How we overcame

We ran this project like a typical Scrum project, only we quickly resorted to a kind of code and fix mentality that often comes when you realize you have too much to do and not enough time.

In addition to playing coach I also played the role of project tracker (tracking things like our team velocity and keeping an eye on overall deadline). For pretty much the entire project things didn’t look good. We had too many features. Not enough time. Typical project stuff.

Spotify’s approach to situations like these is to leave the team alone and let them sort things out for themselves. If they want extra help it’s there. But really it’s up to the team to solve their own problems.

What was interesting on this project was that the team (mostly the dev’s) didn’t initially want any outside help. Even when it became apparent that there was too much to do.

There was an inflection point in the project were senior management insisted we take someone else on (a fourth developer). This turned out to be a very good choice. Our new addition ended up making an invaluable contribution and we wouldn’t have made it without him.

But this leads to an interesting question for management – how far should you trust the team, and how do you know when to step in and ‘overrule’ their judgement. It’s not an easy call.

Finally, and perhaps most importantly, the one thing we did have going for us is we were the #1 priority project for the entire company. This meant that whatever we wanted – we got. A project this cross cutting and of this size can’t be handled alone. And the entire company was needed to pitch in. So when we needed people, meeting rooms, resources, or just decisions, people on the team just pulled out these ‘magic’ project cards and things got done. It was a fun way to work. But one you must be careful not to abuse.

Finally, this project required a lot of overtime, weekend, and late nights to bring across the finish line. We made it. We delivered. But there was definitely a human cost in terms of time, capital, and technical debt.

The launch

Now for the fun stuff! Here are some pictures of the war room on launch day.

There was a lot of coordination and monitoring that needed to happen on launch day. We created a war room, with each team represented at their own workstation.

spotify-sony-playstation-1

We also had sample Sony Playstation on standby, so we could log in to different regions around the world and make sure things were working.

spotify-sony-playstation-2

We also had a real cool analytics team that specializes in tracking usage in real time around the world.

spotify-sony-playstation-3

The Apollo launch feeling was created by design. Several pictures a phones like these were in full force that day.

spotify-sony-playstation-4

And this is what things looked like just before launch.

spotify-sony-playstation-7

spotify-sony-playstation-8

We kept the war room going for about 48 hours. Once Sony pushed their package out to all the PlayStations, we slowly saw people start to come online around the world, and it was great to see no major hiccups or disruption our backend services couldn’t handle.

To be honest I was amazed. I wasn’t expecting anything to go smoothly. What isn’t easy to see is the amount of work that goes on behind the scenes to make something like this happen. So many things can go wrong.

But somehow it worked. It all came together. It wasn’t Agile. It wasn’t any process, or secret sauce. It’s that same old cliche – it was the people. The talented, passionate group that went the extra mile, stayed the extra night, and worked the extra weekend to make this thing happen.

In Summary

I don’t know if I will ever get to participate in an event of this magnitude ever again. It’s pretty incredible to see this much focused work come together in this timeframe, in a single effort.

But I am thankful to have been a part of it. And I want to thank my team for all their hard work and making it happen.

umbrella-team

Newsletter

If you like this newsletter, you can find more here:

http://www.agilenutshell.com/newsletter