Just got back from an excellent session titled: Acceptance testing in the land of the startup by Joseph Wilk.
In it Joseph shared some challenges around:
– long build times with Cucumber (8 hrs but they got it down to 20min)
– the power of continual pairing of dev/ux and qa
– and the blurring of roles and how the ux designer was picking up rails skills
One of my favorite moments was when Joseph showed how they collapsed their kanban board down from this:
down to this:
Excellent stuff. Thanks for presenting Joseph.
And best of luck to your startup SongKick.
Advertisements
Aug 13, 2011 @ 00:37:45
Can you point to a resource on how to get the cucumber testing time down? That would be really cool.
Back in our course (“Best software engineering course ever!” – remember?) we kind of had the same problem, tests were taking more than half an hour to run locally (for a relatively small CRM) and I suppose like an hour on the CI-Server. So I can see the problem and would be very interested in a solution.
Thanks for the post!
Aug 13, 2011 @ 02:44:34
Hi Tobi,
Joseph said they spin up a whole bunch of amazon services and partitioned their build to run across them all in parallel.
They also manually ordered their tests in such a way that those tests that almost never failed ran last, while new tests ran first (so if there was a problem they would know in the first 10 min).
But the real root cause they believe is they wrote too many Cucumber tests and not enough unit tests. Joseph even mentioned he wanted to add a shim layer to his architecture to avoid firing up the UI every time they wanted to run a test.
Cheers – JR
Aug 13, 2011 @ 08:28:13
Thanks for the summary, some good information in there!