The Way of the Web Tester

Leave a comment

Screen Shot 2016-09-15 at 5.45.37 AM.png

Hello everyone. A book I have been working on diligently for the last two years has finally hit the shelves. It’s called The Way of the Web Tester and it’s a book for anyone who wants to learn how to write automated tests for the web.

What’s in this book?

The book starts out by explaining what the three kinds of tests are that we typically see in web projects, along with some rules of them and guidance on where and when to use each.

We they spend a full two chapters at each level of something called the testing pyramid seeing how to add UI tests to legacy systems, how one can test web services, along with some rules of thumb around unit testing. That’s Part I.

Screen Shot 2016-09-15 at 6.25.50 AM.png

In Part II we build on the concepts outlined in the testing pyramid, but shift our focus to those skills specifically geared towards writing good tests.

First we look at some of the basics of programming, and show how by learning a few simple principles and practices we can dramatically increase the readability and maintainability of our automated tests.

We then look at some practices around test organization, and see how by simply grouping and organizing our tests in certain ways can make maintaining our tests so much easier.

And in the chapter on effective mocking, we look at some of the perils that come with this style of unit testing, and see how we can use mocks effectively while staying out of the swamp of mocking.

And we conclude with a section on the art of writing tests first. And see how beginning with the end in mind not only helps you test your systems, it can actually help you iterate on your design.

Who is this book for?

Screen Shot 2016-09-15 at 6.52.26 AM.png

If you are a tester who’s always wanted to get into test automation, this is the perfect book for taking that first step into a larger world.

If you are a developer, and you’d like to learn how to make your code even more robust, this book will show you how to move fast without breaking stuff.

And if you are team lead, looking better ways to get your testers and developers together, his is the Rosetta Stone you’ve been looking for.

Once your team has a common framework, language and understanding around automated testing, they will be able to better coordinate they’ll their efforts and write the best automated tests for your project.

So wait no further. Crush bugs. Move fast. And have fun. I hope you enjoy The Way of the Web Tester.

On sale now at all reputable book stores.

https://pragprog.com/book/jrtest/the-way-of-the-web-tester

https://www.amazon.com/Way-Web-Tester-Beginners-Automating/dp/1680501836

 

Advertisements

How to make Objective-C base test class

Leave a comment

Sometimes you want a base class for your unit tests. You can make one in objective-c like this.

FooBaseTest.h

#import 

@interface FooBaseTest : XCTestCase
@property (nonatomic, strong) SomeProperty *someProperty;
@end

FooBaseTest.m

@interface FooBaseTest ()
@end

@implementation FooBaseTest

- (void)setUp {
    [super setUp];
    self.someProperty = // whatever
}

@end

Then you extend it in your test class like this

SomeTest.m

@interface SubclassTest : FooBaseTest

@end

You can do it this way, or you do it is also via composition.

How to define inner class objective-c

Leave a comment

If you have a test class, and you just want to define another class within it (say for test purposes), define an interface and an implementation near the bottom of the test file. Like this. Note the order here matters. Define the test class first in the file, before you use it in the test (objective-c!).


#import <XCTest/XCTest.h>
#import "SPTAppRemoteServerConnector.h"

// test class to act as the delegate

@interface FakeConnector : NSObject

@end

@interface FakeConnector () <SPTAppRemoteServerConnectorDelegate>

@end


@implementation FakeConnector
- (void)didConnectWithServer
{

}
@end



@interface SPTAppRemoteServerConnectorTests : XCTestCase

@end

@implementation SPTAppRemoteServerConnectorTests

- (void)setUp
{
    [super setUp];
}

- (void)testDelegateGetsFiredWhenServerConnects
{
    FakeConnector *fake = [FakeConnector new];
    fake = nil;

    SPTAppRemoteServerConnector *connector = [SPTAppRemoteServerConnector new];
    connector = nil;
}


@end



Classical vs Mockist testing

4 Comments

I’ve been looking for language to help articulate the difference between two styles of unit testing, and really like how Martin describes it in Mocks Aren’t Stubs.

Mocks use behaviour verification.
Classical uses state verification.

 

Classicist Mockist
Like working with real objects Prefer working with fake objects
State verification Behavior verification
Use mocks to test collaborations Use mocks all the time
Will hard code collaboration Will mock collaborations
TDD from middle out TDD from the outside in
Use ObjectMothers/Factories for test setup Will mock only what they need for test collaboration
Test tend to be more coarse grained – approaching more integration style tests Tests tend to be very fine grained – may miss integrations
Classists don’t couple tests to implementation Mockists do
Classists don’t like thinking about implementation when writing tests Mockists do
Don’t mind creating query methods to support testing Mockists typically don’t have to
Classists style can encourage Asking Not Telling design Mockists style encourages Tell Don’t Ask

Classical vs Mocking

I generally prefer the classical style to the mocking mostly because I don’t like thinking about implementation when writing a test.

I do however like having the mocking option in my back pocket, for those cases when a system is difficult to test, an object has a nasty setup, or I am refactoring legacy code.

The other piece of advice I recommend (my ex colleague Jonas Claesson turned me onto this) is to keep your calculations separate from your orchestrations.

If you can do this you get the best of both worlds. Calculations can be classically testing. Thin orchestrations mocked. Should keep both set easy to read, and easy to setup.

Here are some more summary notes from Martins article.

Article summary

Regular tests

  • state verification

Mock tests

  • behavior verification

Classical and Mockist Testing

  • Classical style uses real objects
  • Mocking style is to do everything with mocks
  • Classicists will use mocks for creating doubles

Choosing between the differences

  • If collaboration of objects is easy – classic is obvious choice
  • If collaboration between objects is awkward – mock
  • No big choice to be made here
  • Where things get more interesting is when we talk TDD

Driving TDD

  • Mockist will TDD there systems from the outside in
  • They will start with the GUI, describe their collaborations, and progressively marching deeper and deeper into the stack one layer at a time
  • Classical TDD doesn’t offer that kind of guidance
  • Whenever a classist requires something from a collaborator they simply hard code it
  • But classists take a slightly different approach – middle out
  • Here a classist will create an object for whatever domain they need and once they are working then you layer on the UI
  • Doing this you may never need to fake anything
  • Focuses the attention on the domain model, which helps keep the logic from leaking into the UI

Fixture Setup

  • Classic TDD needs to create SUT with all it’s collaborators
  • Mockists need only create SUT and mocks for immediate neighbours
  • Classics will do this using ObjectMothers – reuse whole objects
  • Mockists will say this is more work
  • Classists will say we do this once and reuse – you setup every time

Test Isolation

  • If you introduce a bug mocking, only those tests whose SUT contains the bug will fail
  • In classic, any tests of client objects also fail – ripple effect throughout system
  • Mocksts will say make finding bug harder to find
  • Classists say its usually pretty obvious – little bit more noise

Granularity

  • Classic tests can be more coarse grained than mockists as they test overall interaction of objects
  • In essence classic xunit tests are not just unit tests, but also mini-integration tests
  • Classists like this
  • Mocksts lose that quality – also run the risk that the mocks are incorrect masking inherent errors
  • Which ever way you go you need both – coarse and fine grained

Coupling Tests to implementation

  • Mockists couple their tests to implementation
  • Classiss don’t care about implementation – only the final state – not how it was arrived
  • With mock testing, writing the test makes you think about the implementation behavior
  • Mocksist like this – classists dont
  • Coupling implementation also interfere with refactoring, since implementaiton changes are more likely to break tests than classic testing

Design style

  • Classist are more comfortable creating query methods to support testing
  • Mockists don’t have to do that as much
  • Mocking also encourages Tell Don’t Ask
  • Classis can encourage Asking Not Telling – though in practice this is easily hadnleld
  • Mocking can also result in more classes (interfaces in various languages)
  • May not be as true now, but i remember many more interfaces and classes being ccreated than was necseary when mocking
  • Also DHH dislikes the designs mocking sometimes creates – mockists like it

Classic or mockist?

  • I have never seen a reason to go all mock
  • Mockists focus solely on implementation details of the SUT – never felt nature to me

Jonas

  • Don’t combine logic with orchestrations
  • Logic classis – classic mock
  • Orchestrations – simple as possible, no calcs, mockist

Links that help

http://www.martinfowler.com/articles/mocksArentStubs.html

http://googletesting.blogspot.se/2015/01/testing-on-toilet-change-detector-tests.html

How to access a private method in objective-c for testing

2 Comments

Say you’ve got a private method

-(NSUInteger)stackCount {
    return [self.myStack count];
}
@end

And you would like access to it in your test.

- (void)testAddNumberToStack {
    Calculator *calc = [Calculator new];
    [calc pushOperand:1.0];
    XCTAssertEqual(1, calc.stackCount);
}

You could expose it publicly in your header, but that would means exposing our privates (something we prefer to avoid). One neat thing I didn’t appreciate Objective-C could do was simply re-create the interface you want for your object under test in your test class, exposing the method you want to call, and then calling it from there.

@interface Calculator (Tests)
-(NSUInteger)stackCount;
@end

@implementation CalculatorTest

- (void)testAddNumberToStack {
    Calculator *calc = [Calculator new];
    [calc pushOperand:1.0];
    XCTAssertEqual(1, calc.stackCount);
}
@end

It may seem like cheating a bit. But because Objective-c is just a method based language of communication, we can define the message we want to send the object, and just send it assuming that when the tests runs, the method is going to be there.

Thanks to Mikael for showing me this trick. Means I can keep certain methods private, yet get access to them when I want for testing. Var kul!

Complete source

Calculator.h

#import &lt;Foundation/Foundation.h&gt;

@interface Calculator : NSObject
// operations
-(double)result;
-(void)pushOperand:(double)number;
@end

Calculator.m

#import &quot;Calculator.h&quot;

@interface Calculator()
@property (nonatomic, strong) NSMutableArray *myStack;
@end

@implementation Calculator

-(NSMutableArray *)myStack {
    if (!_myStack) _myStack = [NSMutableArray new];
    return _myStack;
}

-(double) result {
    return 0;
}

-(NSUInteger)stackCount {
    return [self.myStack count];
}

-(void)pushOperand:(double)number {
    NSNumber *operandNumber = [NSNumber numberWithDouble:number];
    [self.myStack addObject:operandNumber];
}

@end

Calculatortest.m

#import &lt;XCTest/XCTest.h&gt;
#import &quot;Calculator.h&quot;

@interface CalculatorTest : XCTestCase
@end

@interface Calculator (Tests)
-(NSUInteger)stackCount;
@end

@implementation CalculatorTest

- (void)setUp {
    [super setUp];
}

- (void)testCreateEmptyStack {
    Calculator *calc = [Calculator new];
    XCTAssertEqual(0, calc.result);
}

- (void)testAddNumberToStack {
    Calculator *calc = [Calculator new];
    [calc pushOperand:1.0];
    XCTAssertEqual(0, calc.result);
    XCTAssertEqual(1, calc.stackCount);
}

@end

The Automated Tester

7 Comments

Write. Code. Automate.

In the last couple of years I’ve seen a huge spike in demand for people who can write, code, and automate their own acceptance tests. I call these folks
automated testers. Not only do these people automate their own tests, they also do their own analysis and enjoy talking with customers.

In this talk I would like to explore the role of the automated tester, show how it is different than traditional Quality Assurance (QA), and share a few ideas on how to get there.

Software is eating the world

In his seminal article, Software Is Eating the world, Mark Andreesen makes a strong case for how software is taking over huge swathes of the modern economy and eating the world. Amazon. Apple. Google. Twitter. All these companies are building highly defensible, highly profitable businesses, with software at the heart of what they do.

This is great for the disruptors, but painful for the incumbents.

  • Amazon killed Barnes and Noble.
  • Netflix ended BlockBuster.
  • And AirBnb and Uber are redefining the meaning of hotel and taxi.

And it’s not just companies that are getting disrupted. It’s people.

With the rise of Agile, and the popularity of running projects with a more entrepreneurial startup spirit, traditional software development roles are being challenged. None more so than traditional QA.

Everyone is a tester

Today everyone is a tester.

  • Analysts test.
  • Developers test.
  • Designers test.
  • Customers tests.

Everyone tests. Quality is no longer the sole domain or responsibility of a single group or entity. It’s a team responsibility. And that’s a big change for the traditional tester. Everyone is in their house.

roles-eating-qa-lunch

This wouldn’t be a big deal if testers could show up on a project, be given a well written spec, and then be left alone to create comprehensive test plans to be executed every time someone changed the software. But those days are gone.

That role doesn’t exist anymore.

So if analysts are testing. And developers are testing. What’s traditional QA doing?

Leveling Up

I see two options. Traditional QA can dig in, defend their turf, and fight for the mantle of sole custodians of quality (in other words try to keep the status quo). Or they can branch out and get into analysis and development they way BAs and dev’s have gotten into testing.

the-qa-opportunity

To do this, the trick is to stop viewing analysis, development as testing as siloed roles. These are things that need to be done. Who does them doesn’t really matter.

I know many testers who are wonderful analysts. They bring an attention to detail and outside the box way of thinking that’s invaluable in flushing out requirements and necessary for doing thorough analysis.

I also know testers who are great at automating their own test cases. I don’t mean simple click and record playback bloat ware. I mean actually getting into code, using open source frameworks like Watir, Selenium, Cucumber, and RSpec.

By expanding their skills and getting into these other areas testers can quickly become doubly valuable on projects. Not only can do their own analysis and write great acceptance tests. They can code them up, verify things work, and reduce their reliance on developers for automation.

Shorter queues. Fewer handoffs. Less waste. More value.

Don’t let yourself be limited by yesterday’s job definitions.

Enter The Automated Tester

automated-tester-robot

This is where the automated tester comes in. These are testing experts with a passion for business and a thirst for automation.

They like knowing the business because it helps them write better test cases. And the more they automate, the more time they have for edges cases and exploratory testing.

They also like technology because a good understanding of the tech gives them insight the limits and capabilities of their testing frameworks.

The experienced ones know they can’t (and shouldn’t) automate everything. Every test has a cost and they think carefully about what to automated. They also work closely with developers to to see what unit tests the developers have written and make sure they aren’t duplicating effort.

And of course they still love exploratory testing. Automation is fine. But it’s not perfect. To truly be effective you still can’t beat the intuition and creativity that comes from firing up the browser, clicking through the app, and looking for ways to bust things along the way.

Getting there

Now if all this sounds easy – I assure you it isn’t. This is a tough, demanding role. You’ve got know the business. You’ve got to have a passion for automation and technology. And you have to enjoy continuously learning and retraining yourself.

But it’s a huge opportunity. It’s an invaluable role and one I think we are going to start seeing a lot more demand for in the near future.

Here are some ideas on how to get there.

1. Build apps.

You don’t have to be a developer. The goal is to get just deep enough to understand at a high-level how things work.

If you are testing web apps, you should know how HTML/CSS and JavaScript work. If you are building iPhone applications, you should have XCode and be really familiar with the emulator.

Did you know, that if you build a simple app holding a table with some numbers from a database, you’ve basically replicated 99% of the business applications being built today!

Building simple apps and understanding the technology is going to give you insight. Insight into how these testing frameworks works, their limitations, and where they need to be abandoned.

2. Master one testing automation framework.

There are a lot of automated tested frameworks to choose from. Don’t sweat it. Just pick one. Learn it really well. Then move onto another.

They all work pretty much the same, and once you’ve got one figured out they all work pretty much the same.

If you are testing the web, start with Selenium. Then try Watir, RSpec, Cucumber, or whatever else your team likes. If you are on the iPhone try Frank. The framework doesn’t really matter. Just pick one and go.

3. Hug a developer.

A good developer can be your best friend. The good ones are already doing what you getting into and the best will be more than happy to show you how it all works. They can set you up, help get you going, and even augment the frameworks you are using by making them easier to use.

Study what they do. Learn how they set things up. And soon you will be able to do it on your own.

4. Automate everything.

I know I just said the good tester don’t automate everything before their eyes, but in the beginning you should try, just to get the practice and see what’s possible.

You are going to write a lot of bad tests when you first start out. Don’t worry about it – we all do. Just automated everything you can, and you’ll soon start to get a feel for which tests add value, which ones don’t, and where to draw that line.

5. Fear nothing.

It’s going to be scary when you start (I am talking to you traditional QA testers). You are going to be leaving a space of traditional, comfort, strength, and responsibility, and entering something completely foreign and new. I’ll let you in on a little secret. It’s like this for everybody.

Developers go through this every time they learn a new language. BA’s go through it when learning a new business domain. And PMs go through it when they drop the Gantt chart in favor of the burndown.

You have nothing to fear. Everyone is going through it. Just be humble. Ask for help when you need it. And you will fit right in.

One challenge

stand-out-of-crowd

One challenge of becoming an automated tester is people are going to look at you a little bit funny. Companies don’t like things that don’t fall neatly into slots, and you are going to be an anomaly.

You are going to go to conferences, explain what you do to people and they are going ask:

So are you a analyst, tester, or developer?

Conferences, papers, titles, pay scales, expectations, and bodies of knowledge are not going to be ready for you. Nobody is expecting you to work like this. We’ve got 50 years of history working against us. But it’s like that for the pioneers.

All I can say is hang in there. Do whatever feels right. Start slow. Start fast. Look for the opportunity and seize it when it comes.

In the meantime practice, be prepared, and keep taking on more responsibility. Once you demonstrate the value this role brings (through action – not words) project managers will soon be getting into fist fights for your services, and you’ll have the freedom to contribute however you wish.

Remember the goal isn’t to test or even build great software. It’s to build something wonderful for our customers. And you are just doing whatever it takes to make that happen.

Summary

There’s a new role for testers out there. One that blends the art of traditional testing, with analysis, development, and automation.

If you have a passion for testing, are curious about technology, and are looking for more ways to contribute to your project, consider becoming an automated tester. It’s an exciting role, one that every Agile project needs, and one I hope we see a lot more people entering in the near future.

It’s not about the unit tests

46 Comments

iOS developers generally don’t unit test. So why then do they as a community seem to enjoy a reputation of quality.

No unit tests. No continuous integration. No TDD.

That pretty much summarizes my last project. It was my first paying iOS gig, and not only did we not apply these cherished practices, we shipped a high quality product.

This really bugged me.

I had always thought unit tests were essential. But this experience with iOS seemed to challenge some deeply cherished assumptions I had developed over the years about writing software, and led me to ask myself some very uncomfortable questions.

Was I regressing in my development practices?
Were unit tests not essential?
Was there something different about iOS development?
And how did the iOS community ship quality product, without unit tests at their core?

These questions kept me up and night, and led to the soul searching contained within this article.

Challenged assumptions

Ten years ago I used to debate people on the merits of unit testing. Not any more. Unit testing has become such a common practice that you’d struggle to find a modern development platform that doesn’t have some sort of automated testing framework.

And why not? The benefits are obvious. You can release, change, and refactor your code with confidence. You save a ton on regression testing. And the feeling of security that comes from knowing you have a suite automated unit tests backing your every change is undeniable.

So imagine my surprise when I entered a community responsible for some of the worlds most loved mobile applications, only to discover they don’t unit test. Even more disturbing, they seem to be getting away with it!

What’s different about iOS?

While I won’t say developing apps on iOS is easier, there are a few things iOS developers have going for them.

1. Smaller screen size.

You can’t fit a lot on a mobile phone screen — there just isn’t a lot of room. The apps also tend to be smaller (many don’t even have a backend). This, when combined with a culture of minimalism, dramatically simplifies things in terms of code and data. There is simply less of both.

2. No legacy.

Mobile apps are so new, iOS developers typically aren’t burdened with 100,000 lines of legacy code the way enterprise developers sometimes are. This let’s them start from scratch unencumbered.

3. One language.

You can build an iOS application knowing only one language: Objective-C. The typical web developer needs no fewer than four or five languages to do anything other than static HTML.

4. A mature platform.

A lot of the heavy lifting is done for you in iOS. If you need to do something with photos, music, or Facebook it’s there. You just plugin.

5. Very visual.

This was probably the biggest difference for me coming from the enterprise. Instead of spending days wading through layers of architecture (mocking and unit testing every step of the way), iOS developer have almost no layers of architecture. They spend almost all their time at the UI layer.

That means the nature of the code they write is often visual. The only way to see if it works is to fire up the simulator and try it out.

As interesting as these differences are however, they still don’t tell the whole story. If they did, every mobile platform would enjoy this higher level of quality. Nope there’s something else going on. Something bigger.

Who cares?

When you do something for a long time, it’s easy to forget what people in your field once did without.

  • Architects used to work with slide rules.
  • Seafarers used to navigate by the stars.
  • Artists used to work solely with their bare hands.
  • Programmers used to program without unit tests (often at a much higher quality they you see today).

And yet in all these fields, practitioners were able to achieve what we today, with all our modern tools, would admit was a high level of quality in their work. What was their secret?

They cared.

These people simply cared more about their craft, and what they were doing, than their contemporaries. They ‘out cared’ the competition. And that is what I see in the iOS community.

They care more about the art.
They care a lot about the exact wording and spacing of text on buttons.
They care a lot of the speed and performance.
And they care a ton about affordances (like remembering where in the scroll list a user was when they put the application in the background).

Apple works very hard to make sure every developer in their ecosystem cares, and they give them the tools so they have no excuse not to.

They walk them painstakingly through how to creating beautiful art for their apps.
They share (and enforce) human interface guidelines for mobile application development.
They curate and block apps that don’t meet certain quality or standards.
And their former CEO was known for calling people up in the middle of the night and tweaking the color of a logo.

It’s in the communities DNA. Here is the letter you get on your first day starting at Apple.

These guys care. They care like artists care. And—I’m just citing my own experience here— the same can’t be said for other platforms I have been a part of.

What does this have to do with unit tests?

Absolutely nothing. And that’s my point.

What leads to quality is something much bigger—more than a collection of software engineering techniques or a collection of practices.

When I entered this community I was under the false impression that if you didn’t write software the way I did, you must be doing it wrong.

Instead I discovered a community that cared more about quality than I did, and that I still had a lot to learn about crafting a quality experience.

That’s what this whole things has taught me. It’s not about the practices. It’s about the spirit, intent, and in which they are applied. Used when applicable. Quickly abandoned when not.

Are unit tests an invaluable tool for writing great software? Heck yes.
Am I going to produce a poor product if I can’t unit test? Hell no.

And that is what this experience taught me. I need to be more than a collection of practices. I will unit test where I can (including iOS) and I will doing whatever else it takes where I can’t.

All I can say is to keep growing sometimes we need to challenge our most cherished assumptions. It doesn’t always feel good, but that’s how we grow, gain experience, and turn knowledge into wisdom.

The second you think you’ve got it all figured out you’ve stopped living.

If you want to see how far down the rabbit hole this whole discussion on quality can go, I suggest picking up a copy of Zen and the Art of Motorcycle Maintenance. It’s not an easy read. But it may change your life.

Oh ya, and keep unit testing.

Update

portugal-flag
Portuguese translation courtesy of Sylvestre Mergulhão.

Older Entries

%d bloggers like this: