The Way of the Web Tester

Leave a comment

Screen Shot 2016-09-15 at 5.45.37 AM.png

Hello everyone. A book I have been working on diligently for the last two years has finally hit the shelves. It’s called The Way of the Web Tester and it’s a book for anyone who wants to learn how to write automated tests for the web.

What’s in this book?

The book starts out by explaining what the three kinds of tests are that we typically see in web projects, along with some rules of them and guidance on where and when to use each.

We they spend a full two chapters at each level of something called the testing pyramid seeing how to add UI tests to legacy systems, how one can test web services, along with some rules of thumb around unit testing. That’s Part I.

Screen Shot 2016-09-15 at 6.25.50 AM.png

In Part II we build on the concepts outlined in the testing pyramid, but shift our focus to those skills specifically geared towards writing good tests.

First we look at some of the basics of programming, and show how by learning a few simple principles and practices we can dramatically increase the readability and maintainability of our automated tests.

We then look at some practices around test organization, and see how by simply grouping and organizing our tests in certain ways can make maintaining our tests so much easier.

And in the chapter on effective mocking, we look at some of the perils that come with this style of unit testing, and see how we can use mocks effectively while staying out of the swamp of mocking.

And we conclude with a section on the art of writing tests first. And see how beginning with the end in mind not only helps you test your systems, it can actually help you iterate on your design.

Who is this book for?

Screen Shot 2016-09-15 at 6.52.26 AM.png

If you are a tester who’s always wanted to get into test automation, this is the perfect book for taking that first step into a larger world.

If you are a developer, and you’d like to learn how to make your code even more robust, this book will show you how to move fast without breaking stuff.

And if you are team lead, looking better ways to get your testers and developers together, his is the Rosetta Stone you’ve been looking for.

Once your team has a common framework, language and understanding around automated testing, they will be able to better coordinate they’ll their efforts and write the best automated tests for your project.

So wait no further. Crush bugs. Move fast. And have fun. I hope you enjoy The Way of the Web Tester.

On sale now at all reputable book stores.

https://pragprog.com/book/jrtest/the-way-of-the-web-tester

https://www.amazon.com/Way-Web-Tester-Beginners-Automating/dp/1680501836

 

Objective-C – How to stub out an NSError unit test

Leave a comment

#import <XCTest/XCTest.h>

#import "Player.h"

// Mocking somethings (like NSError's) can be tricky in OCMockito.
// One alternative, if your mocks get too complex, is to simply manually stub.

// create a test class for the behaviour your want to stub,
// override the methods
// and then inject that into your SUT

// For example

// Test class we want to stub
@interface FakeHandler : Handler
- (BOOL)playTrack:(NSString *)track error:(NSError **)errorPtr;
- (BOOL)playAlbum:(NSString *)album error:(NSError **)errorPtr;
@end

@implementation FakeHandler

// Method / behaviour we want to override
- (BOOL)playTrack:(NSString *)track error:(NSError **)errorPtr {

    NSString *domain = @"com.MyCompany.MyApplication.ErrorDomain";
    NSString *desc = NSLocalizedString(@"Unable to connect to network.", @"");
    NSDictionary *userInfo = @{ NSLocalizedDescriptionKey : desc };

    *errorPtr = [NSError errorWithDomain:domain
                                         code:-101
                                     userInfo:userInfo];

    return NO;
}

- (BOOL)playAlbum:(NSString *)album error:(NSError **)errorPtr {
    return NO;
}

@end


// Now we can use all of this in our test.

@interface PlayerStubTest : XCTestCase
@property (nonatomic, strong) Player *player;
@property (nonatomic, strong) Handler *fakeHandler;
@end

@implementation PlayerStubTest

- (void)setUp {
    [super setUp];
    self.fakeHandler = [FakeHandler new];
    self.player = [[Player alloc] initWithHandler:self.fakeHandler]; // Dependence inject our stub
}

- (void)testNetworkFailure {
    // Now when we call our play method, we can expect a failure and test that it bubbles up
    NSError *expectedError;
    [self.player playURL:@"track.xxx" error:&expectedError];
    XCTAssertNotNil(expectedError);
}

@end

How to throw and test exceptions in Objective-C

Leave a comment

PlayerTest.m

- (void)testInvalidUrl {
    XCTAssertThrows([self.player playURL:@"foobar" error:NULL], @"Invlaid url.");
}

Player.m

- (BOOL)playURL:(NSString *)url error:(NSError **)errorPtr {
    if ([url containsString:@"track"]) {
        return [self.handler playTrack:url error:errorPtr];
    } else if ([url containsString:@"album"]) {
        return [self.handler playAlbum:url error:errorPtr];
    } else {
        [NSException raise:@"Invalid url" format:@"%@ must be a track or an album.", url];
    }
    return NO;
}

Dealing with Errors in Objective-C

Leave a comment

A reminder to myself around where and when to use NSError vs throwing exception in Objective-C.

This article gives a nice summary

https://developer.apple.com/library/content/documentation/Cocoa/Conceptual/ProgrammingWithObjectiveC/ErrorHandling/ErrorHandling.html

Use NSError for Most errors

When things go wrong in your app, due to

x network connectivity
x database been down etc

Use NSError and pass errors by reference.

- (BOOL)writeToURL:(NSURL *)aURL
 options:(NSDataWritingOptions)mask
 error:(NSError **)errorPtr;

Create an error pointer/reference and pass it to your method

NSError *anyError;
BOOL success = [receivedData writeToURL:someLocalFileURL options:0
 error:&anyError];

if (!success) {
 NSLog(@"Write failed with error: %@", anyError);
 // present error to user
}

The recover if possible and display an error to the user.

Generating your own errors

To create your own error define your own error domain.

com.companyName.appOrFrameworkName.ErrorDomain

Along with a unique error code for each error that may occur.

NSString *domain = @"com.MyCompany.MyApplication.ErrorDomain";
 NSString *desc = NSLocalizedString(@"Unable to…", @"");
 NSDictionary *userInfo = @{ NSLocalizedDescriptionKey : desc };
 
 NSError *error = [NSError errorWithDomain:domain
 code:-101
 userInfo:userInfo];

Then use as follows

- (BOOL)doSomethingThatMayGenerateAnError:(NSError **)errorPtr;

- (BOOL)doSomethingThatMayGenerateAnError:(NSError **)errorPtr {
 ...
 // error occurred
 if (errorPtr) {
 *errorPtr = [NSError errorWithDomain:...
 code:...
 userInfo:...];
 }
 return NO;
}

Exceptions are Used for Programmer Errors

- (void)testInvalidUrl {
 XCTAssertThrows([self.player playURL:@"foobar" error:NULL], @"Invlaid url.");
}

- (BOOL)playURL:(NSString *)url error:(NSError **)errorPtr {
 if ([url containsString:@"track"]) {
 return [self.handler playTrack:url error:errorPtr];
 } else if ([url containsString:@"album"]) {
 return [self.handler playAlbum:url error:errorPtr];
 } else {
 [NSException raise:@"Invalid url" format:@"%@ must be a track or an album.", url];
 }
 return NO;
}

How to stub a class for unit testing

Leave a comment

Here is an example of how to override an existing class and create a stub for your unit tests in objective-c.

PlayerStubTest.m

#import <XCTest/XCTest.h>

#import "Player.h"
#import "Handler.h"

@interface PlayerStubTest : XCTestCase
@property (nonatomic, strong) Player *player;
@property (nonatomic, strong) Handler *stubHandler;
@end

// overide default behavior of handler for those methds you want to test
@interface StubHandler : Handler
- (BOOL)playTrack:(NSString *)track;
- (BOOL)playAlbum:(NSString *)album;
@end


@implementation StubHandler

- (BOOL)playTrack:(NSString *)track {
 NSLog(@"Stub track");
 return NO;
}

- (BOOL)playAlbum:(NSString *)album {
 NSLog(@"Stub albumn");
 return NO;
}

@end

@implementation PlayerStubTest

- (void)setUp {
 [super setUp];
 self.stubHandler = [StubHandler new];
 self.player = [[Player alloc] initWithHandler:self.stubHandler];
}

- (void)testExample {
 [self.player playURL:@"track"];
}

@end

 

When you run this you should see ‘Stub track’ in the console.

 

How to setup new iOS workflow

Leave a comment

  1. Install Xcode from the app store.
  2. Install Homebrew and its dependencies.
    $ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
  3. Install a recent version of Git and CMake.
    $ brew install git cmake
  4. Upload your SSH key to Github
    1. Generate one if you haven’t already.
      $ ssh-keygen -t rsa -b 4096 -C "<email address>"
  5. Install a good .gitignore file.
    $ curl https://www.gitignore.io/api/objective-c >> ~/.gitignore
  6. Make sure git knows about it.
    $ git config --global core.excludesfile "~/.gitignore"
  7. Make sure Xcode removes trailing whitespace.
    $ defaults write com.apple.dt.Xcode DVTTextEditorTrimTrailingWhitespace 1
    $ defaults write com.apple.dt.Xcode DVTTextEditorTrimWhitespaceOnlyLines 1
    
    FORK

    1. Fork
    2. Clone your fork
    3. Add main (real repository) as your upstream remote.

    git remote add upstream git@ghe.xxx.net:/iOS/xxx-ios.git
    

    4. Disable pushes to upstream, to minimise the risk for mistakes

    git remote set-url --push upstream no_push
    

    5. Make sure git pushes to your fork by default.

    git config remote.pushdefault origin
    

    6. Make sure git only pushes the current branch.

    git config --global push.default simple
    

    7. set your master branch’s remote to upstream.

    git fetch upstream
    git branch --set-upstream-to=upstream/master master
    

    8. Then work

    git checkout master
    git pull upstream master
    git checkout -b my-fix
    

    9. Push the changes to your fork.

    git add .
    git commit -m
    git push -u origin my-fix
    

    10. If merging a PR to upstream conflicts, merge master to your branch and update the PR.

    git fetch upstream master
    git merge upstream/master
    

     

Lessons learned in TDD

Leave a comment

This was an excellent talk Ian Cooper gave at NDC a couple years ago.
It hits on a lot of the challenges and common problems Ian and others have seen with test suites and TDD in general.

It’s a great talk. Which you can watch in it’s entirety here.

Here are some notes.

The Problem

Screen Shot 2016-02-16 at 8.54.34 AM

Where did it all go wrong

Screen Shot 2016-02-16 at 8.54.40 AM

Two keep points
Not writing tests against behaviors.
Coupling our implementation details to our tests.

If we fix these we will have smaller test suites, much more self explaining, and much less painful to own.

There was advice in early TDD that said when you are given a new method on a class, that was the trigger point for writing a new test.

And that’s really wrong.

The trigger for writing a new test is to have some piece of behavior that I want to implement. The test needs to capture that behavior.

The reason why when you go back to your tests that you find them so hard to read is there is a lost connection between the low level test you are writing and the high-level feature you are trying to solve. It’s lost in the noise.

What you need to do is express in your test that we are testing a given behavior of our system.

Before you get to the point where you are going to put the implementation details down, you first put down the behaviours you are trying to capture, and then the implementation. Don’t jump ahead too early.

Testing outside in

Start with the use case, the story, the scenario or feature we want to solve, on the outside, then work your way in from there.

Recommendation: don’t start this at the UI level. Start one level beneath at the plain old object level. Then if your UI changes, it won’t matter. Focus on the domain models.

Test the public API of classes. Not the internals. Test the surface only, and it should be quite narrow.

As soon as you start testing the internals you are coupling your tests to your implementation details. To change your implementation details you now have to change your tests. And that’s the key problem.

So your surface area should be much narrowly than many people are testing today. Just the API.

That means you will write less tests, tests against the use cases, and refactoring the contract remains the same, the internals change, and you don’t break any tests.

BDD

This is what BDD is all about. Testing behavior, and not low level details, and methods on classes.

What is a unit test

Screen Shot 2016-02-16 at 8.54.48 AM

The simplest thing

Screen Shot 2016-02-16 at 8.54.55 AM

The real problem with TDD is Kent asked us to go green as fast as possible, committing as many sins as we want in that step. So go to project, cut and paste some code, and stick in application. Hack it. Make it work. Don’t worry about the design. Get it green ASAP.

Kent’s point here is that it’s difficult to do two things at once. Make the test pass and design at the same time. Better to separate out.

Green is about solving the problem. Refactoring is about doing the design.

Screen Shot 2016-02-16 at 8.55.00 AM

You do not write new tests here when refactoring. You already have the high-level behavior tests written. So long as those pass, you are OK. If you need another test, write it at that level.

But when you are refactoring, you are free to focus on design and engineer it right.

If you write tests here you bind your implementation details to your tests.

Coupling is the biggest problem in software. It is the enemy. Forget DRY. Coupling will kill you. Do you couple tests to implementation details.

Screen Shot 2016-02-16 at 8.55.06 AM

Dependency is the key problem in software development at all scales.

We need to eliminate the dependency between our tests and our code. Tests should not depend on details. Tests should depend on contracts or public interfaces. This allows us to change implementations without changes tests.

Test behaviors not implementations.

Means less tests.
Moving faster.
Going quickly to green.
Now you are refactoring and making it cleaner.
Better forward progress, the tests aren’t slowing you down so much.

Screen Shot 2016-02-16 at 8.55.13 AM

Older Entries

%d bloggers like this: