10 Things About the SDET Role

Unfortunately, there are still lots of developers that remain unconvinced of the usefulness of automated testing, particularly if they have to author the tests themselves. Enter the Software Development Engineer in Test (SDET), or Software Engineer in Test (SET). There are varying opinions about the roles and responsibilities of an SDET. This post is not likely to conform to [your favorite company]’s description, but I expect that it will have a lot in common.

SDETs are subject-matter experts in the testability of code.

This requires them to know …

1) … what kinds of tests to write.

There are different ways to test a product. Most products require a combination of different types of tests. Common types include unit, integration, and functional–Google uses small, medium, and large.

2) … approximately how many of each kind of test is needed.

Different types of tests have different strengths, weaknesses, and ROIs. Things like how long a test takes to execute, maintain, or create affects how many of each of the various types yield the greatest value.

3) … techniques for refactoring product code so that it is testable.

Especially with legacy code, it is common to have to break dependencies before you can properly test a unit of code. For functional tests, the product needs to include hooks (e.g. identifiers for UI components, backdoors for initializing state, etc.) to facilitate testing.

4) … that bad tests are much worse than no tests.

Bad tests might require excessive amount of work to maintain, all fail at once making it hard to locate the source of the issue, or render false positives/negatives leading to distrust of the tests.

I think bad tests are a major source of skepticism among developers. A lot of developers have been burned by their first attempts to automate testing that resulted in unreliable, unmaintainable tests.

5) … how to choose great tools.

Different products require different testing tools. For example, MOQ and RhinoMocks are sufficient mocking libraries for green field development in C#. However, a legacy system in C# might require something like TypeMock Isolator to be able to get poorly designed code under tests.

You want to choose tools that make it easy to write good tests.

6) … how to mentor others.

The benefits of testable code go beyond testability. SDETs need to be able to explain why the code changes are not merely in support of testing, but actually improve the quality of the code.

SDETs are not …

7) … lesser/beginner developers.

Based on the items above, I wouldn’t expect a beginner programmer to fill those shoes. Would you? I think this comes from developers that think product code is more important than test code. In my experience, writing tests and making code testable is often more challenging than writing product code that works.

8) … the only ones writing tests.

Quality needs to be a priority of the entire team. It is hard to have high levels with quality without tests. Also, product developers nearly always outnumber SDETs by a significant ratio (1:4 on my current team) and the amount of test code usually exceeds the amount of product code. That being the case, developers have to write tests, too, if you want a shot at adequate test coverage.

9) … forbidden to touch product code.

As mentioned in #3, a product often has to be changed to make it testable. Even when using TDD, you might have to add hooks for functional tests to facilitate testing.

Also, it shouldn’t surprise you that working within a code base makes you better equipped to test it. Refactoring has always been a quick way for me to familiarize myself with a code base. I think that SDETs should spend time implementing features, too–just not as much time as product developers.

10) … permanent as a role.

I believe that if the SDET role is properly realized in an organization that it will eventually phase itself out. As product developers begin to see the value of testing and incorporate it into their daily routine, there will be less of a need for a distinction between product developers and SDETs.

Share

Allergies and Dependency Inversion

A couple months ago, I posted about the Law of Demeter and how to keep your classes from requiring knowledge of its dependencies’ dependencies. The code for that post was contrived and I only refactored it far enough to demonstrate the Law of Demeter. However, after posting the article on LinkedIn, someone correctly pointed out that I neglected the Dependency Inversion Principle.

The Dependency Inversion Principle tells us that our higher-level modules should not depend on our lower-level modules. Let’s give dependency injection a try to demonstrate the principle.

Consider the following code:

class LawnService {
    func serviceLawn() {
        Employee().mowGrass()
    }
}

class Employee {
    func mowGrass(){
        LawnMower().cut()
    }
}

class LawnMower {
    func cut(){
        println("Hello allergies!")
    }
}

LawnService().serviceLawn()

 

So, what problems does the example code present?

  1. Isolating the different classes for testing purposes is not possible because of dependency on concrete classes.
  2. The code is not very flexible. What if I wanted to have multiple kinds of lawn mowers?

Let’s start with making it possible to isolate individual classes for testing purposes and see if that technique helps us with our second problem, too.

I need to be able to inject a stub or mock into LawnService, so that I can test it without testing the concrete Employee class. If I make a protocol for the employee and provide a mechanism for injecting an instance into the LawnService class, I should be able to do just that.

// Added a field for an instance of employee 
// that will use whatever concrete is passed 
// in on construction
class LawnService {
    var employee: LawnServiceEmployee
    
    init(_ employee: LawnServiceEmployee) {
        self.employee = employee
    }
    
    func serviceLawn() {
        self.employee.mowGrass()
    }
}

// Extract protocol for lawn service employees
protocol LawnServiceEmployee {
    func mowGrass()
}

// Employee now implemented the new protocol
class Employee: LawnServiceEmployee {
    func mowGrass(){
        LawnMower().cut()
    }
}

// A stub class used for testing
class EmployeeStub: LawnServiceEmployee {
    func mowGrass(){
        // do nothing, we're a stub
    }
}

// Nothing new here
class LawnMower {
    func cut(){
        println("Hello allergies!")
    }
}

// Injecting our real Employee 
// implementation
LawnService(Employee()).serviceLawn()

// Injecting our fake Employee
// implementation for tests
LawnService(EmployeeStub()).serviceLawn()

 

Also, I find it convenient to provide a default implementation if I know I’ll be using a particular concrete most of the time.

class LawnService {
    var employee: LawnServiceEmployee
    
    // Notice that we default to an instance of 
    // Employee
    init(employee: LawnServiceEmployee = Employee()) {
        self.employee = employee
    }
    
    func serviceLawn() {
        self.employee.mowGrass()
    }
}

// This call instantiates Employee and uses it
// rather that requiring it to be passed into
// the init
LawnService().serviceLawn()

 

Let’s do the same thing for lawn mowers, but this time we’ll provide real types instead of a testing fake.

class LawnService {
    var employee: LawnServiceEmployee
    
    init(employee: LawnServiceEmployee = Employee()) {
        self.employee = employee
    }
    
    func serviceLawn() {
        self.employee.mowGrass()
    }
}

protocol LawnServiceEmployee {
    func mowGrass()
}

// Added a field for an instance of lawn mower
// that will use whatever concrete is passed
// in on init
class Employee: LawnServiceEmployee {
    var lawnMower: LawnMower
    
    // defaulting to RidingMower may not be 
    // a good idea...
    init(lawnMower: LawnMower = RidingMower()){
        self.lawnMower = lawnMower
    }
    
    func mowGrass(){
        RidingMower().cut()
    }
}

// Extract protocol for lawn mowers
protocol LawnMower {
    func cut()
}

// One type of lawn mower
class RidingMower: LawnMower {
    func cut(){
        println("Riding along on my gas hog")
    }
}

// Another type of lawn mower
class ReelMower: LawnMower {
    func cut(){
        println("Getting some exercise")
    }
}

var mower = ReelMower()
var employee = Employee(lawnMower: mower)
LawnService(employee: employee).serviceLawn()

 

That’s all well and good, but I think I can clean things up a little more by creating a different kinds of lawn services: an eco-friendly lawn service and a standard lawn service.

class LawnService {
    private var employee: LawnServiceEmployee
    
    init(employee: LawnServiceEmployee) {
        self.employee = employee
    }
    
    func serviceLawn() {
        self.employee.mowGrass()
    }
}

// Standard lawn service has employees
// using riding lawn mowers
class StandardLawnService: LawnService {
    init() {
        var employee = Employee(lawnMower: RidingMower())
        super.init(employee: employee)
    }
}

// Eco-friendly lawn service has employees 
// using reel mowers
class EcoFriendlyLawnService: LawnService {
    init() {
        var employee = Employee(lawnMower: ReelMower())
        super.init(employee: employee)
    }
}

protocol LawnServiceEmployee {
    func mowGrass()
}

class Employee: LawnServiceEmployee {
    var lawnMower: LawnMower
    
    init(lawnMower: LawnMower){
        self.lawnMower = lawnMower
    }
    
    func mowGrass(){
        self.lawnMower.cut()
    }
}

protocol LawnMower {
    func cut()
}

class RidingMower: LawnMower {
    func cut(){
        println("Riding along on my gas hog")
    }
}

class ReelMower: LawnMower {
    func cut(){
        println("Getting some exercise")
    }
}

StandardLawnService().serviceLawn()
EcoFriendlyLawnService().serviceLawn()
Share

Private: It’s Not What You Think

So, here is an interesting tidbit for any of you Swifters that have experience in other languages with access modifiers: “private” doesn’t mean what you assume it means.

Put these to classes in the same file:

public class Secrets {
    private var shhh = "Cannot Hide"
}

public class Exposed : Secrets {
    public func CannotRun() -> String {
        return self.shhh // no compiler error?!
    }
}

 
And, call this:

// This has to be in the same file as the classes
Secrets().shhh // no compiler error?! value is "Cannot Hide"

// This, of course, can be put in a different file
Exposed().CannotRun() // returns "Cannot Hide"

 

Ok. That wasn’t what I was expecting. However, if we read the manual, we find the following: Private access restricts the use of an entity to its own defining source file. Use private access to hide the implementation details of a specific piece of functionality.

I have to admit that I didn’t discover this by reading Apple’s documentation. Actually, it happened by accident when I was playing in a playground. The thing about a playground is that it is just one file, which means that the “private” access modifier has no effect. So, when I made a class field private and was still able to access it, I assumed that there must be a bug in the playground. A quick search on Google straightened me out, though.

So, the next question I asked was “Why?” Why would you want “private” to scope variables, methods, and classes to the current file?

It just so happened that I needed a way to quickly create some fake instances of classes for testing. One of the requirements was that the faked instances should have unique IDs. Prior to Swift 1.2, the “static” keyword wasn’t available, so I came up with the following solution instead:

// UniqueItem.swift file

public class UniqueItem {
    public var id : Int
    
    init() {
        id = 0 // maps to unique database value
    }
}
// FakeUniqueItem.swift file

private var autoId: Int = 0

private func getNextId() -> Int {
    return autoId++
}

public class FakeUniqueItem : UniqueItem {
    override init(){
        super.init()
        id = getNextId()
    }
}
// MyTests.swift file
 
println(FakeUniqueItem().id) // 0
println(FakeUniqueItem().id) // 1
println(FakeUniqueItem().id) // 2

No “static”? No problem. Because both the autoId variable and getNextId() function were private, they weren’t accessible outside the file. I got the effect of a static variable because I was sharing the instance of autoId across instances of my FakeUniqueItem class.

Fortunately, the release of Swift 1.2 added static variables to the mix, so I was able to get rid of my global, private variable and function:

public class FakeUniqueItem : UniqueItem {
    private static var autoId : Int = 0
    
    override init(){
        super.init()
        id = FakeUniqueItem.autoId++
    }
}

So there you have it. My story of how I discovered what “private” means to Swift. If you can think of other ways to leverage it, please leave a comment!

Share

The Law of Demeter and the 9th Circle of Hell

There are ways to make your code a living nightmare to test. Most of them involve tight coupling. To me, there is nothing worse than trying to mock multiple layers of dependencies in order to test some otherwise simple block of code.

Consider the following code:

func doubleSomethingUseful(does: Does) -> Int {
    return 2 * does.something().useful().eventually()
}

class Does {
    func something() -> Something {
        return Something()
    }
}

class Something {
    func useful() -> Useful {
        return Useful()
    }
}

class Useful {
    func eventually() -> Int {
        return 1
    }
}

If I’m trying to test doubleSomethingUseful(), I have to mock an instance of the Does class so that the something() method can return a mock of the Something class. The mock of the Something class has to return a mock instance of the Useful class when the useful() method is called. And, all this so that the mock instance of the Useful class can return some predetermined test value when the eventually() method is called.

Enter the Law of Demeter (or principle of least knowledge.) The general idea is that a class should not expose its dependencies to its consumers. Instead, it should expose the functionality of its dependencies through wrapper methods. This is a form of information hiding. Click here for an in-depth explanation.

Here is what it might look like for my contrived example:

class Does {    
    func somethingUsefulNow() -> Int {
        return Something().usefulNow()
    }
}

class Something {
    func usefulNow() -> Int {
        return Useful().eventually()
    }
}

class Useful {
    func eventually() -> Int {
        return 1
    }
}

func doubleSomethingUseful(does: Does) -> Int {
    return 2 * does.somethingUsefulNow()
}

Notice that now I only have to mock the somethingUsefulNow() method on the Does class and have it return the predetermined test value. I don’t have to worry about any of its dependencies.

In Dante’s Inferno, the 9th circle of hell is reserved for traitors. Traitors are there because “their acts involve betraying a special relationship of some kind.” See more on the Wikipedia article. Let’s not betray that special relationship between objects, keeping the details where they belong.

UPDATE: This post addresses the Law of Demeter, but there is more to properly decoupling code! Click here to learn about Dependency Inversion.

Share

A Metric for Test Quality

One thing that I’ve seen over and over again is that best practices or coding standards that aren’t backed up with some sort of tooling are not consistently followed. I’m sure everyone has been on the team where there is one person that insists on using an ‘m’ instead of an ‘_’ in front of a private field name despite everyone else’s consistent opposite.

Fortunately, there are tools that can red flag this non-conformance and even prevent someone from checking in code that isn’t consistent with those standards. A lot of these tools are known and readily available for use on product code. Less tools and techniques are known for encouraging good testing practices.

One good rule for writing automated tests–unit, functional, or otherwise–is to avoid logic in tests. After all, we’re testing code to validate the logic in our products. We don’t write tests to validate the logic in our tests, so it is best to simply avoid it. Check out Google’s testing blog for a more in-depth treatment: Don’t Put Logic in Tests

So, now that we have agreed on this rule, what metric can we use to ensure we follow it? Cyclomatic complexity. For any of you not familiar with cyclomatic complexity, it is the number of paths through a particular block of code. If there are no decision points (if, switch, etc.) in a given method, then the complexity of that method is 1. You can find static code analyzers that can calculate this metric for just about any language, so what are you waiting for?

Share