Automation in Testing

Preface

1. Becoming familiar with an elephant
2. Automation and chicken soup
3. Lasting value
4. A bit of terminology

Reading guide

1. If you want it all
2. If you want to set up or improve automation
3. If you have a specific question

About the author

1. Part one: Automation in Testing

1 The PUPPET model

1. (Automation as a) Product
2. Understanding (the essence)
3. People
4. Process
5. Execution
6. Techniques
7. A complete and simple overview

2 Success (through strategy)

1. The goal (or: the why)
2. The vision (or: the what)
3. The plan (or: the how)
4. Part two: Understanding

3 Testing

1. Testing a car
2. Many components
3. Where to test
4. What does the business need?
5. Repeated testing

4 Automation in testing

1. Automation
2. Testing cannot be automated

5 Automating is not testing

1. Automating is software engineering
2. Skill
3. Technical debt
4. Other skills relating to automation
5. Three roles

6 It is not about the tool

1. The tool is not the solution
2. The tool is (probably) not the problem

7 The goal and scope

1. Good goals
2. Pick one – but feel free to adjust later
3. Scope

8 The value of an automated check

1. Is it being run?
2. Is it telling you something interesting?
3. Is it reliable?
4. Is it fast enough?
5. What happened here?
6. Is it small enough?
7. Is it independent?
8. Two versions?
9. Effective and efficient

9 Writing good checks

1. Readable
2. Understandable
3. Effort
4. Tooling
5. Test data
6. Data-driven

10 The secrets of the great pyramid

1. Which one is the real pyramid?
2. Using the original pyramid effectively
3. What if there is no UI?
4. Checks or instructions?
5. The pyramid for other goals
6. Many checks do not use one interface

11 The design of an automation solution

1. The conceptual picture
2. The technical picture
3. Integrated tools
4. Part three: Automation as a product

12 Automation is a product

1. A paint factory
2. The machine is a product
3. Automation is also a product

13 Decisions (and issues)

1. What?
2. Who?
3. What about the product owner or an external expert?
4. How

14 The automation lifecycle

1. The right time to (re-)start
2. Further preparation
3. Lifecycle stages

15 The goal

1. Productivity
2. Efficiency
3. Fast feedback
4. Quality
5. Cost
6. Sustainability

16 Scope

1. Product
2. People
3. Processes
4. Execution
5. Techniques

17 Who automates?

1. Decentralized internal automation
2. Centralized internal automation
3. Hybrid internal automation
4. Outsourcing

18 The risk management stages

1. The Proof of Concept
2. Tool selection
3. The roadmap
4. The pilot stage

19 The later stages: Stability and decline

20 Additional support

??? Additional valid goals

1. Happy testers
2. Executable specifications and living documentation

??? Poor goals

1. Automation because of Agile or Continuous Integration
2. Let automation find more bugs
3. ‘Automate everything’ and the like
4. Focus on business value using ‘why?’
5. Part four: People

21 Roles, knowledge, and skills

22 Learning and teaching

1. Knowledge
2. Skills
3. Teaching

23 Culture

Team roles

Other roles

Manager

1. Goal and objectives
2. Support
3. Culture
4. Provide training and coaching

Hiring manager

??? The tester

1. Part five: Processes

The test case lifecycle

Defining objectives

Optimizing value

Agile specification

Automating

Executing

Analysis

1. Part six: Execution

Interfaces and deployment

1. The kinds of interfaces
2. Synchronous outgoing interfaces
3. Considering all available interfaces
4. Testability
5. Deployment

Exploring further: The attributes of automated checks

1. What does all this mean for you?

Test environments

1. Usage
2. Infrastructure
3. Maintenance

Data

1. Static and dynamic data

Selection and priority

Reporting

CI / CD

Speed

1. Fully automated and reliable
2. Automate or deploy differently
3. Concurrent execution
4. Prioritising checks
5. Additional considerations
6. Part seven: Techniques
7. Related topics

Architecture

1. Value
2. Lasting value (at reasonable cost)
3. Cost

Designing the automation solution

1. Step one: Who will work with it?
2. Step two: Which SUT interfaces?
3. Step three: Integrated or not?
4. Step four: Proof of Concept
5. Step five: Tool selection
6. Separation of concerns
7. Tool and vendor independence
8. Test engine independence

Selecting the tools

1. The runner
2. Interface tools

Engineering

1. Page object

Test data

Introduction

1. A bit of terminology