Why in-house Test Automation Framework Development Fails?

Automated Testing has become an essential activity within any modern software development practice. In agile methodology, software is released on a regular basis to stay ahead of competitors. To release software frequently and reliably, one must rely on an automated build system, which minimizes human error and speeds up the deployment process.

One step within the automated build system is to execute automated tests. The purpose of the automated tests is to gain confidence that what we have built is functionally correct. In an ideal build process, there are different types of tests which target the application at different levels.

At the lowest level, unit tests target the individual classes and functions within each class. Their purpose is to catch developer errors. Unit tests are often the responsibility of the developers who wrote the software. The next layer of tests, integration tests, target the interaction of various components, namely APIs and third-party applications. These tests aim to find integration issues. Integration tests are also mostly written by developers. At the last layer, we have end-to-end tests which target the system as a whole and mimic user scenarios to validate various business cases. End-to-end tests are often executed via a user interface and for this reason, are referred to as UI tests. UI tests are generally the responsibility of testers in the team.

This article focuses on the problems associated with UI test automation and the in-house test automation framework development.

QAs do not have development background

As mentioned, end-to-end testing is often seen as the responsibility of testers. Coupled with the need to release faster, then the obvious route is to automate these tests. Historically, testers were seen as not technical enough to take on development tasks. Testers were not required to code or to build a test framework.

Agile development methodology demands testers to be more technical and take on more technical tasks. At least they should be able to code and use a programming language to script automated tests. However, this puts huge pressure on testers. Due to their lack of development skills and background, most testers struggle to keep up with the pressure to automate tests quickly and efficiently.

This has a negative effect on development because the effort is now shifted from testing to struggling with code. Development is hard. You can learn a programming language syntax in a week but will take years to master. Developing a test automation framework from scratch is not an easy task by any stretch. Most testers are certainly not technically capable to take on the challenge.

Developers are not interested in UI test automation

Due to the technical challenges testers face, sometimes the responsibility of developing a test automation framework along with UI automated testing is put on the developers. Unfortunately, in some organizations, there is still a large divide between developers and testers. Developers do not enjoy testing and testers do not enjoy developing. Also, there is enough pressure on developers to develop the software and they do not have enough time to work on the tasks which are seen as tester’s responsibility. Further, once developers learn how a specific open source product or framework works, they lose interest in mastering it. Developers would also well know that once an in-house framework is developed by them, it will be theirs forever to maintain and enhance in addition to the development tasks they were already responsible for.

Most of the time, management misunderstands the cost and effort involved in creating an in-house test automation framework. This is clearly not a simple task that most QAs can take and requires a separate development team to build and maintain.

Talent acquisition is hard and more expensive

This new hybrid role of a technical QA (testers who are good at testing as well as coding also called SDET which is an acronym for Software Development Engineers in Test) cannot easily be fulfilled. In some ways, it is like looking for a unicorn. What really differentiates testers from developers is the mindset. Good testers have the right mindset to test a product and find creative ways to find bugs. If we ask them to focus on automated testing, this takes away their time to explore the system effectively. Instead, they will be trying to automate tests with their limited coding skills. It is extremely hard to have the two mindsets in one role, and for this reason, finding such people is extremely challenging. And when we do find one of these unicorns, they tend to be very expensive and when they leave the project, the entire in-house framework development goes into jeopardy.

Summary

Building an in-house test automation framework proves to be very challenging and expensive and often takes many months to develop and maintain forever.

Platforms such as Subject7, alleviates these challenges. It makes testers focus on their main role of testing the product, while being able to turn their manual tests into robust end-to-end automated tests without the need to code and create as powerful of automated tests as what these unicorns can create with coding.