3 simple ways to start automated testing plans for mobile

It is my pleasure once again to present next week at the Software Test Professional Conference fall 2013. I’ll be hosting a discussion on mobile testing in an agile environment, where I plan to bring up something that’s been on my mind a lot lately: Barriers to entry for mobile test automation.

I was recently challenged by both my client and internal project leadership to implement test automation on my current Agile mobile project. My first task was to request, and wait for, our sole iOS automation subject matter expert (SME) to grant us some of his valuable time. Once it became clear I was not going to get this scarce resource, I had the choice to either make automation happen using my existing project resources or inform my stakeholders that I was unable to fulfill their request. Thankfully I was able to meet this request with the assistance of two other QA engineers by using a groundbreaking process created by Dr. Leo Marvin: Baby steps.

In my experience, people often think of software test automation as an all-or-nothing endeavor—as though they have to ignore other testing duties to focus full-time on an automation framework and infrastructure. At my first mobile job, there were two well-meaning SDETs who proactively took it upon themselves to come up with a system that QA engineers could use to get up to speed in mobile automation. After weeks of designing and writing, they created a tutorial for this purpose. However, the unintended result was a tutorial with a huge barrier to entry.

While I can see the potential benefit for a full-featured automation tutorial, I’m a simple man, and I prefer baby steps. There is low-hanging automation fruit to be picked on every project. Below are examples of three low-hanging automation “fruits” that we picked and the process we used to implement them. The exact solutions might not translate directly to your project, but the baby steps likely will.

First thing to do was to get our project connected to an instance of Jenkins. This little man will sit around and wait for a check-in and immediately build the project, giving us instant feedback on the build. Now we have a continuous integration system alerting us to build breaks—baby steps! Next step is to get Jenkins to step his game up a bit and provide a build verification test upon check-in.

We initially wanted to use Calabash, which is a Ruby-based solution for automation that is very user-friendly. However, we were unable to use Calabash due to technical challenges specific to our project. Therefore, we decided to use Apple’s UI Automation framework, part of the Instruments suite of test tools included with Xcode.

This solution can have a higher learning curve than Ruby-based solutions, but with that challenge comes more functionality. Using UI Automation along with a little JavaScript knowledge that we learned online, we were able to quickly implement a basic BVT script that simply launched the application, logged in and out, and closed the application. Now we have a daily automated BVT alerting us to broken backend services. Again, baby steps! Next step, which we are currently working on, is to extend the basic BVT to add more functionality and additional logging.

Where the first two low-hanging fruits build on each other, this third one is a bit different. It has been my experience that Agile teams like to talk about unit testing, but no one really likes to do it. I’ve heard many development teams, at many sprint retrospective meetings, challenge themselves and each other to write more unit tests. Oftentimes developers will write unit tests, but no one continues to execute them after the developer has moved on to coding another feature.

I was surprised when I asked a developer to teach me how to execute the unit tests in Xcode that it took about 5 seconds for all 43 tests to complete. In fact, some of the tests run so fast that Xcode does not know they were executed and reports them as “Not Complete.” It made sense to me that if I’m already running BVTs everyday, I could easily execute and report on the unit tests as well. Once again, baby steps!

Next step here is for QA to write unit tests for new functionality. I’m pretty sure my development team will have a parade in our honor once we are able to take unit test creation off their to-do list or at least assist in the effort.

Again, these exact solutions might not be viable or even possible on your particular project. Regardless, here is your low-hanging fruit recipe:

1. Baby step your way toward an immature automation system;
2. Wow your Project Leadership with how much you can do with little resources;
3. Watch them give you more resources. Like me, you may suddenly find yourself as the “unintentional automation specialist.”

There are more of these low-hanging fruits in the future.  Over the short term, we’re working to monitor and investigate code coverage and minor bugs in both BVTs and unit tests. In the medium term, we want to maintain the BVT script by keeping up with functionality as it is added and create a process for doing so. And over the long term, we’re working toward making QA responsible for writing unit tests.

We’ll talk about this and more next at my Lean Coffee session at the Software Test Professional Conference fall 2013. Not able to join us there? Let us know what you think in the comments!

Todd Schultz is Quality Assurance competency lead at Deloitte Digital’s Seattle Studio.