Mobile test automation basics. Uncovered

Test automation has become a trend in software development & testing industry. The approach has lots of adherents as well as opponents. Still, when a testing range cannot be covered manually, test automation is an excellent way to meet the challenge.

Today automated tests are successfully used in multiple areas including mobile testing. Since the time when mobile market was literally seized by several major platforms – Android, iOS and Windows - mobile test automation has gained epic coverage.

The influence of those mobile platforms affected range, specifics and quality of web and native test automation tools.

Automated tests can be performed by free tools like Appium, Calabash, or paid ones like HP UFT and Ranorex. Moreover, test engineers can choose tools suitable for testing several platforms or single-platform testing tools.

But it`s not only the tools I want to tell about, I am going to pay attention to some Dos and Don`ts of mobile test automation. And in this overview I will rely on my experience with the Appium tool.

First, want to say that running automated tests via some other tools doesn`t have much difference from Appium or Calabash. The whole process is kind of a cliché.

I should also say that scripts for automation tests can be written on different programming languages: choose any from VBScript and to Java and C#.

Emulators and real devices

Mobile test automation can be performed on real devices or emulators. Which is better for mobile automated testing: emulators or real devices? Usually the choice between those two options is defined by marketing or economic terms. Obviously testing on emulators is less expensive as there are lots of free or inexpensive emulators. They allow testers reproducing behavior of multiple devices without any need to buy real ones.

Still, emulators will always be emulators. In some cases it is hard to predict how application will behave on this or that device though some say the chance of error is minor.

Despite the mentioned fears emulators imitate devices to almost 100%. Automation tools fully reproduce user activities within an app, and with API you can emulate any conditions of the real devices. These capacities give emulators advantages over the real devices.

But when emulator is not a good idea? You shouldn`t use emulators when you need to reproduce device characteristics accurately or reconstruct display resolution very precisely. There can also be some test requirements that would difficult to fulfill on emulators. It is always better to use real devices for that purpose.

Application requirements

In fact, there are no specific requirements for an application to run automated testing. The processes of builds delivery for automation and manual testing are identical.

Test speed

Automated tests are very efficient and do not require specific hardware capacity. Depending on a test or an app the time of test execution can last for seconds or for minutes, unlike automated testing of large systems that sometimes can last for hours.

Distributed testing

Even though testing speed and performance of automated tests is high, when the number of tests increases testers have to shrink the time for tests execution. To avoid that, I would recommend applying the distributed testing approach. Distributed testing is used to run tests or parts of the tests on different workstations.

Distributed testing is applicable to several emulators per host.

There is a very interesting thing about distributed testing. If it seems quite expensive just use cloud services to cut costs. Most popular server for that purpose is SauceLabs, it works only with emulators. Test engineers may also like AWS Device Farm by Amazon, which allows using real devices. Some choose other available services, but they offer quite similar opportunities.

Systems of Continuous Integration (CI systems) are also a good option for execution of distributed testing. If you choose CI systems the scheme of your test stand should look like on the image below.  The scheme shows ‘CI Jenkins system and Appium tool’ option for test stand.

As you may see on the scheme every host agent has a pre-installed Appium server.

If you use Appium for testing iOS applications, you have to install it on every MacOS server. Pay attention to an important limitation on testing iOS applications (on real devices or emulators) you need to install Appium on a server host by MacOS.

Automated testing for Windows Mobile applications

There is a special thing about testing Windows applications – most mobile test automation tools are effective only with iOS and Android applications. For that reason the range of tools for testing Windows application is quite limited.

I would recommend using Ranorex or MS CodedUI by VisualStudio Enterprise tools for automated testing of Windows applications.

Still, these instruments allow running tests for applications developed for Windows 8 and upper versions.

Advantages of test automation

With this long “talk” about mobile test automation I almost skipped the actual advantages of test automation. Some of them would seem quite obvious but they make testing routine quicker and more effective.

So, here what I am talking about:

• Desired result. As long as tests are automated, human factor is excluded. You get clear and unbiased results.
• Costs cutting and time decrease. Automated tests can run over night, which allows obtaining information of every build on time along the whole development cycle.  Above that running automated tests doesn’t influence the whole test coverage.
•  Reports. Client receives reports on held automated testing of every build.

So, even being a bit more expensive than manual testing, test automation is definitely beneficial. When you need to get rid of identical routine tests and get unbiased result on your application quality, test automation is a real silver bullet.