This post get from our friend atGilt, who are usingAppiumto automate their mobile testing. Check out the original post on theGilt Tech blog! [prezi id= & quot; p68cqocmm5lx & quot;]

Just a few years ago, mobile purchases create barely a dent in Gilt ’ s revenues. Today, mobile represents more than 40 percent of our sales, and will soon reach 50 percent. With so many of our customer interacting with us through their peregrine devices, it ’ s imperative that we proffer them a stable and enjoyable shopping experience. One bad bug can drive away customers for full.

Given the increasing importance of mobile to our business, and therefore the need to expand the number of teams that can conduce to our mobile covering, the Gilt mobile team has been hard at work improving and streamlining our testing procedure. This station will account our automated testing efforts, the technologies we use, and what lies on the horizon—both for us, and for the automation tools we use.

Testing at Gilt: A Brief Overview

In the former days of Gilt Mobile, none of our examination was automated. This was workable at the time because only one team—the mobile team—worked on the applications. We postdate a fairly simple development cycle, as follows:

• Our crew of peregrine technologist would implement a series of new features, as determined by product direction.

• Known issues would be prioritized by severity, and fixed accordingly.

• We would internally unloose versions of the application for testing function on a veritable basis.

• An overseas team of quizzer performed high level feature testing, regression examination, and other test not covered by engineering during growth.

• The QA team would be creditworthy for the testing of new feature, stress-testing the app in an effort to find new subject and acting of a case of saneness tests to make certain that basic app functionality remained intact.

• Once QA gave the go-ahead, we ’ d sign the body-build appropriately and state to the iOS App Store.

As mobile has turn more critical to the business and more teams have started to lead to our nomadic covering, QA has become progressively important—particularly in iOS, where we concentrate most of our development efforts. With an increasing install base, more contributors, and more features, comes increased complexity. Ensuring that the app is issue-free when we subject to the App Store for approval has become more significant than ever. Understandably, the execution on mobile has render a lot of excitement within Gilt. This has led to increased emphasis on mobile development within Gilt tech in general. We ’ re starting to see increased interest from engineer on early teams, and having a loosely defined development process makes it difficult for fledgling to get up to speed and contribute to our efforts. How can we help them? With well defined process and a dash of automation!

For autonomous testing across multiple user personas, check out SUSATest — it explores your app like 10 different real users.

Our Development Process Today

We ’ re gradually migrating to more of a test-driven workflow. We still depend very heavily on manual QA, but this is now supplemented by a cortege of automated test. Our development process is increasingly starting to look like this:

• Developers are encouraged to write tests for their current features and fixes.

• Instead of hand-building test releases at random, we now have Jenkins performing nightly builds.

• Builds are followed by a run of functional tests.

• Generated test reports are delivered to all team appendage, and can afford detailed information on exactly where and how a failure occurred.

The intent is to free up QA from having to do repetitive and time-consuming sanity tests, which allows them to focus on testing new features and find issues before our users do. The automated exam framework we started with wasKIF: Keep It Functional. Maintained by Square, KIF is quite mature. Using KIF was something of a proof of concept for us—more of a first step toward getting our machine-driven testing situation under control. As such, we didn ’ t go through the utilization of pen an entire sanity test retinue, but instead produced a couple little tests of basic functionality. What we like about KIF: Tests run in the same process as the app, and so feature entree to notifications. This is pretty handy when testing asynchronous component of your app. What we don ’ t like: it ’ s heavy on private and undocumented approachability APIs. There may be some disagreement on how big a deal this is, but Apple ’ s under no obligation to keep these APIs consistent, and can do away with KIF dependencies without notifying anyone—which would make things pretty difficult to fix. Setting up KIF with Continuous Integration—while not impossible—could be easygoing. Lately we ’ ve be judge outAppium, a tool we started search after one of our engineer hear about it at this year ’ s Selenium conference in Boston. Appium is construct on top of UIAutomation: a model, provided by Apple as part of Instruments, that enables you to interact with apps programmatically. We ’ ve used UIAutomation quite a bit in exam prototyping and debugging. So, Appium doesn ’ t use any individual APIs or resort to any cloak-and-dagger hackery to get the job done. Great! But that ’ s only the get-go of how Appium charm our care. Appium is built on the idea that testing native apps shouldn ’ t require include SDKs or recompiling your app. You should be able to use your prefered tryout practices, frameworks, and instrument. Appium is able to achieve all this by implementing a large portion of the Selenium JSON wire protocol, and basically translating these calls into sets of native framework commands—UIAutomation and uiautomator, for iOS and Android respectively. It ’ s really this panorama alone that has us hooked. Our Web team has been down this path before, and has already build out a testing infrastructure revolve around Selenium, Scala, and ScalaTest. Using Appium has allowed us to take advantage of tumid chunks of our preexist work. No reinventing the wheel, and no learning the difficult way. This also provides us with a nice entry point for other Gilt engineers interested in act on mobile. While it didn ’ t take long for us to get up and running with Appium, I still can ’ t say that it suits the demand of everyone out there make apps. Smaller teams with no Selenium experience or existing infrastructure might sense a little more comfortable sticking with something like KIF orCalabash.

Can we do better? (Always)

Like everything else, Appium isn ’ t perfect. Areas where Appium could benefit from important improvement:

• For us, the Appium XPath engine is rather circumscribed, and can only evaluate unproblematic XPath expressions. It might be nice to see Appium use something like Cameron McCormack/Yaron Naveh ’ sXPath parsing package for node.

• Another, more minor gripe is that tag names apply on the web don ’ t correspond with their mobile equivalents in Appium. For representative, a text field on the web has the tag gens “ input. ” Appium calls these tags either “ textfield ” or “ UIATextfield, ” which play us to another issue…

• I say “ either, ” because doing something like driver.findElementsByTagName (“ textfield ”) .getTagName () regress “ UITextField. ” Nothing shocking hither, but perhaps the tag name we search for and the tag gens regress should be the like thing?

The good news is that development activity on Appium is really eminent, and its (notably friendly) community is rapidly addressing its shortcomings. You developers out there who are looking for projects can perpetually get affect in fixing some of this stuff. Some of us on Gilt ’ s mobile team have recently put some fix into Appium.

Final Thoughts

At Gilt, we ’ re trying to create a culture that promotes a proactive approach to testing. For now we ’ re focus on taking a load off of the QA team by automating UI test. Gradually we ’ ll move on to integration testing. Long-term, we ’ d like to borrow a TDD-centric workflow, with developers create tests from the outset, and taking responsibility for tryout maintenance. While fabric like OCUnit and UIAutomation are comparatively easily documented, it doesn ’ t seem like any heavy emphasis has be placed on testing as a part of the growing rhythm. The tools are provided, but not evangelise. Fortunately, this is changing. Xcode 5 will boast some terrific test and automation centric enhancements such as XCTest, and the Bots Continuous Integration system. KIF is revamping its API with KIF-Next to get in line withXcode 5and take advantage of its new features. And Selenium 3, which is in the spec stage, look set to become a tool for user-focused mechanisation of mobile and web apps. All-round, the future for automation and essay aboriginal nomadic apps is looking brighter.

Products

Global puppet

Set up and integrate

Resources

Company