This article is the ninth in a multi-part series on test speed and reliability, inspired by a webinar I gave on the same subject (you canwatch the webinar hither). You might also require to check out the previous episodes on,,,,,,, and.

We arrive finally to the last episode in this not-so-little series on test speed and stableness. This edition is not about specific speed and reliability techniques; instead, it & # x27; s about what to do when none of the previously-discussed suggestions have been helpful. There are no question stacks of early useful tips hiding in the wings, and they & # x27; ll get their appearance hither in due time. But still if there & # x27; s not an obvious solvent to a particular problem, it & # x27; s very useful to be capable to nail the problem so that you can seem for a reasonable workaround or notify the appropriate authorities (Appium, Apple, Google, etc ...).

In order to nail any kind of job with an Appium examination (whether it & # x27; s a straight-up error, or undesired awkwardness or imbalance), it & # x27; s necessary to understand something about the complexness of the technology stack that comes into drama during test executing. Take a look at this diagram of (most of) the pieces of Appium & # x27; s XCUITest stack:

You & # x27; ve got your test code, the Appium client code, the Appium host code, WebDriverAgent, XCUITest itself, and who knows how much proprietary Apple codification shoved in between XCUITest and the respective bits of iOS. A trouble on any one of these layers will eventually manifest itself on the level of exam behavior. It & # x27; s not impossible to figure out what & # x27; s going on, and there are typically lots of helpful clues lying about, but my point is that a slight work is required to get to a useful answer about your problem. So, here is what I advocate doing to name issues, in roughly this order:

Test Code Exceptions

The first investigation should ever be into your own exam code. Oftentimes an exception arises that has cipher to do with Appium whatsoever, and is a simple programming error in your exam code itself. Always read your test code exceptions in detail! Since your examination codification imports the Appium client in your language, this is also where you will get the first indicant of any other errors. Whenever possible, Appium puts useful information in the error messages it returns to the client, and these messages should end up in your IDE or console.

In the example of exceptions generate by the Appium client, the exception will often belong to a certain course(NoSuchElementException, for example), which will afford you a cue as to what has gone incorrect. In the case of a generic exception, aspect for the content itself to see if the Appium server has transmit anything about what could be wrong. Often, system configuration error will be mark by the Appium server and expressed in this manner, and a quick look at the message may even give you specific instructions as to how to fix the issue.

Appium Logs

The Appium waiter writes atonof messages to the console (or to a file if that & # x27; s how you & # x27; ve set it up). In the example of normal test execution, this information is mostly superfluous. It recite you all about what Appium is doing, and give you a very elaborated icon of the flow of HTTP requests/responses, commands Appium is running under the hood, and much more. Check out this clause on (by Appium maintainer Isaac Murchie) for a more detailed introduction to the logs. Here are some of the things I ordinarily appear at as a first quick pass of the logarithm:

For autonomous testing across multiple user personas, check out SUSATest — it explores your app like 10 different real users.

1. The session start loglines, to double-check that all the capabilities be sent in as I expected.
2. The very end of the log, to see the terminal things Appium did (especially in the case of an Appium wreck).
3. Any stacktraces Appium has written to the logs (since these will often have information directly link to an error).
4. Any log lines order & quot; warn & quot; or & quot; error & quot;.

If nothing pops out after following those steps, I read through the logs in more detail to try and jibe up my test measure with loglines, and then look for the country in the log matching the tough country in my trial.

Log Timestamps

In the causa of slowness issues, it & # x27; s essential to see how long different commands take. It & # x27; s possible to determine this on the guest side by adding time mensuration before and after item-by-item commands. Often, nonetheless, this doesn & # x27; t reveal anything useful. We already know that a certain command is slow -- -that & # x27; s why we & # x27; re having trouble! But using the Appium logs, we can dig deep and try to isolate the slowness to a particular Appium subroutine, which can be much more utile.

By default, the Appium log does not show timestamps, in order to keep the log lines short and uncluttered. But if you start the Appium server with-- log-timestamp, each line will be prepended with a timestamp. You can use these to good effect by insulate the subdivision of the log that pair the problematical area in your test, and part looking for large jumps in clip in the log. You might discover a single line or a small set of lines which are taking a long time. If it is surprising to find such a lag with those particular line, you can always hit out to the Appium issue tracker and share your findings with the maintainers to see if there & # x27; s a potential bug causing the slowness.

Device Logs

When it come to problems whose stem cause lies with your app, the mobile OS, or vendor-owned fabric (like XCUITest or UiAutomator2), Appium is typically unable to provide any insight. This doesn & # x27; t mean we & # x27; re out of pick, however. On both iOS and Android, apps can write to system-level logarithm, and often the system logarithm will too capture any exceptions thrown in your app (which might have led to a crash, say).

So, if the Appium logs don & # x27; t supply any help in your investigating, the next place to go is to these logs. On Android, you can runadb logcatto access this log stream, and on iOS you can merely tail the system log (for iOS simulators) or use a instrument likeidevicesyslogfor existent device. In fact, you can also set theshowIOSLogcapability totrue, and Appium will scrape data from these logarithm and interleave it with the veritable Appium logarithm, which often helps to put device problems into the context of the Appium session.

Use a Different Appium Client

If you don & # x27; t notice anything untoward in the Appium server log, and surmise that the subject might lie with the Appium guest you are using, you could incessantly try to rewrite your test playscript (or the problematic portion of it) using a different client (either in the same language or a different words). If after doing this the trouble goes away, you have pretty strong grounds that the trouble was with the Appium client.

Happily, the Appium squad is often capable to fix bugs with the Appium client very chop-chop (as long as the client is officially maintained by the project). Sometimes, though, a bug might exist in the underlying customer library which the Appium client is built on, and so the bug will need to be reported to the Selenium project. The Appium squad can assist in that process, too.

Use the Underlying Automation Engine Directly

Appium is construct on top of former mechanisation technologies, care XCUITest as we saw in the diagram above. In many cases, issue that appear to be related to Appium are actually issues with the underlie automation technology. Before deciding that the Appium codification is at fault for errors or slowness and creating a bug in the Appium number tracker, it can be utile to write up a small testcase using the underlying engine, expressing the same test steps as with your Appium hand. If you find the same kind of problem (subnormality, errors, etc ...), then it & # x27; s unlikely Appium will be able to do anything to directly fix the matter, and it should instead be reported to Apple or Google (and outstanding -- -now you have a staring minimal repro case for them!)

It & # x27; s still useful to ask around the Appium forums in such a case, because there are often workarounds to these kinds of problems. If the workaround is useful plenty, we can also build it into Appium as a first-class citizen, do life easier for all Appium user (at least until Apple or Google fixes the original job).

Ask for Help

If none of the strategies above have leave in clarity on the problem or a satisfactory resolution, it & # x27; s time to hit out for help. A full first pace is always searching the Appium issue tracker, StackOverflow, or the Internet more generally for any signaling of your trouble. Maybe someone else has already discussed a workaround for it.

If that & # x27; s not the event, it & # x27; s a good idea to make an issue at theAppium subject tracker. At this point, if you & # x27; ve follow all the steps above, prepare to be showered with love and congratulations by the Appium maintainers, who will appreciate the extremely detailed report you will be able to include in your number.

Hopefully the Appium maintainers will be able to render a resoluteness, specially if it & # x27; s determined that your issue reflects an Appium bug. If not, so I suppose you could always put on your crazy cyber-terrorist hat and start decompiling some XCUITest beginning code! Yeah -- -that & # x27; s no fun. Anyway, hopefully you & # x27; ll find resolution somewhere else along the way, using the method we & # x27; ve discourse here.

I hope you & # x27; ve enjoy this serial on test speed and reliability. If in the class of your Appium test writing you arrive across any former full trace, please send them along to me, and I & # x27; ll be happy to have them in a future article. Even though the serial is over, I & # x27; ll surely continue to cover tips to do with speed and reliability moving forward.

4 Parts

Connet Now