How to test your user interface using Xcode

An introduction to testing with XCUITest

Paul Hudson March 29th 2019 @twostraws

User interface testing can be tricky to get right, so in this tutorial we’re going to take an app that has no UI tests and upgrade it so you can see exactly how the process work. The project we’ll be using is a simple sandbox with a variety of controls – you can download it here.

Give the example app a try now – it’s a simple sandbox app with a small amount of functionality:

If you enter text into the text box you’ll see the same text appear in the label below.
Dragging the slider will update the progress view; they should move in opposite directions.
Toggling the segmented control with change the title in the navigation bar.
Clicking any of the three colored buttons will present an alert.

This example gives us a variety of controls to work with, which should allow for some interesting UI tests.

If you didn’t see the on-screen keyboard when the text field was selected, you should go to the Hardware menu and uncheck Keyboard > Connect Hardware Keyboard. XCUITest does not work great with hardware keyboards, so you should use the on-screen one to be sure.

Note: This sandbox app is specifically about user interface testing. Where possible regular unit tests are preferable. UI testing is notoriously flaky, and ideally you should make most of your app independent of the user interface – if you can avoid using import UIKit that’s a good sign!

Right now the app doesn’t have anywhere to add UI tests, so the first step is to add a new target. Go to the File menu and choose New > Target. You’ll see a long list of possible targets, but I’d like you to scroll down and select iOS UI Testing Bundle then click Next. Xcode’s default options might be OK, but check the team option to make it’s configured for your team.

Finally, look inside the XCUITestSandboxUITests group and open XCUITestSandboxUITests.swift for editing.

Sponsor Hacking with Swift and reach the world's largest Swift community!

Setting an initial state

Like regular tests you’ll see setUp() and tearDown() methods that run before and after your tests to make sure they are in a consistent state. You’ll also see an empty test method to get us started, called testExample().

For now I want to focus on the setUp() method, which should contain code like this:

// UI tests must launch the application that they test. Doing this in setup will make sure it happens for each test method.
XCUIApplication().launch()

// In UI tests it’s important to set the initial state - such as interface orientation - required for your tests before they run. The setUp method is a good place to do this.

That second comment is important: you need to make sure the app is configured in a very specific way, otherwise you’re likely to have tests fail sporadically.

While you can put code after that comment to force your app to be in a specific state, most good tests I’ve seen use a much more robust approach: pass in a command-line parameter that the app can read and adapt to.

To pass in a testing flag, replace the call to XCUIApplication().launch() with this:

let app = XCUIApplication()
app.launchArguments = ["enable-testing"]
app.launch()

That will pass “enable-testing” to the app when it’s launched, which we can then make it detect and respond to. Our sandbox app doesn’t have any initial state to worry about, but if you needed to configure your app in a certain way then you would add something like this to your main app:

#if DEBUG
if CommandLine.arguments.contains("enable-testing") {
    configureTestingState()
}
#endif

What configureTestingState() does is down to you – you might load some preconfigured data, for example.

Warning: If you’re shipping a macOS app that enables a testing mode using command-line flags, make sure you wrap your command-line arguments check using #if DEBUG otherwise real users can enable your testing mode.

Recording tests

To start with, we’re going to use Xcode’s test recording system to write some code. Xcode watches what you do inside the simulator and translates your taps and swipes into Swift code – or at least tries to.

Click inside the testExample() method, and you should see a red recording button become active directly below the code area – next to where the breakpoint and debugging buttons are. Go ahead and press that button now, and you should see Xcode build and launch your app.

Here’s what I’d like you to do when the app launches:

Click inside the text field, enter “test”, then press enter.
Drag the slider all the way to the right.
Click “Omega” from the segmented control.
Click the Blue button then dismiss the alert.

As you take those actions, code will be written directly into the testExample() method, more or less matching what you’re doing on-screen. I say “more or less” because in practice this is the flakiest part of XCTest: it generates terrible code that often won’t compile.

Here’s the code it generated for me when I followed the above actions:

let app = app2
app.otherElements.containing(.navigationBar, identifier: "Alpha").children(matching: .other).element.children(matching: .other).element.children(matching: .other).element.children(matching: .other).element.children(matching: .textField).element.tap()

let tKey = app.keys["t"]
tKey.tap()
tKey.tap()

let tKey = app.keys["e"]
eKey.tap()
eKey.tap()

let sKey = app.keys["s"]
sKey.tap()
sKey.tap()
tKey.tap()
tKey.tap()

let app2 = app
app2.buttons["Return"].tap()
app.sliders["50%"].swipeRight()
app2.buttons["Omega"].tap()
app.buttons["Blue"].tap()
app.alerts["Blue"].buttons["OK"].tap()

Wherever you see some code with a blue background color, click the micro-sized arrow to the right of it to see other possible interpretations of that code.

That code has a number of problems:

The very first line, let app = app2, is meaningless because app2 isn’t defined.
The second line inserts .element.children(matching: .other) repeatedly for no real reason.
It claims I tapped each keyboard key twice.
The return key is accessed as a button rather than a key. While this might work, it means any other buttons with the text “Return” will confuse the system.
Our slider is referred to as sliders["50%"] – that’s its value, which means it will change.
The slider movement is referred to simply as swipeRight(), rather than a precise movement to a value.
The “Omega” segmented title is under “Buttons”, which again will work but might confuse things later on.

Not only is this code so bad that it won’t compile, but it’s so bad it actually crashed my Swift compiler – see SR-7517.

This code doesn’t actually test anything yet, but before we can write tests we need to make it work. So, please change it to this:

func testExample() {
    let app = XCUIApplication()
    app.textFields.element.tap()

    app.keys["t"].tap()
    app.keys["e"].tap()
    app.keys["s"].tap()
    app.keys["t"].tap()
    app.keyboards.buttons["Return"].tap()

    app.sliders["50%"].swipeRight()
    app.segmentedControls.buttons["Omega"].tap()
    app.buttons["Blue"].tap()
    app.alerts["Blue"].buttons["OK"].tap()
}

That does all the work we want, but it’s significantly less code, and for bonus points actually compiles.

Note: In case you were unfamiliar, XCTest expects all test cases to start with the word “test” and return nothing.

Finding elements to test

XCUITest is built around the concept of queries: you navigate through your UI using calls that get evaluated at runtime. You can navigate in a variety of ways, which is why both app.buttons["Omega"].tap() and app.segmentedControls.buttons["Omega"].tap() are valid – as long as XCUITest finds a button with the title “Omega” somewhere then it’s happy.

If you noticed, there are two ways of accessing a specific element:

app.textFields.element.tap()
app.buttons["Blue"].tap()

The first option is used when the query – textFields – only matches one item. If our app had only one button, we could have used app.buttons.element.tap() to tap it.

The second option is used when you need to select a specific item. Using app.buttons["Blue"] effectively means “the button that has the title ‘Blue’”, but this approach is problematic as can be seen here:

app.sliders["50%"].swipeRight()

Sliders don’t have titles, so Xcode identified it using its value. This was 50% to begin with, but that’s a really confusing way to refer to user interface elements, so iOS gives us a better solution called accessibility identifiers. All UIKit interface objects can be given these text strings to identify them uniquely for user interface testing.

To try this out, open Main.storyboard and select the slider. Select the identity inspector, then enter “Completion” for its accessibility identifier.

Note: The accessibility identifier is designed for internal use only, unlike the other two accessibility text fields that are read to the user when Voiceover is activated.

With that change we can refer to our slider regardless of what value it might have, like this:

app.sliders["Completion"].swipeRight()

Writing our own tests

So far we’ve used Xcode’s event recorder, cleaned up the code it generated, and added an accessibility identifier to avoid problems when accessing the slider. However, we still don’t have any tests, so let’s write those now.

Take a copy of the testExample() method and call it testLabelCopiesTextField(), like this:

func testLabelCopiesTextField() {
    let app = XCUIApplication()
    app.textFields.element.tap()

    app.keys["t"].tap()
    app.keys["e"].tap()
    app.keys["s"].tap()
    app.keys["t"].tap()
    app.keyboards.buttons["Return"].tap()

    app.sliders["Completion"].swipeRight()
    app.segmentedControls.buttons["Omega"].tap()
    app.buttons["Blue"].tap()
    app.alerts["Blue"].buttons["OK"].tap()
}

Now delete its last four lines, but make some space where we can write a test:

func testLabelCopiesTextField() {
    let app = XCUIApplication()
    app.textFields.element.tap()

    app.keys["t"].tap()
    app.keys["e"].tap()
    app.keys["s"].tap()
    app.keys["t"].tap()
    app.keyboards.buttons["Return"].tap()        

    // test goes here
}

As for the test itself, this uses the same XCTAssert() functions you should already be using with unit tests. For example, XCTAssertTrue() will consider your test to have passed if its condition is true.

In this case we want to check whether the label contains the text “test”, which is done like this:

XCTAssertTrue(app.staticTexts.element.label == "test")

Put that directly below the // test goes here comment. You might be wondering why we must use staticTexts rather than labels, but keep in mind XCUITest is cross-platform – AppKit on macOS blurs the lines between text fields and labels, so using something generic like “staticTexts” makes sense on both platforms.

You can try that test now if you want – if you press Cmd+B to build your code you should see an empty gray diamond to the left of func testLabelCopiesTextField(), and clicking that will run the test.

Next we’re going to test the slider and progress view, because moving the slider to the right should move the progress view to the left. Copy the app.sliders["Completion"].swipeRight() line into its own test called testSliderControlsProgress(), like this:

func testSliderControlsProgress() {
    app.sliders["Completion"].swipeRight()
}

You’ll need to copy the definition for app too, making this:

func testSliderControlsProgress() {
    let app = XCUIApplication()
    app.sliders["Completion"].swipeRight()
}

The swipeRight() method call was generated by Xcode, but it’s really not fit for purpose here because it doesn’t mention how far the test should swipe. To fix this we need to replace swipeRight() with a call to adjust(toNormalizedSliderPosition:). As the name suggests, this takes normalized slider positions, meaning that you refer to the leading edge as 0 and the trailing edge as 1 even if your slider counts from 0 to 100.

Here’s how that looks in code:

func testSliderControlsProgress() {
    let app = XCUIApplication()
    app.sliders["Completion"].adjust(toNormalizedSliderPosition: 1)
}

Now for the complicated part: writing a test. This is complicated by three things:

You won’t find a progressViews property inside app. Instead, you must use progressIndicators.
Once you find your progress view, its value is stored as Any? – anything at all, or perhaps nothing.
In this case, the actual value is stored as a string with “%” on the end. This means we should be checking for “0%”, because our slider was dragged all the way to the right.
We need to typecast the Any? to a string, and if that fails for some reason we’ll call XCTFail() then end the test.

Here’s how the test should look:

func testSliderControlsProgress() {
    let app = XCUIApplication()
    app.sliders["Completion"].adjust(toNormalizedSliderPosition: 1)

    guard let completion = app.progressIndicators.element.value as? String else {
        XCTFail()
        return
    }

    XCTAssertTrue(completion == "0%")
}

For our last test we’re going to make sure that tapping colors shows an alert. Start by copying app.buttons["Blue"].tap() and app.alerts["Blue"].buttons["OK"].tap() into their own testButtonsShowAlerts() test, then app the usual let app = XCUIApplication() line to the top. It should look like this:

func testButtonsShowAlerts() {
    let app = XCUIApplication()
    app.buttons["Blue"].tap()
    app.alerts["Blue"].buttons["OK"].tap()
}

Between the second and third lines of that method we need to write a new test: does an alert exist with the title “Blue”? XCUITest has a dedicated exists property for this purpose, so we can test for the alert in just one line of code:

func testButtonsShowAlerts() {
    let app = XCUIApplication()
    app.buttons["Blue"].tap()
    XCTAssertTrue(app.alerts["Blue"].exists)
    app.alerts["Blue"].buttons["OK"].tap()
}

At this point we’ve refactored all the code from testExample() into individual tests, so you can delete testExample() entirely.

That’s all our code tested, so we can now run all the tests at the same time. To make that happen, scroll up to the top of XCUITestSandboxUITests and click to the left of class XCUITestSandboxUITests – it might be a green checkmark or a red cross depending on what state your tests are in.

As Xcode runs all the tests you’ll see it start and stop the app multiple times, ensuring that your code runs from a clean state each time. If everything has gone to plan, once the test runs finish you should see a green checkmark next to your class – everything passed. Good job!

Where next?

If you'd like to learn more about Xcode UI testing, you should read my Xcode UI Testing Cheat Sheet – it's full of solutions to common real-world problems.

Sponsor Hacking with Swift and reach the world's largest Swift community!