I screwed up one key accessibility behavior, and now I'm on a mission to do better

Accessibility labels, hints, and grouping are great, but don’t forget to think about Voice Control too.

Paul Hudson    June 23rd 2023    @twostraws

If you’ve followed any of my tutorials, you’ll know I make a point of teaching accessibility to everyone – I want to make sure that whether someone is learning SwiftUI or already has a lot of experience with it, they know that accessibility needs to be baked into their work.

As part of that, I teach five key concepts that – so I believed – helped make sure most apps provided a great accessibility experience out of the box:

1. Using native controls rather than making your own, including using VStack and HStack in SwiftUI to provide a clear structure for assistive tools to follow.
2. Adding short descriptions of views using accessibilityLabel(), keeping longer descriptions for accessibilityHint() for users who have that enabled.
3. Grouping views together using accessibilityElement() so VoiceOver treats them as atomic units.
4. Marking images as decorative when they shouldn’t be read by VoiceOver.
5. Using Dynamic Type everywhere, to make sure our interfaces scale smoothly no matter what someone’s settings are.

Those five do help provide a great experience for many people, but while at WWDC23 one of the teams at Apple very gently pointed out that I was missing a key component – and in doing so creating a pretty awful accessibility experience for many people.

So, here’s the sixth item I’m putting on the list of things I’ll be teaching everywhere: using accessibilityInputLabels() to provide custom activation commands for Voice Control.

This modifier accepts an array of localizable strings that can be used by the user to activate a control. You can provide as many as you want, and the system will listen for all of them – it gives the user a little leeway, and hopefully means they never need to use “Show Names” to figure out how to activate something.

You can see the problem, and also how accessibilityInputLabels() solves it, in this code:

NavigationStack {
    List {
        NavigationLink {
            Text("Details about today's workouts")
        } label: {
            VStack(alignment: .leading) {
                Text("Today's Workouts")
                    .font(.title2)

                Label("Running: 1km", systemImage: "figure.run")

                Label("Walking: 3km", systemImage: "figure.walk")
            }
        }
        .accessibilityInputLabels(["Today's Workouts", "Today"])
    }
    .navigationTitle("Overview")
}

That shows an example row in a list of workouts, which the user might tap to navigate to more information. I’ve used accessibilityInputLabels() to give it two input labels: “Today’s Workouts” and “Today”.

Without that modifier, someone using Voice Control would need to activate this link by saying something like “Tap Today’s Workouts Running 1 kilometer Walking 3 kilometers” – they’d need to read all of the view’s content just to navigate to the details, which is clearly not a great experience. But with the modifier in place they can just say “Tap Today’s Workouts” or “Tap Today”.

Other situations can be even trickier, such as views that change their contents frequently. For example, imagine a stock trading app that had a live pricing button such as this one:

Button("Buy $AAPL at $184.92") {
    // buy
}

The exact price will likely fluctuate every few seconds, which means the activation phrase will also fluctuate. Having that title there is important for sighted users and for VoiceOver users, but it’s difficult for Voice Control users. Instead, something like this be a big improvement:

Button("Buy $AAPL at $184.92") {
    // buy
}
.accessibilityInputLabels(["Buy Apple", "Buy AAPL"])

The system will automatically pick the first one to show when the user says “Show Names”, but you can provide as many alternatives as you need – just make sure you keep them short, because users need to read them out.

Anyway, until recently I wasn’t using accessibilityInputLabels() at all, and on learning how important it was I genuinely felt terrible – thinking of all the apps I’ve shipped that were tricky to use with Voice Control, I cringe even now.

If you found any of my apps difficult to use with Voice Control, I sincerely apologize.

So, I’m trying to change, which means four things:

1. Publishing this article. It’s okay to make mistakes – everyone makes mistakes! – but being part of the solution means acknowledging the mistake in the first place.
2. Adding a new entry in SwiftUI by Example: How to add custom activation commands for Voice Control so that folks can find the solution easily.
3. Modifying the 100 Days of SwiftUI so this extra modifier is included. This update will happen as part of the iOS 17 update coming in September.
4. Changing my own apps in the future so I take Voice Control support into account.

I firmly believe that all app developers should be taking accessibility needs into account in everything we build, but we can always do better – I can always do better, both in terms of what I build and what I teach. I’m grateful to the Apple team for giving me a nudge, and I hope together we can improve Voice Control support for everyone!

Sponsor Hacking with Swift and reach the world's largest Swift community!