WWDC22 SALE: Save 50% on all my Swift books and bundles! >>

Lumina is the instant Core ML video framework you're looking for

Want Core ML? Want video? Lumina puts the two together.

Paul Hudson       @twostraws

The Core ML and Vision frameworks are two of the most impressively powerful additions in iOS 11, but they are also two of the most complex – there's a big learning curve to get even basic applications working, and that can be off-putting for newcomers. While one option is to buy a good book (gratuitous plug), another is just to dive in with some code, and that's where David Okun's new Lumina framework comes in.

Lumina is like hitting hyperspace for your ML idea: it gets rid of all the AVFoundation heavy lifting that's required to get off the ground, replaces tricky image formats like CVImageBuffer with standard types like UIImage, and provides recognition for common data types such as QR codes, barcodes, and faces. But where Lumina goes from "neat" to "awesome" is that it comes with Core ML built in for devices running iOS 11 or later: provide it with any Core ML-compatible model and have it stream predictions directly to your delegate as they come in from the camera.

All the magic is done using a single LuminaViewController that you configure and present. Should it stream frames or deliver them on completion? Which camera do you want to use? Do you need maximum quality or can you sacrifice quality for speed? Should the user be allowed to zoom, and by how much? And of course the important one: which Core ML model should it use for predicting the contents of the camera feed?

You can do all this yourself – of course you can – but why would you when Lumina does all the hard work for you? By removing most if not all of the barriers from entry into the world of Core ML and Vision apps, I'm hopeful projects like Lumina will open this fascinating new area of development to more people.

Link: Lumina.

Hacking with Swift is sponsored by RevenueCat

SPONSORED You know StoreKit, but you don’t want to do StoreKit. RevenueCat makes it easy to deploy, manage, and analyze in-app subscriptions on iOS and Android so you can focus on building your app.

Explore the docs

Sponsor Hacking with Swift and reach the world's largest Swift community!

Buy Pro Swift Buy Swift Design Patterns Buy Testing Swift Buy Hacking with iOS Buy Swift Coding Challenges Buy Swift on Sundays Volume One Buy Server-Side Swift (Vapor Edition) Buy Advanced iOS Volume One Buy Advanced iOS Volume Two Buy Advanced iOS Volume Three Buy Hacking with watchOS Buy Hacking with tvOS Buy Hacking with macOS Buy Dive Into SpriteKit Buy Swift in Sixty Seconds Buy Objective-C for Swift Developers Buy Server-Side Swift (Kitura Edition) Buy Beyond Code

About the author

Paul Hudson is the creator of Hacking with Swift, the most comprehensive series of Swift books in the world. He's also the editor of Swift Developer News, the maintainer of the Swift Knowledge Base, and a speaker at Swift events around the world. If you're curious you can learn more here.

Was this page useful? Let us know!

Unknown user

You are not logged in

Log in or create account

Link copied to your pasteboard.