TEAM LICENSES: Save money and learn new skills through a Hacking with Swift+ team license >>

Video over video - CGContext error on draw(CGImage, CGRect)

Forums > Swift

@miff  

Hi, I have follow this instructions to achieve video (small) over video (big).

Getting assets, create composition... like this

...
let videoComposition = AVMutableVideoComposition()
videoComposition.customVideoCompositorClass = CustomComposition.self
...

and the main problem is when need to CGContext to draw cgimage:

class CustomComposition: NSObject, AVVideoCompositing {

    var requiredPixelBufferAttributesForRenderContext: [String : Any] = [
        kCVPixelBufferPixelFormatTypeKey as String : NSNumber(value: kCVPixelFormatType_32BGRA),
        kCVPixelBufferOpenGLESCompatibilityKey as String : NSNumber(value: true),
        kCVPixelBufferOpenGLCompatibilityKey as String : NSNumber(value: true)
    ]
    var sourcePixelBufferAttributes: [String : Any]? = [
        kCVPixelBufferPixelFormatTypeKey as String : NSNumber(value: kCVPixelFormatType_32BGRA),
        kCVPixelBufferOpenGLESCompatibilityKey as String : NSNumber(value: true),
        kCVPixelBufferOpenGLCompatibilityKey as String : NSNumber(value: true)
    ]

    override init() {
        super.init()
    }

    func startRequest(_ request: AVAsynchronousVideoCompositionRequest) {

        guard let destination: CVPixelBuffer = request.renderContext.newPixelBuffer() else { return request.finish(with: NSError(domain: "", code: 3, userInfo: nil)) }

        if request.sourceTrackIDs.count == 2 {
            guard let front = request.sourceFrame(byTrackID: 2) else { return request.finish(with: NSError(domain: "", code: 1, userInfo: nil)) }
            guard let back = request.sourceFrame(byTrackID: 1) else { return request.finish(with: NSError(domain: "", code: 2, userInfo: nil)) }

            CVPixelBufferLockBaseAddress(front, .readOnly)
            CVPixelBufferLockBaseAddress(back, .readOnly)
            CVPixelBufferLockBaseAddress(destination, [])

            renderFrontBuffer(front, back: back, to: destination)

            CVPixelBufferUnlockBaseAddress(destination, [])
            CVPixelBufferUnlockBaseAddress(back, .readOnly)
            CVPixelBufferUnlockBaseAddress(front, .readOnly)
        }

        request.finish(withComposedVideoFrame: destination)

        CVBufferRemoveAllAttachments(destination)
    }

    func renderFrontBuffer(_ front: CVPixelBuffer, back: CVPixelBuffer, to destination: CVPixelBuffer) {

        var gc: CGContext? = nil

        guard let frontImage: CGImage = createSourceImage(from: front) else { return }
        guard let backImage: CGImage = createSourceImage(from: back) else { return }

        let width: Int = CVPixelBufferGetWidth(destination) //Int(Config.renderSize.width)
        let height: Int = CVPixelBufferGetHeight(destination)

        let frame = CGRect(x: 0, y: 0, width: CGFloat(width), height: CGFloat(height))

        let colorSpace = backImage.colorSpace!

        gc = CGContext(data: CVPixelBufferGetBaseAddress(destination), width: width, height: height, bitsPerComponent: 8, bytesPerRow: CVPixelBufferGetBytesPerRow(destination), space: colorSpace, bitmapInfo: CGImageAlphaInfo.premultipliedLast.rawValue)

        // MARK: - Place Back (big)
        //gc?.draw(backImage, in: frame) // <- Problem: EXC_BAD_ACCESS

        gc?.beginPath()
        gc?.addRect(CGRect.init(x: 24, y: 64, width: 67, height: 120))
        gc?.setFillColor(UIColor.yellow.cgColor)
        gc?.fillPath()

        // MARK: - Place Front (small)
        //gc?.draw(frontImage, in: frame) // <- Problem: EXC_BAD_ACCESS
    }

    func createSourceImage(from buffer: CVPixelBuffer) -> CGImage? {

        let width: Int = CVPixelBufferGetWidth(buffer)
        let height: Int = CVPixelBufferGetHeight(buffer)
        let stride: Int = CVPixelBufferGetBytesPerRow(buffer)

        var data = CVPixelBufferGetBaseAddress(buffer)

        let rgb = CGColorSpaceCreateDeviceRGB()

        let provider = CGDataProvider(dataInfo: nil, data: &data, size: height * stride) { (_, _, _) in }

        var image: CGImage? = nil

        let last = CGBitmapInfo(rawValue: CGBitmapInfo.byteOrder32Big.rawValue | CGImageAlphaInfo.premultipliedLast.rawValue)
        if let provider = provider {
            image = CGImage(width: width, height: height, bitsPerComponent: 8, bitsPerPixel: 32, bytesPerRow: stride, space: rgb, bitmapInfo: last, provider: provider, decode: nil, shouldInterpolate: false, intent: .defaultIntent)
        }

        return image
    }

    func renderContextChanged(_ newRenderContext: AVVideoCompositionRenderContext) { }

}

When the .draw is not called, render something like FaceTimecall and have audio of both videos (that's ok), otherwise when is called get immediately EXC_BAD_ACCESS error.

I already try to debug with Zombie Object, but nothing.

I also try to save png of cgimage and the error is the same

...
let uimg = UIImage(cgImage: frontImage)
let data = uimg.pngData() <- EXC_BAD_ACCESS
...

My guess is that I missing something in createSourceImage with CGDataProvider, or something with requiredPixelBufferAttributesForRenderContext or sourcePixelBufferAttributes arrays.

Or if someone have other idea how to achieve this (video over video), thanks.

3      

Hi, do you want to achieve this effect for playback or do you need the final video exported? If it is the first then I would just overlay AVPlayer over another

3      

@miff  

@nemecek-filip no, just for export

3      

Hacking with Swift is sponsored by Superwall.

SPONSORED Superwall lets you build & test paywalls without shipping updates. Run experiments, offer sales, segment users, update locked features and more at the click of button. Best part? It's FREE for up to 250 conversions / mo and the Superwall team builds out 100% custom paywalls – free of charge.

Learn More

Sponsor Hacking with Swift and reach the world's largest Swift community!

Archived topic

This topic has been closed due to inactivity, so you can't reply. Please create a new topic if you need to.

All interactions here are governed by our code of conduct.

 
Unknown user

You are not logged in

Log in or create account
 

Link copied to your pasteboard.