Scan barcodes with ML Kit on iOS

You can use ML Kit to recognize and decode barcodes.

Try it out

• Play around with the sample app to see an example usage of this API.

Before you begin

1. Include the following ML Kit pods in your Podfile:

   pod 'GoogleMLKit/BarcodeScanning', '15.5.0'
   

2. After you install or update your project's Pods, open your Xcode project using its .xcworkspace. ML Kit is supported in Xcode version 12.4 or greater.

Input image guidelines

• For ML Kit to accurately read barcodes, input images must contain barcodes that are represented by sufficient pixel data.

  The specific pixel data requirements are dependent on both the type of barcode and the amount of data that's encoded in it, since many barcodes support a variable size payload. In general, the smallest meaningful unit of the barcode should be at least 2 pixels wide, and for 2-dimensional codes, 2 pixels tall.

  For example, EAN-13 barcodes are made up of bars and spaces that are 1, 2, 3, or 4 units wide, so an EAN-13 barcode image ideally has bars and spaces that are at least 2, 4, 6, and 8 pixels wide. Because an EAN-13 barcode is 95 units wide in total, the barcode should be at least 190 pixels wide.

  Denser formats, such as PDF417, need greater pixel dimensions for ML Kit to reliably read them. For example, a PDF417 code can have up to 34 17-unit wide "words" in a single row, which would ideally be at least 1156 pixels wide.

• Poor image focus can impact scanning accuracy. If your app isn't getting acceptable results, ask the user to recapture the image.

• For typical applications, it's recommended to provide a higher resolution image, such as 1280x720 or 1920x1080, which makes barcodes scannable from a larger distance away from the camera.

  However, in applications where latency is critical, you can improve performance by capturing images at a lower resolution, but requiring that the barcode make up the majority of the input image. Also see Tips to improve real-time performance.

1. Configure the barcode scanner

For example, to scan only Aztec code and QR codes, build a BarcodeScannerOptions object as in the following example:

let format = .all
let barcodeOptions = BarcodeScannerOptions(formats: format)
  

MLKBarcodeScannerOptions *options =
  [[MLKBarcodeScannerOptions alloc]
   initWithFormats: MLKBarcodeFormatQRCode | MLKBarcodeFormatAztec];

2. Prepare the input image

Create a VisionImage object using a UIImage or a CMSampleBuffer.

If you use a UIImage, follow these steps:

• Create a VisionImage object with the UIImage. Make sure to specify the correct .orientation.

  Swift

  let image = VisionImage(image: UIImage)
  visionImage.orientation = image.imageOrientation

  Objective-C

  MLKVisionImage *visionImage = [[MLKVisionImage alloc] initWithImage:image];
  visionImage.orientation = image.imageOrientation;

If you use a CMSampleBuffer, follow these steps:

• Specify the orientation of the image data contained in the CMSampleBuffer.

  To get the image orientation:

  Swift

  func imageOrientation(
    deviceOrientation: UIDeviceOrientation,
    cameraPosition: AVCaptureDevice.Position
  ) -> UIImage.Orientation {
    switch deviceOrientation {
    case .portrait:
      return cameraPosition == .front ? .leftMirrored : .right
    case .landscapeLeft:
      return cameraPosition == .front ? .downMirrored : .up
    case .portraitUpsideDown:
      return cameraPosition == .front ? .rightMirrored : .left
    case .landscapeRight:
      return cameraPosition == .front ? .upMirrored : .down
    case .faceDown, .faceUp, .unknown:
      return .up
    }
  }
        

  Objective-C

  - (UIImageOrientation)
    imageOrientationFromDeviceOrientation:(UIDeviceOrientation)deviceOrientation
                           cameraPosition:(AVCaptureDevicePosition)cameraPosition {
    switch (deviceOrientation) {
      case UIDeviceOrientationPortrait:
        return cameraPosition == AVCaptureDevicePositionFront ? UIImageOrientationLeftMirrored
                                                              : UIImageOrientationRight;
  
      case UIDeviceOrientationLandscapeLeft:
        return cameraPosition == AVCaptureDevicePositionFront ? UIImageOrientationDownMirrored
                                                              : UIImageOrientationUp;
      case UIDeviceOrientationPortraitUpsideDown:
        return cameraPosition == AVCaptureDevicePositionFront ? UIImageOrientationRightMirrored
                                                              : UIImageOrientationLeft;
      case UIDeviceOrientationLandscapeRight:
        return cameraPosition == AVCaptureDevicePositionFront ? UIImageOrientationUpMirrored
                                                              : UIImageOrientationDown;
      case UIDeviceOrientationUnknown:
      case UIDeviceOrientationFaceUp:
      case UIDeviceOrientationFaceDown:
        return UIImageOrientationUp;
    }
  }
        

• Create a VisionImage object using the CMSampleBuffer object and orientation:

  Swift

  let image = VisionImage(buffer: sampleBuffer)
  image.orientation = imageOrientation(
    deviceOrientation: UIDevice.current.orientation,
    cameraPosition: cameraPosition)

  Objective-C

   MLKVisionImage *image = [[MLKVisionImage alloc] initWithBuffer:sampleBuffer];
   image.orientation =
     [self imageOrientationFromDeviceOrientation:UIDevice.currentDevice.orientation
                                  cameraPosition:cameraPosition];

3. Get an instance of BarcodeScanner

let barcodeScanner = BarcodeScanner.barcodeScanner()
// Or, to change the default settings:
// let barcodeScanner = BarcodeScanner.barcodeScanner(options: barcodeOptions)

MLKBarcodeScanner *barcodeScanner = [MLKBarcodeScanner barcodeScanner];
// Or, to change the default settings:
// MLKBarcodeScanner *barcodeScanner =
//     [MLKBarcodeScanner barcodeScannerWithOptions:options];

4. Process the image

barcodeScanner.process(visionImage) { features, error in
  guard error == nil, let features = features, !features.isEmpty else {
    // Error handling
    return
  }
  // Recognized barcodes
}

[barcodeScanner processImage:image
                  completion:^(NSArray<MLKBarcode *> *_Nullable barcodes,
                               NSError *_Nullable error) {
  if (error != nil) {
    // Error handling
    return;
  }
  if (barcodes.count > 0) {
    // Recognized barcodes
  }
}];

5. Get information from barcodes

For example:

for barcode in barcodes {
  let corners = barcode.cornerPoints

  let displayValue = barcode.displayValue
  let rawValue = barcode.rawValue

  let valueType = barcode.valueType
  switch valueType {
  case .wiFi:
    let ssid = barcode.wifi?.ssid
    let password = barcode.wifi?.password
    let encryptionType = barcode.wifi?.type
  case .URL:
    let title = barcode.url!.title
    let url = barcode.url!.url
  default:
    // See API reference for all supported value types
  }
}

for (MLKBarcode *barcode in barcodes) {
   NSArray *corners = barcode.cornerPoints;

   NSString *displayValue = barcode.displayValue;
   NSString *rawValue = barcode.rawValue;

   MLKBarcodeValueType valueType = barcode.valueType;
   switch (valueType) {
     case MLKBarcodeValueTypeWiFi:
       ssid = barcode.wifi.ssid;
       password = barcode.wifi.password;
       encryptionType = barcode.wifi.type;
       break;
     case MLKBarcodeValueTypeURL:
       url = barcode.URL.url;
       title = barcode.URL.title;
       break;
     // ...
     default:
       break;
   }
 }

Tips to improve real-time performance

If you want to scan barcodes in a real-time application, follow these guidelines to achieve the best framerates:

• Don't capture input at the camera’s native resolution. On some devices, capturing input at the native resolution produces extremely large (10+ megapixels) images, which results in very poor latency with no benefit to accuracy. Instead, only request the size from the camera that's required for barcode scanning, which is usually no more than 2 megapixels.

  The named capture session presets—AVCaptureSessionPresetDefault, AVCaptureSessionPresetLow, AVCaptureSessionPresetMedium, and so on)—are not recommended, however, as they can map to unsuitable resolutions on some devices. Instead, use the specific presets such as AVCaptureSessionPreset1280x720.

  If scanning speed is important, you can further lower the image capture resolution. However, bear in mind the minimum barcode size requirements outlined above.

  If you are trying to recognize barcodes from a sequence of streaming video frames, the recognizer might produce different results from frame to frame. You should wait until you get a consecutive series of the same value to be confident you are returning a good result.

  The Checksum digit is not supported for ITF and CODE-39.

• For processing video frames, use the results(in:) synchronous API of the detector. Call this method from the AVCaptureVideoDataOutputSampleBufferDelegate's captureOutput(_, didOutput:from:) function to synchronously get results from the given video frame. Keep AVCaptureVideoDataOutput's alwaysDiscardsLateVideoFrames as true to throttle calls to the detector. If a new video frame becomes available while the detector is running, it will be dropped.
• If you use the output of the detector to overlay graphics on the input image, first get the result from ML Kit, then render the image and overlay in a single step. By doing so, you render to the display surface only once for each processed input frame. See the updatePreviewOverlayViewWithLastFrame in the ML Kit quickstart sample for an example.