Apple ARKit

Posted on Edited on In

ARKit is a software framework for iOS to create AR applications. It support features of device tracking, plane detection, and could allow apps to place virtual objects on top of real-world environment.

Compare with RealityKit

RealityKit is a newer framework which is built on the top of ARKit. RealityKit provides higher-level features like adding animations like physics to objects, and lighting effects.

World Tracking AR Session

World tracking APIs

The world tracking AR experience requires the underlying techniques called VIO (visual-inertial odometry), which takes in data from visual sensor and inertial sensor, and output the device’s position and orientation in world coordinate space. Specially,

  • Device’s pose is provided via ARSession transform property, which is the transformation matrix between camera coordinate space and world coordinate space
  • Camera coordinate space, relative to the phone: +x: right, +y: upwards, +z: backwards
  • World coordinate space, relative to environment: the origin is the initial position of device when session starts, and the orientation could be defined by ARWorldAlignment of ARConfiguration. Specially, it includes
    • +y: gravity upwards
    • +x: east, +y: gravity upwards, +z: south

Tracking State

ARSession is the basic unit to hold an AR scene. During an AR session, the tracking state of the device could have following values

The typical flows are

  • Start a session: nonAvailable -> limited (initialization) -> normal
  • Recover a session during app interrupted / relaunch: nonAvailable -> limited (relocalizing) -> normal

Relocalizing requires the user to go back to the previous physical positions when the AR session gets interrupted. Otherwise, the session remains in the relocalizing state.

Persistent AR experience

Using ARWorldMap could stores the information of a session, mainly the anchor data, so that all the content of the app could be recovered in the next launch, and still needs to return to the previous physical position.

This is different from the scenario where the app is interrupted, because the app didn’t get shutdown and the session data are still maintained.

Reference