Microsoft Azure Spatial Anchors Service

Posted on Edited on In

Anchor: a kind of data structure in AR field, which represents a precise point of interest in 3D space that can be shared across users, devices, and time

Microsoft Azure Spatial Anchors is a service that can store the local world anchors to the cloud and share the anchors among different platforms, see the table blow. ASA (Azure Spatial Anchors) enables building apps that can map, persist, and share 3D content from room-scale to real world-scale.

Device System SDK
Android Phone Android ARCore
iPhoen, iPad iOS ARKit
HoloLens Windows Windows SDK

In this article, I will go through the working pipeline of the ASA service in Android system with Unity development environment.

Test Environment

  • Google Pixel 2 XL
  • Unity 2019.2.9f1
  • Azure-Spatial-Anchors-Samples 2.0.0

Note. ASA needs to use some APIs from ARFoundation of Unity, and ARFoundation only intergrated ARCore and ARKit, thus the phones using ASA must support ARCore and ARKit.

Unity Scripts Pipeline Analysis

In the sample project, there are several scripts. There function are as follow.

Scripts Functions
SpatialAnchorManager.cs called as CloudManager below. It includes a lot of functions that related to the cloud anchor session control.
CloudNativeAnchor.cs Contain the native and cloud anchor for a object in Unity
AnchorExchanger.cs POST and GET the anchor identifier to the cloud

Upload Anchor Pipeline

  • Use CloudManager to create a session if there is not session created before
    • Create a session and configure it with AccountKey and AccountKey
    • Subscribe to session events, including
      • OnLogDebug
      • SessionUpdated
      • AnchorLocated
      • LocateAnchorsCompleted
      • Error
  • [Optional] Set the LocationProvider for the session, this allows to use NearDevice criteria to find anchors in the downloading pipeline
  • Use CloudManager to start a session if the session is not started before.
  • Prepare the anchor to upload
    • Let user place an object in the world.
    • Add a CloudNativeAnchor to the object to add a native anchor to the object.
    • Use the CloudNativeAnchor to create a cloud portion of the native anchor.
  • Use CloudManager to upload the anchor
    • Let user scan the environment to get enough features until meets the required info of the session.
    • Real upload to the cloud.
    • (Callback) If the CloudManager uploads the anchor successfully, store the identifier of the cloud anchor for the later use.
  • Use CloudManager to stop the session.
  • Use CloudManager to reset the session.

Download Anchors Pipeline

  • Use CloudManager to create a session (Same with upload pipeline)
  • [Optional] Set the LocationProvider for the session, this allows to use NearDevice criteria to find anchors
  • Use CloudManager to start a session if the session is not started before
  • Look for anchor
    • Set the AnchorLocateCriteria
      • Use identifier for anchors to find
      • Use NearDevice for anchors to find
    • Use CloudManager to create a watcher by passing the AnchorLocateCriteria.
  • Wait until the CloudManager found anchor
    • (Callback) Get the cloud anchor passed by CloudManager.
    • Use CloudNativeAnchor to created a native anchor from the cloud anchor.
    • Created an object, and apply the native anchor to it.
    • Set the pose of the object according to the native anchor.
  • Stop the watcher looking for object.
  • Use CloudManager to stop the session.
  • Use CloudManager to reset the session

Delete Anchors

  • Use CloudManager to delete a specific cloud anchor after the session has started.
1
CloudManager.DeleteAnchorAsync(currentCloudAnchor);

POST and GET of the anchor identifiers

  • Use AnchorExchanger.cs post the identifier to the cloud, and get a anchor index
  • Use AnchorExchanger.cs get the identifier from the cloud, according to the anchor index
1
_anchorKeyToFind = await anchorExchanger.RetrieveAnchorKey(_anchorNumber);

More Analysis

What content in the Anchor

In my opinion, an anchor contains a set of 3d point cloud. The point cloud includes the target position, which is the real position the anchor located, and the surronding 3d map points. In SLAM term, this is like a local map. Each point contains its 3d position, descriptors and etc.

Create and locate Anchor

Creation of Anchor just like the mapping process in SLAM. Once the user selects a point to place anchor, it is required to scan nearby environment to gain enough surronding points to store in the Anchor data structure.

Localization of an anchor is just the point cloud matching. The process may like the following process

  1. The device scans the environment, and builds a local point clouds map
  2. The user selects a specific Anchor to locate in the environment, the device gains a set of point cloud
  3. Use the descriptors to establish the corresponding relationship between Anchor points and local map points
  4. Use ICP algorithm to canculate the relative pose change from Anchor points cloud to local point cloud
  5. Use the mapping relationship to map the target position of Anchor to the local map. This position is the Anchor’s position. Also the rotation information is also persisted because the point cloud also has the rotation alignment.

For more information about ICP, refer this and this post.

What ASA service does

  1. Creates and locates the Anchor in client devices
  2. Establishs a cloud server for clients to upload and download the Anchor data
  3. Store the Anchor data in database of cloud