Bowen's Tech Blog

XR devices leverages variety of algorithms, mainly include computer vision and computer graphics categories.

VR, AR introduction.

Introduction for software development in Apple platforms

Opencv has a lot of useful functions.

Show image/video stream

1
2
3

while True:
    cv2.imshow(img)
    cv2.waitKey(1)

Color Related

Color Space Convert

1

gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

Histogram and Backprojection

1
2

hist = cv2.calcHist([img], [0, 1], None, [180, 256], [0, 180, 0, 256])
mask = cv2.calcBackProject([target], [0, 1], hist, [0, 180, 0, 256], 1)

Otsu threshold

1

_, mask = cv2.threshold(img[:, :, 1], 0, 255, cv2.THRESH_BINARY | cv2.THRESH_OTSU)

Geometric

Contour of images

1

_, contours, _ = cv2.findContours(mask, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_SIMPLE)

Convex Hull and Convex Defect

1
2

hull = cv2.convexHull(contour, returnPoints=False)
defects = cv2.convexityDefects(contour, hull)

There are a lot of application development paths for Microsoft MixedReality platforms. This post covers things including OpenXR, Unity, Stereo Kit, MRTK and etc.

Captures some features of Microsoft HoloLens.

Anchor: a kind of data structure in AR field, which represents a precise point of interest in 3D space that can be shared across users, devices, and time

Microsoft Azure Spatial Anchors is a service that can store the local world anchors to the cloud and share the anchors among different platforms, see the table blow. ASA (Azure Spatial Anchors) enables building apps that can map, persist, and share 3D content from room-scale to real world-scale.

Microsoft Azure Remote Rendering is a service that can render high-quality, interactive 3D content, and stream it to your devices in real time.

ARKit is a software framework for iOS to create AR applications. It support features of device tracking, plane detection, and could allow apps to place virtual objects on top of real-world environment.

As the development of LLM and Agentic AI, the concept of spatial AI is attracting increasing attention. Spatial AI combines machine learning/deep learning with 3D geometry and sensor data, allowing computers to perceive, map, and reason about physical space. Unlike traditional AI that focuses on text or 2D images, Spatial AI understands 3D space and real-world physics.

SLAM, as a fundamental technique, is very powerful to understand and perceive the 3D space and physical environment. SLAM has a long history over the traditional computer vision and state estimation theories, and now it’s getting new opportunities to evolve with the latest LLM development.