Depth cameras for volumetric capture and spatial computing research: how academic and production teams are using iToF

For about a decade, a large share of body tracking studies, volumetric video shoots, and spatial computing prototypes ran on Kinect hardware. When Microsoft ended production of the Azure Kinect Developer Kit in 2023, it pointed its users toward the Femto Bolt from Orbbec, which uses the same indirect time-of-flight (iToF) depth camera and keeps the Azure Kinect’s depth operating modes. For a lab with years of recorded data, or a studio with a calibrated multi-camera rig, that continuity matters as much as any headline spec. It also raises a practical question for anyone designing a new capture setup. What does iToF do well for capturing people and spaces, and where does it need help?

What iToF measures and why it suits capture work

An iToF camera floods the scene with modulated near-infrared light and measures the phase shift of the return at every pixel. The emitter and the sensor sit close together, and depth comes from timing rather than from triangulation between two viewpoints. Three properties follow from that design, and each one maps to a common pain point in capture work.

The first is independence from surface texture. Passive stereo needs visible texture to find correspondences, so it struggles on bare skin, solid-colored clothing, and painted studio walls, which are exactly the surfaces volumetric and body tracking work deals with most. iToF measures each pixel directly, so a plain gray sweatshirt returns depth as readily as a patterned one.

The second is the absence of a stereo baseline. Triangulation-based systems, including structured light and stereo, view the scene from two positions. That creates regions near object edges that one viewpoint sees and the other doesn’t, leaving holes beside arms, fingers, and the outline of a torso. With the emitter and sensor nearly co-located in an iToF camera, those baseline-driven occlusion bands largely disappear. Some edge artifacts remain, as discussed below, but they come from a different source.

The third is depth error that doesn’t grow with the square of distance. Stereo and structured light error rises quickly as range increases because of the triangulation geometry. The Femto Bolt’s published accuracy is a systematic error of less than 11mm plus 0.1% of distance, with random error standard deviation of 17mm or less. Over the 1 to 3m distances typical of a capture volume, that profile stays fairly flat.

Where iToF needs care

iToF has known failure modes, and capture pipelines should plan for them. Multipath interference occurs when light bounces off more than one surface before returning, typically in concave corners where a wall meets the floor, and it biases depth there. Flying pixels appear at depth discontinuities, where a single pixel averages returns from a foreground edge and the background behind it. Fast motion can smear depth, because the phase measurement spans several exposures. Very dark or low-reflectivity materials, including some hair and black fabrics, return weak signal. Sunlight adds near-infrared noise. That’s one reason the Femto Bolt is specified for indoor use, at 10°C to 25°C and 8% to 90% relative humidity, non-condensing.

Most of these are manageable with standard practice. Edge filtering removes flying pixels, controlled studio lighting avoids sunlight, and careful set design avoids hard reflective corners inside the capture volume.

The Femto Bolt as a reference capture camera

The Femto Bolt pairs a 1-megapixel iToF depth sensor at 850nm with a 4K RGB camera that supports HDR, plus a 6DoF IMU, over a single USB 3.2 Gen 1 Type-C connection. Its depth modes match the Azure Kinect Developer Kit exactly, and each mode trades field of view against range.

The wide-field mode covers 120° × 120°. Unbinned at 1024 × 1024, it runs at up to 15fps over 0.25m to 2.21m. Binned to 512 × 512, it reaches 30fps and extends to 2.88m. The narrow-field mode covers 75° × 65°. Unbinned at 640 × 576, it runs at up to 30fps over 0.5m to 3.86m. With software 2 × 2 binning, it reaches 5.46m. A passive IR mode at 1024 × 1024 is also available.

For capture, the wide-field binned mode suits close volumes where a performer moves within arm’s reach of several cameras. The narrow-field unbinned mode suits full-body capture at a few meters, where resolution on the subject matters more than coverage.

Raw-to-depth processing runs on the host computer, as it did on the Azure Kinect. A rig with many cameras therefore needs to budget host compute and USB bandwidth per camera. Teams that would rather move that processing to the sensor can use the Femto Mega, which shares the same depth modes and runs depth processing on a built-in NVIDIA Jetson Nano, with PoE as a connection option.

Volumetric video production

Volumetric capture surrounds a performer with synchronized RGB-D cameras, fuses their depth into a mesh or point cloud per frame, and textures it with the color streams. The sensor requirements are specific. Every camera has to expose at the same moment, depth has to be registered to color, and the cameras can’t interfere with each other.

Synchronization is where the Femto Bolt differs most from its predecessor. The Azure Kinect used 3.5mm audio jacks for frame sync. The Bolt uses an 8-pin connector, and Orbbec’s sync hub carries the trigger over standard RJ45 CAT-5 or better cabling with switchable 1.8V, 3.3V, or 5V trigger levels. Longer and more reliable trigger runs make larger capture stages practical. Any camera or an external sensor can be designated as the primary.

Interference is handled with timing. Like any active illumination system, multiple iToF cameras pointed at the same volume can corrupt each other’s measurements. Production rigs stagger the exposures of subordinate cameras by small offsets so that no two cameras illuminate the scene at the same instant.

Production tools have followed the hardware. Orbbec’s published case studies describe Depthkit and ScannedReality building volumetric capture workflows on the Femto Bolt. Depthkit reports higher color resolution than its earlier setup while keeping depth accuracy.

Human body tracking research

Body tracking research has a reproducibility problem whenever hardware changes. A gait study or motion dataset collected on one depth camera can’t simply be extended on another camera with a different noise profile, field of view, and depth algorithm. That makes the Femto Bolt’s identical depth modes useful well beyond convenience. A lab can keep collecting data under the same depth conditions it has used since the Azure Kinect.

Software continuity follows the same logic. Orbbec’s K4A Wrapper lets applications written for the Azure Kinect Sensor SDK run on the Femto Bolt, and Orbbec documents running Microsoft’s Body Tracking SDK through the same wrapper. Existing skeleton tracking pipelines, Unity body tracking samples, and analysis scripts can carry over with minimal changes.

One caveat deserves attention in a methods section. The color camera is not identical. In widescreen modes, the Femto Bolt’s color field of view is 80° × 51°, compared with 90° × 59° on the Azure Kinect, and the sensor and optics differ. Pipelines that rely on color, such as RGB-based pose estimation fused with depth, should be recalibrated and validated rather than assumed equivalent.

Medical and rehabilitation prototypes

Depth cameras are attractive for clinical research prototypes because they measure posture and movement without markers or contact. Orbbec lists fall detection, gait analysis, rehabilitation, and patient positioning among its healthtech applications, and the Femto Bolt’s product page highlights body movement tracking for elderly care and rehabilitation.

iToF’s characteristics line up with these settings. Patient rooms and therapy spaces are indoor and lit artificially. Subjects often wear plain clothing or gowns that give passive stereo little texture. The wide field of view can cover a bed or a walkway from a single mounting point. Prototypes still need validation against reference systems such as optical motion capture before their measurements support clinical conclusions, and a research prototype is not a regulated medical device.

Spatial computing and robotics research

Spatial computing labs use RGB-D cameras for scene reconstruction, AR and XR anchoring, and human-robot interaction studies. These projects increasingly share infrastructure with robotics groups, which makes ROS 2 support a practical requirement. Orbbec’s ROS 2 wrapper supports the Femto Bolt with a dedicated launch file, and the open-source Orbbec SDK v2 supports it on Windows and Ubuntu. Labs running Azure Kinect code can keep it through the K4A Wrapper while new work moves to the native SDK.

Frequently asked questions

Is iToF better than stereo for volumetric capture?

For controlled indoor capture of people, iToF has clear advantages. It doesn’t depend on surface texture, it avoids baseline-driven occlusion at body edges, and its error stays fairly flat across a typical capture volume. Stereo is the better choice outdoors or where sunlight can’t be controlled. Many studios choose based on the stage environment rather than on the depth method in the abstract.

Can existing Azure Kinect datasets be extended with Femto Bolt data?

For depth, largely yes, because the depth operating modes and performance are identical to the Azure Kinect. Researchers should still record the hardware change in their methods and run a short side-by-side validation. The color camera’s field of view and optics differ, so color-dependent features need recalibration.

How many Femto Bolt cameras can run in one capture volume?

The practical limit comes from synchronization, interference, and host resources rather than from a fixed number. Cameras need hardware sync with staggered exposure offsets to avoid illuminating the scene at the same moment. Because depth processing runs on the host, each camera adds compute and USB bandwidth load. Large rigs often spread cameras across several capture PCs or use the Femto Mega to move processing onto the camera.