The main challenge in autonomous driving today is the transition to the concept of “embodied AI” and the implementation of complex sensor fusion. It is not enough for drones to see the road through a camera; they need to instantly synchronize video streams with lidars, radars, and motion sensors. To teach a car to respond safely to unpredictable real-world situations, developers need billions of precisely labeled frames and spatial point clouds.
That is why modern machine learning teams are abandoning the maintenance of in-house annotator staff, handing over these tasks to specialized providers. This article explores why outsourcing annotation has become critical for scaling a business and what criteria determine the leaders in the autonomous vehicle data market.
Types of Data Annotation in Autonomous Driving
Training autonomous vehicles requires data providers to have a variety of annotation techniques. Currently, annotation companies use three main types of annotation that cover all the needs of modern ML models:
- 3D point cloud & LiDAR annotation: This direction is important for the car’s spatial orientation. Providers track and surround objects with three-dimensional cubes (3D bounding boxes) defined by coordinates (x, y, z), dimensions, and a rotation angle. A more complex task is 3D semantic segmentation, where each point is classified as part of the road surface, a building, a car, or vegetation.
- Video & camera-based annotation: Cameras provide AI with semantic context (color, text, texture). Here, pixel-by-pixel semantic segmentation is required to determine the boundaries of the road surface, sidewalks, and road signs. Dynamic scenes require end-to-end object tracking across consecutive video frames, enabling the model to predict the trajectories of pedestrians and other vehicles.
- Sensor fusion alignment: This is a technologically demanding and complex annotation step. It is necessary to merge flat 2D video streams from cameras with 3D data from spatial sensors (LiDAR, radar, IMU). The goal is to ensure precise time synchronization with millisecond accuracy. This ensures that the object label in the video matches its spatial position in the point cloud.
Criteria for Choosing an AV Data Provider
When choosing an annotation company, it is essential to look at its technological maturity. A reliable partner must meet three criteria:
- Tooling & automation: AV projects now require scalable SaaS platforms. Therefore, providers integrate advanced AI models (for example, SAM 3D for point clouds or PointPillars for fast object detection) to automate the pre-labeling stage. Active learning pipelines enable the system to independently select only complex or blurry frames for human review, thereby reducing the time required to prepare datasets.
- Scalability & workforce management: Unmanned systems generate terabytes of raw data every month. The provider must have a proven infrastructure to instantly scale annotator staff to high volumes without compromising quality. The main role is played by internal management: a clear hierarchy of roles (annotators, reviewers, QA engineers) and multi-level control systems (human-in-the-loop), which minimize the human factor.
- Data security & compliance: Data from drone cameras contains confidential information (passers-by’s faces, license plates). Therefore, the supplier is obliged to comply with international security standards, such as ISO 27001, SOC 2, and GDPR, to ensure content is depersonalized. In addition, legal compliance (Participant Consent) is important: obtaining official consent from the participants in the filming, especially in complex egocentric or custom field scenarios, fully protects the client from legal and reputational risks.
Leading AV Data Annotation Companies:
Scale AI
Scale AI targets the Fortune 500 segment, providing scalable annotation for automakers, robo-taxi developers, and the defense sector. The Scale platform combines its own generative AI and computer vision models for automatic pre-labeling with a network of validators that verify the final result. The main advantage of Scale AI is its ability to process millions of multimodal data streams without losing speed. However, for many mid-level companies or startups, collaboration remains inaccessible due to the high entry threshold and rigid pricing policy. In addition, larger providers often rely on standardized contracts and workflows, which may offer less flexibility for highly specialized engineering requirements.
Keymakr
As a comprehensive solution for computer vision teams, Keymakr offers B2B customers a full production cycle: from organizing their own complex field data collection (including egocentric first-person RGB videos from body cameras) to selling ready-made commercially pure datasets (OTS). Their proprietary Keylabs platform effectively solves the “granularity trap” with a hierarchical markup system, in which long processes are represented as continuous activities with nested micro-actions.
Keymakr’s technology stack supports integrating the latest models (such as SAM 3 for 3D segmentation and PointPillars), ensuring sensor fusion alignment. Special temporal alignment and frame tolerance parameters ensure millisecond-accurate synchronization of video, lidar, and IMU without accidental mark shifts at frame junctions. Thanks to the combination of high-tech automation, strict legal compliance, and a flexible B2B approach, Keymakr has become a preferred partner for teams that need human-in-the-loop quality control for a justified and transparent budget.
SuperAnnotate
SuperAnnotate has gained popularity among autonomous vehicle engineers due to its advanced tools for handling large amounts of spatial data. It offers specific infrastructure for segmenting 3D point clouds and integrating automated annotation systems, which allows teams to accelerate the tracking of dynamic objects on the road. The company relies on process orchestration: its software allows customers to integrate the work of automatic AI pipelines and internal marketing teams. The platform is suitable for projects with a high degree of automation, where you need to set up active learning quickly.
Sama
Sama has a distinct business model that relies on its own staff of certified specialists. The company refused crowdsourcing, which allowed it to achieve a minimum level of defect when marking important road scenarios. Sama relies on ethical data sourcing and compliance with international security standards, including SOC 2 Type II and ISO 27001. However, maintaining a large workforce across multiple regions may result in higher service costs, while scaling to very large data volumes can take more time than with highly automated crowd-based platforms.
iMerit
iMerit offers expertise in sensor fusion, ensuring accurate calibration and fusion of data simultaneously from LiDAR, radar, and thermal cameras. Their expert teams are trained for the specifics of a particular client, which guarantees high accuracy in marking edge cases. The main advantage lies in high-quality analytics and custom workflows developed for complex road conditions. On the other hand, due to its focus on complex enterprise solutions and deep manual expert analytics, iMerit may be slower on simple 2D markup tasks, where other providers benefit from massive automation.
Encord
Encord focuses on automating video markup and large sets of visual data for computer vision. The company has built its ecosystem around the concept of active learning pipelines and micro-models. Instead of manually marking every second of video, Encord automatically creates high-quality object masks from several reference frames. The platform is highly utilized among engineering teams that seek to minimize human participation in routine work and direct human resources exclusively to auditing complex errors. The Encord ecosystem offers analytical tools for assessing dataset quality, helping detect duplicates or uninformative frames before the model is trained. However, for projects that require specific physical data collection “in the field” or intensive work with hierarchical text descriptions of scenes in natural language, the platform functionality may require additional third-party integrations.
Conclusion
The further progress and safety of the autonomous driving industry directly depend on the accuracy of processing critical edge cases—rare and unpredictable situations on the road. It is in these scenarios that AI requires high-quality marking. Given this, the best data provider is a technology partner with its own flexible SaaS infrastructure, advanced AI automation tools, and strict legal compliance. Only such a symbiosis between technology and Human-in-the-Loop control can guarantee the necessary precision and accuracy needed to integrate datasets into the training cycle of next-generation AI.
