The performance of embodied intelligence models heavily depends on the quality of training data, yet quality acceptance of datasets is often reduced to merely seeing if the annotations are accurate or not. That is far from sufficient. Based on practical project experience, I believe the acceptance process should be governed by at least four dimensions and more than ten specific metrics.
First is data completeness and solvability. Each sample must not only include multimodal raw data such as images, point clouds, and force/torque sensor readings, but also ensure strict synchronization of sensor timestamps and a unified spatial coordinate system. Many datasets are collected with uncalibrated devices, resulting in a distorted physical world that the model sees during training. During acceptance, you should spot-check whether all modalities can be aligned and replayed, instead of merely checking if files exist.
Second is fine-grained verification of annotation quality. In addition to common annotations like bounding boxes, segmentation, and keypoints, embodied tasks care more about the semantic accuracy of action labels—for example, whether the boundary between grasp and press is clear, and whether object pose ground truth is smoothed across frames rather than jumping. It is recommended to adopt a cross-blind validation approach, where different annotators re-check the same batch of data, and to compute inter-class consistency coefficients. This has a direct impact on grasp success rate metrics.
Third is diversity coverage of scenes and tasks. The biggest fear in embodied intelligence is overfitting to laboratory environments. During acceptance, the distribution of object categories, illumination variation ranges, number of background textures, robot pose dispersion, and frequency distribution of human demonstration actions should all be statistically analyzed. A qualified dataset should contain a large number of failed attempts and recovery actions; otherwise, the model will never learn to correct errors. Checking inter-group variance is more meaningful than merely looking at total volume.
Fourth is dynamic temporal consistency and physical plausibility. Many datasets have high quality static frames, but when played continuously, issues like object penetration, hand jitter, and abrupt action changes appear. It is recommended to use optical flow methods to detect motion smoothness between consecutive frames, and use physics engines to inversely verify whether contact forces along the grasp trajectory are within reasonable ranges. Extreme acceleration values and joint angle limits during motion are easily overlooked yet critical indicators.
Finally, do not forget to preserve traceable metadata. The robot model, sensor version, and collection date for each data source should be saved along with samples to facilitate domain discrepancy analysis later. Quality acceptance should not be a one-time pass; these metrics should be solidified into automated scripts and continuously monitored across collection batches. Only by guarding this gate well can subsequent model iterations avoid wasting effort on dirty data.