The study of complex human interactions and group activities has become a focal point in human-centric computer vision. However, progress in related tasks is often hindered by the challenges of obtaining large-scale labeled datasets from real-world scenarios. To address the limitation, we introduce M3Act, a synthetic data generator for multi-view multi-group multi-person human atomic actions and group activities. Powered by Unity Engine, M3Act features multiple semantic groups, highly diverse and photorealistic images, and a comprehensive set of annotations, which facilitates the learning of human-centered tasks across single-person, multi-person, and multi-group conditions. We demonstrate the advantages of M3Act across three core experiments. The results suggest our synthetic dataset can significantly improve the performance of several downstream methods and replace real-world datasets to reduce cost. Notably, M3Act improves the state-of-the-art MOTRv2 on DanceTrack dataset, leading to a hop on the leaderboard from 10th to 2nd place. Moreover, M3Act opens new research for controllable 3D group activity generation. We define multiple metrics and propose a competitive baseline for the novel task.
@misc{chang2023learning,
title={Learning from Synthetic Human Group Activities},
author={Che-Jui Chang and Danrui Li and Deep Patel and Parth Goel and Honglu Zhou and Seonghyeon Moon and Samuel S. Sohn and Sejong Yoon and Vladimir Pavlovic and Mubbasir Kapadia},
year={2023},
eprint={2306.16772},
archivePrefix={arXiv},
primaryClass={cs.CV}}
@misc{chang2024equivalency,
title={On the Equivalency, Substitutability, and Flexibility of Synthetic Data},
author={Che-Jui Chang and Danrui Li and Seonghyeon Moon and Mubbasir Kapadia},
year={2024},
eprint={2403.16244},
archivePrefix={arXiv},
primaryClass={cs.LG}}