Siamese Multi-View Masked Autoencoders for Skeleton-based Action Representation Learning
ID:30 Submission ID:37 View Protection:ATTENDEE Updated Time:2024-10-23 11:32:49 Hits:47 Oral Presentation

Start Time:2024-11-02 08:50 (Asia/Shanghai)

Duration:20min

Session:[P5] Parallel Session 5 » [P5-2] Parallel Session 5(November 2 AM)

No files

Abstract
  In recent years, supervised skeleton-based action recognition has achieved notable results. However, these methods rely on labeled data, which is both resource-intensive and time-demanding to obtain. Self-supervised methods do not require data labels and has attracted considerable interest within the academic community. Masked Autoencoders are a self-supervised learning paradigm, and Siamese networks are a common structure in computer vision tasks. Combining these two approaches is natural. However, most existing research has applied these methods to image or video tasks, while relatively limited attention has been given to skeleton-based action recognition. Additionally, current methods tend to ignore differences in how the same action appears from different views, which limits the model's spatial representation capabilities. To address this, we introduce the Siamese Masked Autoencoders framework into skeleton-based action representation learning, named SiamMVMAE. To encourage the model to capture action features across various viewpoints, both the original skeleton sequences and rotation-augmented sequences are used as independent inputs for the Siamese networks. These inputs are then processed with a transformer encoder and decoder, enabling effective learning of action representations. Experiments on the NTU-RGB+D 60, NTU-RGB+D 120, and PKU-MMD benchmark datasets show that our method is highly competitive compared to existing approaches.
Keywords
action recognition, masked autoencoders, self-supervised learning, Siamese networks
Speaker
LiuRonggui
None Dongguan University of Technology

Submission Author
LiuRonggui Dongguan University of Technology
RenZiliang Dongguan University of Technology
WeiWenhong Dongguan University of Technology
ZhengZiyang Dongguan University of Technology
ZhangQieshi Shenzhen Institute of Advanced Technology,Chinese Academy of Sciences
Comment submit
Verification code Change another
All comments

Important Dates

15th August 2024   31st August 2024- Manuscript Submission

15th September 2024 - Acceptance Notification

1st October 2024 - Camera Ready Submission

1st October 2024  – Early Bird Registration

 

Contact Us

Website:

https://icsmd2024.aconf.org/

Email:
icsmd2024@163.com