Dual-branch spatiotemporal disentanglement and reconstruction for robust remote photoplethysmography
DOI:
CSTR:
Author:
Affiliation:

1.School of Computer Science, Hangzhou Dianzi University, Hangzhou 310018, China; 2.Zhejiang Provincial Key Laboratory of Brain Computer Collaborative Intelligence Technology and Applications, Hangzhou 310018, China

Clc Number:

TN911.73;TP391

Fund Project:

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
  • |
  • Comments
    Abstract:

    Remote photoplethysmography (rPPG) can recover pulse-related physiological signals from facial videos captured by ordinary cameras. However, the cardiac-induced color variations are extremely subtle and are easily corrupted by head motion, pose variation, illumination fluctuation, shadow occlusion, and imaging noise. Most existing deep learning methods formulate rPPG estimation as an end-to-end regression problem, but they usually lack explicit modeling of the generation mechanisms of pulse and noise components, making the intermediate representations difficult to interpret. To address this problem, this paper proposes PNRNet, a physics-guided dual-branch spatiotemporal disentanglement and reconstruction network for robust rPPG. The proposed method models facial RGB temporal variations as a superposition of physiological pulse components and non-physiological noise components. Two independent 3D convolutional branches are used to learn pulse-related and interference-related dynamics, and a feature orthogonality constraint is introduced to reduce redundant coupling between the two latent representations. Inspired by the dichromatic reflection model, an RGB temporal reconstruction module projects the predicted pulse and noise signals back to the RGB observation domain, enabling the network to be jointly supervised by reference PPG waveforms and video-domain reconstruction. Experimental results show that PNRNet achieves MAE of 3.02, 1.18, and 1.78 bpm on the iBVP, UBFC-rPPG, and PURE datasets, respectively, with an 8.8% MAE reduction over the best competing method on iBVP. These results indicate that the proposed pulse-noise disentanglement and RGB observation-domain reconstruction mechanism helps improve the robustness and interpretability of rPPG models under complex interference conditions.

    Reference
    Related
    Cited by
Get Citation
Related Videos

Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:
  • Revised:
  • Adopted:
  • Online: September 20,2026
  • Published:
Article QR Code