rPPG facial video synthesis via generative animation and biophysical embedding
DOI:
CSTR:
Author:
Affiliation:

1.School of Computer Science, Hangzhou Dianzi University, Hangzhou 310018, China; 2.Zhejiang Provincial Key Laboratory of Brain Computer Collaborative Intelligence Technology and Applications, Hangzhou 310018, China

Clc Number:

TN911.73;TP391

Fund Project:

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
  • |
  • Comments
    Abstract:

    Remote photoplethysmography (rPPG), as a non-contact physiological monitoring technique, is of great value in clinical monitoring and human-computer interaction. However, the performance of deep learning models in rPPG tasks heavily relies on large-scale and diverse annotated data. Existing benchmark datasets, such as PURE and UBFC-rPPG, are limited by subject scale and relatively simple motion scenarios, which leads to insufficient generalization when models face complex nonlinear motion interference. Therefore, this paper proposes a facial video synthesis framework based on generative animation and biophysical embedding. The framework adopts a two-stage strategy of motion generation and signal embedding. First, the generative framework LivePortrait is used to transfer large-scale motions and facial expressions from driving videos to static images, constructing high-fidelity dynamic facial videos and effectively simulating macroscopic displacement noise in real scenes. Then, a layer-separated biophysical embedding model is proposed to implant physiological signals into videos. This model uses a large-kernel Gaussian operator to decompose the image into a base layer and a detail layer, and introduces multiplicative modulation into the low-frequency component, which better conforms to skin optical characteristics and blood-flow variations. While preserving detailed skin texture, it ensures the physical accuracy of weak physiological signals in both spectral and temporal dimensions. Experimental results on the UBFC-rPPG and PURE datasets show that, under sample-limited conditions, after introducing the synthetic data generated by this paper, the MAE of PhysNet on the UBFC-rPPG dataset decreases from 1.11 bpm to 0.78 bpm, and the SNR increases from 5.50 dB to 5.99 dB; on the PURE dataset, the MAE decreases from 1.16 bpm to 0.99 bpm, and the SNR increases from 8.17 dB to 8.36 dB. The results demonstrate that the synthetic data generated by this paper can effectively improve the heart-rate estimation accuracy and signal quality of rPPG models. This study provides a feasible approach with both physical determinacy and motion realism for addressing data scarcity in the rPPG field.

    Reference
    Related
    Cited by
Get Citation
Related Videos

Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:
  • Revised:
  • Adopted:
  • Online: September 20,2026
  • Published:
Article QR Code