Abstract: Quantifying sports energy consumption is the foundation of scientific sports. At present, there are many limitations in the detection equipment of daily exercise energy consumption. To address this problem, this paper adopts the image recognition technology that has been developing rapidly in recent years, and designs a set of human movement energy consumption monitoring system based on the mobile phone terminal, the system contains four kinds of movement and fitness actions, through the mobile phone front camera to shoot the video of the human body completing the corresponding action under the voice and video guidance, using the image analysis technology to calculate the human movement energy consumption value in real time, to realize the human movement energy consumption monitoring. The correlation coefficients between the experimental results and the measured value of indirect calorimetry, which is called the "gold standard", reached more than 0.6 (P<0.01), and the energy consumption prediction equations for four actions were proposed. The methods are more consistent and can be used as an alternative to traditional methods.