In order to improve the multitarget tracking robustness and enhance the difference between the targets, this paper uses an energy minimization method for multitarget tracking. Different to the existing algorithm, the algorithm focuses on the representation of the complex problem in multitarget tracking as energy function model, which includes a better target segmentation strategy (similarity model). By assigns every possible solutions a cost (the “energy”), the algorithm transforms the multiple target tracking problem into an energy minimization problem. In the energy minimization optimization method, the algorithm uses the conjugate gradient algorithm and a series of jump moves to find the minimum energy value. The experimental results of open data demonstrate the effectiveness. And the quantitative analysis results show that this algorithm can improve the difference between targets or between target and background so as to obtain better robust performance compared with other algorithms.