In order to satisfy the requirements of real-time, low-power and miniaturization for dynamic target recognition and tracking system, a system based on FPGA is designed, which is supplemented by OV7670 video acquisition module and VGA interface displayer. The image is captured, processed and marked with a rectangle. Consequently, the dynamic recognition, tracking and display are realized. Among them, the image processing part is completed by FPGA, including cache, gray processing, improved bit operation median filtering, background and foreground separation, inter-frame difference method and background difference method compromising. It making full use of the merits of high-speed and parallel processing ability of FPGA, high-speed RAM on-chip, large capacity of off-chip SDRAM to achieve the processing and storage of video data. At the same time, the system has strong anti-interference, compact, flexible, low power consumption, versatility and scalability, which is suitable for industrial areas, but also suitable for home use.