Aiming at the problem that the traditional floating small target feature detection method is difficult to extract the target feature effectively, this paper analyzes the feature of small target on the sea surface, and studies the principle of fine composite multi-scale dispersion entropy (RCMDE). A small target detection method based on RCMDE-XGBoost is proposed. The signal was de-noised by using variational mode decomposition, the multi-scale features of the target were extracted by fine composite multi-scale dispersion entropy, the multi-dimensional feature matrix was constructed and input into XGBoost network for feature classification, and the small target detection on the sea surface was realized through model training. Using the IPIX radar measurement database, the detection rate of #54, #311, #320 HV polarization mode reaches 93. 33%, 92. 38%, 95% respectively, which is 12% higher than the graph connected density detection method on average, proving the effectiveness of RCMDE-XGBoost detection method.