A fiber optic magnetic field sensor with FBG F-P magnetically sensitive structure combined with optical coherence detection is reported and applied in the field of casing loss detection. The fiber optic FBG F-P interferometer is designed with a long cavity structure with weak reflection to improve the phase shift sensitivity of the magnetic sensor. Based on the differential anti-interference detection structure to eliminate the cross-sensitivity between magnetic field and temperature, the double-layer packaging structure can adapt to the harsh environment. The test platform for casing damage detection was built and the test scheme was formulated. The experimental results showed that the measured results of the magnetic sensor were basically consistent with the actual model defects. The fiber optic magnetic sensor is an all-fiber structure, which can meet the application of underground measurement working environment after being sealed with pressure-proof and water-proof.