In this work, a Michelson interferometer based sensing demodulation scheme using an improved phase generated carrier (PGC) technique is proposed to eliminate the influence of interference amplitude on the phase demodulation of the distributed optical fiber vibration sensing system. Firstly, a high frequency carrier signal is applied to the reference fiber of the Michelson interferometer based fiber vibration sensing system to modulate the vibration signal which is acts on the sensing fiber link of the sensing system. Second, the sinusoidal and cosine terms of the applied signal can be acquired by the zero order harmonic, the first order harmonic carrier signal and the low-pass filter. Finally, the demodulated signal without the interference amplitude is demodulated by a proper transformation and the division of sine term and cosine term. The simulation results and experimental results show that the improved PGC demodulation technique can effectively eliminate the influence of interference amplitude on the final demodulation output signal.