Nonlinear friction negatively impacts the dynamic and static performance of hydraulic servo systems in robotic excavators, leading to issues such as trajectory creep, flat peaks, and steady-state errors. The traditional LuGre friction model, which relies solely on velocity and internal bristle state variables that cannot be accurately measured, fails to comprehensively describe the complex friction characteristics of excavator hydraulic servo systems. Considering the position, velocity, and direction of the excavator hydraulic servo system, we propose an enhanced LuGre friction model and introduce a velocity threshold to address the instability issue of the elastic bristle average deformation state observer in the friction model. Secondly, to address the issues of traditional optimization algorithms getting stuck in local optimal solutions and having slow convergence speeds, the basic particle swarm optimization algorithm has been enhanced. This enhancement involves the introduction of inertia weight, asynchronous change, and elite mutation operations to accurately and rapidly identify the six unknown parameters in the improved LuGre friction model. Subsequently, using the identified friction model, a friction compensation controller based on the principle of structural invariance is designed. Three different operating condition trajectory tracking experiments were conducted on a 23-ton excavator. The conventional proportional-integral-differential controller exhibits the highest tracking error, with the maximum tracking error for the triangular trajectory reaching 29.68 mm. In contrast, the feedforward friction compensation controller, which is based on the enhanced LuGre model, achieves a significantly lower error of 9.70 mm, representing a 67.31% reduction in error. The experimental results demonstrate that the proposed friction compensation controller significantly enhances the trajectory tracking accuracy of the excavator.