For the problem that it is difficult to achieve model-free optimal tracking control in the dynamic system of wheeled mobile robot, a new online integral reinforcement learning control algorithm based on actor-critic framework is proposed in this paper. Firstly, the critic neural network based on RBF is constructed to fit the quadratic tracking control performance index function and the weight updating law of the network is designed based on the approximate Behrman error. Secondly, the RBF actor neural network is constructed to compensate the unknown terms in the dynamic system and the weight updating law is designed to minimize the performance index function. Finally, it is proved by Lyapunov theory that the proposed integral reinforcement learning control algorithm can make the value function, the critic and actor neural network weights error uniformly and finally bounded. Simulation and experimental results show that the algorithm not only realizes the tracking of constant or time-varying velocity, but also can be implemented on the embedded platform.