This paper proposes a new MPEG-4 rate control algorithm for single or multiple object video sequences. The algorithm aims to achieve an accurate bit rate with the maximum picture quality while efficiently handling buffer fullness and scene change. In addition to estimating the bit budget of a frame based on its global coding complexity, the algorithm dynamically distributes the target bits for each object within a frame according to its coding complexity. Even though the VM8 solution and other algorithms adopt a simple proportional buffer controller, their control ability is rather ineffective. The proposed algorithm exploits a novel Proportional Integrated Differential (PID) buffer controller to effectively minimize the buffer overflow or underflow. The PID based controller reduces the deviation between the current buffer fullness and the target buffer fullness, mitigates the overshoots, and improves the transient response. The combined effect is a more smooth and effective buffer control. Furthermore, the algorithm defines a new and effective coding complexity of an object and dynamically optimizes several parameters. Overall, the proposed algorithm successfully achieves accurate target bit rate, provides promising coding quality, decreases buffer overflow/underflow and lowers the impact of a scene change.