RC5 is a well-known block cipher notable for its simplicity, speed, suitability for hardware and software implementation, and its low memory requirement. However, RC5 can be broken with various attacks, revealing weak security. This is mainly due to the poor diffusion operation that allows the cipher to be attacked with less complexity than that of the exhaustive search. In this work, we modify the RC5 to give high level security, and adopt it for image encryption by adjusting the structure of both the encryption routine and the key schedule. We generate round keys based on chaos. Then, we strengthen the diffusion operation by making heavy use of rotations. The security analysis of the modified cipher against several attacks is explored from a strict cryptographic viewpoint. Experimental results demonstrate that the modified RC5 provides an efficient and secure way for image encryption.