The double-random phase encoding (DRPE) scheme, which is based on a 4f optical correlator system, is considered as a reference for the optical encryption field. We propose a modification of the classical DRPE scheme based on the use of a class of structured phase masks, the deterministic phase masks. In particular, we propose to conduct the encryption process by using two deterministic phase masks, which are built from linear combinations of several subkeys. For the decryption step, the input image is retrieved by using the complex conjugate of the deterministic phase masks, which were set in the encryption process. This concept of structured masks gives rise to encryption–decryption keys which are smaller and more compact than those required in the classical DRPE. In addition, we show that our method significantly improves the tolerance of the DRPE method to shifts of the decrypting phase mask—when no shift is applied, it provides similar performance to the DRPE scheme in terms of encryption–decryption results. This enhanced tolerance to the shift, which is proven by providing numerical simulation results for grayscale and binary images, may relax the rigidity of an encryption–decryption experimental implementation setup. To evaluate the effectiveness of the described method, the mean-square-error and the peak signal-to-noise ratio between the input images and the recovered images are calculated. Different studies based on simulated data are also provided to highlight the suitability and robustness of the method when applied to the image encryption–decryption processes.