Square and circular light source layouts conventionally employed in visible light communication (VLC) systems suffer from communication blind spots and received signal nonuniformity as well as from large power fluctuations in the middle of the room. To address these shortcomings, we use the cross square and annulus layouts in conjunction with the cross-fertilize particle swarm optimization algorithm to improve VLC system performance. The distribution of the unit area received power (RPUA), the distribution of the signal-to-noise ratio (SNR), and the bit error rate (BER) performance were simulated and the rationality of the algorithm was compared. The results show that after optimization, the RPUA fluctuation value of the cross square layout decreases from 2.32 to 1.59 dB / m2, the SNR fluctuation value reduces from 8.5 to 4.2 dB, and the RPUA fluctuation value of the annulus layout decreases from 4.69 to 3.46 dB / m2, the SNR fluctuation value decreases from 13.3 to 8.8 dB. The RPUA value of cross-square layout fluctuates between −1.5 and −0.5 dB / m2, and the receiving area accounts for 93.4% in this range while in the case of the square layout, this area is only 86.7%. Therefore, it suggests that this scheme is reasonable for the equality of communication quality and system performance heightens greatly.