Fourier Ptychographic Microscopy (FPM) is a super-resolution microscopy technology, in which a set of low-resolution images containing different frequency components of the sample can be obtained by changing the angle of the light source in this technology, and then the iterative algorithm is used to reconstruct high-resolution intensity and phase information. The reconstruction usually takes a long time and is not suitable for real-time FPM imaging. It has been recognized recently that the potential fast image reconstruction algorithm is the use of deep learning algorithms. We designed a conditional generative adversarial network (cGAN) which has multi-branch input and multi-branch output which can expand the frequency spectrum of the reconstructed image very well. Based on the convolutional neural network (CNN), the brightfield and darkfield images obtained by FPM imaging can be regarded as different image features obtained by different convolutional kernel, and the skip connection of U-net can effectively utilize this information. The brightfield and darkfield images in FPM imaging are input to different branches, which can avoid missing the darkfield signal information. Importantly, the neural network we designed will continue to perform simulation process of FPM imaging from the recovered high-resolution intensity and phase to obtain low-resolution images and make them correspond one-to-one with the input low-resolution images. These corresponded images will enter loss function, making it easier for the neural network to learn relation between the low-resolution images and the high-resolution images. We validated the deep learning algorithm through simulated experimental research on biological cell imaging.
Structure Illumination Microscopy (SIM) is a wide-field super-resolution fluorescence imaging technology with characteristics such as fast imaging speed and low phototoxicity. By projecting sinusoidal patterns at the sample plane, the high-frequency information in Fourier space which is out of the optical transfer function of the optical system is loaded into the low-frequency information and collected by the objective lens. However, due to the mechanical error of the system, the fringes in the collected data often have some deviation from the presupposed initial values. These systemic errors of fringe will directly affect the quality of the reconstructed SIM image, among which Fringe modulation depth is a very important parameter. Here, we explored the SIM reconstruction method based on the U-net neural network architecture recently reported by Luhong Jin et al.We performed a simulation to validate the method. Specifically, we use an open source fluorescent-bead images for the training and testing. We found that after training, the output of the trained neural network is very close to the ground truth, and then the super-resolution information can be well recovered from the low-modulation SIM raw images. We then further performed the similar study on the images of real biological structures which are also available as an open source dataset. Our study thus demonstrates that the deep learning neural network algorithm can significantly relax the requirement on the fringe modulation depth.Therefore, the simplified SIM system without any polarization modulation can be expected.