A novel method is proposed in this paper for light field depth estimation by using a convolutional neural network. Many approaches have been proposed to make light field depth estimation, while most of them have a contradiction between accuracy and runtime. In order to solve this problem, we proposed a method which can get more accurate light field depth estimation results with faster speed. First, the light field data is augmented by proposed method considering the light field geometry. Because of the large amount of the light field data, the number of images needs to be reduced appropriately to improve the operation speed, while maintaining the confidence of the estimation. Next, light field images are inputted into our network after data augmentation. The features of the images are extracted during the process, which could be used to calculate the disparity value. Finally, our network can generate an accurate depth map from the input light field image after training. Using this accurate depth map, the 3D structure in real world could be accurately reconstructed. Our method is verified by the HCI 4D Light Field Benchmark and real-world light field images captured with a Lytro light field camera.
Convolutional neural networks (CNNs) is becoming a critical role for deep learning-based computer vision applications. Through CNN we can extract meaningful information out of massive sensor data. Vision applications use this information to analyze the mainstream trends of the data and take immediate action based on these trends. However, CNN's energy consumption and bandwidth limitations make it difficult for CNN network systems to deploy in mobile systems with stringent energy limits. In this paper, we explore the simulation and the hardware method of optical convolution for low power image processing, which is inspired by the bio-image sensor Angle sensitive pixel (ASP). This optical computation method may be used to substitute the first convolution layer of the CNN network due to its energy-saving features and speed of light processing time. We adopt two-layer of customized transmission grating to perform this optical convolution computing. By generating the Talbot effect, the two-layer grating structure can perform optical convolution computation using Gabor wavelet filters, which will cause zero electrical power. We demonstrate both simulation and experiment results for optical convolution through our algorithm and prototype system, the convolution results can extract different meaningful information about the original image, which is very similar to edge filtering. This optical operation will hopefully be used to replace the first convolution layer of CNN since it can effectively reduce both the consumption of the energy power and the performing time.
To obtain surreal and richer visual experience, augmented reality (AR) technology has been widely used in various areas. As a popular solution of AR, display using computer generated hologram (CGH) is often accompanied by blurring which is caused by uncontrolled interference. In this paper, a modified algorithm based on double-phase hologram (DPH) algorithm is proposed to reduce speckle noise in holographic reconstruction. The macro-pixels in the original hologram are separated into multiple sub-holograms, and these sub-holograms are displayed alternately in high frequency, which reduces the speckle noise generated from the interference between adjacent macro-pixels. Meanwhile, the method is less time-consuming than the traditional Gerchberg-Saxton algorithm because no iteration is needed. The simulation and the optical experiment based on liquid crystal on silicon (LCoS) have been conducted, and the results confirm the feasibility of the proposed method to improve the image quality.
Artificial neural networks are computational models enlightened by biological neural networks, playing a significant role in image recognition, language translation and computer vision fields, etc. In this paper, we propose a fully optical neural network based on programmable nanophotonic processor (PNP) to realize digit recognition. The architecture includes 4 layers cascaded Mach–Zehnder interferometers (MZIs), which could theoretically execute matrix functions corresponding to a two-layer fully connected ANN with four inputs. We simulate cascaded MZIs and adjust phase shifters to match weight matrices calculated by ANN in computer beforehand. The accuracy of 4-class handwritten digits in ONN is 80.29% due to the compressed input data. The accuracy of 10-class digits could achieve 99.23% when the input node merely increases to 36. The results demonstrate the handwritten digits could be recognized effectively through PNP in ONN and the construction of PNP could be extended for more complex recognition systems.