Generating imagery using gaming engines has become a popular method to both augment or completely replace the need for real data. This is due largely to the fact that gaming engines, such as Unity3D and Unreal, have the ability to produce novel scenes and ground-truth labels quickly and with low-cost. However, there is a disparity between rendering imagery in the digital domain and testing in the real domain on a deep learning task. This disparity/gap is commonly known as domain mismatch or domain shift, and without a solution, renders synthetic imagery impractical and ineffective for deep learning tasks. Recently, Generative Adversarial Networks (GANs) have shown success at generating novel imagery and overcoming this gap between two different distributions by performing cross-domain transfer. In this research, we explore the use of state-of-the-art GANs to perform a domain transfer between a rendered synthetic domain to a real domain. We evaluate the data generated using an image-to-image translation GAN on a classification task as well as by qualitative analysis.
Capsule networks have shown promise in their ability to perform classification tasks with viewpoint invariance; outperforming the accuracy of other models in some cases. This capability applies to maritime classification tasks where there is a lack of labeled data and an inability to collect all viewpoints of objects that are needed to train machine learning algorithms. Capsule Networks lend themselves well to applying their unique network architecture to the maritime vessel BCCT dataset, which exhibits characteristics aligned with the theorized strengths of Capsule Networks. Comparing these with respect to traditional CNN architectures and data augmentation techniques provides a potential roadmap for incorporation into future classification tasks involving imagery in data starved domains relying heavily on viewpoint invariance. We present our results on the classification of ship using Capsule Networks and explore their usefulness at this task given their current state of development.
Rendering synthetic imagery from gaming engine environments allows us to create data featuring any number of object orientations, conditions, and lighting variations. This capability is particularly useful in classification tasks, where there is an overwhelming lack of labeled data needed to train state-of-the-art machine learning algorithms. However, the use of synthetic data is not without limit: in the case of imagery, training a deep learning model on purely synthetic data typically yields poor results when applied to real world imagery. Previous work shows that "domain adaptation," mixing real-world and synthetic data, improves performance on a target dataset. In this paper, we train a deep neural network with synthetic imagery, including ordnance and overhead ship imagery and investigate a variety of methods to adapt our model to a dataset of real images.