Automatic target acquisition (ATA) and automatic target recognition (ATR) are two vital tasks for missile systems, and
having a robust detection and recognition algorithm is crucial for overall system performance. In order to have a robust
target detection and recognition algorithm, an extensive image database is required. Automatic target recognition
algorithms use the database of images in training and testing steps of algorithm. This directly affects the recognition
performance, since the training accuracy is driven by the quality of the image database. In addition, the performance of
an automatic target detection algorithm can be measured effectively by using an image database. There are two main
ways for designing an ATA / ATR database. The first and easy way is by using a scene generator. A scene generator can
model the objects by considering its material information, the atmospheric conditions, detector type and the territory.
Designing image database by using a scene generator is inexpensive and it allows creating many different scenarios
quickly and easily. However the major drawback of using a scene generator is its low fidelity, since the images are
created virtually. The second and difficult way is designing it using real-world images. Designing image database with
real-world images is a lot more costly and time consuming; however it offers high fidelity, which is critical for missile
algorithms. In this paper, critical concepts in ATA / ATR database design with real-world images are discussed. Each
concept is discussed in the perspective of ATA and ATR separately. For the implementation stage, some possible
solutions and trade-offs for creating the database are proposed, and all proposed approaches are compared to each other
with regards to their pros and cons.