Optical inspection systems require fast image acquisition at significantly enhanced resolution when utilized for advanced machine vision tasks. Examples are quality assurance in print inspection, printed circuit board inspection, wafer inspection, real-time surveillance of railroad tracks, and in-line monitoring in flat panel fabrication lines. Ultra-highspeed is an often demanded feature in modern industrial production facilities, especially, where it comes to high volume production. A novel technology in this context is the new high-speed sensor for line-scan camera applications with unmatched line rates up to 200 kHz (tri-linear RGB) and 600 kHz (b/w), presented in this paper. At this speed, the multiline- scan sensor provides full color images with, e.g., a spatial resolution of 50 μm at a transport speed of 10 m/s. In contrast to conventional Bayer pattern or three-chip approaches, the sensor presented here utilizes the tri-linear principle, where the color filters are organized line-wise on the chip. With almost 100% fill-factor, the tri-linear technology assures high image quality because of its robustness against aliasing and Moiré effects leading to improved inspection quality, less false positives and thus less waste in the production lines.
For industrial print flaw detection images are acquired and then compared to a specimen (master image). Due to the
production process, the images are not exactly aligned to each other. Therefore, preceding a pixel-by-pixel comparison,
the acquired image has to be rectified in order to match the master image' properties-it has to be warped into the
master image' coordinate system. To achieve the required detection speed, several Megapixels per second have to be
processed. It proved to be very advantageous to continuously process the stream of image data in an image processing
pipeline. The first stage is the warping process. In this paper we introduce a streaming warper unit which implements
affine backward mapping and cubic spline interpolation. Since a complete pixel transformation is computed per clock
cycle the performance-implemented on contemporary FPGA
devices--can be up to 200 Megapixels per second. The
implementation of several streaming warper units within a single FPGA is possible. This enables image processing
systems which allow high data rates even under real-time constraints.
Dealing with high-speed image acquisition and processing systems, the speed of operation is often limited by the amount
of available light, due to short exposure times. Therefore, high-speed applications often use line-scan cameras, based on
charge-coupled device (CCD) sensors with time delayed integration (TDI). Synchronous shift and accumulation of
photoelectric charges on the CCD chip - according to the objects' movement - result in a longer effective exposure time
without introducing additional motion blur. This paper presents a high-speed color line-scan camera based on a
commercial complementary metal oxide semiconductor (CMOS) area image sensor with a Bayer filter matrix and a field
programmable gate array (FPGA). The camera implements a digital equivalent to the TDI effect exploited with CCD
cameras. The proposed design benefits from the high frame rates of CMOS sensors and from the possibility of arbitrarily
addressing the rows of the sensor's pixel array. For the digital TDI just a small number of rows are read out from the area
sensor which are then shifted and accumulated according to the movement of the inspected objects. This paper gives a
detailed description of the digital TDI algorithm implemented on the FPGA. Relevant aspects for the practical
application are discussed and key features of the camera are listed.
Today, printing products which must meet highest quality standards, e.g., banknotes, stamps, or vouchers, are automatically checked by optical inspection systems. Typically, the examination of fine details of the print or security features demands images taken from various perspectives, with different spectral sensitivity (visible, infrared, ultraviolet), and with high resolution. Consequently, the inspection system is equipped with several cameras and has to cope with an enormous data rate to be processed in real-time. Hence, it is desirable to move image processing tasks into the camera to reduce the amount of data which has to be transferred to the (central) image processing system. The idea is to transfer relevant information only, i.e., features of the image instead of the raw image data from the sensor. These features are then further processed. In this paper a color line-scan camera for line rates up to 100 kHz is presented. The camera is based on a commercial CMOS (complementary metal oxide semiconductor) area image sensor and a field programmable gate array (FPGA). It implements extraction of image features which are well suited to detect print flaws like blotches of ink, color smears, splashes, spots and scratches. The camera design and several image processing methods implemented on the FPGA are described, including flat field correction, compensation of geometric distortions, color transformation, as well as decimation and neighborhood operations.
Requirements for contemporary print inspection systems for industrial applications include, among others, high throughput, examination of fine details of the print, and inspection from various perspectives and different spectral sensitivity. Therefore, an optical inspection system for such tasks has to be equipped with several high-speed/high-resolution cameras, each acquiring hundreds of megabytes of data per second. This paper presents an inspection system which meets the given requirements by exploiting data parallelism and algorithmic parallelism. This is achieved by using complex field-programmable gate arrays (FPGA) for image processing. The scalable system consists of several processing modules, each representing a pair of a FPGA and a digital signal processor. The main chapters of the paper focus on the functionality implemented in the FPGA. The image processing algorithms include flat-field correction, lens distortion correction, image pyramid generation, neighborhood operations, a programmable arithmetic unit, and a geometry unit. Due to shortage of on-chip memory, a multi-port memory concept for buffering streams of data between off-chip and on-chip memories is used. Furthermore, performance measurements of the processing module are presented.
In this paper we present a new bus protocol satisfying extreme real time demands. It has been applied to a high
performance quality inspection system which can involve up to eight sensors of various types. Thanks to the modular
configuration this multi-sensor inspection system acts on the outside as a single sensor image processing system.
In general, image processing systems comprise three basic functions (i) image acquisition, (ii) image processing and (iii)
output of processed data. The data transfers for these three fundamental functions can be accomplished either by
individual bus systems or by a single bus. In case of using a single bus the system complexity of the implementation, i.e.
Development of protocols, hardware employment and EMC technical considerations, is far smaller. An important goal
of the new protocol design is to support extremely fast communication between individual processing modules. For
example, the input data (image acquisition) is transferred in real time to individual processing modules. Concurrent to
this communication the processed data are being transferred to the output module. Therefore, the key function of this
protocol is to realize concurrent data paths (data rates over 1.2 Gbit/s) by using principles of pipeline architectures and
methods of time division multiplex.
Moreover, the new bus protocol enables concurrent data transfers via a single bus system. In this paper the function of
the new bus protocol including hardware layout and innovative bus arbiter are described in details.
The function of image processing systems usually consists of three basic steps: (i) image acquisition, (ii) image processing, and (iii) output of the results with possible intermediate image data presentation. 'Off the shelf' architectures, used, e.g., in the standard personal computers, achieve the necessary bus performance in the required real time only in few inline applications. Highly innovative bus systems with pipeline architectures are therefore needed for ensuring huge data throughput in the specifically oriented optical quality inspection systems. The imaging systems for quality inline inspection have to meet specific requirements on data acquisition and communications, mainly: -to ensure a connection of various sensors of heterogeneous nature, such as area scan cameras, line scan cameras, fast CMOS cameras with selectable readout capabilities, -to incorporate carefully designed high speed communication buses between the processing nodes and the sensor adapters, -to manifest a good balance between real time behavior, flexibility, performance and cost efficiency. We have developed innovative real time communication buses for the real-time imaging system designed in the ARC Seibersdorf research Ltd. and used in optical quality inspection of printed matter. In this paper we address in details: -the hardware layout and the individual functions of the innovative buses, -the evaluation of the behavior of the buses in a quality inspection system with extreme real-time demands. All benefits are explained and numerically documented that can be of interest for reader dealing with other applications.