Biological studies suggest that neurons in the mammalian retina accomplish a dynamic segmentation of the visual input. When activated by large, high contrast spots, retinal spike trains exhibit high frequency oscillations in the upper gamma band, between 60 to 120 Hz. Despite random phase variations over time, the oscillations recorded from regions responding to the same spot remain phase locked with zero lag whereas the phases recorded from regions activated by separate spots rapidly become uncorrelated. Here, a model of the mammalian retina is used to explore the segmentation of high contrast, gray-scale images containing several well-separated objects. Frequency spectra were computed from lumped spike trains containing 2×2 clusters of neighboring retinal output neurons. Cross-correlation functions were computed between all cell clusters exhibiting significant peaks in the upper gamma band. For each pair of oscillatory cell clusters, the cross-correlation between the lumped spike trains was used to estimate a functional connectivity, given by the peak amplitude in the upper gamma band of the associated frequency spectra. There was a good correspondence between the largest eigenvalues/eigenvectors of the resulting sparse functional connectivity matrix and the individual objects making up the original image, yielding an overall segmentation comparable to that generated by a standard watershed algorithm.

Inertial navigation systems (INS) incorporating three mutually orthogonal accelerometers and three mutually orthogonal gyroscopes are integrated with global positioning systems (GPS) to provide reliable and accurate positioning information for vehicular navigation. Because of their high reliability and accuracy, ring laser gyroscopes (RLG) and fiber optic gyroscopes (FOG) are usually utilized inside most of the present INS. However, bias drift at the output of these optical gyroscopes may deteriorate the performance of the overall INS/GPS navigation system. This paper introduces a method to enhance the performance of optical gyros in two phases. The first phase utilizes wavelet multi-resolution analysis to band limit the gyro measurement and improves its signal-to-noise ratio. The second phase employs radial-basis function (RBF) neural networks to predict drift errors. The drift model provided by the RBF network is established using the gyro raw measurement and time as inputs and provides the drift error at its output. The RBF neural networks are utilized in this study since they generally have simpler architecture and faster training procedure than other neural network types. The proposed method is applied to E-core 2000 FOG (KVH Industries Inc., Rhode Island, USA).

The basic task of mathematical processing of reflectance spectra is the calculation of the dielectric function of the substance, which describe response of a crystal to an external electromagnetic field. The most modern and perspective way of the solution of this task is the dispersion analysis [Lorentz model (LM)]. However LM requires large volume of computing works at phonons optimum parameters selection. The rapid computer facilities development promotes overcoming of this difficulty. However without application of effective methods of optimization practically it is impossible to execute DA for composite reflectance spectra. The efficiency GA strongly depends on such details, as a solutions coding method, genetic operations embodying, selection mechanisms, other algorithm parameters adjustment, success criterion. In this paper we offer modification GA for the solution of the reflectance spectra processing problem and results of the obtained algorithm work.

This paper illustrates the use of genetic algorithms (GA) in optimizing mask and illumination source geometries for lithographic imaging systems. The main goal of the proposed optimization process is to find optimum conditions for the generation of certain features like lines and spaces patterns or arrays of contact holes by optical projection lithography. Therefore, different optical resolution enhancement techniques, such as optical proximity correction (OPC) by sub-resolution assists, phase shift masks, and off-axis illumination techniques are combined and mutually optimized. This paper focuses on improving both the genetic algorithm's settings and the representation of the mask and source geometries. It is shown that these two issues have a significant impact on the convergence behavior of the GA. Different representation types for the mask and source geometry are introduced, and their advantages and problems are discussed. One of the most critical tasks in formulating the optimization problem is to set up an appropriate fitness function. In our case, the fitness function consists of five sub-functions, which ensure valid geometries, correct feature dimensions, a stable process for different defocus settings, the mask's manufacturability and inspectability, and that no other features besides the specified target are printed. In order to obtain a stable and fast convergence these criteria have to be assessed. Different weight settings are introduced and their impact on the convergence behavior is discussed. Several results show the potential of the proposed approach and directions for further improvements.

This paper presents a learning based codebook design algorithm for vector quantization of digital images using evolution strategies (ES). This technique embeds evolution strategies into the standard competitive learning vector quantization algorithm (CLVQ) and efficiently overcomes its problems of under-utilization of neurons and initial codebook dependency. The embedding of ES greatly increases the algorithm’s capability of avoiding the local minimums, leading to global optimization. Experimental results demonstrate that it can obtain significant improvement over CLVQ and other comparable algorithms in image compression applications. In comparison with the FSLVQ and KSOM algorithms, this new technique is computationally more efficient and requires less training time.

In this paper, we discuss two important topics: the first one, soft computing, is directly related to the digital image processing, while the second one, Small Systems Integration (SSI), is a broad-range extension of the first one, dealing with such interdisciplinary subjects, as optics, electronics, mechanics, materials, and heat management.

In multispectral imaging, automated cross-spectral (band-to-band) image registration is difficult to achieve with a reliability approaching 100%. This is particularly true when registering infrared to visible imagery, where contrast reversals are common and similarity is often lacking. Algorithms that use mutual information as a similarity measure have been shown to work well in the presence of contrast reversal. However, weak similarity between the long-wave infrared (LWIR) bands and shorter wavelengths remains a problem. A method is presented in this paper for registering multiple images simultaneously rather than one pair at a time using a multivariate extension of the mutual information measure. This approach improves the success rate of automated registration by making use of the information available in multiple images rather than a single pair. This approach is further enhanced by including a cyclic consistency check, for example registering band A to B, B to C, and C to A. The cyclic consistency check provides an automated measure of success allowing a different combination of bands to be used in the event of a failure. Experiments were conducted using imagery from the Department of Energy’s Multispectral Thermal Imager satellite. The results show a significantly improved success rate.

The inherent nonuniformity in the photoresponse and readout-circuitry of the individual detectors in infrared focal-plane-array imagers result in the notorious fixed-pattern noise (FPN). FPN generally degrades the performance of infrared imagers and it is particularly problematic in the midwavelength and longwavelength infrared regimes. In many applications, employing signal-processing techniques to combat FPN may be preferred over hard calibration (e.g., two-point calibration), as they are less expensive and, more importantly, do not require halting the operation of the camera. In this paper, a new technique that uses knowledge of global motion in a video sequence to restore the true scene in the presence of FPN is introduced. In the proposed setting, the entire video sequence is regarded as an output of a motion-dependent linear transformation, which acts collectively on the true scene and the unknown bias elements (which represent the FPN) in each detector. The true scene is then estimated from the video sequence according to a minimum mean-square-error criterion. Two modes of operation are considered. First, we consider non-radiometric restoration, in which case the true scene is estimated by performing a regularized minimization, since the problem is ill-posed. The other mode of operation is radiometric, in which case we assume that only the perimeter detectors have been calibrated. This latter mode does not require regularization and therefore avoids compromising the radiometric accuracy of the restored scene. The algorithm is demonstrated through preliminary results from simulated and real infrared imagery.

Recursive partitioned architectures in the spatial domain and block-based discrete transform domain algorithms are developed to retrieve information from digital image data. In the former case, the data is represented in forms which facilitate efficient optimization of the objective criterion. In the latter case, discrete, real, and circular Fourier transforms of the data blocks are filtered by coefficients chosen in the discrete frequency domain. This has led to improvements in the feature detection and localization process. In this paper, recursive and iterative approaches are applied to achieve the restoration of digital images from linearly degraded samples. In addition, block-based algorithms are employed to segment images.

In this paper we introduce the affine spatial overlap operation defined as a generalization of the two-dimensional convolution and cross-correlation operations. Our attention is focused on the rotational overlap operation and some of its mathematical properties as well as its application to shape description. Based on the auto-rotational overlap operation, a one-dimensional signature is proposed as a shape representation for planar binary objects with bounded support. We provide illustrative examples of its digital computation using different binary objects. In addition, physical realization of the 2-D rotational overlap operation is demonstrated with a hybrid optical-digital system for real time processing. The experimental setup uses a microcomputer controlled high-precision rotatory stage for performing analog rotations at the input plane of an incoherent two lens correlator architecture.

Adaptive nonlinear filters based on nonparametric Spearman’s correlation between ranks of an input scene computed in a moving window and ranks of a target for illumination-invariant pattern recognition are proposed. Several properties of the correlations are investigated. Their performance for detection of noisy objects is compared to the conventional linear correlation in terms of noise robustness and discrimination capability. Computer simulation results for a test image corrupted by mixed additive and impulsive noise are provided and discussed.

Multiple description video coding is one method that can be used to reduce detrimental effects caused by transmission over lossy packet networks. In a multiple description system, a video sequence is segmented into two or more complimentary streams in such a way that each stream is independently decodable. When combined, the streams provide the highest level of quality, yet if one of the streams is lost or delivered late the video can be played out with only a slight reduction in overall quality. Each approach to multiple description coding consists of a tradeoff between compression efficiency and robustness. How efficiently each method achieves this tradeoff depends on the level of quality and robustness desired and on the characteristics of the video itself. Previous approaches to multiple description coding have made the assumption that a single segmentation method would be used for an entire sequence. Yet, the optimal method of segmentation can vary depending on the goals of the system, it can change over time, and it can vary within a frame. This work introduces a unique approach to multiple description coding through the use of adaptive segmentation. By selecting from a set of segmentation methods, the system adapts to the local characteristics of the video and maximizes tradeoff efficiency. We present an overview of this system and analyze its performance on real video sequences.

Computer networks and the internet have taken an important role in modern society. Together with their development, the need for digital video transmission over these networks has grown. To cope with the user demands and limitations of the network, compression of the video material has become an important issue. Additionally, many video-applications require flexibility in terms of scalability and complexity (e.g. HD/SD-TV, video-surveillance). Current ITU-T and ISO/IEC video compression standards (MPEG-x, H.26-x) lack efficient support for these types of scalability. Wavelet-based compression techniques have been proposed to tackle this problem, of which the Motion Compensated Temporal Filtering (MCTF)-based architectures couple state-of-the-art performance with full (quality, resolution, and frame-rate) scalability. However, a significant drawback of these architectures is their high complexity. The computational and memory complexity of both spatial domain (SD) MCTF and in-band (IB) MCTF video codec instantiations are examined in this study. Comparisons in terms of complexity versus performance are presented for both types of codecs. The paper indicates how complexity scalability can be achieved in such video-codecs, and analyses some of the trade-offs between complexity and coding performance. Finally, guidelines on how to implement a fully scalable video-codec that incorporates quality, temporal, resolution and complexity scalability are proposed.

A wide range of image processing studies are based on the extraction of texture features, the analysis of input data and the identification and design of appropriate classifiers given a particular application, for instance, in the fields of industrial inspection, remote sensing, medicine or biology amongst others. In this paper, we introduce a novel generalized classification framework for texture imagery based on a novel building blocks system architecture and present the advantages of such a system to tackle a variety of image analysis problems at the same time of obtaining good classification performances. Firstly, an overview of the system architecture is described from the texture feature extraction module to the data analysis and the classification building blocks. Thus, we obtain an optimized and generic classification framework which is highly flexible due to its scalable building blocks system approach and provides the facility to extend easily the study obtained for textural images to other kind of imagery. The results of this generalized classification framework are validated using imagery from two different application fields where texture plays a key role. The first one is in the field of remote sensing for agriculture crops classification and the second one, in the area of non-destructive industrial inspection.

Recently available commercial high-resolution satellite imaging sensors provide an important source for urban remote sensing applications. The high spatial image resolution reveals very fine details in urban areas and greatly facilitates the extraction of urban-related features such as roads, buildings, and vehicles. Since many urban land cover types have significant spectral overlap, structural information obtained using mathematical morphologic operators can provide complementary information to improve discrimination of different urban features. Here we present research demonstrating new applications of mathematical morphology for urban feature extraction from high-resolution satellite imagery. For image preprocessing, an alternating sequential filter is used to eliminate small spatial-scale disturbances to facilitate the extraction of larger-scale structures. For road extraction, directional morphological filtering is exploited to mask out those structures shorter than the distance of a typical city block. For building extraction, a recently introduced concept called the differential morphological profile (DMP) is used to generate building and shadow hypotheses. For vehicle detection, a morphological shared-weight neural network is used to classify image pixels on roads into target and non-target. Thus, mathematical morphology has a wide variety of useful applications for urban feature extraction from high-resolution satellite imagery.

Automatic delineation of buildings is very attractive for both civilian and military applications. Such applications include general mapping, detection of unauthorized constructions, change detection, etc. For military applications, high demand exists for accurate building change updates, covering large areas, and over short time periods. We present two algorithms coupled together. The height image algorithm is a fast coarse algorithm operating on large areas. This algorithm is capable of defining blocks of buildings and regions of interest. The point-cloud algorithm is a fine, 3D-based, accurate algorithm for building delineation. Since buildings may be separated by alleys, whose width is similar or narrower than the LADAR resolution, the height image algorithm marks those crowded buildings as a single object. The point-cloud algorithm separates and accurately delineates individual building boundaries and building sub-sections utilizing roof shape analysis in 3D. Our focus is on the ability to cover large areas with accuracy and high rejection of non-building objects, like trees. We report a very good detection performance with only few misses and false alarms. It is believed that LADAR measurements, coupled with good segmentation algorithms, may replace older systems and methods that require considerable manual work for such applications.

This paper proposed a knowledge-based methodology for determining the resolvability of N occluded vehicles seen in a monocular image sequence. The resolvability of each vehicle is determined by: firstly, deriving the relationship between the camera position and the number of vertices of a projected cuboid on the image; secondly, finding the direction of the edges of the projected cuboid in the image; and thirdly, modeling the maximum number of occluded cuboid edges of which the occluded cuboid is irresolvable. The proposed methodology has been tested rigorously on a number of real world monocular traffic image sequences that involves multiple vehicle occlusions, and is found to be able to successfully determine the number of occluded vehicles as well as the resolvability of each vehicle. We believe the proposed methodology will form the foundation for a more accurate traffic flow estimation and recognition system.

Advances in hyperspectral sensor technology increasingly provide higher resolution and higher quality data for the accurate generation of terrain categorization/classification (TERCAT) maps. The generation of TERCAT maps from hyperspectral imagery can be accomplished using a variety of spectral pattern analysis algorithms; however, the algorithms are sometimes complex, and the training of such algorithms can be tedious. Further, hyperspectral imagery contains a voluminous amount of data with contiguous spectral bands being highly correlated. These highly correlated bands tend to provide redundant information for classification/feature extraction computations. In this paper, we introduce the use of wavelets to generate a set of Generalized Difference Feature Index (GDFI) measures, which transforms a hyperspectral image cube into a derived set of GDFI bands. A commonly known special case of the proposed GDFI approach is the Normalized Difference Vegetation Index (NDVI) measure, which seeks to emphasize vegetation in a scene. Numerous other band-ratio measures that emphasize other specific ground features can be shown to be a special case of the proposed GDFI approach. Generating a set of GDFI bands is fast and simple. However, the number of possible bands is capacious and only a few of these “generalized ratios” will be useful. Judicious data mining of the large set of GDFI bands produces a small subset of GDFI bands designed to extract specific TERCAT features. We extract/classify several terrain features and we compare our results with the results of a more sophisticated neural network feature extraction routine.

Efficient analysis of medical images to assist physician’s decision making is an important task. However, the analysis of such images often requires sophisticated segmentation and classification algorithms. An approach to speed up these time consuming operations is to use parallel processing. In this paper a new parallel system for medical image analysis is presented. The system combines distributed and shared memory architectures using MPI and the inter-process communication switching mechanism (IPC). MPI is used to communicate between nodes and shared-memory IPC is used to perform shared memory operations among processors within a node. We show how to map a clinical endoscopic image analysis algorithm efficiently onto this architecture. This results in an implementation with significant runtime savings.

In this paper, a new intelligent surveillance system for robust detection and tracking of a moving person by using the pan/tilt-embedded stereo camera system is suggested and implemented. In the proposed system, face coordinates of a target person is detected from the sequential input stereo image pairs by using the YCbCr color model and phase-type correlation methods and then, using this data as well as the geometric information of the stereo tracking system, distance to the target from the stereo camera and 3-dimensional location information of a target person are extracted. Basing on these extracted data the pan/tilt system embedded in the stereo camera is controlled to adaptively track a moving person and as a result, moving trajectory of a target person can be obtained. From some experiments using 780 frames of the sequential stereo image pairs, it is analyzed that standard deviation of the position displacement of the target in the horizontal and vertical directions after tracking is kept to be very low value of 1.5, 0.42 for 780 frames on average, and error ratio between the measured and computed 3D coordinate values of the target is also kept to be very low value of 0.5% on average. These good experimental results suggest a possibility of implementation of a new stereo target tracking system having a high degree of accuracy and a very fast response time with this proposed algorithm.

Registration of individual images remains a significant problem in the generation of accurate images collected using coherent imaging systems. An investigation of the performance of eight distinct image registration algorithms was conducted using data collected from a coherent optical imaging system developed by the Air Force Research Laboratories, Sensors Division, ARFL/SNJT. A total of 400 images of three distinct scenes were collected by SRJT and made available to the Air Force Institute of Technology (AFIT) for this study. Scenery was collected at 3 and 10 kilometers of wheeled vehicles supporting resolution and uniform target boards. The algorithms under study were developed by scientists and engineers at AFRL, and had varying levels of performance in terms of image mis-registration and execution time. These eight algorithms were implemented on a general-purpose computer running the MATLAB simulation environment. The algorithms compared included: block-match, cross-correlation, cross-search, directional-search, gradient-based, hierarchical-block, three-step, and vector-block methods. It was found that the cross-correlation, gradient-based and vector-block search techniques typically had the lowest error metric. The vector-block and cross-correlation methods proved to have the fastest execution times, while not suffering significant error degradation when estimating the registration shift of the test images.

3-D imaging LADAR systems that are capable of rapid frame acquisition may suffer from a loss of range resolution due to the duration of the pulse transmitted to the target. Because of the tradeoff between the requirement to produce sufficient illumination and the desire to obtain high range resolution, these systems may sacrifice range resolution in favor of improved signal to noise ratio of the detected signal. In this paper, deconvolution techniques are employed in order to obtain improved range resolution from a sequence of laser radar return images collected at extremely high speeds. The study pursued in this paper explores the degree to which range resolution can be improved in the presence of photon and speckle noise. Noise amplification in the deconvolution process serves to degrade the signal to noise ratio of the reconstructed laser radar return images. The performance of the reconstruction algorithm is quantified through the estimation of the probability of detection. It will be shown that both the probability of detection and range resolution can be improved in imaging laser radar systems.

Recently surveillance and Automatic Target Recognition (ATR) applications are increasing as the cost of computing power needed to process the massive amount of information continues to fall. This computing power has been made possible partly by the latest advances in FPGAs and SOPCs. In particular, to design and implement state-of-the-Art electro-optical imaging systems to provide advanced surveillance capabilities, there is a need to integrate several technologies (e.g. telescope, precise optics, cameras, image/compute vision algorithms, which can be geographically distributed or sharing distributed resources) into a programmable system and DSP systems. Additionally, pattern recognition techniques and fast information retrieval, are often important components of intelligent systems. The aim of this work is using embedded FPGA as a fast, configurable and synthesizable search engine in fast image pattern recognition/retrieval in a distributed hardware/software co-design environment. In particular, we propose and show a low cost Content Addressable Memory (CAM)-based distributed embedded FPGA hardware architecture solution with real time recognition capabilities and computing for pattern look-up, pattern recognition, and image retrieval. We show how the distributed CAM-based architecture offers a performance advantage of an order-of-magnitude over RAM-based architecture (Random Access Memory) search for implementing high speed pattern recognition for image retrieval. The methods of designing, implementing, and analyzing the proposed CAM based embedded architecture are described here. Other SOPC solutions/design issues are covered. Finally, experimental results, hardware verification, and performance evaluations using both the Xilinx Virtex-II and the Altera Apex20k are provided to show the potential and power of the proposed method for low cost reconfigurable fast image pattern recognition/retrieval at the hardware/software co-design level.

A weakened portion of an artery in the brain leads to a medical condition known as a cerebral aneurysm. A subarachnoid hemorrhage (SAH) occurs when an aneurysm ruptures. For those individuals suspected of having a SAH, a computerized tomography (CT) scan of the brain usually demonstrates evidence of the bleeding. However, in a considerable portion of people, the CT scan is unable to detect the blood that has escaped from the blood vessel. Recent studies have indicated nearly 30% of patients with a SAH are initially misdiagnosed. For circumstances when a SAH is suspected despite a normal CT scan, physicians make the diagnosis of SAH by performing a spinal tap. A spinal tap uses a needle to sample the cerebrospinal fluid (CSF) collected from the patient’s lumbar spine. However, it is also possible for blood to be introduced into the CSF as a result of the spinal tap procedure. Therefore, an effective solution is required to help medical personnel differentiate between the blood that results from a tap and that from a ruptured aneurysm. In this paper, the development of a prototype is described which is sensitive and specific for measuring bilirubin in CSF, hemorrhagic-CSF and CSF-like solutions. To develop this instrument a combination of spectrophotometric analysis, custom data analysis software and other hardware interfaces are assembled that lay the foundation for the development of portable and user-friendly equipment suitable for assisting trained medical personnel with the diagnosis of a ruptured cerebral aneurysm.

Optical systems designed for some defense, environmental, and commercial remote-sensing applications must simultaneously have a high dynamic range, high sensitivity, and low noise-equivalent contrast. We have adapted James Janesick’s photon transfer technique for characterizing the noise performance of an electron multiplication CCD (EMCCD), and we have developed methods for characterizing performance parameters in a lab environment. We have defined a new figure of merit to complement the traditionally used dynamic range that quantifies the usefulness of EMCCD imagers. We use the results for EMCCDs to predict their performance with hyperspectral and multispectral imaging systems.

We present a framework for optimal rate allocation to image subbands to minimize the distortion in the joint compression and classification of JPEG2000-compressed images. The distortion due to compression is defined as a weighted linear combination of the mean-square error (MSE) and the loss in the Bhattacharyya distance (BD) between the class-conditional distributions of the classes. Lossy compression with JPEG2000 is accomplished via deadzone uniform quantization of wavelet subbands. Neglecting the effect of the deadzone, expressions are derived for the distortion in the case of two classes with generalized Gaussian distributions (GGDs), based on the high-rate analysis of Poor. In this regime, the distortion function takes the form of a weighted MSE (WMSE) function, which can be minimized using reverse water-filling. We present experimental results based on synthetic data to evaluate the efficacy of the proposed rate allocation scheme. The results indicate that by varying the weight factor balancing the MSE and the Bhattacharyya distance, we can control the trade-off between these two terms in the distortion function.

This paper describes a video surveillance system which is composed of three key components, smart cameras, a server, and clients, connected through IP-networks in wired or wireless configurations. The system has been designed so as to protect the privacy of people under surveillance. Smart cameras are based on JPEG 2000 compression where an analysis module allows for events detection and regions of interest identification. The resulting regions of interest can then be encoded with better quality and scrambled. Compressed video streams are scrambled and signed for the purpose of privacy and data integrity verification using JPSEC compliant methods. The same bitstream may also be protected for robustness to transmission errors based on JPWL compliant methods. The server receives, stores, manages and transmits the video sequences on wired and wireless channels to a variety of clients and users with different device capabilities, channel characteristics and preferences. Use of seamless scalable coding of video sequences prevents any need for transcoding operations at any point in the system.

JPEG 2000 Part 10 is a new work part of the ISO/IEC JPEG Committee dealing with the extension of JPEG 2000 technologies to three-dimensional data. One of the issues in Part 10 is the ability to encode floating point datasets. Many Part 10 use cases come from the scientific and engineering communities, where floating point data is often produced either from numerical simulations or from remote sensing instruments. This paper presents the technologies that are currently being developed to accommodate this Part 10 requirement. The coding of floating point datasets with JPEG 2000 requires two changes to the coding pipeline. Firstly, the wavelet transformation stage is optimized to correctly decorrelate data represented with the IEEE 754 floating point standard. Special IEEE 754 floating point values like Infinities and NaN's are signaled beforehand as they do not correlate well with other floating point values. Secondly, computation of distortion measures on the encoder side is performed in floating point space, rather than in integer space, in order to correctly perform rate allocation. Results will show that these enhancements to the JPEG 2000 coding pipeline lead to better compression results than Part 1 encoding where the floating point data had been retyped as integers.

JPEG 2000 Part 10 is a new work part of the ISO/IEC JPEG Committee dealing with the extension of JPEG 2000 technologies to three-dimensional data. One of the issues in Part 10 is the ability to encode non-uniform data grids having variable resolution across its domain. Some parts of the grid can be more finely sampled than others in accordance with some pre-specified criteria. Of particular interest to the scientific and engineering communities are variable resolution grids resulting from a process of adaptive mesh refinement of the grid cells. This paper presents the technologies that are currently being developed to accommodate this Part 10 requirement. The coding of adaptive mesh refinement grids with JPEG 2000 works as a two step process. In the first pass, the grid is scanned and its refinement structure is entropy coded. In the second pass, the grid samples are wavelet transformed and quantized. The difference with Part 1 is that wavelet transformation must be done over regions of irregular shape. Results will be shown for adaptive refinement grids with cell-centered or corner-centered samples. It will be shown how the Part 10 coding of an adaptive refinement grid is backwards compatible with a Part 1 decoder.

This paper describes an objective comparison of the image quality of different encoders. Our approach is based on estimating the visual impact of compression artifacts on perceived quality. We present a tool that measures these artifacts in an image and uses them to compute a prediction of the Mean Opinion Score (MOS) obtained in subjective experiments. We show that the MOS predictions by our proposed tool are a better indicator of perceived image quality than PSNR, especially for highly compressed images. For the encoder comparison, we compress a set of 29 test images with two JPEG encoders (Adobe Photoshop and IrfanView) and three JPEG2000 encoders (JasPer, Kakadu, and IrfanView) at various compression ratios. We compute blockiness, blur, and MOS predictions as well as PSNR of the compressed images. Our results show that the IrfanView JPEG encoder produces consistently better images than the Adobe Photoshop JPEG encoder at the same data rate. The differences between the JPEG2000 encoders in our test are less pronounced; JasPer comes out as the best codec, closely followed by IrfanView and Kakadu. Comparing the JPEG- and JPEG2000-encoding quality of IrfanView, we find that JPEG has a slight edge at low compression ratios, while JPEG2000 is the clear winner at medium and high compression ratios.

In this paper, we present the current status of the JPWL standardization work item. JPWL is an extension of the JPEG 2000 baseline specification in order to enable the efficient transmission of JPEG 2000 codestream over an error-prone network. In particular, JPWL supports a set of tools and methods for error protection and correction such as Forward Error Correcting (FEC) codes, Unequal Error Protection (UEP), and data partitioning and interleaving. We then evaluate the performance of the JPWL Error Protection Block (EPB) tool. We consider two configurations of EPB: to protect the Main and Tile-part headers, or to protect the whole codestream using UEP. Experimental results show a significant quality improvement when using EPB compared to baseline JPEG 2000.

In this paper, we first review the on-going JPSEC standardization activity. Its goal is to extend the baseline JPEG 2000 specification to provide a standardized framework for secure imaging, in order to support tools needed to secure digital images, such as content protection, data integrity check, authentication, and conditional access control. We then present two examples of JPSEC tools. The first one is a technique for secure scalable streaming and secure transcoding. It allows the protected JPSEC codestream to be transcoded while preserving the protection, i.e. without requiring unprotecting (e.g. decrypting) the codestream. The second one is a technique for conditional access control. It can be used for access control by resolution or quality, but also by regions of interest.

In Motion JPEG2000 each frame of a video is encoded with JPEG2000. In JPEG2000, the post-coding rate distortion optimisation (PCRD-opt) technique is recommended for rate control. The advantage of using PCRD-opt for rate control is that at any given rate the optimum visual quality can be achieved for that rate. However, implied in the optimisation procedure each code-block is over-coded. Since the encoding process in JPEG2000 is one of the slower operations, too much over-coding is undesirable especially in video applications. Over-coding also results in more memory than necessary for buffering the intermediate compressed data. This paper proposes a technique that minimises the amount of over-coding in the arithmetic encoding step while maintaining image quality. This is achieved by terminating the encoding process of a code-block once terminating conditions have been met. The terminating conditions are a set of rules based on the coding of the current code-block and other heuristics. The terminating conditions will be discussed fully in detail in this paper. Visually the image quality using the proposed rate control scheme is comparable to that of using PCRD-opt. Furthermore, the proposed rate control strategy produces a fully compliant JPEG2000 codestream.

In this paper, we present an MPEG-4 contents streaming system and propose MPEG-4 contents streaming scheme by using priority. The presented streaming system which consists of a server and a client supports MPEG-4 contents compliant with ISO/IEC 14496-1 and enables a user to interact with MPEG-4 contents over IP networks. The server consists of GUI, Server Management Layer, Sync Layer, and Delivery Layer. The client supports to display MPEG-4 contents stored in local storage or received through IP networks. Moreover, we propose an MPEG-4 contents streaming scheme that the object a user prefers to watch is sent first by increasing priority and objects with low priority are dropped at a server side when network bandwidth is not enough to transmit all objects that are supposed to appear in the scene. We made experiment of the proposed scheme with the presented MPEG-4 contents streaming system, and the experiment results are shown in this paper. If we use the proposed scheme for MPEG-4 contents streaming, it is possible for a user to watch a video of interest in high quality and video of indifference in low quality.

Part 7 of MPEG-21 entitled Digital Item Adaptation (DIA), is an emerging metadata standard defining protocols and descriptions enabling content adaptation for a wide variety of networks and terminals, with attention to format-independent mechanisms. The descriptions standardized in DIA provide a standardized interface not only to a variety of format-specific adaptation engines, but also to format-independent adaptation engines for scalable bit-streams. A fully format-independent engine contains a decision-taking module operating in a media-type and context independent manner, cascaded with a bit-stream adaptation module that models the bit-stream adaptation process as an XML transformation operating on a high-level syntax description of the bit-stream, with parameters derived from decisions taken. In this paper, we describe the DIA descriptions and underlying mechanisms that enable such fully format-independent scalable bit-stream adaptation. Further, a new model-based, compact and lightweight transformation language for scalable bit-streams is described for use in the bit-stream adaptation module. Fully format-independent adaptation mechanisms lead to universal adaptation engines that substantially reduce adoption costs for new media types and formats because the same delivery and adaptation infrastructure can be used for different types of scalable media, including proprietary and encrypted content.

In this paper we outline a post-processing system for compressed video sources, aimed at reducing the visibility of coding artifacts. To achieve optimal video quality for compressed sources, it addresses artifact reduction and video enhancement functions as well as their interdependency. The system is based on the Unified Metric for Digital Video Processing (UMDVP), a quality metric that estimates the level of coding artifacts on a per-pixel basis. Experiments on MPEG-2 encoded video sequences showed significant improvement in picture quality compared to systems that do not have UMDVP control or that do not consider the interdependency between artifact reduction and video enhancement.

Within the last years, the enormous growth of Internet based communication as well as the rapid increase of available processing power has lead to the widespread use of multimedia streaming as a means to convey information. This work aims at providing an open architecture designed to support scalable streaming to a large number of clients using application layer multicast. The architecture is based on media relay nodes that can be deployed transparently to any existing media distribution scheme, which can support media streamed using the RTP and RTSP protocols. The architecture is based on overlay networks at application level, featuring rate adaptation mechanisms for responding to network congestion.

Rate and distortion models can play a very important role in real-time video encoding, since they can be used to obtain near optimal operation performance in terms of the RD tradeoff without the drawback of having to encode multiple times the same VOP to find the best combination of coding parameters. In the context of object-based video encoding, notably in MPEG-4 video encoding, rate and distortion models characterize the relation between the average number of bits/pixel to code a given Video Object Plane (VOP), the average VOP distortion, and the relevant coding parameters. These models are usually defined in terms of rate-quantization (RQ), distortion-quantization (DQ), and rate-distortion (RD) functions. This paper addresses the problem of rate and distortion modeling for Intra and Inter coding in the context of object-based MPEG-4 video encoding. In the case of Intra coding, the VOP to encode does not depend on other past or future VOPs; therefore, its rate and distortion characteristics depend exclusively on the current quantizer parameter(s) and VOP statistics. In the case of Inter coding, the rate and distortion functions depend not only on the current VOP but also on its reference VOP(s); therefore the rate and distortion functions become bidimensional and consequently more difficult to estimate during encoding. In this paper, a new approach is proposed where the rate and distortion functions for Inter coding are modeled as one-dimensional functions plus an adaptation term.

Modern video coding applications require transmission of video data over variable-bandwidth channels to a variety of terminals with different screen resolutions and available computational power. Scalable video coding is needed to optimally support these applications. Recently proposed wavelet-based video codecs employing spatial domain motion compensated temporal filtering (SDMCTF) provide quality, resolution and frame-rate scalability while delivering compression performance comparable to that of the state-of-the-art non-scalable H.264-codec. These codecs require scalable coding of the motion vectors in order to support a large range of bit-rates with optimal compression efficiency. Scalable motion vector coding algorithms based on the integer wavelet transform followed by embedded coding of the wavelet coefficients were recently proposed. In this paper, a new and fundamentally different scalable motion vector codec (MVC) using median-based motion vector prediction is proposed. Extensive experimental results demonstrate that the proposed MVC systematically outperforms the wavelet-based state-of-the-art solutions. To be able to take advantage of the proposed scalable MVC, a rate allocation mechanism capable of optimally dividing the available rate among texture and motion information is required. Two rate allocation strategies are proposed and compared. The proposed MVC and rate allocation schemes are incorporated into an SDMCTF-based video codec and the benefits of scalable motion vector coding are experimentally demonstrated.

A principal challenge for reducing the cost of complex systems-on-chip is to pursue more generic systems for a broad range of products. For this purpose, we explore three new architectural concepts for state-of-the-art video applications. First, we discuss a reusable scalable hardware architecture employing a hierarchical communication network fitting with the natural hierarchy of the application. In a case study, we show that MPEG streaming in DTV occurs at high level, while subsystems communicate at lower levels. The second concept is a software design that scales over a number of processors to enable reuse over a range of VLSI process technologies. We explore this via an H.264 decoder implementation that scales nearly linearly over up to eight processors by applying data partitioning. The third concept is resource-scalability, which is required to satisfy real-time constraints in a system with a high amount of shared resources. An example complexity-scalable MPEG-2 encoder scales the required cycle budget with a factor of three, in parallel with a smooth degradation of quality.

This paper investigates multimedia streaming over wireless local area networks. Physical layer sigmoid analytical models are presented for 802.11a/g and for 2x3 MIMO 802.11n MIMO-based systems are presented. Performance results in a wireless LAN environment are presented for traffic using UDP and TCP transport mechanisms. Packet losses are observed in WLAN environments which affects the overall throughput available. Possibilities for performance improvements with the use of 802.11e and MIMO technologies are discussed. System platform architecture performance issues for wireless video conferencing between Intel PXA27x processor-based handheld platforms are presented and results with retry-limit adaptation are also presented.

Multimedia content distribution through a distributed system, a peer-to-peer (P2P) network for instance, is attractive since it harnesses the resources available with the numerous peers in the network. Another advantage of such a system is that the potentially available resources scale in proportion to the demand as more and more peers join the system. Recent studies have concentrated mainly on such aspects of these distributed networks as querying, indexing, etc. These studies however take for granted the voluntary contribution of resources by peers in the system. Empirical evidence however points to the contrary, i.e. in existing P2P systems, a substantial fraction of peers do not contribute resources to the system, while benefiting from the services it provides at the expense of the contributing peers. In this paper we analyze a P2P system in a game-theoretic setting in which games involving content exchange are played repeatedly. The model takes into account the manner in which a peer adapts his contribution to the system depending on the benefit he has derived from the system so far and expects to derive in the long run. The model enables us to formulate an optimization problem that ields optimal content sharing trategies that a peer should adopt in rder to maximize his net benefit by participating in the system.

H.264/MPEG-4 AVC is the latest international video coding standard. It was jointly developed by the Video Coding Experts Group (VCEG) of the ITU-T and the Moving Picture Experts Group (MPEG) of ISO/IEC. It uses state-of-the-art coding tools and provides enhanced coding efficiency for a wide range of applications, including video telephony, video conferencing, TV, storage (DVD and/or hard disk based, especially high-definition DVD), streaming video, digital video authoring, digital cinema, and many others. The work on a new set of extensions to this standard has recently been completed. These extensions, known as the Fidelity Range Extensions (FRExt), provide a number of enhanced capabilities relative to the base specification as approved in the Spring of 2003. In this paper, an overview of this standard is provided, including the highlights of the capabilities of the new FRExt features. Some comparisons with the existing MPEG-2 and MPEG-4 Part 2 standards are also provided.

We examine various issues related to demonstrating real-time channel adaptive video communications for UAVs using the latest-generation H.264 video compression technology. These issues include among others: real-time channel estimation techniques, real-time data rate adaptation techniques in H.264/AVC, latency in encoding, current encoding speeds, transcoding, and scalable video developments in H.264, all as essential steps along the way. These demonstrations will be conducted in a communication laboratory and a limited operational testing environment.

H.264/MPEG-4 AVC is expected to cover wide range of applications not only for smaller images like CIF/QCIF but also for SDTV/HDTV. As encoding/decoding of larger images requires more number of macroblocks to be processed, any kind of technique for speeding up is indispensable for real-time applications. Pipeline processing is an effective tool for hardware implementation. However, when we apply it to intra prediction, additional constraint will be necessary, which causes a severe loss in coding efficiency. In this paper we propose modifications on intra prediction that enable us to apply pipeline processing with an ignorable level of loss in coding efficiency.

MPEG LA, LLC recently announced terms of a joint patent license for the AVC (a/k/a H.264) Standard (ISO/IEC IS 14496-10: Information technology -- Coding of audio-visual objects -- Part 10: Advanced Video Coding|ITU-T Rec. H.264: Series H: Audiovisual and Multimedia Systems: Infrastructure of audiovisual services -- Coding of moving video: Advanced video coding for generic audiovisual services). Like MPEG LA’s other licenses, the AVC Patent Portfolio License is offered for the convenience of the marketplace as an alternative enabling users to access essential intellectual property owned by many patent holders under a single license rather than negotiating licenses with each of them individually. The AVC Patent Portfolio License includes essential patents owned by Columbia Innovations Enterprises; Electronics and Communications Research Institute (ETRI); France Télécom, société anonyme; Fujitsu Limited; Koninklijke Philips Electronics N.V.; Matsushita Electric Industrial Co., Ltd.; Microsoft Corporation; Mitsubishi Electric Corporation; Robert Bosch GmbH; Samsung Electronics Co., Ltd.; Sharp Kabushiki Kaisha; Sony Corporation; Toshiba Corporation; and Victor Company of Japan, Limited. MPEG LA’s objective is to provide worldwide access to as much AVC essential intellectual property as possible for the benefit of AVC users. Therefore, any party that believes it has essential patents is welcome to submit them for evaluation of their essentiality and inclusion in the License if found essential.

The development of new video processing, new displays, and new modes of dissemination and usage enables a variety of moving picture applications intended for mobile and desktop devices as well as the more conventional platforms. These applications include multimedia as well as traditional video and require novel lighting environments and bit rates previously unplumbed in Moving Picture Experts Group (MPEG) video compression. The migration to new environments poses a methodological challenge to testers of video quality. Both the viewing environment and the display characteristics differ dramatically from those used in well-established subjective testing methods for television. The MPEG Test Committee has adapted the television-centric methodology to the new testing environments. The adaptations that are examined here include: (1) The display of progressive scan pictures in the Common Intermediate Format (CIF at 352x288 pixel/frame) and Quarter CIF (QCIF at 176x144 pixel/frame) as well as other, larger moving pictures requires new ways of testing the subjects including different viewing distances and altered ambient lighting. (2) The advent of new varieties of display technologies suggests there is a need for methods of characterizing them to assure the results of the testing do not depend strongly on the display. (3) The use of non-parametric statistical tests in test data analysis. In MPEG testing these appear to provide rigorous confidence statements more in line with testing experience than those provided by classical parametric tests. These issues have been addressed in a recent MPEG subjective test. Some of the test results are reviewed; they suggest that these adaptations of long-established subjective testing methodology for TV are capable of providing practical and reliable measures of subjective video quality for a new generation of technology.

Finding out the better parameter set (OffsetA and OffsetB) for conducting the de-blocking process of H.264/AVC, is capable of improving visual quality, said eliminating the resultant blocking artifact. Identifying which edges belong to blocking regions relies on the perceptual judging process of human beings. In fact, this subjective assessment may not exactly match existing objective assessing measurements, and the meaning of high PSNR does not always stand for less blocking artifacts. In this paper, we first introduce a new criterion for measuring the block boundary distortion by comparing the source video and the reconstructed video prior to the deblocking process. By jointly optimizing the objective picture quality and the blocky energy, the deblocking parameters decision process can find out a good balance between signal matching and blocky elimination, and therefore, maximize the effect of the built-in deblocking process. In our experiments, the proposed method can efficiently pick a better deblocking parameter set from all 169 possibilities for each coded frame and result in a better visual quality.

The emerging video coding standard, H.264/AVC, exhibits the unprecedented coding performance. Comparing to traditional coders, e.g., MPEG-2 and MEPG-4 ASP, about half bitrate saving is shown in the official verification test. Such outstanding performance makes it become the video compression candidate for the upcoming HD-DVD. As a side effect, it was also blamed that H.264/AVC is much more logically complex and requires more computation power than any of the existing standards. A low-cost and efficient implementation of the international standard hence plays an important role of its success. In this paper, we realize an H.264/AVC baseline decoder by a low-cost DSP processor, i.e., Philips’ TriMedia TM-1300, and illustrate that less computation demand for H.264/AVC decoding becomes feasible by using effective software core. To this end, we first consider different approaches and take advantage of SIMD instruction set to optimize critical time-consuming coding modules, such as the fractional motion compensation, spatial prediction and inverse transform. Next, we also present some other optimization approaches for entropy decoding and in-loop deblocking filtering, even though they cannot get benefits from utilizing SIMD. In our experiments, by exploiting appropriate instruction level parallelism and efficient algorithms, the decoding speed can be improved by a factor of 8~10; a CIF video sequence can be decoded at up to 19.74~28.97 fps on a 166-MHz TriMedia TM-1300 processor compared to 2.40~2.98 fps by the standard reference software.

A fast intra/inter mode decision method using a risk-minimization criterion is proposed to reduce the encoder complexity of the H.264 encoder in this work. The current H.264 reference codes employ exhaustive search to find the best mode that optimizes the rate-distortion performance among all possible intra/inter predictive modes. To develop a fast binary mode decision scheme (i.e. either the inter- or intra-prediction mode to be used), we consider the risk of choosing the wrong predictive mode. If the cost of choosing the wrong mode in terms of the averaged rate-distortion (RD) performance loss is low, then the risk is tolerable. The fast algorithm consists of three steps. First, three features are extracted from the current macroblock to form a 3D feature vector. Second, the feature space is partitioned into three regions, i.e. risk-free, risk-tolerable, and risk-intolerable regions. Finally, depending on the location of the feature vector in the feature space, we can apply mechanisms of different complexities for the final mode decision. The proposed algorithm can select either the correct mode or the wrong mode but with low RD performance degradation. It is demonstrated by experimental results that the proposed algorithm can save approximately 19-25% of the total encoding time of H.264 (JM7.3a) with little degradation in the rate-distortion performance.

In H.264/AVC, an integer 4×4 transform is used instead of traditional float DCT transform due to its low complexity and exact reversibility. Combined with the normalization for the integer transform together, a division-free quantization scheme is used in H.264/AVC. H.264/AVC is the most outstanding video coding standard at far. But at first H.264/AVC targets to low bit-rate coding, and almost all experimental results of the proposals for H.264/AVC are tested at low bit-rate. In the near, experimental results show that 8×8 transform can further improve the coding efficiency on high definition (HD) coding. In this paper a kind of 8×8 low complexity integer transforms are studied and corresponding quantization schemes are developed for HD coding. Compared with 4×4 transform/prediction based coder, the proposed 8×8 based coder can achieve better performance on HD coding while with much lower encoder/decoder complexity.

H.264/AVC is a new specification for digital video coding that aims at a deployment in a lot of multimedia applications, such as video conferencing, digital television broadcasting, and internet streaming. This is for instance reflected by the design goals of the standard, which are about the provision of an efficient compression scheme and a network-friendly representation of the compressed data. Those requirements have resulted in a very flexible syntax and architecture that is fundamentally different from previous standards for video compression. In this paper, a detailed discussion will be provided on how to apply an extended version of the MPEG-21 Bitstream Syntax Description Language (MPEG-21 BSDL) to the Annex B syntax of the H.264/AVC specification. This XML based language will facilitate the high-level manipulation of an H.264/AVC bitstream in order to take into account the constraints and requirements of a particular usage environment. Our performance measurements and optimizations show that it is possible to make use of MPEG-21 BSDL in the context of the current H.264/AVC standard with a feasible computational complexity when exploiting temporal scalability.

H.264/MPEG-4 AVC is the latest video-coding standard jointly developed by the Video Coding Expert Group (VCEG) of ITU-T and the Moving Picture Expert Group (MPEG) of ISO/IEC JTC1. It uses to enhance significantly the compression rate when compared to previous standard in the field, especially the largely adopted MPEG-2 one. However, MPEG-2 is presently overwhelmed used by the industry, and as a consequence most of the existing digital video material is encoded in this standard. This paper discuss efficient way to convert existing MPEG-2 and beyond it MPEG-1 and MPEG-4 visual material, toward H.264, what is a critical factor toward a successful H.264 deployment.

A viable lossy bandwidth compression for hyperspectral imagery is presented. The algorithm is leveraged on the standard JPEG 2000 technology. The component decorrelation of the JPEG 2000 (extension 2) is replaced with a two-level Karhunen-Loeve Transformation (KLT) operation resulting in a reduction in the computation complexity. The set of n² hyperspectral imagery is arranged as an n by n mosaic. Each of the n columns of the mosaic is spectrally uncorrelated via a first-level KLT operation. The resulting n principal component (PC) images for each column are placed next to one another to form an n by n mosaic of PC images. A second-level KLT is then applied to the first four rows of the n by n PC mosaic to approximate a full spectral decorrelation. This approach reduces the computational complexity of the KLT spectral decorrelation process of JPEG 2000 since, 1) it uses a smaller and computationally more feasible KLT matrix (i.e., n by n KLT matrix instead of size n² by n²) and 2) it reduces the number of required computations for spectral decorrelation by a factor of n/4.

For consumer imaging applications, multi-spectral color refers to capturing and displaying images in more than three primary colors in order to achieve color gamuts significantly larger than those produced by RGB devices. In this paper, we describe the building of both a multi-camera recording system and multi-projector display system using off-the-shelf components that, unlike existing multi-camera/projector systems that rely on expensive and time consuming optical alignment of camera/projector views, relies upon the virtual alignment of views performed in software. Once images are properly aligned, the described systems represent recording/display platforms that scale linearly in cost with the number of color primaries where new colors are added