Because of the lack of mammography databases with a large amount of codified images and identified characteristics
like pathology, type of breast tissue, and abnormality, there is a problem for the development of robust systems for
computer-aided diagnosis. Integrated to the Image Retrieval in Medical Applications (IRMA) project, we present an
available mammography database developed from the union of: The Mammographic Image Analysis Society Digital
Mammogram Database (MIAS), The Digital Database for Screening Mammography (DDSM), the Lawrence Livermore
National Laboratory (LLNL), and routine images from the Rheinisch-Westfälische Technische Hochschule (RWTH)
Aachen. Using the IRMA code, standardized coding of tissue type, tumor staging, and lesion description was developed
according to the American College of Radiology (ACR) tissue codes and the ACR breast imaging reporting and data
system (BI-RADS). The import was done automatically using scripts for image download, file format conversion, file
name, web page and information file browsing. Disregarding the resolution, this resulted in a total of 10,509 reference
images, and 6,767 images are associated with an IRMA contour information feature file. In accordance to the respective
license agreements, the database will be made freely available for research purposes, and may be used for image based
evaluation campaigns such as the Cross Language Evaluation Forum (CLEF). We have also shown that it can be
extended easily with further cases imported from a picture archiving and communication system (PACS).
The content of medical images can often be described as a composition of relevant objects with distinct relationships. Each single object can then be represented as a graph node, and local features of the objects are associated as node attributes, e.g. the centroid coordinates. The relations between these objects are represented as graph edges with annotated relational features, e.g. their relative size. Nodes and edges build an attributed relational graph (ARG). For a given setting, e.g. a hand radiograph, a generalization of the relevant objects, e.g. individual bone segments, can be obtained by the statistical distributions of all attributes computed from training images. These yield a structural prototype graph consisting of one attributed node per relevant object and of their relations represented as attributed edges. In contrast to the ARG, the mean and standard deviation of each local or relational feature are used to annotate the prototype nodes or edges, respectively. The prototype graph can then be used to identify the generalized objects in new images. As new image content is represented by hierarchical attributed region adjacency graphs (HARAGs) which are obtained by region-growing, the task of object or scene identification corresponds to the problem of inexact sub-graph matching between a small prototype and the current HARAG. For this purpose, five approaches are evaluated in an example application of bone-identification in 96 radiographs: Nested Earth Mover's Distance, Graph Edit Distance, a Hopfield Neural Network, Pott's Mean Field Annealing and Similarity Flooding. The discriminative power of 34 local and 12 relational features is judged for each object by sequential forward selection. The structural prototypes improve recall by up to 17% in comparison to the approach without relational information.
Content-based image retrieval requires a formal description of visual information. In medical applications, all relevant biological objects have to be represented by this description. Although color as the primary feature has proven successful in publicly available retrieval systems of general purpose, this description is not applicable to most medical images. Additionally, it has been shown that global features characterizing the whole image do not lead to acceptable results in the medical context or that they are only suitable for specific applications. For a general purpose content-based comparison of medical images, local, i.e. regional features that are collected on multiple scales must be used. A hierarchical attributed region adjacency graph (HARAG) provides such a representation and transfers image comparison to graph matching. However, building a HARAG from an image requires a restriction in size to be computationally feasible while at the same time all visually plausible information must be preserved. For this purpose, mechanisms for the reduction of the graph size are presented. Even with a reduced graph, the problem of graph matching remains NP-complete. In this paper, the Similarity Flooding approach and Hopfield-style neural networks are adapted from the graph matching community to the needs of HARAG comparison. Based on synthetic image material build from simple geometric objects, all visually similar regions were matched accordingly showing the framework's general applicability to content-based image retrieval of medical images.
We present an evaluation of methods for the automatic categorization of medical images. The properties of medical images render some otherwise very successful discriminate features for images (e.g. color) inapplicable. Therefore, we evaluate several feature types: texture, structure, and down-scaled representations. The classification is done using a nearest neighbor classifier with various distance measures as well as the automatic combination of classifier results. A corpus of 6,335 images selected arbitrarily from the clinical routine was encoded using a multi-axial, mono-hierarchical code. The reference categorization was done by experienced radiologists familiar with the code. The code's hierarchy allows the analysis of the automatic categorization performance (depending on the features and the classifier used) at different levels of differentiation. Experiments were done for 54 and 57 categories or 70 and 81 categories focussing on radiographs only or for all images, respectively. A maximum classification accuracy of 86% was obtained using the winner-takes-all rule and a one nearest neighbor classifier. Accuracy is increased to 93% and 95% if the correct category is only required to be within the 5 or 10 best matches, respectively. In this case, the best rate of 98% is
obtained. This is sufficient for most applications in content-based image retrieval.
Proc. SPIE. 5150, Visual Communications and Image Processing 2003
KEYWORDS: Content based image retrieval, Databases, Image processing, Computing systems, Feature extraction, Medical imaging, Image retrieval, Image storage, Algorithm development, Picture Archiving and Communication System
We describe a platform for the implementation of a system for content-based image retrieval in medical applications (IRMA). To cope with the constantly evolving medical knowledge, the platform offers a flexible feature model to store and uniformly access all feature types required within a multi-step retrieval approach. A structured generation history for each feature allows the automatic identification and re-use of already computed features. The platform uses directed acyclic graphs composed of processing steps and
control elements to model arbitrary retrieval algorithms. This visually intuitive, data-flow oriented representation vastly improves the interdisciplinary communication between computer scientists and physicians during the development of new retrieval algorithms.
The execution of the graphs is fully automated within the platform.
Each processing step is modeled as a feature transformation. Due to a high degree of system transparency, both the implementation and the
evaluation of retrieval algorithms are accelerated significantly.
The platform uses a client-server architecture consisting of a central database, a central job scheduler, instances of a daemon
service, and clients which embed user-implemented feature ansformations. Automatically distributed batch processing and distributed feature storage enable the cost-efficient use of an existing workstation cluster.
Proc. SPIE. 5033, Medical Imaging 2003: PACS and Integrated Medical Information Systems: Design and Evaluation
KEYWORDS: Content based image retrieval, Human-machine interfaces, Transparency, Imaging systems, Databases, Image segmentation, Image processing, Feature extraction, Medical imaging, Picture Archiving and Communication System
Picture archiving and communication systems (PACS) aim to efficiently provide the radiologists with all images in a suitable quality for diagnosis. Modern standards for digital imaging and communication in medicine (DICOM) comprise alphanumerical descriptions of study, patient, and technical parameters. Currently, this is the only information used to select relevant images within PACS. Since textual descriptions insufficiently describe the great variety of details in medical images, content-based image retrieval (CBIR) is expected to have a strong impact when integrated into PACS. However, existing CBIR approaches usually are limited to a distinct modality, organ, or diagnostic study. In this state-of-the-art report, we present first results implementing a general approach to content-based image retrieval in medical applications (IRMA) and discuss its integration into PACS environments. Usually, a PACS consists of a DICOM image server and several DICOM-compliant workstations, which are used by radiologists for reading the images and reporting the findings. Basic IRMA components are the relational database, the scheduler, and the web server, which all may be installed on the DICOM image server, and the IRMA daemons running on distributed machines, e.g., the radiologists’ workstations. These workstations can also host the web-based front-ends of IRMA applications. Integrating CBIR and PACS, a special focus is put on (a) location and access transparency for data, methods, and experiments, (b) replication transparency for methods in development, (c) concurrency transparency for job processing and feature extraction, (d) system transparency at method implementation time, and (e) job distribution transparency when issuing a query. Transparent integration will have a certain impact on diagnostic quality supporting both evidence-based medicine and case-based reasoning.
Automatic identification of frontal (posteroanterior/anteroposterior) vs. lateral chest radiographs is an important preprocessing step in medical imaging. A recent approach by Amura et al. (Procs SPIE 2002; 4684: 308-315) is based on manual selection and combination of about 500 radiographs to generate as much as 24 templates by pixel-wise summing up the references, and a correctness rate of 99,99 % is reported. In order to design a fully automated procedure, 1,867 images were arbitrarily selected from clinical routine as reference for this work: 1,266 in frontal and 601 in lateral view position. The size of the radiographs varies between 2,000 and 4,000 pixels in each direction. Automatic categorization is done in two steps. At first, the image is reduced substantially in size. Regardless of the initial aspect ratio, a squared version is obtained, where the number h of pixels in both directions is a power of two. In the second step, the normalized cross correlation function at the optimal displacement is used for 5-nearest-neighbor classification. Leaving-one-out experiments were performed for h = 4, 8, 16, 32, and 64 resulting in mean correctness of 92.0 %, 99.3 %, 99.3 %, 99.6 % and 99.4 %, respectively. With respect to the approach of Amura et al., these results show that the determination of the view position of chest radiographs can be fully automated and substantially simplified if the correlation function is used directly for 5-NN classification.
The widely used DICOM 3.0 imaging protocol specifies optional tags to store specific information on modality and body region within the header: Body Part Examined and Anatomic Structure. We investigate whether this information can be used for the automated categorization of medical images, as this is an important first step for medical image retrieval. Our survey examines the headers generated by four digital image modalities (2 CTs, 2 MRIs) in clinical routine at the Aachen University Hospital within a period of four months. The manufacturing dates of the modalities range from 1995 to 1999, with software revisions from 1999 and 2000. Only one modality sets the DICOM tag Body Part Examined. 90 out of 580 images (15.5%) contained false tag entries causing a wrong categorization. This result was verified during a second evaluation period of one month one year later (562 images, 15.3% error rate). The main reason is the dependency of the tag on the examination protocol of the modality, which controls all relevant parameters of the imaging process. In routine, the clinical personnel often applies an examination protocol outside its normal context to improve the imaging quality. This is, however, done without manually adjusting the categorization specific tag values. The values specified by DICOM for the tag Body Part Examined are insufficient to encode the anatomic region precisely. Thus, an automated categorization relying on DICOM tags alone is impossible.