Content-based Retrieval of Medical Images: Landmarking, Indexing, and Relevance Feedback

Content-based image retrieval (CBIR) is the process of retrieval of images from a database that are similar to a query image, using measures derived from the images themselves, rather than relying on accompanying text or annotation. To achieve CBIR, the contents of the images need to be characterized by quantitative features; the features of the query image are compared with the features of each image in the database and images having high similarity with respect to the query image are retrieved and displayed. CBIR of medical images is a useful tool and could provide radiologists with assistance in the form of a display of relevant past cases. One of the challenging aspects of CBIR is to extract features from the images to represent their visual, diagnostic, or application-specific information content. In this book, methods are presented for preprocessing, segmentation, landmarking, feature extraction, and indexing of mammograms for CBIR. The preprocessing steps include anisotropic diffusion and the Wiener filter to remove noise and perform image enhancement. Techniques are described for segmentation of the breast and fibroglandular disk, including maximum entropy, a moment-preserving method, and Otsu's method. Image processing techniques are described for automatic detection of the nipple and the edge of the pectoral muscle via analysis in the Radon domain. By using the nipple and the pectoral muscle as landmarks, mammograms are divided into their internal, external, upper, and lower parts for further analysis. Methods are presented for feature extraction using texture analysis, shape analysis, granulometric analysis, moments, and statistical measures. The CBIR system presented provides options for retrieval using the Kohonen self-organizing map and the k-nearest-neighbor method. Methods are described for inclusion of expert knowledge to reduce the semantic gap in CBIR, including the query point movement method for relevance feedback (RFb). Analysis of performance is described in terms of precision, recall, and relevance-weighted precision of retrieval. Results of application to a clinical database of mammograms are presented, including the input of expert radiologists into the CBIR and RFb processes. Models are presented for integration of CBIR and computer-aided diagnosis (CAD) with a picture archival and communication system (PACS) for efficient workflow in a hospital.