CN105956198A - Nidus position and content-based mammary image retrieval system and method - Google Patents

Abstract

The present invention provides a breast image retrieval system and method based on lesion position and content. The system includes an image preprocessing unit, an image lesion position similarity measurement unit, an image content similarity measurement unit, and an image comprehensive similarity measurer. In order to obtain the image to be retrieved and the historical image set of mammography x-ray images, select the standard image, preprocess the image to be retrieved and the historical image set, and perform image processing on the preprocessed image to be retrieved and the preprocessed historical image set Similarity measurement of lesion position, the image content similarity measurement is performed on the preprocessed image to be retrieved and the preprocessed historical image set of mammography x-ray images, and the image sequence number of the comprehensive similarity of the image is obtained, and the image retrieval to be retrieved is obtained result. The invention adds a measurement method based on the similarity of the lesion position, effectively improves the retrieval performance of mammogram x-ray images, and further assists doctors in diagnosing breast diseases.

Description

A breast image retrieval system and method based on lesion location and content

technical field

The invention belongs to the technical field of medical image post-processing, and in particular relates to a breast image retrieval system and method based on lesion location and content.

Background technique

At present, breast cancer screening is an important means to achieve early diagnosis and treatment of breast cancer, which can reduce the mortality rate by 30%. Mammography mammography is an important basis for early detection and diagnosis of breast cancer. The different manifestations of lesions in breast images have become the only standard for early diagnosis of breast cancer, but its diagnosis is relatively difficult. Retrieval of breast images can Effectively assist doctors in diagnosis.

Content-based image retrieval technology started in the early 1990s, and now it is used more and more widely in medicine. Among them, it is of great significance in breast image retrieval, which is based on the characteristics of breast images. The results are considered useful for medical diagnosis. Up to now, the retrieval technology has developed relatively maturely, but there are still defects in the retrieval performance, which is mainly manifested in the low retrieval performance. The main reason is the amount of information contained in the image and the similarity measurement method, that is, the extracted The content of useful information in images and the method of similarity retrieval. Therefore, how to improve the retrieval performance still needs further discussion and research.

Contents of the invention

Aiming at the deficiencies of the prior art, the present invention proposes a breast image retrieval system and method based on lesion location and content.

The technical solution of the present invention is: a breast image retrieval system based on lesion position and content, including an image preprocessing unit, an image lesion position similarity measurement unit, an image content similarity measurement unit, and an image comprehensive similarity measurer;

The image preprocessing unit is used to obtain the image to be retrieved F ₀ of the mammography X-ray image, the historical image set (F ₁ , F ₂ ,...,F _n ), select the standard image F _C , and the image to be retrieved F ₀ Preprocess with the historical image set (F ₁ ,F ₂ ,…,F _n ) to get the preprocessed image set (I ₀ ,I ₁ ,I ₂ ,…, _In ), including the preprocessed The image I ₀ to be retrieved and the preprocessed historical image set (I ₁ , I ₂ ,...,In ); the standard image F _{C is} a mammography X-ray image with normal shape and moderate size;

The image lesion position similarity measurement unit _is used to perform preprocessing on the preprocessed image to be retrieved I ₀ and the preprocessed historical image set (I ₁ , I ₂ ,...,In ) of the mammogram X-ray image Measure the similarity of image lesion positions to obtain a set of image lesion position similarities (S ₁ , S ₂ ,...,S _n );

The image content similarity measurement unit is used to image the preprocessed image to be retrieved I ₀ and the preprocessed historical image set (I ₁ , I ₂ ,...,In ) of the _mammogram X-ray image Content similarity measurement to obtain image content similarity set (E ₁ ,E ₂ ,…,E _n );

The image comprehensive similarity measurer is used to sort the image lesion position similarity set (S ₁ , S ₂ ,...,S _n ) according to the image lesion position similarity from large to small, and mark the serial number, and sort the The sequence number of the image lesion position similarity set is assigned the weight of A%, and the image content similarity set (E ₁ , E ₂ ,...,E _n ) is sorted according to the image content similarity from small to large, and the sequence number is marked, and the sorted The serial number of the image content similarity set is assigned (100-A)% weight, and the image serial number of the image lesion position similarity and the image content similarity image serial number are integrated to obtain the image comprehensive similarity image serial number, and the retrieval result of the image to be retrieved is obtained: the image The smaller the serial number of the comprehensive similarity image, the more similar the image is to the image to be retrieved.

Preferably, the image preprocessing unit includes: an image denoiser and an image intensifier;

The image denoiser is used to perform denoising processing on the image to be retrieved F ₀ and the historical image set (F ₁ , F ₂ ,...,F _n ) respectively, to obtain the denoised image to be retrieved P ₀ and the denoised image The historical image set (P ₁ ,P ₂ ,…,P _n );

The image intensifier is used to perform image enhancement processing on the noise-reduced image to be retrieved P ₀ and the noise-reduced historical image set (P ₁ , P ₂ ,...,P _n ), to obtain the pre-processed image to be retrieved Retrieve the image I ₀ and the preprocessed historical image set (I ₁ ,I ₂ ,…, _In );

Preferably, the image lesion position similarity measurement unit includes: a lesion position center point and radius determiner, an image to be retrieved and an image library and a standard image registration unit, and an image lesion position similarity determiner;

The lesion position center point and radius determiner is used to determine the lesion position center point coordinate set {(x ₀ , y ₀ ) of the preprocessed image set (I ₀ , I ₁ , I ₂ ,...,In ₎ ,(x ₁ ,y ₁ ),(x ₂ ,y ₂ )…,(x _n ,y _n )} and the lesion position radius set (R ₀ ,R ₁ ,R ₂ ,…,R _n );

Preferably, the determination of the center point coordinate set {(x ₀ ,y ₀ ),( _{x 1} ,y ₁ ) of the lesion position of the preprocessed image set (I ₀ , I ₁ , I ₂ ,...,In ) ,(x ₂ ,y ₂ )…,(x _n ,y _n )} and the lesion location radius set (R ₀ ,R ₁ ,R ₂ ,…,R _n ) are as follows:

The preprocessed image set (I ₀ , I ₁ , I ₂ ,…,In ) _is binarized using the classic Otsu method threshold segmentation algorithm, and the highlighted areas in the binarized image are retained, and the high Half of the maximum value in the X-axis direction of the bright area is taken as the abscissa of the center point of the lesion, and half of the maximum value in the Y-axis direction of the highlighted area is taken as the ordinate of the center point of the lesion, and the coordinate set of the center point of the lesion is obtained {(x ₀ ,y ₀ ),(x ₁ ,y ₁ ),(x ₂ ,y ₂ )…,(x _n ,y _n )}, traverse all the points in the highlighted area, start from the first point, use the right triangle Pythagorean theorem, find The distance from the point to the center point is calculated sequentially from all points in the highlighted area to the center point, and the maximum value is used as the radius of the lesion position to obtain the radius set of the lesion position (R ₀ , R ₁ , R ₂ ,…,R _n ).

The image to be retrieved and the image library and standard image registration device are used to combine the preprocessed image set (I ₀ , I ₁ , I ₂ ,..., _In ) with the standard image F _C performs matching to obtain the transformed focal point coordinate set {(X ₀ ,Y ₀ ),(X ₁ ,Y ₁ ),(X ₂ ,Y ₂ ),…,(X _n ,Y _n )}, and The transformed lesion center point coordinate set {(X ₀ ,Y ₀ ),(X ₁ ,Y ₁ ),(X ₂ ,Y ₂ ),…,(X _n ,Y _n )} and the lesion position radius set ( R ₀ , R ₁ , R ₂ ,…,R _n ) are displayed on the standard image F _C , using the transformed focal point coordinate set {(X ₀ ,Y ₀ ),(X ₁ ,Y ₁ ),(X ₂ ,Y ₂ ),…,(X _n ,Y _n )} and the lesion position radius set (R ₀ ,R ₁ ,R ₂ ,…,R _n ) determine the lesion area (Circle ₀ ,Circle ₁ ,Circle ₂ ,… , Circle _n );

The image lesion position similarity determiner is used to calculate the lesion area Circle ₀ of the image to be retrieved and the lesion area of the historical image (Circle ₁ , Circle _n ) in the lesion area (Circle ₀ , Circle ₁ , Circle ₂ ,..., Circle ₂ ,…,Circle _n ) intersection and union, let Obtain the image lesion position similarity set (S ₁ , S ₂ , S ₃ ,...,S _n );

Preferably, the image content similarity measurement unit includes: an image feature histogram extractor and an image content similarity determiner;

The image feature histogram extractor is used to extract the _grayscale features, shape features and texture features of the preprocessed image set (I ₀ , I ₁ , I ₂ ,...,In ), and construct its image grayscale histogram Figure, based on the edge direction histogram, direction gradient histogram and local binary pattern histogram, the gray feature vector (α ₀ ,α ₁ ,α ₂ ,…,α _n ), the shape feature vector (β ₀ ,β ₁ ,β ₂ ,…,β _n ) and texture feature vectors (γ ₀ ,γ ₁ ,γ ₂ ,…,γ _n ), merged grayscale feature vectors (α ₀ ,α ₁ ,α ₂ ,…,α _n ), Shape eigenvectors (β ₀ , β ₁ , β ₂ ,…,β _n ) and texture eigenvectors (γ ₀ , γ ₁ ,γ ₂ ,…,γ _n ), get the integrated histogram feature vector (ω ₀ ,ω ₁ ,ω ₂ ,…,ω _n );

The image content similarity determiner is used to perform similarity between the integrated histogram feature vector ω ₀ of the image to be retrieved and the integrated histogram feature vector (ω ₁ , ω ₂ ,...,ω _n ) of the historical image by using the EMD method Measured to obtain the image content similarity set (E ₁ , E ₂ ,...,E _n ).

The image retrieval method using a mammary gland image retrieval system based on lesion position and content comprises the following steps:

Step 1: Obtain the image F ₀ to be retrieved and the historical image set (F ₁ , F ₂ ,...,F _n ) of mammography X-ray images, and select the standard image F _C ;

Step 2: Preprocess the image to _be retrieved F ₀ and the historical image set (F ₁ , F ₂ ,...,F _n ), and obtain the preprocessed image set (I ₀ , I ₁ , I ₂ ,...,In ) ;

Step 2.1: Perform noise reduction processing on the image to be retrieved F ₀ and the historical image set (F ₁ , F ₂ ,...,F _n ) respectively, and obtain the image to be retrieved after noise reduction P ₀ and the historical image set after noise reduction (P ₁ ,P ₂ ,...,P _n );

Step 2.2: Carry out image enhancement processing on the denoised image to be retrieved P ₀ and the denoised historical image set (P ₁ , P ₂ ,...,P _n ) to obtain the preprocessed image to be retrieved I ₀ and Preprocessed historical image set (I ₁ ,I ₂ ,…, _In );

Step 3: _Carry out image lesion position similarity measurement on the preprocessed mammography image to be retrieved I ₀ and the preprocessed historical image set (I ₁ , I ₂ ,...,In );

Step 3.1: Determine the center point coordinate set {(x ₀ ,y _{0 )} , _{(x 1 , y 1 )} ,( x ₂ ,y ₂ )…,(x _n ,y _n )} and the radius set of the lesion position (R ₀ ,R ₁ ,R ₂ ,…,R _n );

Step 3.2: Use the CPD-based registration method to match the preprocessed image set (I ₀ , I ₁ , I ₂ ,...,In ) with the standard image F _C to obtain the transformed lesion center point coordinate set _{ (X ₀ ,Y ₀ ),(X ₁ ,Y ₁ ),(X ₂ ,Y ₂ ),…,(X _n ,Y _n )}, and the transformed focal point coordinate set {(X ₀ , Y ₀ ),(X ₁ ,Y ₁ ),(X ₂ ,Y ₂ ),…,(X _n ,Y _n )} and lesion position radius set (R ₀ ,R ₁ ,R ₂ ,…,R _n ) Displayed on the standard image F _C , using the transformed focal point coordinate set {(X ₀ ,Y ₀ ),(X ₁ ,Y ₁ ),(X ₂ ,Y ₂ ),…,(X _n ,Y _n )} and the lesion location radius set (R ₀ , R ₁ , R ₂ ,…,R _n ) determine the lesion area (Circle ₀ , Circle ₁ , Circle ₂ ,…,Circle _n );

Step 3.3: Calculate the difference between the lesion area Circle ₀ of the image to be retrieved in the lesion area (Circle ₀ , Circle ₁ , Circle ₂ ,...,Circle _n ) and the lesion area of the historical image (Circle ₁ , Circle ₂ ,...,Circle _n ) intersection and union, let Obtain the image lesion position similarity set (S ₁ , S ₂ , S ₃ ,...,S _n );

Step 4: Perform image content similarity measurement on the preprocessed image to be retrieved I ₀ and the preprocessed historical image set (I ₁ , I ₂ ,...,In ) of the _mammography image;

Step 4.1: _Extract the grayscale features, shape features and texture features of the preprocessed image set (I ₀ , I ₁ , I ₂ ,...,In ), and construct its image grayscale histogram, based on the edge direction histogram, Oriented gradient histogram and local binary pattern histogram to obtain gray feature vector (α ₀ ,α ₁ ,α ₂ ,…,α _n ), shape feature vector (β ₀ ,β ₁ ,β ₂ ,…,β _n ) and texture feature vectors (γ ₀ ,γ ₁ ,γ ₂ ,…,γ _n );

Step 4.2: Combine gray feature vectors (α ₀ ,α ₁ ,α ₂ ,…,α _n ), shape feature vectors (β ₀ ,β ₁ ,β ₂ ,…,β _n ) and texture feature vectors (γ ₀ , γ ₁ ,γ ₂ ,…,γ _n ), get the integrated histogram feature vector (ω ₀ ,ω ₁ ,ω ₂ ,…,ω _n );

Step 4.3: Use the EMD method to measure the similarity between the integrated histogram feature vector ω ₀ of the image to be retrieved and the integrated histogram feature vector (ω ₁ ,ω ₂ ,…,ω _n ) of the historical image, and obtain the image content similarity set (E ₁ ,E ₂ ,…,E _n );

Step 5: Sort the image lesion position similarity set (S ₁ , S ₂ ,...,S _n ) according to the image lesion position similarity from large to small, and mark the sequence number, and put the sequence number of the sorted image lesion position similarity set Assign the weight of A%, sort the image content similarity set (E ₁ , E ₂ ,...,E _n ) according to the image content similarity from small to large, and mark the sequence number, assign the sequence number of the sorted image content similarity set (100-A)% weight, the integrated image lesion position similarity image serial number and image content similarity image serial number obtains the image comprehensive similarity image serial number, obtains the retrieval result of the image to be retrieved: the smaller the image comprehensive similarity image serial number, the smaller the image comprehensive similarity image serial number, Indicates that the image is more similar to the image to be retrieved.

Preferably, said step 3.2 includes the following steps:

Step 3.2.1: Extract the breast contour in the _preprocessed image set (I ₀ , I ₁ , I ₂ ,...,In ) and the breast contour of the standard image F _C ;

Step 3.2.2: Using CPD-based affine transformation, _register the breast contour in the preprocessed image set (I ₀ , I ₁ , I ₂ ,...,In ) with the breast contour of the standard image F _C , Get the registration transformation matrix (T ₀ ,T ₁ ,T ₂ ,…,T _n );

Step 3.2.3: Use the registration transformation matrix (T ₀ ,T ₁ ,T ₂ ,…,T _n ) to coordinate the center point coordinate set of the lesion position {(x ₀ ,y ₀ ),(x ₁ ,y ₁ ),( x ₂ ,y ₂ )…,(x _n ,y _n )} and the radius set of the lesion position (R ₀ ,R ₁ ,R ₂ ,…,R _n ) are transformed to obtain the center point coordinate set of the lesion position { (X ₀ ,Y ₀ ),(X ₁ ,Y ₁ ),(X ₂ ,Y ₂ ),…,(X _n ,Y _n )};

Step 3.2.4: Set the center point coordinates of the transformed lesion position {(X ₀ ,Y ₀ ),(X ₁ ,Y ₁ ),(X ₂ ,Y ₂ ),…,(X _n ,Y _n )} and the lesion position radius set (R ₀ , R ₁ , R ₂ ,…,R _n ) are displayed on the standard image F _C , and the transformed lesion position central point coordinate set {(X ₀ ,Y ₀ ),(X ₁ ,Y ₁ ),(X ₂ ,Y ₂ ),…,(X _n , _{Y n ) }} and the circle ₍ Circle ₀ , Circle ₁ , Circle ₂ ,...,Circle _n ) as the lesion area.

Beneficial effects of the present invention:

The present invention proposes a breast image retrieval system and method based on lesion location and content. On the basis of the traditional content-based image retrieval method, a similarity measurement method based on lesion location is added, which can effectively improve mammography X-ray images. The image retrieval performance can further assist doctors in the diagnosis of breast diseases.

Description of drawings

Fig. 1 is the structural block diagram of the mammary gland image retrieval system based on lesion position and content in the specific embodiment of the present invention;

Fig. 2 is the flow chart of the breast image retrieval method based on lesion position and content in the specific embodiment of the present invention;

Fig. 3 is a flow chart of performing image lesion position similarity measurement on the image to be retrieved after the preprocessing of the mammography image and the historical image set after the preprocessing in the specific embodiment of the present invention;

Fig. 4 is a flow chart of determining a lesion area using a CPD-based registration method in a specific embodiment of the present invention;

5 is a flow chart of performing image content similarity measurement on mammogram images to be retrieved after preprocessing and historical image sets after preprocessing in a specific embodiment of the present invention;

Fig. 6 is a flow chart of the image processing process for obtaining the retrieval result of the image to be retrieved according to the historical image set in the specific embodiment of the present invention.

detailed description

The specific embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings.

A breast image retrieval system based on lesion position and content, as shown in Figure 1, includes an image preprocessing unit, an image lesion position similarity measurement unit, an image content similarity measurement unit, and an image comprehensive similarity measurer.

The image preprocessing unit is used to obtain the image to be retrieved F ₀ and the historical image set (F ₁ , F ₂ ,...,F _n ) of the mammography X-ray image, select the standard image F _C , and the image to be retrieved F ₀ and the historical image set The image set (F ₁ , F ₂ ,…,F _n ) is preprocessed to obtain the preprocessed image set (I ₀ ,I ₁ ,I ₂ ,…,In ), which includes the preprocessed image set to _be retrieved Image I ₀ and the preprocessed historical image set (I ₁ , I ₂ ,..., _In ).

The standard image F _C is a mammography X-ray image with normal shape and moderate size.

The image preprocessing unit includes: image denoiser and image enhancer.

The image denoiser is used to perform denoising processing on the image to be retrieved F ₀ and the historical image set (F ₁ , F ₂ ,...,F _n ) respectively, to obtain the denoised image P ₀ to be retrieved and the denoised history Image set (P ₁ ,P ₂ ,...,P _n ).

An image intensifier, which is used to perform image enhancement processing on the denoised image to be retrieved P ₀ and the denoised historical image set (P ₁ , P ₂ ,...,P _n ) to obtain the preprocessed image to be retrieved I ₀ and the preprocessed historical image set (I ₁ , I ₂ ,…, _In ).

The image lesion position similarity measurement unit _is used to perform image lesion detection on the preprocessed image I ₀ to be retrieved and the preprocessed historical image set (I ₁ , I ₂ ,...,In ) of the mammography image. The location similarity measurement is used to obtain the image lesion location similarity set (S ₁ , S ₂ ,...,S _n ).

The image lesion position similarity measurement unit includes: a lesion position center point and radius determiner, an image to be retrieved and an image database and a standard image registration unit, and an image lesion position similarity determiner.

The center point and radius determiner of the lesion position is used to determine the coordinate set of the center point of the lesion position _{ ( _{x 0} ,y _{0 )} , ₍ x ₁ ,y ₁ ),(x ₂ ,y ₂ )…,(x _n ,y _n )} and the radius set of the lesion position (R ₀ ,R ₁ ,R ₂ ,…,R _n ).

In this embodiment, the center point coordinate set {(x ₀ , y ₀ ), ( _{x 1} , y ₁ ) of the lesion position of the preprocessed image set (I ₀ , I ₁ , I ₂ ,...,In ) is determined ,(x ₂ ,y ₂ )…,(x _n ,y _n )} and the lesion location radius set (R ₀ ,R ₁ ,R ₂ ,…,R _n ) are as follows:

The preprocessed image set (I ₀ , I ₁ , I ₂ ,…,In ) _is binarized using the classic Otsu method threshold segmentation algorithm, and the highlighted areas in the binarized image are retained, and the high Half of the maximum value in the X-axis direction of the bright area is taken as the abscissa of the center point of the lesion, and half of the maximum value in the Y-axis direction of the highlighted area is taken as the ordinate of the center point of the lesion, and the coordinate set of the center point of the lesion is obtained {(x ₀ ,y ₀ ),(x ₁ ,y ₁ ),(x ₂ ,y ₂ )…,(x _n ,y _n )}, traverse all the points in the highlighted area, start from the first point, use the right triangle Pythagorean theorem, find The distance from the point to the center point is calculated sequentially from all points in the highlighted area to the center point, and the maximum value is used as the radius of the lesion position to obtain the radius set of the lesion position (R ₀ , R ₁ , R ₂ ,…,R _n ).

The image to be retrieved and the image library and the standard image registration device are used to align the preprocessed image set (I ₀ , I ₁ , I ₂ ,…, _In ) with the standard image F _C using the CPD-based registration method District matching, to obtain the transformed lesion center point coordinate set {(X ₀ ,Y ₀ ),(X ₁ ,Y ₁ ),(X ₂ ,Y ₂ ),…,(X _n ,Y _n )}, and The transformed lesion center point coordinate set {(X ₀ ,Y ₀ ),(X ₁ ,Y ₁ ),(X ₂ ,Y ₂ ),…,(X _n ,Y _n )} and the lesion position radius set (R ₀ ,R ₁ ,R ₂ ,…,R _n ) are displayed on the standard image F _C , using the transformed lesion center point coordinate set {(X ₀ ,Y ₀ ),(X ₁ ,Y ₁ ),(X ₂ ,Y ₂ ),…,(X _n ,Y _n )} and the lesion position radius set (R ₀ ,R ₁ ,R ₂ ,…,R _n ) determine the lesion area (Circle ₀ ,Circle ₁ ,Circle ₂ ,…, Circle _n ).

Image lesion position similarity determiner, used to calculate the lesion area Circle ₀ of the image to be retrieved in the lesion area (Circle ₀ , Circle ₁ , Circle ₂ ,...,Circle _n ) and the lesion area (Circle ₁ , Circle ₂ ,…,Circle _n ) intersection and union, let A similarity set of image lesion positions (S ₁ , S ₂ , S ₃ ,...,S _n ) is obtained.

The image content similarity measurement unit _is used to perform image content similarity on the preprocessed image to be retrieved I ₀ and the preprocessed historical image set (I ₁ , I ₂ ,...,In ) of the mammography image Similarity measure to get image content similarity set (E ₁ ,E ₂ ,…,E _n ).

The image content similarity measurement unit includes: an image feature histogram extractor and an image content similarity determiner.

The image feature histogram extractor is used to extract the _grayscale features, shape features and texture features of the preprocessed image set (I ₀ , I ₁ , I ₂ ,...,In ), and construct its image grayscale histogram, Based on the edge orientation histogram, orientation gradient histogram and local binary pattern histogram, the gray feature vector (α ₀ ,α ₁ ,α ₂ ,…,α _n ), shape feature vector (β ₀ ,β ₁ ,β ₂ ,…,β _n ) and texture feature vectors (γ ₀ ,γ ₁ ,γ ₂ ,…,γ _n ), combined gray feature vectors (α ₀ ,α ₁ ,α ₂ ,…,α _n ), shape features vector (β ₀ ,β ₁ ,β ₂ ,…,β _n ) and texture feature vector (γ ₀ ,γ ₁ ,γ ₂ ,…,γ _n ), to obtain the integrated histogram feature vector (ω ₀ ,ω ₁ ,ω ₂ ,…,ω _n ).

The image content similarity determiner is used to measure the similarity between the integrated histogram feature vector ω ₀ of the image to be retrieved and the integrated histogram feature vector (ω ₁ , ω ₂ ,…,ω _n ) of the historical image by using the EMD method, Obtain the image content similarity set (E ₁ , E ₂ ,...,E _n ).

The image comprehensive similarity measurer is used to sort the image lesion position similarity set (S ₁ , S ₂ ,...,S _n ) according to the image lesion position similarity from large to small, and mark the serial number, and sort the image lesion The sequence number of the position similarity set is assigned a weight of A%, and the image content similarity set (E ₁ , E ₂ ,…,E _n ) is sorted according to the image content similarity from small to large, and the sequence number is marked, and the sorted image content The serial number of the similarity set is assigned (100-A)% weight, and the image serial number of the image lesion position similarity and the image content similarity image serial number are combined to obtain the image comprehensive similarity image serial number, and the retrieval result of the image to be retrieved is obtained: that is, the image comprehensive similarity The smaller the serial number of the image, the more similar the image is to the image to be retrieved.

A method for image retrieval using a breast image retrieval system based on lesion position and content, as shown in Figure 2, includes the following steps:

Step 1: Obtain the image F ₀ to be retrieved and the historical image set (F ₁ , F ₂ ,…,F _n ) of mammography X-ray images, and select the standard image _{F C} , which is normal in shape and moderate in size Mammography X-ray image.

Step 2: Preprocess the image to _be retrieved F ₀ and the historical image set (F ₁ , F ₂ ,...,F _n ), and obtain the preprocessed image set (I ₀ , I ₁ , I ₂ ,...,In ) , including the preprocessed image to be retrieved I ₀ and the preprocessed historical image set (I ₁ , I ₂ ,..., _In ).

Step 2.1: Perform noise reduction processing on the image to be retrieved F ₀ and the historical image set (F ₁ , F ₂ ,...,F _n ) respectively, and obtain the image to be retrieved after noise reduction P ₀ and the historical image set after noise reduction (P ₁ ,P ₂ ,...,P _n ).

In this embodiment, the spatial domain transformation method is used to select the median filter for filtering, so as to realize the noise reduction processing of the image to be retrieved F ₀ and the historical image set (F ₁ , F ₂ ,...,F _n ), and reduce the number of images to be retrieved. Noise in F ₀ and the historical image set (F ₁ ,F ₂ ,…,F _n ).

Step 2.2: Carry out image enhancement processing on the denoised image to be retrieved P ₀ and the denoised historical image set (P ₁ , P ₂ ,...,P _n ) to obtain the preprocessed image to be retrieved I ₀ and Preprocessed historical image set (I ₁ ,I ₂ ,…, _In ).

In this embodiment, image enhancement processing is performed on the noise-reduced image to be retrieved P ₀ and the noise-reduced historical image set (P ₁ , P ₂ , . . . , P _n ) by using a contrast enhancement method. Emphasize the overall or local characteristics of mammography X-ray images, expand the difference between different object features in the image, suppress uninteresting features, and increase the contrast between suspected lesions and surrounding tissues.

Step 3: Carry out image lesion position similarity measurement on the preprocessed image I ₀ to be retrieved and the preprocessed historical image set (I ₁ , I ₂ ,...,In ) of the _mammography image, as shown in the figure 3.

Step 3.1: Determine the center point coordinate set {(x ₀ ,y _{0 )} , _{(x 1 , y 1 )} ,( x ₂ ,y ₂ )…,(x _n ,y _n )} and the radius set of the lesion position (R ₀ ,R ₁ ,R ₂ ,…,R _n ).

In this embodiment, the classic Otsu method threshold segmentation algorithm _is used to perform binarization processing on the preprocessed image set (I ₀ , I ₁ , I ₂ ,...,In ), and retain the high In the bright area, half of the maximum value in the X-axis direction of the highlighted area is taken as the abscissa of the center point of the lesion, and half of the maximum value in the Y-axis direction of the highlighted area is taken as the ordinate of the center point of the lesion, and the coordinate set of the center point of the lesion is obtained {( x ₀ ,y ₀ ),(x ₁ ,y ₁ ),(x ₂ ,y ₂ )…,(x _n ,y _n )}, traverse all points in the highlighted area, start from the first point, use right triangle Pythagorean theorem, find the distance from the point to the center point, calculate the distance between all the points in the highlighted area and the center point in turn, take the maximum value as the radius of the lesion position, and obtain the radius set of the lesion position (R ₀ , R ₁ , R ₂ ,...,R _n ).

Step 3.2: Use the CPD-based registration method to align the preprocessed image set (I ₀ , I ₁ , I ₂ ,…, _In ) with the standard image F _C to obtain the transformed lesion center point coordinate set {(X ₀ ,Y ₀ ),(X ₁ ,Y ₁ ),(X ₂ ,Y ₂ ),…,(X _n ,Y _n )}, and the transformed focal point coordinate set {(X ₀ ,Y ₀ ),(X ₁ ,Y ₁ ),(X ₂ ,Y ₂ ),…,(X _n ,Y _n )} and the lesion position radius set (R ₀ ,R ₁ ,R ₂ ,…,R _n ) is displayed on the standard image F _C , using the transformed focal point coordinate set {(X ₀ ,Y ₀ ),(X ₁ ,Y ₁ ),(X ₂ ,Y ₂ ),…,(X _n ,Y _n )} and the lesion location radius set (R ₀ , R ₁ , R ₂ ,...,R _n ) determine the lesion area (Circle ₀ , Circle ₁ , Circle ₂ ,...,Circle _n ), as shown in Figure 4.

In this embodiment, the CPD (Coherence Point Drift)-based registration method can be used to standardize breast contours of different sizes and reduce the error in calculating the similarity of lesion positions.

Step 3.2.1: Extract the breast contour in the _preprocessed image set (I ₀ , I ₁ , I ₂ ,...,In ) and the breast contour of the standard image F _C .

Step 3.2.2: Using CPD (Coherence Point Drift)-based affine transformation, _register the breast contour in the preprocessed image set (I ₀ , I ₁ , I ₂ ,…,In ) with the standard image F The breast contour of _C , and the registration transformation matrix (T ₀ , T ₁ , T ₂ ,...,T _n ) is obtained.

Step 3.2.3: Use the registration transformation matrix (T ₀ ,T ₁ ,T ₂ ,…,T _n ) to coordinate the center point coordinate set of the lesion position {(x ₀ ,y ₀ ),(x ₁ ,y ₁ ),( x ₂ ,y ₂ )…,(x _n ,y _n )} and the radius set of the lesion position (R ₀ ,R ₁ ,R ₂ ,…,R _n ) are transformed to obtain the center point coordinate set of the lesion position { (X ₀ ,Y ₀ ),(X ₁ ,Y ₁ ),(X ₂ ,Y ₂ ),…,(X _n ,Y _n )}.

Step 3.2.4: Set the center point coordinates of the transformed lesion position {(X ₀ ,Y ₀ ),(X ₁ ,Y ₁ ),(X ₂ ,Y ₂ ),…,(X _n ,Y _n )} and the lesion position radius set (R ₀ , R ₁ , R ₂ ,…,R _n ) are displayed on the standard image F _C , and the transformed lesion position central point coordinate set {(X ₀ ,Y ₀ ),(X ₁ ,Y ₁ ),(X ₂ ,Y ₂ ),…,(X _n , _{Y n ) }} and the circle ₍ Circle ₀ , Circle ₁ , Circle ₂ ,...,Circle _n ) as the lesion area.

Step 3.3: Calculate the difference between the lesion area Circle ₀ of the image to be retrieved in the lesion area (Circle ₀ , Circle ₁ , Circle ₂ ,...,Circle _n ) and the lesion area of the historical image (Circle ₁ , Circle ₂ ,...,Circle _n ) intersection and union, let A similarity set of image lesion positions (S ₁ , S ₂ , S ₃ ,...,S _n ) is obtained.

In this embodiment, The larger the similarity ratio of the lesion position, the more similar it is.

Step 4: Perform image content similarity measurement on the preprocessed image I ₀ to be retrieved and the preprocessed historical image set (I ₁ , I ₂ ,...,In ) of _mammography images, as shown in Figure 5 shown.

Step 4.1: _Extract the grayscale features, shape features and texture features of the preprocessed image set (I ₀ , I ₁ , I ₂ ,...,In ), construct the image grayscale histogram, and construct the image grayscale histogram based on the edge direction histogram (Edge Direction Histogram, EDH), histogram of oriented gradients (HOG) and local binary pattern histogram (LocalBinary Pattern, LBP), to obtain the gray feature vector (α ₀ ,α ₁ ,α ₂ ,…,α _n ), shape feature vectors (β ₀ ,β ₁ ,β ₂ ,…,β _n ) and texture feature vectors (γ ₀ ,γ ₁ ,γ ₂ ,…,γ _n ).

Step 4.2: Combine gray feature vectors (α ₀ ,α ₁ ,α ₂ ,…,α _n ), shape feature vectors (β ₀ ,β ₁ ,β ₂ ,…,β _n ) and texture feature vectors (γ ₀ , γ ₁ ,γ ₂ ,…,γ _n ) to get the integrated histogram feature vector (ω ₀ ,ω ₁ ,ω ₂ ,…,ω _n ).

Step 4.3: Use the EMD (Earth Mover's Distance) method to measure the similarity between the integrated histogram feature vector ω ₀ of the image to be retrieved and the integrated histogram feature vector (ω ₁ ,ω ₂ ,…,ω _n ) of the historical image, and obtain Image content similarity set (E ₁ , E ₂ ,...,E _n ).

In this embodiment, (E ₁ , E ₂ , . . . , E _n )=(0.0223, 0.0668, 0.2641, 0.3103, 0.3179, . . . , 0.6624). The smaller the value, the more similar the feature histograms, that is, the content similarity of mammography images.

Step 5: Sort the image lesion position similarity set (S ₁ , S ₂ ,...,S _n ) according to the image lesion position similarity from large to small, and mark the sequence number, and put the sequence number of the sorted image lesion position similarity set Assign a weight of 40%, sort the image content similarity set (E ₁ , E ₂ ,...,E _n ) in ascending order of image content similarity, and mark the sequence number, assign the sequence number of the sorted image content similarity set With a weight of 60%, the image serial number of the similarity of the image lesion position and the image content similarity are combined to obtain the image serial number of the image comprehensive similarity, and the retrieval result of the image to be retrieved is obtained: that is, the smaller the serial number of the image comprehensive similarity, it means that the image is similar to that of the image. The images to be retrieved are more similar.

In this embodiment, on the basis of the traditional content-based image retrieval method, a lesion position-based similarity measurement method is added, which can effectively improve the retrieval performance of mammography X-ray images, thereby further assisting doctors in mammography. For disease diagnosis, the image processing process of obtaining the image retrieval results to be retrieved based on the historical image set is shown in Figure 6.

Claims (7)

1. A mammary gland image retrieval system based on lesion position and content, is characterized in that, comprises image preprocessing unit, image lesion position similarity measurement unit, image content similarity measurement unit and image comprehensive similarity measurer; The image preprocessing unit is used to obtain the image to be retrieved F ₀ of the mammogram X-ray image, the historical image set (F ₁ , F ₂ ,..., F _n ), select the standard image F _C , and the image to be retrieved F ₀ Preprocess with the historical image set (F ₁ , F ₂ , ..., F _n ) to obtain the preprocessed image set (I ₀ , I ₁ , I ₂ , ..., _In ), including the preprocessed The image I ₀ to be retrieved and the preprocessed historical image set (I ₁ , I ₂ , ..., In ); the standard image F _{C is} a mammography X-ray image with normal shape and moderate size; The image lesion position similarity measurement unit _is used to perform preprocessing on the preprocessed image to be retrieved I ₀ and the preprocessed historical image set (I ₁ , I ₂ , ..., In ) of the mammogram X-ray image Measure the similarity of the image lesion position to obtain the image lesion position similarity set (S ₁ , S ₂ , ..., S _n ); The image content similarity measurement unit is used to image the preprocessed image to be retrieved I ₀ and the preprocessed historical image set (I ₁ , I ₂ , ..., In ) of the _mammogram X-ray image Content similarity measurement to obtain image content similarity set (E ₁ , E ₂ ,..., E _n ); The image comprehensive similarity measurer is used to sort the image lesion position similarity set (S ₁ , S ₂ , ..., S _n ) according to the image lesion position similarity from large to small, and mark the serial number, and sort the The sequence number of the image lesion position similarity set is assigned a weight of A%, and the image content similarity set (E ₁ , E ₂ ,..., E _n ) is sorted according to the image content similarity from small to large, and the sequence number is marked, and the sorted The serial number of the image content similarity set is assigned (100-A)% weight, and the image serial number of the image lesion position similarity and the image content similarity image serial number are integrated to obtain the image comprehensive similarity image serial number, and the retrieval result of the image to be retrieved is obtained: the image The smaller the serial number of the comprehensive similarity image, the more similar the image is to the image to be retrieved. 2. the mammary gland image retrieval system based on lesion position and content according to claim 1, is characterized in that, described image preprocessing unit comprises: image denoiser and image intensifier; The image denoiser is used to perform denoising processing on the image to be retrieved F ₀ and the historical image set (F ₁ , F ₂ , ..., F _n ) respectively, to obtain the denoised image to be retrieved P ₀ and the denoised image The historical image set (P ₁ , P ₂ , ..., P _n ); The image intensifier is used to perform image enhancement processing on the noise-reduced image to be retrieved P ₀ and the noise-reduced historical image set (P ₁ , P ₂ , ..., P _n ) to obtain the pre-processed image to be retrieved. Retrieve the image I ₀ and the preprocessed historical image set (I ₁ , I ₂ , ..., _In ). 3. The breast image retrieval system based on lesion position and content according to claim 1, wherein the image lesion position similarity measurement unit comprises: lesion position center point and radius determiner, image to be retrieved and image library Similarity determiner with standard image registration device and image lesion position; The lesion position center point and radius determiner is used to determine the lesion position center point coordinate set {(x ₀ , y ₀ ) of the preprocessed image set (I ₀ , I ₁ , I ₂ , ..., In ₎ , (x ₁ , y ₁ ), (x ₂ , y ₂ )…, (x _n , y _n )} and the lesion position radius set (R ₀ , R ₁ , R ₂ ,…, R _n ); The image to be retrieved and the image library and standard image registration device are used to combine the preprocessed image set (I ₀ , I ₁ , I ₂ ,..., _In ) with the standard image F using the CPD-based registration method _C performs matching to obtain the transformed lesion center point coordinate set {(X ₀ , Y ₀ ), (X ₁ , Y ₁ ), (X ₂ , Y ₂ ), ..., (X _n , Y _n )}, and The transformed lesion center point coordinate set {(X ₀ , Y ₀ ), (X ₁ , Y ₁ ), (X ₂ , Y ₂ ), ..., (X _n , Y _n )} and the lesion position radius set ( R ₀ , R ₁ , R ₂ ,…, R _n ) are displayed on the standard image F _C , using the transformed focal point coordinate set {(X ₀ , Y ₀ ), (X ₁ , Y ₁ ), (X ₂ , Y ₂ ), ..., (X _n , Y _n )} and the lesion position radius set (R ₀ , R ₁ , R ₂ , ..., R _n ) determine the lesion area (Circle ₀ , Circle ₁ , Circle ₂ , ... , Circle _n ); The image focus position similarity determiner is used to calculate the focus area Circle ₀ of the image to be retrieved and the focus area (Circle ₁ , Circle _n ) of the image to be retrieved in the focus area (Circle ₀ , Circle ₁ , Circle ₂ ,... Circle ₂ ,..., Circle _n ) intersection and union, let A similarity set of image lesion positions (S ₁ , S ₂ , S ₃ , . . . , S _n ) is obtained. 4. The breast image retrieval system based on lesion position and content according to claim 1, wherein said image content similarity measurement unit comprises: image feature histogram extractor and image content similarity determiner; The image feature histogram extractor is used to extract the _grayscale features, shape features and texture features of the preprocessed image set (I ₀ , I ₁ , I ₂ , ..., In ), and construct its image grayscale histogram Figure, based on the edge direction histogram, direction gradient histogram and local binary mode histogram, the gray feature vector (α ₀ , α ₁ , α ₂ ,…, α _n ), the shape feature vector (β ₀ , β ₁ , β ₂ ,…, β _n ) and texture feature vectors (γ ₀ , γ ₁ , γ ₂ ,…, γ _n ), merged grayscale feature vectors (α ₀ , α ₁ , α ₂ ,…, α _n ), Shape feature vectors (β ₀ , β ₁ , β ₂ ,…, β _n ) and texture feature vectors (γ ₀ , γ ₁ , γ ₂ ,…, γ _n ), get the integrated histogram feature vector (ω ₀ , ω ₁ , ω ₂ ,..., ω _n ); The image content similarity determiner is used to perform similarity between the integrated histogram feature vector ω ₀ of the image to be retrieved and the integrated histogram feature vector (ω ₁ , ω ₂ , ..., ω _n ) of the historical image by using the EMD method Measured to obtain the image content similarity set (E ₁ , E ₂ ,..., E _n ). 5. The breast image retrieval system based on lesion position and content according to claim 3, characterized in that said determining the lesion of the _preprocessed image set (I ₀ , I ₁ , I ₂ , ..., In ) Position center point coordinate set {(x ₀ , y ₀ ), (x ₁ , y ₁ ), (x ₂ , y ₂ )…, (x _n , y _n )} and lesion position radius set (R ₀ , R ₁ , R ₂ ,..., R _n ) the specific method is: The preprocessed image set (I ₀ , I ₁ , I ₂ ,..., In ) _is binarized using the classic Otsu method threshold segmentation algorithm, and the highlighted areas in the binarized image are retained, and the high Half of the maximum value in the X-axis direction of the bright area is taken as the abscissa of the center point of the lesion, and half of the maximum value in the Y-axis direction of the highlighted area is taken as the ordinate of the center point of the lesion, and the coordinate set of the center point of the lesion is obtained {(x ₀ , y ₀ ), (x ₁ , y ₁ ), (x ₂ , y ₂ )…, (x _n , y _n )}, traverse all points in the highlighted area, start from the first point, use the right triangle Pythagorean theorem, find The distance from the point to the center point is calculated sequentially from all points in the highlighted area to the center point, and the maximum value is used as the radius of the lesion position to obtain the radius set of the lesion position (R ₀ , R ₁ , R ₂ , ..., R _n ). 6. adopt the method for image retrieval based on the breast image retrieval system of lesion position and content according to claim 1, it is characterized in that, comprising the following steps: Step 1: Obtain the image F ₀ to be retrieved and the historical image set (F ₁ , F ₂ , . . . , F _n ) of mammography X-ray images, and select the standard image F _C ; Step 2: Preprocess the image to _be retrieved F ₀ and the historical image set (F ₁ , F ₂ , ..., F _n ) to obtain the preprocessed image set (I ₀ , I ₁ , I ₂ , ..., In ) ; Step 2.1: Perform denoising processing on the image to be retrieved F ₀ and the historical image set (F ₁ , F ₂ , ..., F _n ) respectively, and obtain the image to be retrieved after denoising P ₀ and the historical image set after denoising (P ₁ , P ₂ ,..., P _n ); Step 2.2: Carry out image enhancement processing on the image to be retrieved after denoising P ₀ and the historical image set (P ₁ , P ₂ , ..., P _n ) after denoising respectively, and obtain the preprocessed image to be retrieved I ₀ and Preprocessed historical image set (I ₁ , I ₂ , ..., _In ); Step 3: _Perform image lesion position similarity measurement on the preprocessed mammography image to be retrieved I ₀ and the preprocessed historical image set (I ₁ , I ₂ , . . . , In ); Step 3.1: Determine the coordinate set {(x ₀ , y _{0 )} , _{(x 1 , y 1 )} , ( x ₂ , y ₂ )…, (x _n , y _n )} and the radius set of the lesion position (R ₀ , R ₁ , R ₂ ,…, R _n ); Step 3.2: Using the CPD-based registration method, match the preprocessed image set (I ₀ , I ₁ , I ₂ , ..., In ) with the standard image F _C to obtain the transformed lesion center point coordinate set _{ (X ₀ , Y ₀ ), (X ₁ , Y ₁ ), (X ₂ , Y ₂ ), ..., (X _n , Y _n )}, and the transformed focal point coordinate set {(X ₀ , Y ₀ ), (X ₁ , Y ₁ ), (X ₂ , Y ₂ ), ..., (X _n , Y _n )} and the radius set of the lesion position (R ₀ , R ₁ , R ₂ , ..., R _n ) Displayed on the standard image F _C , using the transformed focal point coordinate set {(X ₀ , Y ₀ ), (X ₁ , Y ₁ ), (X ₂ , Y ₂ ),..., (X _n , Y _n )} and the lesion location radius set (R ₀ , R ₁ , R ₂ ,..., R _n ) determine the lesion area (Circle ₀ , Circle ₁ , Circle ₂ ,..., Circle _n ); Step 3.3: Calculate the difference between the lesion area Circle ₀ of the image to be retrieved and the lesion area (Circle _{1 , Circle 2 , ..., Circle n ) of the historical image in the lesion area (Circle 0 , Circle 1 , Circle 2 , ... , Circle n} ) respectively intersection and union, let Get image lesion position similarity set (S ₁ , S ₂ , S ₃ ,..., S _n ); Step 4: Perform image content similarity measurement on the preprocessed image to be retrieved I ₀ and the preprocessed historical image set (I ₁ , I ₂ , . . . , In ) of the _mammography image; Step 4.1: _Extract the grayscale features, shape features and texture features of the preprocessed image set (I ₀ , I ₁ , I ₂ , ..., In ), and construct its image grayscale histogram, based on edge direction histogram, Oriented gradient histogram and local binary pattern histogram to get gray feature vector (α ₀ , α ₁ , α ₂ ,…, α _n ), shape feature vector (β ₀ , β ₁ , β ₂ ,…, β _n ) and texture feature vectors (γ ₀ , γ ₁ , γ ₂ ,…, γ _n ); Step 4.2: Combine gray feature vectors (α ₀ , α ₁ , α ₂ , ..., α _n ), shape feature vectors (β ₀ , β ₁ , β ₂ , ..., β _n ) and texture feature vectors (γ ₀ , γ ₁ , γ ₂ ,…,γ _n ), get the integrated histogram feature vector (ω ₀ , ω ₁ , ω ₂ ,…,ω _n ); Step 4.3: Use the EMD method to measure the similarity between the integrated histogram feature vector ω ₀ of the image to be retrieved and the integrated histogram feature vector (ω ₁ , ω ₂ , ..., ω _n ) of the historical image, and obtain the image content similarity set (E ₁ , E ₂ ,..., E _n ); Step 5: Sort the image lesion position similarity sets (S ₁ , S ₂ , ..., S _n ) according to the image lesion position similarity from large to small, and mark the serial number, and put the sequence number of the sorted image lesion position similarity set Assign the weight of A%, sort the image content similarity sets (E ₁ , E ₂ ,..., E _n ) according to the image content similarity from small to large, and mark the serial number, assign the serial number of the sorted image content similarity set (100-A)% of the weight, integrated image lesion position similarity image serial number and image content similarity image serial number to get the image comprehensive similarity image serial number, obtain the image retrieval result to be retrieved: the smaller the image comprehensive similarity image serial number, the smaller the image comprehensive similarity image serial number, Indicates that the image is more similar to the image to be retrieved. 7. the breast image retrieval method based on lesion position and content according to claim 5, is characterized in that, described step 3.2 comprises the following steps: Step 3.2.1: Extract the breast contour in the _preprocessed image set (I ₀ , I ₁ , I ₂ , ..., In ) and the breast contour of the standard image F _C ; Step 3.2.2: Using CPD-based affine transformation, _register the breast contour in the preprocessed image set (I ₀ , I ₁ , I ₂ ,...,In ) with the breast contour of the standard image F _C , Get the registration transformation matrix (T ₀ , T ₁ , T ₂ ,..., T _n ); Step 3.2.3: Use the registration transformation matrix (T ₀ , T ₁ , T ₂ , ..., T _n ) to coordinate the center point coordinate set of the lesion position {(x ₀ , y ₀ ), (x ₁ , y ₁ ), ( x ₂ , y ₂ )..., (x _n , y _n )} and the radius set of the lesion position (R ₀ , R ₁ , R ₂ , ..., R _n ) are transformed to obtain the center point coordinate set of the lesion position { (X ₀ , Y ₀ ), (X ₁ , Y ₁ ), (X ₂ , Y ₂ ),..., (X _n , Y _n )}; Step 3.2.4: Set the center point coordinates of the transformed lesion position {(X ₀ , Y ₀ ), (X ₁ , Y ₁ ), (X ₂ , Y ₂ ), ..., (X _n , Y _n )} and the lesion position radius set (R ₀ , R ₁ , R ₂ ,..., R _n ) are displayed on the standard image F _C , and the transformed lesion position central point coordinate set {(X ₀ , Y ₀ ), (X ₁ , Y ₁ ), (X ₂ , Y ₂ ),…, (X _n , _{Y n ) }} and the circle ₍ Circle ₀ , Circle ₁ , Circle ₂ ,..., Circle _n ) as the lesion area.