CN108710690A - Medical image search method based on geometric verification - Google Patents

Abstract

The invention discloses a medical image retrieval method based on geometric verification, relates to the technical field of medical image processing, aims at the problems in the prior art, and acquires local The spatial relationship between the features is encoded, and then the initial retrieval results are optimized by verifying whether the spatial encoding conforms to the geometric consistency to reduce the chance of false matching and improve the accuracy of breast cancer image retrieval results.

Description

Medical Image Retrieval Method Based on Geometry Verification

technical field

The invention relates to the technical field of medical image processing, in particular to a medical image retrieval method based on geometric verification.

Background technique

Breast cancer is the disease with the highest incidence rate among female malignant tumors in my country, and it shows a trend of younger age. However, it is difficult to treat it effectively until it has spread in the human body. There are a variety of breast examination techniques, such as: mammography, ultrasonography, magnetic resonance imaging and so on. Among them, mammography is the most commonly used examination method. On this basis, many Computer Aided Diagnosis (CAD) methods have been proposed to improve the technical level of detecting masses (important indicators of breast cancer) in mammograms, and content-based image retrieval is one of them. The widely used retrieval method mainly consists of four steps: X-ray film preprocessing, breast image region segmentation, feature extraction, feature clustering and quantification, and similarity measurement.

With the wide application of image retrieval methods in medical diagnosis and the deepening of research, these methods can provide more clinical basis, and the misdiagnosis rate caused by doctors' lack of clinical experience has decreased and the diagnosis efficiency is higher. Therefore, breast cancer image retrieval methods have attracted extensive attention in the field of female breast cancer treatment.

In the study of breast cancer image retrieval methods, many retrieval ideas have been proposed. Tourassi G D proposed a template matching method based on mutual information (mutual information) in 2003 to judge whether the ROI (RegionOf Interest) in the breast image describes a real mass. This method is a knowledge-based method. The computer-aided diagnosis system establishes a breast image ROI knowledge database based on known reference standards. Each ROI in the database is regarded as a template, and the computer-aided system follows a set of template matching methods. Algorithm, using mutual information as a similarity measure, to match query images with breast cancer database images. This is used to determine whether the ROI of the query image describes a real mass. According to their information content, all similar ROIs in the database are retrieved and ranked, and then the decision index is calculated according to the best matching result of the query. The decision index efficiently combines the similarity measure and the best matching template The reference standard of is fused into the judgment of whether there is a mass in the ROI of the query breast image.

In order to improve the accuracy of retrieval results, Tourassi added more similarity measurement methods to breast cancer image ROI template matching in 2007. The research shows that some similarity measurement methods have higher accuracy and others have higher retrieval efficiency. . Narzaez proposed in 2012 to perform curvature transformation on the ROI of breast mass images, and then describe the ROI through its marginal distribution. In order to improve the scalability and retrieval efficiency of the retrieval method, Jiang added the method of vocabulary tree and inverted index in the research of breast cancer image retrieval. The methods introduced above have made outstanding contributions in the research of breast cancer image retrieval methods, but the performance of the retrieval system is affected due to the lack of scalability or the lack of consideration of the spatial attribute information of image features.

Contents of the invention

The embodiment of the present invention provides a medical image retrieval method based on geometric verification, which can solve the problems existing in the prior art.

The invention provides a medical image retrieval method based on geometric verification, the method comprising the following steps:

Step 1, extract SIFT features to all images in the breast cancer image database, and store them in the image feature database;

Step 2, performing hierarchical K-means clustering on the SIFT features of all extracted database images to generate a breast cancer image visual vocabulary tree;

Step 3, establishing an inverted index for the generated breast cancer image visual vocabulary tree;

Step 4, describe all breast cancer database images through the visual words in the breast cancer image visual vocabulary tree;

Step 5, after receiving the query image, extract the SIFT feature of the query image, quantify the feature of the query image according to the established visual vocabulary tree based on the inverted index, and describe the query image in the form of a visual word vector;

Step 6, based on the inverted index, measure the similarity between the quantified query image and the quantified image in the database to form an initial retrieval result;

Step 7, taking the matching visual words in the query image and the initial retrieval result image as reference points to delineate two local circular neighborhoods with the same radius;

Step 8: Carry out geometric verification on the relevant feature points in the two local circular neighborhoods, eliminate the wrongly matched feature points, and obtain the final retrieval result.

The medical image retrieval method based on geometric verification in the embodiment of the present invention aims at the above problems, on the basis of the vocabulary tree (Vacabulary Tree) and the inverted index (Inverted Index), by obtaining the spatial relationship code between the local features of the image, and then by Verify that the spatial encoding conforms to the geometric consistency to optimize the initial retrieval results to reduce the probability of false matching and improve the accuracy of breast cancer image retrieval results.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. Those skilled in the art can also obtain other drawings based on these drawings without creative work.

Fig. 1 is a flowchart of a medical image retrieval method based on geometric verification in an embodiment of the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

With reference to Fig. 1, the embodiment of the present invention provides the medical image retrieval method based on geometric verification, and this method comprises the following steps:

Step 1, extract SIFT features from all images in the breast cancer image database, and store them in the image feature database.

Step 2: Hierarchical K-means clustering is performed on the extracted SIFT features of all database images to generate a visual vocabulary tree for breast cancer images.

Step 3: Establish an inverted index for the generated visual vocabulary tree of breast cancer images.

Step 4, describe all breast cancer database images through the visual words in the breast cancer image visual vocabulary tree.

Step 5: After receiving the query image, extract the SIFT features of the query image, quantify the features of the query image according to the established visual vocabulary tree based on the inverted index, and describe the query image in the form of a visual word (Visual Words) vector.

Step 6, based on the inverted index, perform similarity measurement between the quantified query image and the quantified image in the database to form an initial retrieval result.

Step 7, taking the matching visual words in the query image and the initial retrieval result image as reference points to define two local circular neighborhoods with the same radius.

Step 8: Carry out geometric verification on the relevant feature points in the two local circular neighborhoods, eliminate the wrongly matched feature points, and obtain the final retrieval result.

Wherein, step 2 specifically includes the following sub-steps:

Sub-step 201, setting parameters: first, set a height L and the number K of non-leaf node subtrees for the visual vocabulary tree to be created, that is, the number of cluster centers at each level.

Substep 202, generate a subtree: perform K-means clustering on the extracted breast cancer image SIFT feature data set to obtain K cluster centers, then these K cluster centers are the K child nodes of the root node, and the data The set is correspondingly divided into K subsets.

Sub-step 203, recursive process: perform K-means clustering operation on the obtained K subsets respectively, and obtain K cluster centers of each subset, that is, the child nodes of the node in this layer of the vocabulary tree, iteratively carry out this step operation until The height of the vocabulary tree reaches L.

After the above steps, a visual vocabulary tree of the breast cancer image graph is finally generated. All the leaf nodes of the tree are visual words, and each visual word is a SIFT feature descriptor.

In order to quantitatively express the effect of different visual words on the measure of image lovesickness, the TF-IDF method is used to weight each visual word in the vocabulary tree. The performance of the retrieval system can be improved by assigning different TF-IDF weights to visual words.

TF-IDF works as follows:

First, assuming that there are k visual words in the visual vocabulary tree, the d-th image I _d in the breast cancer image database can be expressed as a k-dimensional vector:

v _i =(t ₁ ,t ₂ ,…,t _k ) ^T (1)

The i-th vector element t _i expression of vector v _i is:

t _i =tf _i,d ×idf _i (2)

In formula (2), tf _i,d represent the frequency of the i-th visual word in the visual dictionary appearing in the d-th image in the image database, and idf _i represents the i-th visual word in the visual dictionary in all images in the image database The Inverse Document Frequency (Inverse Document Frequency), tf _{i, d} and idf _i are calculated as:

In formula (3), n _i,d represent the number of occurrences of the i-th visual word in the visual dictionary in the d-th image of the image database, and n _d represents the total number of occurrences of all visual words counted in the d-th image of the database. In formula (4), N is the total number of images in the image database, and N _i represents the total number of times the i-th visual word appears in the image database images. Therefore, the i-th vector element expression in the image vector v _i is as follows:

From the expression (5), we can see the operation mechanism of the TF-IDF weighting algorithm. When a visual word in the visual dictionary appears more frequently in an image, that is, the greater the value of n _i,d , the word frequency tf The larger _{i and d} are, the stronger the representativeness of the visual word is; at the same time, when the visual word appears less often in other images in the image database, that is, the smaller the value of N _i is, the reverse document frequency idf _i The larger the value, the higher the discrimination of the visual word.

After creating a visual vocabulary tree for breast cancer images in the database and weighting the visual words in the vocabulary tree, both the database image and the query image can be expressed by visual word vectors, and each vector element is the frequency of occurrence of the visual word in the image The weighted value completes the feature quantification of the breast cancer image.

Step 7 specifically includes the following sub-steps:

Sub-step 701, define the ROI of the query image as I _q , define the ROI of an initial retrieval result image as I _d , Q and D represent the SIFT feature point sets on the query image and initial retrieval result image ROI respectively, and we can get The set of matching feature point pairs (hereinafter referred to as matching pairs) M(Q,D)={(q _i ,d _i )|q _i ∈Q,d _i ∈D}, q _i and d _i represent the query image and For the matching feature points on the ROI of the initial retrieval result image, select a pair of matching feature points as the reference feature point pair (q _i , d _i ) to verify whether the matching pair is a wrong match.

Sub-step 702, after the matching pair is selected, the local circular neighborhood of the matching pair is defined, and the radius of the two corresponding circular areas is defined as follows:

Among them, scl _i and scl _i ' represent the scale parameters of the feature points of the query image in the matching pair and the feature points of the initial retrieval result image, and δ is a parameter set to control the range of the local circular neighborhood.

Step 8 specifically includes the following sub-steps:

Sub-step 801, exclude irrelevant feature points outside the circular neighborhood in sub-step 702, the following formula:

in, and Represents the set of feature points located in the local circular neighborhood on the ROI of the query image and the initial retrieval image, and dist(,) represents the Euclidean distance between the relevant feature point and the reference feature point.

Sub-step 802, extracting relevant feature points matched in two local circular neighborhoods, the extraction method is as follows:

In sub-step 803, since the direction of the selected reference feature point is not necessarily parallel to the direction of the coordinate axis, a rotation adjustment needs to be made, and the direction of the reference feature point is used as the new coordinate axis, so that the positions of the relevant feature points are rotated. After the rotation adjustment, the position of the relevant feature points is calculated as follows:

Among them, _θi represents the main gradient direction of the reference feature point SIFT.

Sub-step 804, in order to measure the geometric consistency of the relevant feature points, after the relevant feature points are rotated and adjusted, the geometric positional relationship of each pair of matching relevant feature points relative to their respective reference feature points is encoded on the divided quadrant circle to Verify that it falls within the same segmented region. Calculated as follows:

If the relevant feature point falls on the left side of the reference feature point, the value of C1_LR(q _i , q _j ) is 0, otherwise it falls on the right side, and the value of C1_LR(q _i , q _j ) is 1. Similarly, if the relevant feature point falls below the reference feature point, the value of C1_UD(q _i , q _j ) is 0; otherwise, the value of C1_UD(q _i , q _j ) is 1.

Sub-step 805, in order to generate another four equally divided fan-shaped areas in the local circular neighborhood, the local circular area is rotated clockwise by an angle of π/4 around the reference feature point, and the new position of the relevant feature point becomes:

Then two new codes C2_LR(q _i ,q _j ) and C2_UD(q _i ,q _j ) are calculated:

The geometric encoding of the relevant feature points relative to the reference feature points in the extracted ROI of the query image is calculated above. In the same way, the geometry of the relevant feature points relative to the reference feature points can also be calculated in the initial retrieval result breast cancer image. Coding: C1_LR(d _i ,d _j ), C1_UD(d _i ,d _j ), C2_LR(d _i ,d _j ) and C2_UD(d _i ,d _j ).

Sub-step 806, XOR operation is performed on the 8 geometric codes obtained in the geometric coding process:

where ⊕ is an XOR operation, taking (17) as an example, assuming that a relevant feature point in the local circular neighborhood of the reference feature point in the query image ROI is located in the left fan-shaped segmented area of the reference feature point, if the initial retrieval As a result, the matching related feature points in the breast cancer image are also located in the fan-shaped segmented area in the same orientation as the reference feature point, so the value of V1_LR((q _i ,q _j ),(d _i ,d _j )) is 0 , otherwise the value is 1.

Then, V1_LR((q _i ,q _j ),(d _i ,d _j )) and V2_LR((q _i ,q _j ),(d _i , d _j )) and and Accumulate and then sum, and score the rationality of the geometric relationship of the reference feature points, expressed as follows:

Sub-step 807, by summing S_LR(q _i , d _i ) and S_UD(q _i , d _i ), the final score of the rationality of the geometric relationship of the reference point is obtained, expressed as follows:

Sco(q _i ,d _i )=S_LR(q _i ,d _i )+S_UD(q _i ,d _i ) (23)

At this point, set a threshold Compare Sco(q _i ,d _i ) with it to exclude false matches, expressed as follows:

in, is the number of relevant feature points that match each other in the local circular area of the reference feature point, and ω is the weighting coefficient. If the match is correct, the value of C(q _i ,d _i ) is 0, otherwise it is 1.

The above steps have completed the verification of the geometric relationship of the relevant feature points in the local circular neighborhood of a pair of matching visual words, and then the relevant features in the local circular neighborhood of all matching visual words in the breast cancer query image ROI and the initial retrieval result ROI Verify the geometric relationship of the points, and make statistics on all C(q _i , d _i ), and sort the initial retrieval results in ascending order according to the C(q _i , d _i ) statistical results of each initial retrieval result image ROI, then match The search results with high accuracy will be ranked in the top position.

Those skilled in the art should understand that the embodiments of the present invention may be provided as methods, systems, or computer program products. Accordingly, the present invention can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams.

While preferred embodiments of the invention have been described, additional changes and modifications to these embodiments can be made by those skilled in the art once the basic inventive concept is appreciated. Therefore, it is intended that the appended claims be construed to cover the preferred embodiment as well as all changes and modifications which fall within the scope of the invention.

Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention also intends to include these modifications and variations.

Claims (5)

1. based on the medical image retrieval method of geometric verification, it is characterized in that, the method comprises the following steps: Step 1, extract SIFT features to all images in the breast cancer image database, and store them in the image feature database; Step 2, performing hierarchical K-means clustering on the SIFT features of all extracted database images to generate a breast cancer image visual vocabulary tree; Step 3, establishing an inverted index for the generated breast cancer image visual vocabulary tree; Step 4, describe all breast cancer database images through the visual words in the breast cancer image visual vocabulary tree; Step 5, after receiving the query image, extract the SIFT feature of the query image, quantify the feature of the query image according to the established visual vocabulary tree based on the inverted index, and describe the query image in the form of a visual word vector; Step 6, based on the inverted index, measure the similarity between the quantified query image and the quantified image in the database to form an initial retrieval result; Step 7, taking the matching visual words in the query image and the initial retrieval result image as reference points to delineate two local circular neighborhoods with the same radius; Step 8: Carry out geometric verification on the relevant feature points in the two local circular neighborhoods, eliminate the wrongly matched feature points, and obtain the final retrieval result. 2. The medical image retrieval method based on geometric verification as claimed in claim 1, wherein step 2 specifically comprises the following sub-steps: Sub-step 201, setting parameters: first set a height L and the number K of non-leaf node subtrees for the visual vocabulary tree to be created, that is, the number of cluster centers in each layer; Substep 202, generate a subtree: perform K-means clustering on the extracted breast cancer image SIFT feature data set to obtain K cluster centers, then these K cluster centers are the K child nodes of the root node, and the data The set is correspondingly divided into K subsets; Sub-step 203, recursive process: perform K-means clustering operation on the obtained K subsets respectively, and obtain K cluster centers of each subset, that is, the child nodes of the node in this layer of the vocabulary tree, iteratively carry out this step operation until The height of the vocabulary tree reaches L. 3. The medical image retrieval method based on geometric verification as claimed in claim 2, wherein the TF-IDF method is used to weight each visual word in the vocabulary tree, so as to quantitatively express the lovesickness of different visual words to the image The size of the measurement, the TF-IDF method includes: There are k visual words in the visual vocabulary tree, and the d-th image I _d in the breast cancer image database is expressed as a k-dimensional vector: v _i =(t ₁ ,t ₂ ,…,t _k ) ^T (1) The i-th vector element t _i expression of vector v _i is: t _i =tf _i,d ×idf _i (2) In formula (2), tf _i,d represent the frequency of the i-th visual word in the visual dictionary appearing in the d-th image in the image database, and idf _i represents the i-th visual word in the visual dictionary in all images in the image database The inverse file frequency of , tf _i,d and idf _i are calculated as: In formula (3), n _i,d represent the number of times that the i-th visual word in the visual dictionary appears in the d-th image of the image database, n _d represents the total number of all visual words that are counted in the d-th image of the database, In formula (4), N is the total number of images in the image database, and N _i represents the total number of times the i-th visual word appears in the image database images. Therefore, the expression of the i-th vector element in the image vector v _i is as follows: When a visual word in the visual dictionary appears more frequently in an image, that is, the larger the value of _ni,d , the larger the word frequency tf _i,d , indicating that the representativeness of the visual word is stronger; at the same time, when the The less the visual word appears in other images in the image database, that is, the smaller the value of _{Ni is, the larger the value of inverse document frequency idf i is} , and the higher the degree of discrimination of the visual word is. 4. The medical image retrieval method based on geometric verification as claimed in claim 1, wherein step 7 specifically comprises the following sub-steps: Sub-step 701, define the ROI of the query image as I _q , define the ROI of an initial retrieval result image as I _d , Q and D represent the SIFT feature point sets on the query image and initial retrieval result image ROI respectively, and obtain the matching The set of feature point pairs M(Q,D)={(q _i ,d _i )|q _i ∈Q,d _i ∈D}, q _i and d _i respectively represent the mutual matching on the ROI of the query image and the initial retrieval result image Select one pair of matching feature points as the reference feature point pair (q _i , d _i ) to verify whether the matching pair is a wrong match; Sub-step 702, after the matching pair is selected, the local circular neighborhood of the matching pair is defined, and the radius of the two corresponding circular areas is defined as follows: Among them, scl _i and scl _i ' represent the scale parameters of the feature points of the query image in the matching pair and the feature points of the initial retrieval result image, and δ is a parameter set to control the range of the local circular neighborhood. 5. The medical image retrieval method based on geometric verification as claimed in claim 4, wherein step 8 specifically comprises the following sub-steps: Sub-step 801, exclude irrelevant feature points outside the circular neighborhood in sub-step 702, the following formula: in, and Respectively represent the set of feature points located in the local circular neighborhood on the ROI of the query image and the initial retrieval image, dist(,) represents the Euclidean distance between the relevant feature point and the reference feature point; Sub-step 802, extracting relevant feature points matched in two local circular neighborhoods, the extraction method is as follows: In sub-step 803, the direction of the reference feature point is used as the new coordinate axis, and the position of the relevant feature point after rotation adjustment is calculated as follows: Among them, _θi represents the main gradient direction of the reference feature point SIFT; Sub-step 804, encode the geometric positional relationship of each pair of matching relevant feature points with respect to their respective reference feature points on the segmented quarter circle to verify whether they fall in the same segmented area, the calculation formula is as follows: If the relevant feature point falls on the left side of the reference feature point, the value of _{C1_LR} (q _i , _{q j} ) is 0; If the point falls below the reference feature point, the value of C1_UD(q _i , q _j ) is 0, and on the contrary, the value of C1_UD(q _i , q _j ) is 1; Sub-step 805, rotate the local circular area clockwise by an angle of π/4 around the reference feature point to generate another 4 equally divided fan-shaped areas in the local circular neighborhood, then the new position of the relevant feature point becomes: Then two new codes C2_LR(q _i ,q _j ) and C2_UD(q _i ,q _j ) are calculated: In the same way, calculate the geometric encoding of the relevant feature points relative to the reference feature points in the initial retrieval result breast cancer image: C1_LR(d _i ,d _j ), C1_UD(d _i ,d _j ), C2_LR(d _i , d _j ) and C2_UD(d _i ,d _j ); Sub-step 806, XOR operation is performed on the 8 geometric codes obtained in the geometric coding process: V1_LR((q _i ,q _j ),(d _i ,d _j )) and V2_LR((q _i ,q _j ),(d _i ,d _j ))as well as and Accumulate and then sum, and score the rationality of the geometric relationship of the reference feature points, expressed as follows: Sub-step 807, by summing S_LR(q _i , d _i ) and S_UD(q _i , d _i ), the final score of the rationality of the geometric relationship of the reference point is obtained, expressed as follows: Sco(q _i ,d _i )=S_LR(q _i ,d _i )+S_UD(q _i ,d _i ) (23) set a threshold Compare Sco(q _i ,d _i ) with it to exclude false matches, expressed as follows: in, is the number of related feature points matched with each other in the local circular area of the reference feature point, ω is the weighting coefficient, if the match is correct, the value of C(q _i ,d _i ) is 0, otherwise the value is 1; The above steps have completed the verification of the geometric relationship of the relevant feature points in the local circular neighborhood of a pair of matching visual words, and then the relevant features in the local circular neighborhood of all matching visual words in the breast cancer query image ROI and the initial retrieval result ROI Verify the geometric relationship of the points, and make statistics on all C(q _i , d _i ), and sort the initial retrieval results in ascending order according to the C(q _i , d _i ) statistical results of each initial retrieval result image ROI, then match The search results with high accuracy will be ranked in the top position.