JP4912015B2 - Similar image retrieval apparatus and method, and program - Google Patents

Description

  The present invention relates to a similar image search apparatus and method for searching for an image similar to an image to be processed, and a program therefor.

  Conventionally, image diagnosis for diagnosing a subject by interpreting an X-ray image or CT image of the subject has been performed in various fields. In image diagnosis, it may be difficult to make an appropriate diagnosis if the image reader is inexperienced or there is an experience, but the diagnosis target is an unusual case. In order to solve such a problem, for example, it is considered effective to make a diagnosis while referring to a case of an image similar to an image to be diagnosed in past cases.

Therefore, a large number of images that have been diagnosed in the past and diagnostic reports that include the diagnosis results and findings at that time are stored so that diagnosis can be performed while referring to past cases of images similar to the images to be diagnosed. A method of searching for an image similar to an image to be diagnosed and a diagnosis report thereof from a case database and displaying the search result has been proposed by the present applicant (Patent Documents 1 and 2).
Japanese Patent Laid-Open No. 2005-065728 JP 2004-005364 A

  By the way, recently, there is a concept of improving the reliability of diagnosis by simultaneously interpreting a plurality of images including the same part of the same subject. For example, it is conceivable to predict future changes in symptoms by simultaneously interpreting a plurality of images having the same imaging device (imaging modality) used for imaging and different in time series. Further, for example, it may be possible to narrow down symptoms by simultaneously interpreting a plurality of images having different shooting modalities. As an example of the latter, in Japanese Patent Application Laid-Open No. 2003-000575, a method for determining whether a tumor is benign or malignant using two types of images having different imaging modalities consisting of a mammogram (breast radiation image) and an ultrasound image (echo). Is disclosed.

  Therefore, in the above-described method for searching for images related to past cases similar to the image to be diagnosed and displaying the search results, a plurality of images obtained by photographing the same subject with the image to be diagnosed It is conceivable to search for an image similar to each of the plurality of images. Further, in this case, if there is a subject portion that is suspected of being a lesion on each image to be diagnosed, the region surrounding such a portion is set as a region of interest, and the feature amount of the image of these regions of interest It is conceivable to search for an image similar to the image to be diagnosed based on at least the feature amount.

  However, the work of setting each region of interest surrounding the same subject portion for each of a plurality of images to be diagnosed is very complicated.

  In view of the above circumstances, the present invention sets a region of interest that surrounds the same subject portion on a plurality of images to be diagnosed, and resembles a plurality of images to be diagnosed based on the characteristics of the images of these regions of interest. An object of the present invention is to provide a similar image search device and method, and a program therefor, which can simplify the setting of a region of interest and reduce the burden on an operator when searching similar images related to past cases. It is what.

  The similar image search device of the present invention stores a large number of images obtained by photographing a subject with a first photographing device and images obtained by photographing a subject of the same type as the subject with a second photographing device. A first image obtained by photographing the same kind of subject as the subject with the first photographing device and the same subject as the first image by the second photographing device. Image input means for accepting input of the obtained second image as a search target; alignment means for aligning a subject between the first and second images and obtaining a corresponding positional relationship between the images; A region-of-interest specifying means for accepting specification of a region of interest on the first image, and a region on the second image corresponding to the region of interest specified on the first image on the second image As a region of interest based on the corresponding positional relationship between the images. Corresponding region-of-interest determining means to be determined, and the first feature amount obtained by calculating the feature amount for the first and second images and including the feature amount of the image of the region of interest on the image An acquisition unit; a second feature amount acquisition unit that acquires a feature amount for an image captured by the first and second imaging devices stored in the image database; and the first feature amount The feature amount acquired by the acquisition unit and the feature amount acquired by the second feature amount acquisition unit are compared, and the image stored in the image database is similar to the first image. Search means for searching for an image obtained by photographing with the first photographing apparatus and an image obtained by photographing with the second photographing apparatus similar to the second image, and the retrieval means Search result information showing the search results by It is characterized in that an output means for.

  In the similar image search device of the present invention, an image obtained by photographing with the first and second photographing devices is photographed with an object including a plurality of anatomical structures by the first and second photographing devices. When the plurality of anatomical structures obtained by photographing the subject using the first photographing device is photographed by the first photographing device. Obtained by photographing a subject using a first model image storage means for storing a first standard density distribution model image expressed using a standard density value appearing on the image and the second photographing device. A second model for storing a second standard density distribution model image represented by using standard density values appearing on the image when a plurality of anatomical structures are photographed by the second photographing apparatus. An image storage means; the first standard density distribution model image; and the first image. First alignment means for obtaining a first corresponding position obtained by aligning corresponding anatomical structures with the second standard density distribution model image, the second image, A second alignment means for obtaining a second corresponding position obtained by aligning corresponding anatomical structures between the first image and the second image Corresponding positions of the anatomical structures are defined as corresponding positions of the corresponding anatomical structures between the first standard concentration distribution model image and the second standard concentration distribution model image, and the first corresponding positions. And a corresponding position obtaining means for obtaining from the second corresponding position.

  The similar image retrieval method of the present invention is obtained by photographing a first image obtained by photographing a subject with the first photographing device and a subject identical to the first image with the second photographing device. An image input step for accepting the input of the second image as a search target, an alignment step for aligning a subject between the first and second images and obtaining a corresponding positional relationship between the images, A region-of-interest specifying step for receiving a region-of-interest specification on the first image, and a region on the second image corresponding to the region of interest specified on the first image on the second image As a region of interest, a corresponding region-of-interest determining step that is determined based on the corresponding positional relationship between the images, and a feature amount for the first and second images, and a feature amount of the image of the region of interest on the image Calculating and obtaining what is included A feature amount obtaining step, an image obtained by photographing a subject of the same type as the subject with a first photographing device, and an image obtained by photographing a subject of the same type as the subject with a second photographing device; Acquired by the second feature amount acquisition step for acquiring the feature amount for the image acquired by the first and second imaging devices in the image database storing a large number of images, and the first feature amount acquisition step The first feature similar to the first image is selected from among the images stored in the image database by comparing the feature amount obtained and the feature amount obtained in the second feature amount obtaining step. A search step for searching for an image obtained by photographing with the second photographing device and an image obtained by photographing with the second photographing device similar to the second image, and a search result by the retrieval step Represents search result information It is characterized in that an output step of outputting.

The program of the present invention stores a large number of images obtained by photographing a subject with a first photographing device and images obtained by photographing a subject of the same type as the subject with a second photographing device. A first image obtained by photographing the same kind of subject as the subject with the first photographing device and the same subject as the first image by the second photographing device. Image input means for accepting input of the obtained second image as a search target; alignment means for aligning a subject between the first and second images and obtaining a corresponding positional relationship between the images; ,
A region-of-interest designating unit that accepts designation of a region of interest on the first image, and a region on the second image corresponding to the region of interest designated on the first image on the second image Corresponding region-of-interest determining means for determining the region of interest based on the corresponding positional relationship between the images, and a feature amount for the first and second images, and a feature amount of the image of the region of interest on the image A first feature amount acquisition means for calculating and acquiring an inclusion, and a feature amount for an image obtained by photographing with the first and second photographing devices stored in the image database; The feature amount acquired by the second feature amount acquisition unit, the feature amount acquired by the first feature amount acquisition unit, and the feature amount acquired by the second feature amount acquisition unit are compared and stored in the image database. From the images Retrieval means for retrieving an image obtained by photographing with the first photographing device similar to one image and an image obtained by photographing with the second photographing device similar to the second image The computer is caused to function as a similar image search device by functioning as output means for outputting search result information representing a search result by the search means.

  The image database need not be a single database, and may be a plurality of databases. For example, it may be composed of a plurality of databases existing in a single facility, or may be a plurality of databases scattered in a plurality of different facilities that can be connected via a network. That is, it includes a form in which images and diagnostic data in a database existing in each facility are shared and used.

  Further, the images stored in the image database need not be stored in the same place, and may be stored in different places.

  The search result information output by the output means means a searched image itself, a data file name specifying the searched image, or the like.

  Here, the feature amount is, for example, the shape, size, shading pattern characteristics, position, direction, number of shadows appearing in the image, shading pattern characteristics of the entire image, structure (lung, heart, etc.) In addition to index values representing the shape and size of the image, index values used to determine the degree of malignancy of the detected abnormal shadow (for example, in the case of a mass shadow, circularity, edge information of the contour, region) Density histogram information, etc .; see Japanese Patent Application Laid-Open No. 9-167238), and the like, which can determine similarity based on image data. It includes index values representing the shape, size, shading pattern characteristics, etc. of the appearing shadow.

  The similar image retrieval apparatus of the present invention further includes a diagnostic database storing diagnostic data related to images stored in the image database, and the search means stores diagnostic data related to the searched similar images. It is also possible to further search from the diagnostic database, and the output means outputs the searched diagnostic data as search result information together with related similar images.

  The diagnostic data is a record of various data relating to diagnosis such as disease name, diagnostic information, definitive diagnostic information, treatment policy, findings, doctor name in charge, photographing date and time, follow-up information.

  Note that the first and second imaging devices may be different types of imaging devices or the same type of imaging device. Particularly in the latter case, the first and second images input as search targets may be time-series images.

  According to the similar image search apparatus and method of the present invention, input of a plurality of images representing the same subject is received, alignment between the plurality of images is performed, and a corresponding positional relationship between the images is acquired. The region of interest is designated on one of the images, and the region on the other image corresponding to the designated region of interest is set as the region of interest on the other image based on the corresponding positional relationship between the images. Determining and calculating feature quantities for a plurality of input images that include the feature quantities of the region of interest image, while obtaining similar feature quantities for the images stored in the database, Comparing the feature quantities related to the plurality of input images with the feature quantities related to the images stored in the database, each of the images stored in the database is similar to the plurality of input images. The search result is output and the search result is output. Therefore, by specifying the region of interest once on any of the images to be diagnosed, the corresponding region of interest is set on the other image. Can set a region of interest that surrounds the same subject on a plurality of images to be diagnosed, and based on the characteristics of the images of these regions of interest, past cases similar to the plurality of images to be diagnosed When searching for such a similar image, it is possible to simplify the setting of the region of interest and reduce the burden on the operator.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing the configuration of a similar image search system according to an embodiment of the present invention. This similar image search system captures images when a set of a plurality of medical images having different imaging modalities obtained by imaging the same part of the same patient as the same subject with different types of imaging devices is input as a search target. A similar image set having the same modality as that of this image set and similar features such as an image and diagnostic data related to the similar image set are searched from the database, and the search result is output. Here, the image data represents an image, and the following description will be given without particularly distinguishing the image from the image data.

  The similar image retrieval system according to the present embodiment stores a case database 10 that stores a large number of case image sets including a plurality of images having different imaging modalities representing the same subject and diagnostic data related to the case image sets. And an image input unit (image input means) 20 that accepts input of a plurality of images S1 and S2 having different shooting modalities representing the same subject as search targets, and the images S1 and S2 are aligned with each other. An image registration unit (positioning unit) 30 that acquires the corresponding positional relationship B12 of the two, and a region of interest designating unit (region of interest designating unit) 40 that designates a region of interest (ROI) on one of the images S1 and S2. The region on the other image corresponding to the region of interest on the one image is set as the region of interest on the other image based on the corresponding positional relationship B12. A corresponding region-of-interest determination unit (corresponding region-of-interest determination means) 50, and a first feature amount acquisition unit (first unit) that calculates and acquires feature amounts of the images S1 and S2 and the image of the region of interest. Feature amount acquisition means) 60 and a second feature amount acquisition unit (second feature amount acquisition means) for acquiring feature amounts for images having the same imaging modality as the images S1 and S2 among the images constituting the case image set. ) 70 and the feature amount related to the image set SS composed of the images S1 and S2 and the feature amount related to the case image set are compared, and a similarity calculation unit that calculates the overall similarity LT between the two image sets ( (Similarity calculation means) 80, a search unit (search means) 90 for searching case image sets and their diagnostic data included in the upper predetermined number from the case database 10 in descending order of the total similarity LT, and a search unit 9 And an output unit (output means) 100 for outputting the search result information indicating the search results, and a display unit 110 for displaying the search result information output by the output unit 100 on the monitor screen by.

  The case database 10 includes an image database 11 that stores a large number of case image sets including a plurality of images having different imaging modalities that represent the same subject, and a diagnostic database 12 that stores diagnostic data related to each case image set. ing.

  The image database 11 includes a CT image, an MRI image, a PET image, a simple X-ray image, and the like obtained by imaging the same subject using a CT imaging device, an MRI imaging device, a PET imaging device, an X-ray imaging device, or the like. A large number of case image sets, which are a series of images and have been diagnosed in the past, are stored for each subject and for each diagnosis. Note that processing result information obtained by processing performed at the time of past diagnosis is attached to the images constituting the case image set. This processing result information includes various feature amounts calculated for the image, information on the position of the region of interest set on the image, feature amounts calculated for the image of the region of interest, and the like. .

  The diagnosis database 12 stores diagnosis data related to each case image set stored in the image database 11 in a state associated with each case image set. This diagnosis data includes various data relating to diagnosis such as disease name, diagnosis information, definitive diagnosis information, treatment policy, findings, doctor name in charge, photographing date and time, follow-up information, and the like. When the diagnostic database 12 is constructed, diagnostic data, particularly diagnostic information and definitive diagnostic information may be manually registered and stored in the diagnostic database, or PACS and electronic medical record data may be automatically stored. You may make it memorize | store in the form collected.

  The image input unit 20 receives an input of a plurality of images S1 and S2 having different imaging modalities representing the same subject, which are images to be searched. Here, as an example, a CT image S1 representing the same part of the same patient. And the input of the MRI image S2.

  As shown in FIG. 2, the image aligning unit 30 is a CT standard density distribution model image M1 (first standard density distribution model image) representing a standard density distribution of an image obtained by imaging with a CT imaging apparatus. ) And a standard density distribution model image M2 for MRI (second standard density) representing a standard density distribution of an image obtained by photographing with an MRI photographing apparatus. Corresponding positions of corresponding anatomical structures between the second standard concentration distribution model image storage unit 32 storing the distribution model image), the CT standard concentration distribution model image M1, and the MRI standard concentration distribution model image M2. Corresponding anatomical structures are aligned between the CT image correspondence position storage unit 33 that stores the BM 12, and the CT image S1 and the CT standard density distribution model image M1, and a first corresponding position BS1 is obtained. Take The second position for acquiring the second corresponding position BS2 by aligning the corresponding anatomical structure between the first position matching unit 34 and the MRI image S2 and the standard density distribution model image M2 for MRI. The matching position B12 and the corresponding position B12 of the anatomical structure between the CT image S1 and the MRI image S2 are the same as the corresponding position BM12 between the CT standard density distribution model image M1 and the MRI standard density distribution model image M2. The corresponding position acquisition unit 36 is obtained from the corresponding position BS1 and the second corresponding position BS2.

  The CT image S1, the MRI image S2, the CT standard density distribution model image M1, and the MRI standard density distribution model image M2 are volume data composed of a large number of voxel data, and hold density value information for each voxel data. Anatomical structures such as bones and organs are represented by shades of voxel data.

  The standard density distribution model image is an anatomical structure based on a large number of CT images and MRI images taken in the past, in which a normal person has a standard body shape and tissues such as internal organs in a standard size and position. Is a model image representing a standard density value distribution. Since the density value that appears in each image varies depending on the modality that was taken, the CT standard density distribution model image M1 represented by the standard density value that appears on the image when taken with the CT imaging apparatus, and the MRI imaging apparatus Then, an MRI standard density distribution model image M2 represented by standard density values appearing on the image is prepared.

  These standard density distribution model images can acquire the average position and density value of each organ from a number of images taken in the past. In addition, a model image generated so that each voxel data included in the standard concentration distribution model image belongs to which anatomical structure (bone, viscera, fat, etc.) belongs is preferable. For example, if coordinate values (x, y, z) are given to the CT standardity distribution model image M1 (CT_Model), the density value and the tissue name can be understood as shown in the following equations (1) and (2). 1 standard density distribution model image storage unit 31.

Density value CT_Model_intensity (x, y, z) = 100 (1)
Organization name CT_Model_tissue (x, y, z) = Liver (2)
Similarly, when coordinate values (x, y, z) are given to the MRI standard density distribution model image M2 (MRI_Model), the density value and the tissue name can be understood as shown in the following equations (3) and (4). The data is stored in the second standard density distribution model image storage unit 32.

Concentration value MRI_Model_intensity (x ', y', z ') = 80 (3)
Organization name MRI_Model_tissue (x ', y', z ') = liver (4)
In the inter-model image corresponding position storage unit 33, the corresponding positions BM12 of the anatomical structures of the CT standard density distribution model image M1 and the MRI standard density distribution model image M2 are stored in correspondence with each voxel data. To do. That is, a relationship represented by the following formula (5) is obtained in advance.

CT_Model (x, y, z) = MRI_Model (x ', y', z ') (5)
This corresponding position may be obtained manually.

  Here, a registration method by the image registration unit 30 will be described.

  First, the first alignment unit 34 aligns the CT image S1 and the CT standard density distribution model image M1 to obtain a first corresponding position BS1. Images taken with the same imaging modality are more accurate than the alignment of images taken with different imaging modalities because the density distribution of anatomical structures such as organs and tissues shown on the image are similar. Good alignment is possible.

  For example, the body surface is recognized based on the pixel value of the CT image S1, and the shape of the body surface of the CT standard density distribution model image M1 stored in the first standard degree distribution model image storage unit 31 substantially matches. As described above, alignment is performed using a technique such as warping to obtain the first corresponding position BS1.

  In this way, the first corresponding position BS1 as in the following equation (6) is obtained between the CT image S1 (CT_Data) and the CT standard density distribution model image M1.

CT_Model (x1, y1, z1) = CT_Data (x, y, z) (6)
Similarly, the second alignment unit 35 aligns the MRI image S2 (MRI_Data) and the MRI standard density distribution model image M2 stored in the second standard density distribution model image storage unit 32. Then, a second corresponding position BS2 as shown in the following equation (7) is obtained.

MRI_Model (x2, y2, z2) = MRI_Data (x ', y', z ') (7)
The corresponding position acquisition unit 36 obtains the coordinate values (x ′, y ′,...) Of the MRI image S2 corresponding to the coordinate values (x, y, z) of the CT image S1 from the above equations (5), (6), (7). z ′) can be obtained. By obtaining this for all voxel data, the CT image S1 and the MRI image S2 are aligned, and a corresponding positional relationship B12 between these images is obtained.

  The region-of-interest designating unit 40 designates a region of interest on the CT image S1 or MRI image S2 to be searched, and the operator views the image displayed on the monitor screen of the display unit 110 while viewing the image. For example, a portion suspected of being a lesion is designated using a position indicating means (not shown) such as a mouse.

  When the region of interest is specified on one of the CT image S1 or the MRI image by the region of interest specifying unit 40, the corresponding region of interest determining unit 50 selects the region of interest on the other image corresponding to the region of interest on the one image. The region is determined as a region of interest on the other image based on the corresponding positional relationship B12 between the CT image S1 and the MRI image S2 acquired by the image alignment unit 30, and for example, on the CT image S1. When the region of interest R1 is designated, the region on the MRI image S2 representing the same subject portion as the subject portion corresponding to the region of interest R1 is automatically determined as the region of interest R2.

  In general, when an operator finds a suspected lesion in an image to be diagnosed, the operator designates that portion as a region of interest and calculates the similarity using the feature amount of the image of the region of interest. In the case where there are a plurality of images to be diagnosed, it is conceivable that the same subject portion on the plurality of images is set as a region of interest. In such a case, as described above, if the same subject portion on a plurality of images is set as a region of interest by only one designation, the burden on the operator is reduced.

  The first feature amount acquisition unit 60 calculates and acquires a feature amount C1 for the CT image S1 and the MRI image S2. As the feature amount, for example, the shape of the shadow appearing in the image, Examples include size, shading pattern characteristics, position, direction, number, shading pattern characteristics of the entire image, and index values representing the shape and size of structures (lung, heart, etc.). When the region of interest R1 is set on the CT image S1, or when the region of interest R2 is set on the MRI image S2, the shape of the shadow appearing in the image of the region of interest, An index value representing the size, the density pattern characteristic, etc. is also calculated as a feature amount.

  The second feature amount acquisition unit 70 has images having the same imaging modality as the CT image S1 and the MRI image S2 among the images constituting the case image set, that is, the imaging modalities are “CT imaging apparatus” and “MRI imaging apparatus”. Is obtained in the processing result information attached to the images constituting the case image set, and is set on the various feature amounts already calculated for the image and on the image. When the calculated feature amount is included in the image of the region of interest, the feature amount is read and acquired. When such a feature amount is not included, a new feature amount is calculated. Is what you get.

  The similarity calculation unit 80 includes the feature amount acquired by the first feature amount acquisition unit 60, that is, the feature amount C1 related to the image set SS including the CT image S1 and the MRI image S2, and the second feature amount acquisition unit. The feature amount acquired by 70, that is, the feature amount C2 related to the case image set is compared to calculate the overall similarity LT between the two image sets. From the CT image S1 and the MRI image S2, For each combination of images having the same imaging modality between the image set SS and the case image set, the feature amounts in both images constituting the combination are compared with each other, and the similarities L1 and L2 between the images are compared. Using the similarity calculation unit for each modality (photographing device similarity calculation means) 81 and a plurality of similarities L1 and L2 calculated for each combination. And a general similarity calculation section (general similarity calculation means) 82 for calculating a.

  More specifically, the modality-specific similarity calculation unit 81 sets both of the images constituting the combination with respect to the combination of images having the imaging modality “CT imaging apparatus” between the image set SS and the case image set. The feature quantities acquired for the images are compared with each other to calculate a similarity L1 for this combination. For the combination of images whose imaging modality is “MRI imaging apparatus”, both images constituting this combination are used. Is obtained by comparing the feature quantities acquired for the combination, and calculating the similarity L1 for this combination. Here, various known methods can be considered as the similarity calculation method. For example, each feature amount group acquired for two images is represented by a point on a multidimensional feature amount space. A method is conceivable in which the similarity is calculated by a predetermined formula in which the similarity is higher as the Euclidean distance is smaller, and the similarity is smaller as the distance is larger.

  Further, the total similarity calculation unit 82 adds, for example, the similarity L1 and L2 calculated by the modality similarity calculation unit 81 using a predetermined weighting factor or takes a product, thereby obtaining the image set SS. And the total similarity LT between the case image set and the case image set is calculated.

  The search unit 90 sets the case image sets included in the upper predetermined number in the descending order of the overall similarity LT among the many case image sets stored in the image database 11 as the similar image set LS. And the diagnostic data related to the set of similar images is searched from the diagnostic database 12. Here, it is assumed that the search unit 90 searches for the case image sets included in the top four in descending order of the overall similarity LT as the similar image set LS.

  Next, the operation of the similar image search system of the present embodiment configured as described above will be described. FIG. 3 is a diagram showing a processing flow in the similar image search system according to the present embodiment.

  First, the image input unit 20 receives an input of an MRI image S2 of the same subject as the CT image S1 of a certain subject to be diagnosed, and the display unit 110 displays the CT image S1 and the MRI image S2 on the monitor screen. (Step ST1).

  FIG. 4 shows an MRI image S2 of the chest of the same subject as the CT image S1 of the chest of a subject to be diagnosed. A circular shadow Q suspected of being a tumor shadow appears in the left lung portion of the CT image S1 and the MRI image S2.

  Then, the image alignment unit 30 aligns the subject between the CT image S1 and the MRI image S2 by the above-described processing, and acquires the corresponding positional relationship B12 between the CT image S1 and the MRI image S2 (step S12). ST2).

  Here, when the operator designates a region suspected of being a lesion for one of the CT image S1 or the MRI image S2 displayed on the monitor screen of the display unit 110, the region of interest designating unit 40 The designated region is set as the region of interest on the one image (step ST3), and the corresponding region-of-interest determining unit 50 corresponds to the region designated on the one image based on the correspondence positional relationship B12. The region on the other image is determined as the region of interest on the other image (step ST4).

  In FIG. 5, the operator designates a region surrounding a shadow Q suspected of being a lesion on one of the CT image S1 or MRI image S2 as a region of interest, and a corresponding region on the other image is also a region of interest. The designated state is shown, and the region of interest R1 is set on the CT image S1, and the region of interest R2 is set on the MRI image S2.

  Here, when the operator clicks the “similar image search button” provided on the screen, the first feature quantity acquisition unit 60 performs the feature quantity, the image of the region of interest R1 and the interest of the CT image S1 and the MRI image S2. The feature amount C1 including the feature amount for the image in the region R2 is calculated and acquired (step ST5).

  When the feature quantity relating to the image set SS including the CT image S1 and the MRI image S2 is obtained by the first feature quantity acquisition unit 60, the second feature quantity acquisition unit 70 is stored in the image database 11. For each case image set, images having the same imaging modality as those of the CT image S1 and the MRI image S2 among images constituting the case image set, that is, images having the imaging modality of “CT imaging apparatus” and “MRI imaging apparatus”. In the processing result information identified and attached to these images, various feature quantities already calculated for the image, feature quantities calculated for the image of the region of interest set on the image, etc. Is included, the feature values are read and acquired.If these feature values are not included or if there are not enough feature values to be acquired, It is newly calculated amount, to obtain the feature quantity C2 of case image set (step ST6).

  When the feature amount C1 related to the image set SS including the CT image S1 and the MRI image S2 and the feature amount C2 related to each case image set stored in the image database 11 are acquired, the similarity calculation unit 81 by modality In addition, between the image set SS and each case image set, for the combination of images whose imaging modality is “CT imaging apparatus”, the feature amounts acquired for both images constituting this combination are compared. Then, the similarity L1 for this combination is calculated, and for the combination of images whose imaging modality is “MRI imaging apparatus”, the feature quantities acquired for both images constituting this combination are compared, A similarity L1 for this combination is calculated (step ST7). Then, the total similarity calculation unit 82 adds the similarities L1 and L2 calculated by the modality-specific similarity calculation unit 81 for each case image set using a predetermined weighting coefficient, thereby obtaining the image set SS. An overall similarity LT between the case image set is calculated (step ST8).

  When the overall similarity LT for each case image set is calculated, the search unit 90 selects the top four in descending order of the overall similarity LT from among a large number of case image sets stored in the image database 11. Is searched from the image database 11 as a similar image set LS, and diagnostic data related to the similar image set LS is searched from the diagnostic database 12 (step ST9). Then, the output unit 100 outputs search result information representing the search result, for example, a data file name representing the similar image set LS and related diagnostic data, image data of the similar image set LS, and diagnostic data itself. (Step ST10), the display unit 110 displays the data file name representing the image data of the similar image set LS and the related diagnostic data on the monitor screen, or the image or diagnostic data represented by the image data is displayed. The diagnostic information to be displayed is displayed on the monitor screen.

  FIG. 6 shows an example of the monitor screen of the display unit 110 after the similar image search is performed. On the monitor screen, a CT image S1 and an MRI image S2 in the upper left part, thumbnail images of the four similar image sets LS searched in the upper right part, and one similar image set selected from the similar image sets LS in the lower left part The enlarged image of LS1, and diagnostic data (diagnostic report) RPM1 related to the image selected from the similar image set which is an enlarged image are displayed in the lower right part.

  The operator can update and display an enlarged image of the image by clicking the desired thumbnail image, and can click on any of the enlarged similar image sets to display the image. The diagnostic report can be updated and displayed.

  As described above, according to the similar image search system of the present embodiment, input of a plurality of images S1 and S2 representing the same subject is accepted, alignment is performed between the plurality of images S1 and S2, and correspondence between the images is performed. The positional relationship B12 is acquired, the designation of the region of interest is received on one of the plurality of images S1 and S2, and the region on the other image corresponding to the designated region of interest is represented on the other image. The region is determined based on the corresponding positional relationship B12 between the images, and the feature amount for the plurality of input images S1 and S2 including the feature amount of the image of the region of interest is calculated. 11 is obtained for the images stored in the image 11, and the feature values related to the plurality of images S1 and S2 input are compared with the feature values related to the images stored in the image database. Since an image similar to each of the plurality of input images S1 and S2 is searched from the images stored in the image database 11, and the search result is output, any of the images to be diagnosed The region of interest R1 surrounding the same subject portion on the plurality of images S1 and S2 to be diagnosed can be set by simply designating the region of interest once on the other images. , R2 respectively, and when searching for similar images relating to past cases similar to the plurality of images S1, S2 to be diagnosed based on the characteristics of the images of the regions of interest R1, R2, It is possible to simplify the setting and reduce the burden on the operator.

  In addition, according to the present embodiment, not only a similar image similar to the plurality of images S1 and S2 to be diagnosed but also diagnostic data related to the similar image is output. It is possible to provide effective diagnosis support information when the user is at a loss in diagnosis at the time of image interpretation.

  In this embodiment, a plurality of images to be searched are a plurality of images having different imaging modalities, and these two images are two types of images, “CT imaging apparatus (simple)” and “MRI imaging apparatus”. As described above, there are three or more types of imaging modalities such as “CT imaging apparatus (simple)”, “CT imaging apparatus (with contrast medium)”, “MRI imaging apparatus”, “RI imaging apparatus”, “PET imaging apparatus”, etc. Even in this case, similar image search can be performed in the same manner. In this case, images of all shooting modalities may be searched as search targets, or images of two or more types of shooting modalities may be selected as search targets. For the similar image set LS obtained by the search, only the image of the imaging modality used for the search may be displayed on the screen, or all the images constituting the similar image set LS are displayed on the screen. You may make it do.

  In addition, the plurality of images to be searched may be images (time-series images) of the same shooting modality acquired with time for follow-up observation. In this way, there are many (for example, about 10 to 20) time-series images for the same subject, and the region of interest is set for all the images, and all of them are based on the characteristics of the image of the region of interest. Even in a case where similar images for each of the images are searched, it is sufficient to set a region of interest on one of the images, which is very efficient.

  In the present embodiment, it is assumed that the first feature quantity acquisition unit 60 and the second feature quantity acquisition unit 70 acquire a predetermined type of feature quantity. It is also possible to provide a selection means for selecting several types of the feature quantity types, and to acquire only the feature quantity of the type selected by the selection means.

  Further, in this embodiment, images similar to a plurality of images to be diagnosed are searched for in units of image sets, but images similar to the plurality of images may be searched separately. That is, an image similar to the CT image S1 and an image similar to the MRI image S2 may be searched separately.

Configuration diagram of similar image retrieval system according to this embodiment Configuration diagram of image alignment unit 30 The figure which shows the processing flow in the similar image search system by this embodiment. Diagram showing images with different shooting modalities that represent the same subject that is to be diagnosed and searched The figure which shows a mode that the region of interest was each set on two images used as search object Figure showing a display example of search results

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Case database 11 Image database 12 Diagnostic database 20 Image input part 30 Image registration part 31 1st standard density distribution model image memory | storage part 32 2nd standard density distribution model image memory part 33 Model image corresponding | compatible position memory part 34 1st 1 alignment unit 35 second alignment unit 36 corresponding position acquisition unit 40 region of interest designation unit 50 corresponding region of interest determination unit 60 first feature amount acquisition unit 70 second feature amount acquisition unit 80 similarity calculation unit 81 Modality similarity calculation unit 82 Total similarity calculation unit 90 Search unit 100 Output unit 110 Display unit

Claims (4)

An image database for storing a large number of images obtained by photographing a subject with a first photographing device and images obtained by photographing a subject of the same type as the subject with a second photographing device;
A first image obtained by photographing the same kind of subject as the subject with the first photographing device and a second image obtained by photographing the same subject as the first image with the second photographing device. Image input means for accepting input of the image as a search target;
Alignment means for aligning a subject between the first and second images to obtain a corresponding positional relationship between the images;
A region-of-interest specifying means for receiving specification of a region of interest on the first image;
Corresponding interest determined based on the corresponding positional relationship between the images, with the region on the second image corresponding to the region of interest specified on the first image as the region of interest on the second image An area determination means;
A first feature quantity acquiring means a feature quantity you get to calculate those containing a feature amount of an image of the region of interest on said image for said first and second images,
Second feature amount acquisition means for acquiring a feature amount for an image obtained by photographing with the first and second photographing devices stored in the image database;
The feature amount acquired by the first feature amount acquisition unit and the feature amount acquired by the second feature amount acquisition unit are compared, and the image stored in the image database is Search for searching for an image obtained by photographing with the first photographing device similar to the first image and an image obtained by photographing with the second photographing device similar to the second image Means,
Output means for outputting search result information representing a search result by the search means ,
The image obtained by photographing with the first and second photographing devices is an image obtained by photographing a subject including a plurality of anatomical structures with the first and second photographing devices,
The alignment means;
A plurality of anatomical structures obtained by photographing a subject using the first photographing device are represented by using standard density values appearing on the image when the first photographing device is photographed. First model image storage means for storing a first standard density distribution model image;
A plurality of anatomical structures obtained by photographing a subject using the second imaging device are represented using standard density values that appear on the image when the second imaging device is imaged. Second model image storage means for storing a second standard density distribution model image;
First alignment means for obtaining a first corresponding position obtained by aligning a corresponding anatomical structure between the first standard density distribution model image and the first image;
Second alignment means for obtaining a second corresponding position obtained by aligning a corresponding anatomical structure between the second standard density distribution model image and the second image;
Corresponding positions of corresponding anatomical structures between the first image and the second image correspond to the first standard density distribution model image and the second standard density distribution model image. A similar image retrieval apparatus comprising: a corresponding position acquisition unit that is obtained from a corresponding position of an anatomical structure, the first corresponding position, and the second corresponding position . A diagnostic database storing diagnostic data related to images stored in the image database;
The search means further searches the diagnostic database for diagnostic data related to the searched similar image;
It said output means, retrieved diagnostic data, associated similar image with the search results, characterized in that is output as information claim 1 Symbol placement of the similar image retrieval apparatus. The input of the first image and the second image obtained with the first same object and the image taken with the second imaging device obtained by photographing an object at a first imaging device, search An image input step to accept as a target;
An alignment step of aligning a subject between the first and second images to obtain a corresponding positional relationship between the images;
A region-of-interest specifying step for receiving a region-of-interest specification on the first image;
Corresponding interest determined based on the corresponding positional relationship between the images, with the region on the second image corresponding to the region of interest specified on the first image as the region of interest on the second image An area determination step;
A first feature amount acquisition step you get to calculate those containing a feature amount of an image of the region of interest on said image a feature value for said first and second images,
In an image database storing a large number of images obtained by photographing a subject of the same type as the subject with a first photographing device and images obtained by photographing a subject of the same type as the subject with a second photographing device. A second feature amount acquisition step of acquiring a feature amount with respect to an image obtained by photographing with the first and second photographing devices;
The feature amount acquired by the first feature amount acquisition step is compared with the feature amount acquired by the second feature amount acquisition step, and the image stored in the image database is Search for searching for an image obtained by photographing with the first photographing device similar to the first image and an image obtained by photographing with the second photographing device similar to the second image Steps,
An output step for outputting search result information representing a search result by the search step ;
The image obtained by photographing with the first and second photographing devices is an image obtained by photographing a subject including a plurality of anatomical structures with the first and second photographing devices,
The alignment step comprises:
A plurality of anatomical structures obtained by photographing a subject using the first photographing device are represented by using standard density values appearing on the image when the first photographing device is photographed. A first alignment step of obtaining a first corresponding position obtained by aligning a corresponding anatomical structure between a first standard density distribution model image and the first image;
A plurality of anatomical structures obtained by photographing a subject using the second photographing device are represented by using standard density values that appear on the image when the second photographing device is photographed. A second alignment step of obtaining a second corresponding position obtained by aligning corresponding anatomical structures between the second standard density distribution model image and the second image;
Corresponding positions of corresponding anatomical structures between the first image and the second image correspond to the first standard density distribution model image and the second standard density distribution model image. A similar image search method comprising: a corresponding position acquisition step obtained from a corresponding position of an anatomical structure, the first corresponding position, and the second corresponding position . Computer
An image database for storing a large number of images obtained by photographing a subject with a first photographing device and images obtained by photographing a subject of the same type as the subject with a second photographing device;
A first image obtained by photographing the same kind of subject as the subject with the first photographing device and a second image obtained by photographing the same subject as the first image with the second photographing device. Image input means for accepting input of the image as a search target;
Alignment means for aligning a subject between the first and second images to obtain a corresponding positional relationship between the images;
A region-of-interest specifying means for receiving specification of a region of interest on the first image;
Corresponding interest determined based on the corresponding positional relationship between the images, with the region on the second image corresponding to the region of interest specified on the first image as the region of interest on the second image An area determination means;
A first feature quantity acquiring means a feature quantity you get to calculate those containing a feature amount of an image of the region of interest on said image for said first and second images,
Second feature amount acquisition means for acquiring a feature amount for an image obtained by photographing with the first and second photographing devices stored in the image database;
The feature amount acquired by the first feature amount acquisition unit and the feature amount acquired by the second feature amount acquisition unit are compared, and the image stored in the image database is Search for searching for an image obtained by photographing with the first photographing device similar to the first image and an image obtained by photographing with the second photographing device similar to the second image Means,
A program for causing the computer to function as a similar image search device by functioning as output means for outputting search result information representing a search result by the search means ,
The image obtained by photographing with the first and second photographing devices is an image obtained by photographing a subject including a plurality of anatomical structures with the first and second photographing devices,
The alignment means;
A plurality of anatomical structures obtained by photographing a subject using the first photographing device are represented by using standard density values appearing on the image when the first photographing device is photographed. First model image storage means for storing a first standard density distribution model image;
A plurality of anatomical structures obtained by photographing a subject using the second imaging device are represented using standard density values that appear on the image when the second imaging device is imaged. Second model image storage means for storing a second standard density distribution model image;
First alignment means for obtaining a first corresponding position obtained by aligning a corresponding anatomical structure between the first standard density distribution model image and the first image;
Second alignment means for obtaining a second corresponding position obtained by aligning a corresponding anatomical structure between the second standard density distribution model image and the second image;
Corresponding positions of corresponding anatomical structures between the first image and the second image correspond to the first standard density distribution model image and the second standard density distribution model image. A program comprising: a corresponding position acquisition unit that is obtained from a corresponding position of an anatomical structure, the first corresponding position, and the second corresponding position .

Images