JP2014029644A - Similar case retrieval device and similar case retrieval method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a similar case retrieval device capable of efficiently retrieving similar cases with respect to a disease having plural disease types.SOLUTION: The similar case retrieval device includes: an image feature acquisition section 140 that acquires plural image feature quantity outputs from an image to be read; a primary retrieval section 150 that acquires plural case data which have the similarity with an image feature quantity output equal to or greater than a predetermined value; a reading item extraction section 160 that acquires reading items from a reading report included in the acquired case data; a case candidate cluster generating section 170 that generates plural case candidate clusters using the acquired reading items; a weight determination section 180 that identifies a reading item relative to the case candidate cluster to determine, on the basis of a reading knowledge database in which the relativeness between the respective image feature quantity output and the respective reading items are predetermined, a weight with a larger value to a higher relativeness between the image feature quantity output and the reading item; and a secondary retrieve section 190 that retrieves a case data by comparing while weighing the image feature quantity output of medical image included in the image feature quantity output and the case data acquired by the image feature acquisition section 140.

Description

  The present invention relates to a similar case search apparatus and a similar case search method for automatically searching for a case that serves as a reference for a case to be interpreted.

  In recent years, high-definition medical images that have been digitized can be acquired in a large capacity due to the development and spread of medical imaging devices such as CT (Computed Tomography) and MRI (Magnetic Resonance Imaging). In addition, medical images that have been interpreted by a doctor are being sequentially accumulated in a PACS (Picture Archiving and Communication Systems) together with an interpretation report. Here, in order to refer to a new interpretation, a technique for searching a past medical image similar to a medical image to be interpreted from accumulated past cases has begun to be developed.

  In the similar image search, it is important to optimize the image feature amount that determines the similarity between images according to the search target image. Conventionally, an image feature amount is often designed for each target organ for which a similar image search is performed, and the concept level other than the organ (for example, disease type, disease progression degree, disease severity, etc.) is used. On the other hand, even if the concept level is different, the same image feature amount is mostly used for similar image retrieval.

  A similar image search method is disclosed in which the image feature amount used for the search is dynamically changed with respect to a concept level other than the organ (see, for example, Non-Patent Document 1 and Patent Document 1).

Jennifer G. Dy et al. “Unsupervised Feature Selection Applied to Content-based Retrieval of Lung Images”, IEEE Transactions on Pattern Analysis and Machine Intelligence. 25, no. 3, March 2003

JP 2009-93563 A

  However, the conventional similar image search method cannot sufficiently cope with the case where the appearance of a lesion in an image greatly differs if the disease type is different even if the disease is the same. Also, preparing image feature amounts for each disease type in advance and performing similar image search using the prepared image feature amounts is not realistic from the viewpoint of processing time, considering that the number of disease types is enormous. Absent.

  The present invention solves the above-described conventional problems, and an object of the present invention is to provide a similar case search apparatus that can efficiently search for similar cases for diseases having a plurality of disease types.

  In order to solve the above-described conventional problem, a similar case search apparatus according to an aspect of the present invention includes a plurality of case data including a medical image and an interpretation report that is document data describing a result of interpretation of the medical image. A similar case retrieval apparatus for retrieving case data including a medical image similar to an interpretation target image that is a medical image to be interpreted from an accumulated case database, and an image for acquiring a plurality of image feature amounts from the interpretation target image Based on a feature acquisition unit, the plurality of image feature amounts acquired by the image feature acquisition unit, and a plurality of image feature amounts acquired from medical images included in each case data registered in the case database, Case data including the medical image in which the similarity between the image to be interpreted and the medical image included in each case data is calculated, and the calculated similarity is a predetermined value or more A plurality of primary search sections to be acquired; and an interpretation item extraction section to extract a plurality of interpretation items that are character strings obtained by verbalizing features of medical images from interpretation reports included in each case data acquired by the primary search section; Using a plurality of interpretation items extracted for each case data by the interpretation item extraction unit, a disease type candidate cluster generation unit that generates a plurality of disease type candidate clusters composed of case data groups with similar combinations of interpretation items; Extracted from each image feature amount extracted from the medical image and an interpretation report for the medical image for each of at least one disease type candidate cluster among the plurality of disease type candidate clusters generated by the disease type candidate cluster generation unit. For each image feature amount acquired by the image feature acquisition unit, based on an interpretation knowledge database in which relevance of each interpretation item is determined in advance, the image feature amount and the disease A weight determination unit that determines a greater weight as the relevance between the interpretation items included in the candidate cluster is higher, the plurality of image feature amounts extracted by the image feature acquisition unit, and the case database Similar to the image to be interpreted by weighting and comparing the plurality of image feature amounts acquired from the medical image included in the case data included in the case data with weights for each image feature amount determined by the weight determination unit A secondary search unit that searches case data including medical images to be searched from the case database.

  Note that the present invention can be realized not only as a similar case search apparatus including such a characteristic processing unit, but also by using a process executed by a characteristic processing unit included in the similar case search apparatus as a step. This can be realized as a case search method. It can also be realized as a program for causing a computer to function as a characteristic processing unit included in a similar case search apparatus or a program for causing a computer to execute characteristic steps included in a similar case search method. Such a program can be distributed via a computer-readable non-transitory recording medium such as a CD-ROM (Compact Disc-Read Only Memory) or a communication network such as the Internet. .

  According to the similar case retrieval apparatus of the present invention, it is possible to provide a similar case retrieval apparatus that can efficiently retrieve similar cases for diseases having a plurality of disease types.

The block diagram which shows the structure of the similar case search apparatus in embodiment of this invention The flowchart which shows the procedure of the interpretation knowledge database preparation in embodiment of this invention The flowchart which shows the procedure of the image feature extraction in embodiment of this invention The figure which shows the example of the interpretation report of the chest CT examination in embodiment of this invention The figure which shows the interpretation item and disease name extracted from the interpretation report in embodiment of this invention The figure which shows the information of the interpretation item and disease name extracted from the interpretation report in embodiment of this invention, and the position and presence extracted simultaneously with the interpretation item The figure which shows a data set acquired for the interpretation knowledge extraction in embodiment of this invention. The conceptual diagram of the correlation (binary) between the interpretation item and image feature-value in embodiment of this invention The conceptual diagram of the correlation (multi-value) between the interpretation item and image feature-value in embodiment of this invention The figure which shows the storage format of the correlation between the (image feature-value-interpretation item) extracted as interpretation knowledge in embodiment of this invention. The flowchart explaining the positioning of the similar case search process by the similar case search device in the diagnosis act of the doctor in the embodiment of the present invention The flowchart which shows the procedure of the similar case search in embodiment of this invention The figure which shows an example of the similar case search screen in embodiment of this invention The figure for demonstrating designation | designated of a lesion position or an area | region in embodiment of this invention The conceptual diagram which shows the mode of the primary search in embodiment of this invention The figure which shows an example of the interpretation item contained in the case obtained by the result of the primary search in embodiment of this invention The figure which shows an example of the interpretation item contained in the case obtained by the result of the primary search in embodiment of this invention The conceptual diagram of the clustering result by the interpretation item vector in embodiment of this invention The figure which shows an example of the clustering result by the interpretation item vector in embodiment of this invention The conceptual diagram of the weighting method using the average vector of all the interpretation item vectors which form a disease type candidate cluster in embodiment of this invention The figure which shows an example of the display / input screen for making the user (interpreter) of the similar case search device select an illness type candidate cluster in embodiment of this invention. The figure which shows an example of the display / input screen for making the user (interpreter) of the similar case search device select an illness type candidate cluster in embodiment of this invention. The conceptual diagram which shows the weighting method regarding a disease type candidate cluster in embodiment of this invention The conceptual diagram which performs multiple kinds of weighted searches in embodiment of this invention The figure which shows an example of the display screen of the similar case display part in embodiment of this invention The figure which shows an example of the display screen of the similar case display part in embodiment of this invention The conceptual diagram of the method of specifying an important interpretation item with respect to a disease type candidate cluster in embodiment of this invention. The figure which shows an example of the specified interpretation item in definite diagnosis disease name "MAC disease" in embodiment of this invention The conceptual diagram which shows the weighting method regarding a disease type candidate cluster in embodiment of this invention

(Knowledge that became the basis of the present invention)
The present inventor has found that the following problems occur with respect to Non-Patent Document 1 and Patent Document 1 described in the “Background Art” column.

  Non-Patent Document 1 discloses a two-step search method called “customized-queries” approach (CQA) as a similar image search method that dynamically changes the image feature amount used for search to a concept level other than an organ. is suggesting. In the first step, the query image is classified by using an image feature amount that can best classify a class such as a disease type, a disease progression degree, or a disease severity. In a second step, similar images are searched using image feature quantities optimized for further classifying cases included in the class that is the classification result. At this time, an optimal image feature amount for each class is obtained in advance by unsupervised learning. Further, in this document, CQA is applied to a lung CT image, and the retrieval recall rate is improved as compared with a similar image retrieval using a conventional set of image feature amounts.

  Patent Document 1 proposes a search method for searching for similar cases after converting an image feature space using text information attached to a medical image in a case database. First, a candidate image group similar to the search key image is searched in the initial image feature space. Next, a text feature is extracted from the text information described with respect to the candidate image group, and a text feature that is easy to distinguish from others is selected from the extracted text features. Then, the image feature space is transformed using the image feature associated with the selected text feature. Since the text information reflects the doctor's knowledge, the similar case search can be performed in the image feature space using the doctor's knowledge.

  However, in Non-Patent Document 1, since image feature amounts optimized in advance for each disease type are used, the appearance of lesions in the image varies greatly depending on the disease type even for the same disease type. In some cases, there is a problem that it is not possible to respond sufficiently. Here, the disease type is a “disease type” that is translated into “dease type” in English and is classified according to the cause, symptom, and the like. Examples of various disease type classifications are shown below.

(1) Examples of disease types classified by cause Diabetes: Type 1 and Type 2 (Japan Diabetes Association)
(2) Examples of disease types classified by symptoms Tuberculosis: Type I, Type II, Type III, Type IV, Type V (Classification of the Japanese Tuberculosis Society)
Schizophrenia: Destructive type, tension type, delusion type (WHO International Classification of Diseases 10th edition (ICD-10))
(3) Examples of disease types classified by gene expression pattern Breast cancer: Luminal A, Luminal B, HER2-enriched, Basal like (Suggested by Perou, Sorlie et al.)

  The present disclosure is directed to a situation in which a disease type is classified by a symptom, the symptom is observed in a medical image, and the medical disease type can be estimated by a doctor from the medical image.

  Even in Non-Patent Document 1, it is possible to cope by recognizing a disease type and obtaining an image feature amount in advance for each type of disease type, but this is not practical for the following reasons.

  For example, as a pulmonary disease, there is a disease called MAC (Mycobacterium avium complex) disease which is a kind of non-tuberculous mycobacterial disease. This MAC disease is usually classified into five disease types: 1) nodule / bronchial dilation type, 2) cavity formation type, 3) solitary nodule type, 4) systemic dissemination type, 5) hypersensitivity pneumonia type (non) Patent Document 2: “Thoracic CT 3rd Edition”, edited by Kiyo Murata, Takeshi Kamiko, Sadayuki Murayama, published by Medical Science International).

  For example, 1) Nodules and bronchodilation appear together as nodule / bronchial dilation image findings. 2) A nodular shadow with a cavernous lesion or a disseminated lesion appears as a cavity-forming image. 4) Mediastinal lymph node swelling appears as a whole-seeding image. 5) In the hypersensitivity pneumonia type, a lobular central granular shadow and a panlobular ground glass shadow are mainly used, and a mosaic pattern tends to appear at the same time. In this example of MAC disease, even if the disease is the same, image findings of different combinations depending on the disease type are exhibited. Moreover, in practice, not all of the combinations always appear. In addition, there are several tens to hundreds of tens of disease types in lung diseases, and an image identification unit is created for all disease types possessed by these disease types, and an optimal image for each disease type It is very difficult to obtain the feature amount.

  In Patent Document 1, a text feature that is easily distinguishable from others is selected from text features included in a candidate image group acquired by an initial image search, and an image feature space is converted using the image feature related to the text feature. ing. However, there is no guarantee that a text feature that can be easily distinguished from others is an important clue for identification for a search target image. In addition, the set of candidate images is affected by the similarity threshold in the initial image search, and the text features that are easily distinguished from the other change. Further, since details of processing are not described in Patent Document 1, it is unclear what effect can be actually obtained in a situation where there are a plurality of disease types to be solved by the present application.

  In order to solve such a problem, the similar case retrieval apparatus according to one aspect of the present invention includes a plurality of case data including a medical image and an interpretation report that is document data describing a result of interpretation of the medical image. A similar case retrieval apparatus for retrieving case data including a medical image similar to an interpretation target image that is a medical image to be interpreted from an accumulated case database, and an image for acquiring a plurality of image feature amounts from the interpretation target image Based on a feature acquisition unit, the plurality of image feature amounts acquired by the image feature acquisition unit, and a plurality of image feature amounts acquired from medical images included in each case data registered in the case database, Case data including the medical image in which the similarity between the image to be interpreted and the medical image included in each case data is calculated, and the calculated similarity is a predetermined value or more A plurality of primary search sections to be acquired; and an interpretation item extraction section to extract a plurality of interpretation items that are character strings obtained by verbalizing features of medical images from interpretation reports included in each case data acquired by the primary search section; Using a plurality of interpretation items extracted for each case data by the interpretation item extraction unit, a disease type candidate cluster generation unit that generates a plurality of disease type candidate clusters composed of case data groups with similar combinations of interpretation items; Extracted from each image feature amount extracted from the medical image and an interpretation report for the medical image for each of at least one disease type candidate cluster among the plurality of disease type candidate clusters generated by the disease type candidate cluster generation unit. For each image feature amount acquired by the image feature acquisition unit, based on an interpretation knowledge database in which relevance of each interpretation item is determined in advance, the image feature amount and the disease A weight determination unit that determines a greater weight as the relevance between the interpretation items included in the candidate cluster is higher, the plurality of image feature amounts extracted by the image feature acquisition unit, and the case database Similar to the image to be interpreted by weighting and comparing the plurality of image feature amounts acquired from the medical image included in the case data included in the case data with weights for each image feature amount determined by the weight determination unit A secondary search unit that searches case data including medical images to be searched from the case database.

  According to this configuration, even for diseases having a plurality of disease types, cases obtained as a result of the primary search without defining disease types in advance or preparing search image feature values for each disease type The image feature amount can be optimized using the “interpretation item” included in the. Specifically, using “interpretation items” included in cases obtained by the primary search, a cluster according to the disease type is dynamically generated and related to the “interpretation items” included in the cluster. Perform image search with emphasis on image features. Thereby, it is possible to perform a search with an image feature amount corresponding to a possible disease type for the search source image.

  In this specification, “interpretation item” is defined as “a character string in which the characteristics of a medical image to be interpreted by an interpretation doctor are verbalized”. The terms used as interpretation items are almost limited depending on the medical imaging apparatus or target organ used. Examples of interpretation items include lobed, spiny, irregular, clear border, blurred outline, low density, high density, low absorption, high absorption, ground glass, calcification, mosaic, early dark stain, low echo, There is high echo, fluffing, etc.

  For example, the weight determination unit may determine the image feature based on the interpretation knowledge database for each of at least one disease type candidate cluster among the plurality of disease type candidate clusters generated by the disease type candidate cluster generation unit. For each image feature amount acquired by the acquisition unit, the relationship between the image feature amount and the interpretation item included in the disease type candidate cluster is higher, and the appearance frequency of the interpretation item included in the disease type candidate cluster The higher the value, the larger the weight may be determined.

  According to this configuration, the higher the appearance frequency of the interpretation items included in the disease type candidate cluster, the greater the weight. Thereby, it is possible to perform weighting on the image feature amount in consideration of the appearance frequency of the interpretation item, and the search accuracy can be further improved.

  The secondary search unit is included in the plurality of image feature amounts extracted from the image feature acquisition unit and case data acquired by the primary search unit among case data registered in the case database. A case including a medical image similar to the image to be interpreted by weighting and comparing the plurality of image feature amounts acquired from a medical image to be weighted with a weight for each image feature amount determined by the weight determination unit Data may be searched from a plurality of case data acquired by the primary search unit.

  According to this configuration, case data search is performed by the secondary search unit for the case data acquired by the primary search unit. For this reason, case data can be narrowed down efficiently.

  In addition, the disease type candidate cluster generation unit uses a plurality of interpretation items extracted for each case data from the interpretation item extraction unit, and the combination of interpretation items is similar and the confirmation is given to the case data A plurality of disease type candidate clusters composed of case data groups having the same diagnosis disease name may be generated.

  According to this configuration, it is possible to generate a disease type candidate cluster by clustering case data for each disease name. For this reason, it is considered that the generated disease type candidate cluster conforms to the disease type of each disease.

  The similar case search apparatus described above may further include a presentation unit that presents the search result of the secondary search unit to the user.

  According to this configuration, the search result can be presented to the user.

  The presenting unit may present the case data searched by the secondary search unit to the user for each disease type candidate cluster generated by the disease type candidate cluster generation unit.

  The retrieved case data is presented for each disease type candidate cluster. Therefore, the user can easily associate the disease type candidate with the case data.

  In addition, the weight determination unit includes, for each of at least one disease type candidate cluster among the plurality of disease type candidate clusters generated by the disease type candidate cluster generation unit, an interpretation item included in the disease type candidate cluster. And at least one interpretation item is identified, and based on the interpretation knowledge database, for each image feature amount acquired by the image feature acquisition unit, the relationship between the image feature amount and the identified interpretation item is high A larger value weight may be determined.

  Image feature amounts are weighted based on the specified interpretation items among the interpretation items included in the disease type candidate cluster.

  Further, the similar case search device described above further includes, for each disease type candidate cluster generated by the disease type candidate cluster generation unit, case data searched by the secondary search unit and interpretation items specified by the weight determination unit. May be provided to the user.

  According to this configuration, the user can know what interpretation results are used and what search results are obtained.

  Further, the similar case search apparatus described above further includes an input unit that receives a selection input of any disease type candidate cluster by the user from among the plurality of disease type candidate clusters generated by the disease type candidate cluster generation unit. The weight determination unit, for each of the disease type candidate clusters indicated by the selection input received by the input unit among the plurality of disease type candidate clusters generated by the disease type candidate cluster generation unit, Identify at least one interpretation item among the interpretation items included in the candidate cluster, and for each image feature amount acquired by the image feature acquisition unit based on the interpretation knowledge database, the image feature amount and the specified interpretation item You may determine the weight of a big value, so that the relationship between is high.

  By causing the user to select an interpretation item, it is possible to weight the image feature amount by reflecting the user's point of interest. Thereby, the case data reflecting the user's point of interest can be searched.

  These general or specific modes may be realized by a system, a method, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM. The system, method, integrated circuit, computer program Alternatively, it may be realized by any combination of recording media.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that each of the embodiments described below shows a specific example of the present invention. Numerical values, components, connection modes of components, steps, order of steps, and the like shown in the following embodiments are merely examples, and are not intended to limit the present invention. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims indicating the highest concept are described as optional constituent elements.

(Embodiment)
FIG. 1 is a block diagram of a similar case retrieval apparatus according to an embodiment of the present invention.

<Configuration>
The similar case retrieval apparatus obtains a medical image similar to a medical image to be interpreted from a case database in which a plurality of case data including a medical image and an interpretation report that is document data describing a result of interpretation of the medical image is stored. Search for case data. The similar case search device includes a case database 100, an interpretation knowledge database 110, an interpretation target image reading unit 120, an interpretation target image display unit 130, an image feature acquisition unit 140, a primary search unit 150, and an interpretation item extraction. Unit 160, disease type candidate cluster generation unit 170, weight determination unit 180, secondary search unit 190, and similar case display unit 200. The similar case retrieval apparatus includes an interpretation report input unit (not shown).

  The case database 100 includes a medical image such as CT (Computed Tomography) or MRI (Magnetic Resonance Imaging) (in the present disclosure, “image data” is simply referred to as “image”) and a result obtained by reading the medical image. It is a database storing a plurality of case data (hereinafter simply referred to as “cases”) composed of a pair with a certain interpretation report. The interpretation knowledge database 110 is a database that stores interpretation knowledge obtained by analyzing a plurality of cases. Details of the interpretation knowledge database 110 will be described later. The case database 100 and the interpretation knowledge database 110 are stored in a storage device such as an HDD (Hard Disk Drive).

  The interpretation target image reading unit 120 reads an interpretation target image, which is a medical image to be interpreted, taken by a medical image photographing apparatus such as CT or MRI from a medical image photographing apparatus or an externally connected storage device.

  The interpretation target image display unit 130 includes a medical high-definition monitor or the like, and displays the interpretation target image read by the interpretation target image reading unit 120.

  The image feature acquisition unit 140 acquires image feature amounts by extracting a plurality of types of image feature amounts from the interpretation target images read by the interpretation target image reading unit 120, and generates an image feature vector. Note that the image feature acquisition unit 140 may acquire the image feature amount by receiving the image feature amount from the outside (for example, a terminal device connected to the network) instead of extracting the image feature amount.

  The primary search unit 150 interprets an image by comparing the image feature vector generated by the image feature acquisition unit 140 with an image feature vector extracted from a medical image included in case data stored in the case database 100. The degree of similarity between the target image and the medical image included in the case data is calculated, and a plurality of case data having a degree of similarity equal to or greater than a predetermined value is acquired. In the present embodiment, it is assumed that image feature vectors extracted from medical images included in case data are stored in advance in case database 100. This image feature quantity vector extraction method is the same as the image feature quantity vector extraction method performed by the image feature acquisition unit 140.

  The interpretation item extraction unit 160 extracts, for each case data, an interpretation item that is a character string obtained by verbalizing the characteristics of the medical image from the interpretation report included in the case data acquired by the primary search unit 150.

  The disease type candidate cluster generation unit 170 uses the interpretation items acquired for each case data by the interpretation item extraction unit 160 for a plurality of case data acquired by the primary search unit 150, and the cases having similar combinations of interpretation items A plurality of candidate disease type clusters composed of data groups are generated.

  The weight determination unit 180 applies to each image feature amount and medical image extracted from the medical image for each of at least one disease type candidate cluster among the plurality of disease type candidate clusters generated by the disease type candidate cluster generation unit 170. Based on the interpretation knowledge database 110 in which the relationship between each interpretation item extracted from the interpretation report is determined in advance and the interpretation items included in the case data included in the disease type candidate cluster, the image feature acquisition unit 140 For each extracted image feature amount, a higher weight is determined as the relevance between the image feature amount and the interpretation item is higher.

  The secondary search unit 190 weights the plurality of image feature amounts extracted by the image feature acquisition unit 140 and the plurality of image feature amounts extracted from the medical images included in the case data registered in the case database 100. Case data including a medical image similar to the image to be interpreted is searched from the case database 100 by weighting and comparing with the weight for each image feature amount determined by the determination unit 180.

  The similar case display unit 200 is an example of a presentation unit that presents the search result of the secondary search unit 190 to the user, and displays the case data searched by the secondary search unit 190. The similar case display unit 200 may be separately configured with the same model as the high-definition monitor that constitutes the interpretation target image display unit 130. The similar case display unit 200 is similar to the interpretation target image and the similar case on the high-definition monitor that constitutes the interpretation target image display unit 130. May be displayed simultaneously. Note that the similar case display unit 200 and the interpretation target image display unit 130 may be of different models.

  An interpretation report input unit (not shown) receives an interpretation report input from the user. That is, the user inputs an interpretation report into an interpretation report input unit (not shown) while referring to the interpretation target image displayed on the interpretation target image display unit 130 and the similar case displayed on the similar case display unit 200. To do. The interpretation report input unit is composed of a keyboard, a mouse, and the like.

  The interpretation target image display unit 130, the similar case display unit 200, and the interpretation report input unit (not shown) constitute an interpretation terminal 210.

  The user (interpretator) of the similar case retrieval apparatus is a doctor who performs interpretation such as a radiologist and a clinician. However, the user is not limited to these, and may be a person who does not have a doctor's license, such as a clinical laboratory technician or a medical student.

<Operation>
Hereinafter, the operation of each part of the similar case retrieval apparatus will be described in detail.

<Preparation of interpretation knowledge database>
In performing similar case search, interpretation knowledge is created in advance and stored in the interpretation knowledge database 110. The interpretation knowledge is created from a collection of a plurality of “cases” composed of a pair of a medical image and an interpretation report that is a result of interpretation of the medical image. As cases, those stored in the case database 100 may be used, or cases stored in other databases may be used. The required number of cases is sufficient to obtain some kind of law and knowledge using various data mining algorithms. Usually, hundreds to tens of thousands of data are used. In this embodiment, as the interpretation knowledge stored in the interpretation knowledge database 110, the relationship between each image feature amount extracted from the medical image and each interpretation item extracted from the interpretation report for the medical image, That is, the correlation between the two terms (image feature amount−interpretation item) is used.

  Examples of the “image feature amount” include those related to the shape of an organ or a lesion in a medical image, or those related to a luminance distribution. For example, Non-Patent Document 3: “Nemoto, Shimizu, Sugawara, Obata, Nawano,” Improvement of mass shadow discrimination accuracy on breast X-ray image and high-speed feature selection by feature selection from multiple feature amounts "Proposal of the method", IEICE Transactions D-II, Vol. J88-D-II, No. 2, pp. 416-426, February 2005 describes the use of 490 features. ing. Also in the present embodiment, several tens to several hundreds of image feature amounts determined in advance for each medical image capturing apparatus (modality) used for capturing a medical image or each target organ for interpretation are used.

  Hereinafter, the procedure for creating the interpretation knowledge database will be described with reference to the flowchart of FIG. The medical image capturing apparatus to be used in this embodiment, that is, to be used is a multi-slice CT, and the target organ is a lung.

  In step S10, one case is acquired from a database storing cases for obtaining interpretation knowledge. Here, the total number of cases for obtaining interpretation knowledge is C. One case is composed of a pair of a medical image and an interpretation report that is a result of interpretation of the medical image. When a medical image is acquired by a multi-slice CT apparatus, one case includes a large number of slice images. In general, when a doctor interprets a multi-slice CT image, one to several important slice images are attached to the interpretation report as key images. Hereinafter, a large number of slice image sets or several key images may be simply referred to as “medical images” or “images”.

  In step S11, an image feature amount is extracted from the medical image. The process of step S11 will be described in detail using the flowchart of FIG.

  In step S111, the region of the target organ is extracted. In this embodiment, a lung region is extracted. Non-patent document 4: “Kitazaka, Mori, Hasegawa, Toriwaki,“ Lung field extraction from 3D chest X-ray CT image using shape model with variable Bezier curved surface ”, electronic information communication A method such as “Academic Journal D-II, Vol. J83-D-II, No. 1, pp. 165-174, January 2000” can be used.

  In step S112, a lesion area is extracted from the organ area extracted in step S111. In the present embodiment, a lesion area is extracted from a lung area. As a method for extracting a lesion area, for example, Non-Patent Document 5: “Hasegawa, Mori, Toriwaki, Anno, Katada,“ Automatic extraction of lung cancer candidate areas from chest CT images by three-dimensional digital image processing ”, IEICE Techniques such as “Paper Journal D-II, Vol. J76-D-II, No. 8, pp. 1587-1594, August 1993” can be used. However, including the above documents, at the present time, many automatic extractions of lesions called “localization” in which pathological changes such as inflammation and tumor are limited to a narrow range have been proposed, but “diffuse” A technique for accurately extracting a lesion that has spread widely in a so-called organ has not been established. Therefore, in the case of diffuse lung disease, it is desirable to extract manually by a doctor or the like. In a case having undergone image diagnosis, a slice image including a lesion is designated as a key image, and the position of the lesion is designated by an arrow or a region. Therefore, the lesion region can be extracted using the information. Here, if the number of lesions extracted from the image in the i-th case is Mi, the lesion can be specified by a set (i, j) of (case number, lesion number). Here, 1 ≦ i ≦ C and 1 ≦ j ≦ Mi.

  In step S113, one region is selected from the lesion regions extracted in step S112.

In step S114, an image feature amount is extracted from the lesion area selected in step S113. In this embodiment, non-patent document 6: “Hirano, Hasegawa, Toriwaki, Omatsu, Eguchi,“ Interactive lung tumor region extraction from 3D chest CT images and application to benign / malignant discrimination ”, IEICE Transactions D-II, Vol. J87-D-II, No. 1, pp. 237-247, January 2004 ”and Non-Patent Document 7:“ Yoshikazu Uchiyama, Shigehiko Katsuragawa, Hiroyuki Abe, Jun. ” Shirai, Feng Li, Qiang Li, Chao-Tong Zhang, Kenji Suzuki, and Kunio Doi, “Quantitatively computerized analysis of diffuses -resolution computed tomography ", Med. Phys. 30, 2440 (2003)" using the image feature amount described in. The number of image feature amounts is _NIF . The image feature amount extracted in this step can be specified by a set (i, j, k) of (case number, tumor number extracted from this case (medical image), image feature amount number). Here, 1 ≦ i ≦ C, 1 ≦ j ≦ M _i , and 1 ≦ k ≦ N _IF .

  In step S115, it is checked whether or not there is an unselected lesion among the lesion areas extracted in step S112. If there is an unselected lesion, the process returns to step S113 to select an unselected lesion area. Re-execute S114. When there is no unselected lesion, that is, when the image feature amount selection in step S114 is performed for all the lesion areas extracted in step S112, the processing of the flowchart of FIG. 3 is terminated, and the flowchart of FIG. Return.

  In step S12 of FIG. 2, an interpretation report analysis process is performed. Specifically, an interpretation item and a disease name are extracted from the interpretation report.

  In the present embodiment, morphological analysis and syntax analysis are performed using an interpretation item word dictionary storing interpretation items and a disease name word dictionary storing disease names. Through these processes, a word that matches the word stored in each word dictionary is extracted. As morphological analysis techniques, for example, Non-Patent Document 8: MeCab (http://mecab.sourceforge.net) and Non-Patent Document 9: ChaSen (http://chasen-legacy.sourceforge.jp) are syntactically analyzed. Non-patent document 10: KNP (http://nlp.kuee.kyoto-u.ac.jp/nl-resource/knp.html), Non-patent document 11: CaboCha (http://chasen.org) / ~ Take / software / cabocha /) etc. exist. Interpretation reports are often described by doctors with expressions unique to interpretation reports, so it is desirable to develop morphological analysis technology, syntax analysis technology, and word dictionaries specialized for interpretation reports.

FIG. 4 is an example of an interpretation report of a chest CT examination, and FIG. 5 shows interpretation items and disease names extracted from the interpretation report of FIG. When i-th number of interpretation items extracted from the image interpretation report in case the N _i, interpretation items can be identified by (case number, image interpretation item number) pairs of (i, j). Here, 1 ≦ i ≦ C and 1 ≦ j ≦ N _i .

  In FIG. 5, only the interpretation item and the disease name word are extracted, but a character string indicating the position and presence / absence of a lesion in the interpretation report may be extracted simultaneously. For this reason, extracting not only the interpretation items but also the position and presence / absence information together is effective in extracting interpretation knowledge described later. FIG. 6 shows an example in which position information and presence / absence information are extracted simultaneously with the interpretation items. For example, the interpretation report of FIG. 4 is analyzed, and “right S6” is extracted as the position attribute of “infiltration shadow” and “acknowledgement (recognition)” is present as the presence / absence attribute from the phrase “recognition of infiltration shadow in right S6”. Is done. Also, “not recognized” is extracted as the presence / absence attribute of “lymph node” from the phrase “no lymph node enlargement”.

  In step S13, it is checked whether or not there is an unacquired case in the database storing cases for obtaining interpretation knowledge. If there is an unacquired case, the process returns to step S10 and an unacquired case is acquired. Steps S11 and S12 are executed. If there are no unacquired cases, that is, if image feature extraction in step S11 and report analysis in step S12 have been performed for all cases, the process proceeds to step S14.

  Since the results of step S11 and step S12 do not depend on each other, the execution order may be reversed.

When the process reaches step S14, for example, a set of data represented in FIG. 7 is acquired. That is, an image feature amount, an interpretation item, and a disease name are acquired for each case. For the case of case number 1, M ₁ lesions are included in the medical image, and the number of image feature amounts extracted from each lesion is _NIF . In addition, the number of image interpretation items in the radiology report is _one N. For example, the value of the first image feature value of the first lesion indicated by the lesion number (1, 1) is 0.851. The value of the first interpretation item indicated by the interpretation item number (1, 1) is “infiltration shadow”. In the example of FIG. 7, each image feature amount is a numerical value from 0 to 1, and the interpretation item and the disease name are character strings. An image feature value having a negative value or a value larger than 1 may be used. Further, data may be stored in the form of a predetermined word ID as an interpretation item and a disease name.

  In step S14, interpretation knowledge is extracted from the image feature amount obtained in step S11 and the interpretation item and disease name obtained in step S12. In the present embodiment, the correlation between the two terms of the image feature quantity and the interpretation item is taken as interpretation knowledge.

<Correlation between (image feature-interpretation item)>
A method of obtaining the correlation between a pair of (image feature amount, interpretation item) will be described. Although there are a plurality of expression forms of the correlation, the correlation ratio is used here. The correlation ratio is an index representing a correlation between qualitative data and quantitative data, and is expressed by (Equation 1).

  In the interpretation report, two categories of cases where an interpretation item is included and not included are considered as qualitative data. The value of a certain image feature amount itself extracted from the medical image is used as quantitative data. For example, for all cases included in the case database for extracting interpretation knowledge, the interpretation report is classified into those including or not including a certain interpretation item. Here, a method of obtaining the correlation ratio between the interpretation item “infiltration shadow” and the image feature amount “luminance average value of lesion area” will be described. In (Expression 1), it is assumed that category i = 1 includes “infiltration shadow” and category i = 2 does not include “infiltration shadow”. The j-th observed value that is the “average luminance value of the lesion area” of the lesion image extracted from the case that includes “infiltration shadow” in the interpretation report is assumed to be x1j. Further, the j-th observed value that is the “brightness average value of the lesion area” of the lesion image extracted from the case that does not include “infiltration shadow” in the interpretation report is assumed to be x2j.

  The “infiltration shadow” is a shadow caused by accumulation of cell components and liquid components in the alveoli, and appears due to pneumonia, lung infection, and the like. In the CT image, the shadow is so deep that the blood vessel shadow cannot be seen through, and has a CT value of about −200 HU to +300 HU. When the interpretation report includes “invasion shadow”, there are many cases where the CT value of the lesion is about −200 HU to +300 HU, and when the interpretation report does not include “invasion shadow”, the CT value of the lesion has various values. Take. Therefore, it is expected that the correlation ratio between the image feature amount “brightness average value of the lesion area” and the interpretation item “infiltration shadow” is large (close to 1). Further, since the infiltration shadow does not depend on the shape of the lesion, it is expected that the correlation ratio between the image feature amount “circularity of the lesion” and the interpretation item “infiltration shadow” is small (close to 0). In this way, the correlation ratio between all interpretation items and all image feature amounts is calculated.

  FIG. 8 shows a conceptual diagram of the correlation (here, the correlation ratio) between the interpretation item and the image feature amount. A plurality of interpretation items are listed on the left side, and names of a plurality of image feature amounts are listed on the right side. An interpretation item having a correlation ratio equal to or greater than a threshold and an image feature amount are connected by a solid line. Since the correlation ratio takes a value of 0 or more and 1 or less, a value of about 0.3 to 0.7 can be used as the threshold value. When the calculated correlation ratio is finally binarized with a threshold value, information as shown in FIG. 8 is obtained.

  FIG. 9 shows another conceptual diagram of the correlation (for example, correlation ratio) between the interpretation item and the image feature amount. In this figure, the interpretation item “small granular shadow” and a plurality of image feature amount correlation ratios are expressed in multiple values, and the thickness of the solid line between the interpretation item and the image feature amount corresponds to the magnitude of the correlation ratio. Yes. “Small granular shadow” is characterized by the presence of many fine lesions having a diameter of 2 to 3 mm. Compared with the other interpretation items, the correlation with the image feature amount “lesion area” is large because it can be clearly distinguished by the size of the lesion.

  By paying attention to the value of the correlation ratio, it is possible to specify an image feature amount having a high correlation with a certain interpretation item. Actually, one case often includes a plurality of lesions, and in that case, the interpretation report includes descriptions about the plurality of lesions. For example, a plurality of slice images are obtained by one CT image capturing, and the slice image sequence often includes a plurality of lesions, and a plurality of image feature amounts are extracted from one lesion. Therefore, as many image feature amounts as (number of lesions detected from one patient) × (number of types of image feature amounts) are obtained, and a plurality of image feature amounts and a plurality of image feature values extracted from one interpretation report are obtained. It is necessary to obtain the correlation with the interpretation items and disease names. Of course, there is a possibility that the correspondence can be obtained correctly by using a large number of cases, but the description of the interpretation report and the corresponding image feature amount are handled to some extent in advance by using the lesion position as shown in FIG. If it can be attached, a correlation can be calculated | required more correctly.

  In the above description, the case where the qualitative data is in two categories, including and not including a certain interpretation item, has been described. However, a certain interpretation item (for example, “clear border”) and its interpretation item ( For example, two categories such as “unclear boundary”) may be used. When the interpretation items are ordinal scales such as “low density”, “medium density”, and “high density”, the correlation ratio may be calculated using each of them as a category (three categories in this example).

  For synonyms such as “low density”, “low luminance”, and “low absorption”, a synonym dictionary is created in advance, and these are treated as the same interpretation item.

  When the process of step S14 is performed by the above method, a correlation between (image feature amount−interpretation item) as shown in FIG. 10 is obtained. The numbers in the table are correlation ratios. The correlation ratio takes a value between 0 and 1. Further, the obtained correlation is stored in the interpretation knowledge database 110 in the format of FIG.

<Similar case search>
Hereinafter, the processing procedure of the similar case search will be described.

  FIG. 11 is a flowchart for explaining the positioning of the similar case search process by the similar case search device in the diagnosis act of the doctor.

  The doctor or laboratory technician obtains the image of the patient to be diagnosed by using a CT apparatus or an MRI apparatus (S1).

  At similar intervals, the similar case search device searches for similar cases similar to the image to be interpreted using the image acquired in the processing of S1 as the image to be interpreted (S2). The process of S2 will be described later.

  The doctor diagnoses the patient based on the image to be interpreted acquired in S1 and the similar case retrieved by the similar case retrieval device (S3).

  That is, the search for similar cases by the similar case search apparatus plays a role of providing a judgment material in diagnosis of a patient by a doctor.

  Next, details of the process of S2 of FIG. 11 shown in FIG. 12 will be described.

  The similar case search screen display example shown in FIG. 13 is a screen display example of the interpretation terminal 210 configured by the interpretation target image display unit 130, the similar case display unit 200, and the interpretation report input unit (not shown in FIG. 1) of FIG. It is.

  In step S20, the interpretation target image reading unit 120 acquires an interpretation target image from the medical image photographing apparatus. As in the case of creating the interpretation knowledge database in FIG. 2, the medical imaging apparatus targeted in this embodiment is a multi-slice CT, and the target organ is a lung. The read image (interpretation target image 300 in FIG. 13) is displayed on the interpretation target image display unit 130.

  In step S21, designation of a target lesion for which a similar case search is to be performed is accepted from the radiogram interpreter. The designation is performed by clicking a lesioned part on the screen of the interpretation target image display unit 130 (interpretation target image 300) using a mouse.

  The designation of this lesion will be described. As a designation method, one point near the center of the lesion may be designated, or a lesion area (outline) may be designated. When one point near the center is designated, a detailed lesion area is set using the same method as step S112 in FIG. When the lesion area is roughly specified, a detailed lesion area is set from this area by using the same method as in step S112.

  The designation of the lesion position or region will be described with reference to FIG. An image to be interpreted 400 (here, chest CT) shown in FIG. 14 includes images of the periphery 410 of the chest, the target organ (here, lung) 420, and the lesion area 430. When designating the coordinates, for example, the vicinity of the center position of the lesion (point 440 in FIG. 14) is clicked with the mouse. When specifying an area, a method of surrounding the lesion area 430 in FIG. 14 with a rectangle, circle, or ellipse, or between a lesion area (lesion area 430 in FIG. 14) and a normal tissue (outside the lesion area 430 in FIG. 14). There is a way to specify the boundary with a free curve. When only the center coordinates are specified, or when an area is specified by surrounding it with a rectangle, circle, ellipse, etc., there is an advantage that the burden on the interpreter is small. It is necessary to automatically extract a lesion area by an image processing algorithm. For the lesion area extraction, the same technique as in step S112 can be used.

In step S22, the image feature acquisition unit 140 acquires an image feature amount extracted from the lesion area specified or extracted in step S21 for the image to be interpreted. The types and number of types of image feature amounts are the same as those in step S114. That is, the number of types is _NIF . If there are a plurality of designated or extracted lesion areas, _NIF image feature amounts are acquired for all of them. The extraction of the image feature amount may be performed in the image feature acquisition unit 140 or may be performed by another existing image feature extraction unit.

In step S23, the primary search unit 150 acquires a plurality of cases having an image feature amount similar to the image feature amount acquired in step S22 from the case database 100. Specifically, the Euclidean distance between (Equation 2) between the image feature amount extracted from the medical image included in the case stored in the case database 100 and the image feature amount extracted from the image to be interpreted. And a case whose distance is smaller than a predetermined threshold is acquired from the case database 100 as a search result candidate case. In (Expression 2), x is a vector obtained by concatenating all ( _NIF ) image feature amounts extracted from the image to be interpreted. u ⁱ is an image feature amount extracted from the i-th case among cases stored in the case database 100. When different types of image feature quantities are connected, canonicalization (normalized to 0 average and 1 variance) is performed so as not to be affected by the difference in scale for each image feature quantity.

The state of this primary search is shown in FIG. FIG. 15 is an image diagram showing a state in which an image feature amount within a predetermined distance threshold is specified using the image feature amount acquired in step S22 as a search source in the image feature space of all N _IF dimensions. In step S23, a plurality of past cases having image feature amounts within a predetermined distance threshold are acquired. A simple similarity (cosine similarity) may be used instead of the Euclidean distance. In this case, an image feature amount that is equal to or greater than a predetermined similarity threshold is specified and acquired. Here, since “weighting with respect to important image feature amounts”, which is one of the features of the present disclosure, is not performed, the search relevance rate between the search source case and the candidate case (the doctor who is the user of the similar case search apparatus The percentage of similar cases to be recognized is not so high. At this stage, the purpose is to roughly search for candidate cases and obtain a set of text features included in the candidate cases. For clustering in step S25, which will be described later, at least a number of case groups is acquired so as to include several disease names that should be distinguished from the search source case. When the disease name cannot be narrowed down to one in the image diagnosis, usually one to several suspected disease names are described in the interpretation report. Here, the threshold of similarity is adjusted so that five disease names are included, and a group of cases is acquired.

  In step S24, the interpretation item extraction unit 160 extracts an interpretation item for each case from the case group acquired in step S23. Since each case has been interpreted, an interpretation report is attached. For an interpretation report created with a natural language sentence, an interpretation item is extracted using the same process as in step S12. In the case of a structured report generated using standardized notation and attributes, keywords (interpretation items) corresponding to image findings are extracted. Then, an interpretation item vector is generated using the extracted interpretation items. Here, the interpretation item vector has the number of dimensions (NII dimensions) as many as the number of types of interpretation items, and when the focused case includes the interpretation item k, the element of the k-th dimension is the number of occurrences of the interpretation item. It is represented by. An index such as tf-idf (tf: Term Frequency, idf: Inverse Document Frequency) may be used instead of using the number of appearances. Using this tf-idf, in addition to the number of appearances of words, the number of appearances that lowers the importance of interpretation items appearing in many interpretation reports and increases the importance of interpretation items that appear only in specific interpretation reports Can be corrected. Moreover, since there is a fluctuation of notation in the interpretation item, a synonym dictionary may be prepared in advance, and each dimension of the interpretation item vector may be a dimension reduced by the synonym. An example of the interpretation items extracted from each case is shown in FIG. In FIG. 16, the notation of the interpretation item keyword that appears one or more times is shown in the table. Each interpretation item vector may be given a definite diagnosis disease name (disease name) as its attribute. In the case where a definite diagnosis disease name is given, if the cases are sorted for each definite diagnosis disease name, an example of an interpretation item as shown in FIG. 17 is obtained. The usage of the definite diagnosis disease name will be explained separately.

In step S25, the disease type candidate cluster generation unit 170 generates a plurality of disease type candidate clusters composed of case data groups with similar combinations of interpretation items, using the interpretation item vectors generated for each case in step S24. . Let N _CALL be the total number of generated disease type candidate clusters. A disease type candidate cluster can be generated using a clustering technique such as a K-average method for a set of interpretation item vectors for each case. Other methods such as hierarchical clustering can also be used (for example, Non-Patent Document 12: “Classification of multivariate data—discriminant analysis / cluster analysis”, Yoshiharu Sato, Asakura Shoten, etc.) See the book). As described in step S24, the interpretation item vector is a vector having dimensions corresponding to the number of types of interpretation items, and when the case includes a certain interpretation item, the corresponding element is represented by the number of appearances of the interpretation item. It is. For example, in FIG. 16, assuming that all the interpretation items described appear once, the interpretation item vector of the case of case number 1 (hereinafter referred to as “case 1”) is (1, 0, 0, 1, 0, 0, 0, 0, ...). In case 1, “cavity lesion” and “nodule” appear as interpretation items, so the first and fourth elements are 1. When the “cavity lesion” appears twice, the interpretation item vector of case 1 is (2, 0, 0, 1, 0, 0, 0, 0,...). Clustering is performed on such interpretation item vectors. By clustering, cases with similar combinations of interpretation items form the same cluster. An image of the result of clustering is shown in FIG. Here, each axis extending the NII-dimensional interpretation item space corresponds to each interpretation item. Here, the interpretation item is defined as “a character string obtained by verbalizing the characteristics of a medical image to be interpreted”, and that the interpretation item (combination) is similar is a symptom (combination) that appears in the medical image. Corresponds to the similarity. Therefore, this cluster becomes a set according to the above-mentioned “disease type”. Here, the reason for referring to “according to” is as follows. This cluster is simply a set classified into cases with similar combinations of interpretation items, and does not necessarily match the actual disease type. Moreover, since it includes cases corresponding to different definite diagnosis disease names, it cannot be called “disease type”.

  As shown in FIG. 17, when a definitive diagnosis disease name is given to each interpretation item as an attribute, a disease composed of case data groups in which combinations of interpretation items are similar for each definite diagnosis disease name. Generate multiple type candidate clusters.

An example of the clustering result of the interpretation item vectors corresponding to MAC disease is shown in FIG. In FIG. 19, N _CALL = 4 clusters are generated from 18 cases. Considering that these clusters are composed of cases related to MAC disease, the cluster with cluster number 1 (hereinafter, the cluster with cluster number i is referred to as cluster i) and cluster 2 are “cavity-forming”, cluster It can be estimated from the interpretation items included that 3 is composed mainly of “nodule / bronchial dilation” and cluster 4 is composed of “isolated nodule”. A group of cases similar in the image feature space is likely to have similar interpretation items (combinations) even if the disease names are different. Therefore, the object of the present disclosure can be achieved without using a definite diagnosis disease name.

  Note that when a definite diagnosis disease name is used, the generated cluster can more easily correspond to an actual disease type.

  In step S26, the weight determination unit 180 interprets each image feature amount extracted from the medical image and interpretation of the medical image for each of at least one disease type candidate cluster among the disease type candidate clusters generated in step S25. The image features extracted in step S11 based on the interpretation knowledge database in which the relevance between each interpretation item extracted from the report is predetermined and the interpretation items included in the interpretation item vector included in the disease type candidate cluster. For each quantity, a higher weight is determined as the relationship between the image feature quantity and the interpretation item included in the interpretation item vector included in the disease type candidate cluster is higher.

A method for determining the weight will be described. Here, the average vector of all the interpretation item vectors forming the disease type candidate cluster is named “average interpretation item vector” and is used. Here, it is assumed that the average interpretation item vector of the cluster of cluster number 1 in FIG. 19 is v1 = (0.7, 0, 0, 0.4, 0.1, 0, 0, 0,...). That is, the influence of the interpretation item “cavity lesion” is 0.7, the influence of “bronchial dilation” and “airway” are both 0, the influence of “nodule” is 0.4, and the influence of “substantial shadow” is 0. It can be thought of as 1,. Therefore, a value obtained by multiplying the weight of the interpretation item “cavity lesion” and each image feature amount by 0.7, a value obtained by multiplying the weight of the “nodule” and each image feature amount by 0.4, The weight to each image feature amount can be calculated by adding a value obtained by multiplying the weight of each shadow image and each image feature amount by 0.1 for each corresponding image feature amount. This is illustrated in FIG. A correlation ratio having a value between 0 and 1 is assigned between all the interpretation items and all the image feature amounts. Here, line segments representing weights are displayed except for interpretation items whose corresponding element of the average interpretation item vector is 0. When attention is paid to the disease type candidate cluster 1, the weight corresponding to the image feature amount 1 is calculated as (0.7 × w _1,1 + 0.4 × w _4,1 + 0.1 × w _5,1 ). it can.

Note that some of the disease type candidate clusters may be selected instead of all of the disease type candidate clusters generated in step S25 (N _CALL pieces), and the weight vector may be calculated. Here, let N _CW (1 ≦ N _CW ≦ N _CALL ) be the number of disease type candidate clusters for which weight vectors are calculated. As a reference for automatic selection of disease type candidate clusters by the similar case search device, the distance in the image feature space between the image to be interpreted (search source image) and the disease type candidate cluster can be used. Since the disease type candidate cluster generated in step S25 is a cluster in the interpretation item space as shown in FIG. 18, it is converted into a cluster in the image feature space, and the cases included in each cluster are searched. Compare the original case. For the conversion of the interpretation item space into the image feature space, for each disease type candidate cluster, image feature vectors of cases included in the disease type candidate cluster are obtained, and an average vector thereof, that is, an image representing the disease type candidate cluster. This can be done by creating a feature vector. The representative image feature vector for each disease type candidate cluster is compared with the image feature quantity of the search source case extracted in step S22, and a predetermined number of representative image feature vectors within a predetermined distance or those having a small distance are used. By selecting representative image feature vectors, some disease type candidate clusters can be selected.

  In addition, a user (interpreter) of the similar case search apparatus may select a disease type candidate cluster. In this case, a disease type candidate cluster selection process may be provided between step S25 and step S26. This selection process can be realized by presenting (interpreting) a dominant interpretation item in composing a disease type candidate cluster to a user for a plurality of disease type candidate clusters and allowing the user to select. An input unit (not shown) receives selection input of a disease type candidate cluster by the user. Here, an interpretation item whose average interpretation item vector element is equal to or greater than a predetermined threshold is displayed. FIG. 21 shows an example of the screen display. On the screen of the interpretation terminal 210, a button 501 for accepting a selection input of “search pattern” and an interpretation item (combination) 503 for explaining a disease type candidate cluster are displayed. This “search pattern” corresponds to the interpretation item (combination thereof) and can be regarded as conforming to the disease type. The user refers to the representative interpretation items (combinations) for each disease type candidate cluster identified in step S26, and uses the mouse to select a button 501 corresponding to the search pattern determined by the user that the interpretation target image is well represented. Click to select a disease type candidate cluster. When a disease type candidate cluster is generated for each confirmed diagnosis disease name in step S25, the confirmed diagnosis disease name 502 is also displayed on the screen of the interpretation terminal 210 together with the search pattern as shown in FIG. Can do. Thereby, it becomes more useful auxiliary information when the user selects a disease type candidate cluster.

In step S27, the secondary search unit 190 searches for a similar case from the case database 100 using the image feature quantity extracted in step S22 and the N _CW weight vectors determined in step S26. The N _CW weight vectors are used for weighted searches corresponding to N _CW disease type candidate clusters, respectively. First, a weighted search method using one weight vector will be described.

<Weighted search using one weight vector>
The secondary search unit 190 calculates a weighted distance between the image feature amount extracted from the medical image included in the case stored in the case database 100 and the image feature amount extracted from the interpretation target image. The secondary search unit 190 searches the case database 100 for cases having a weighted distance smaller than a predetermined threshold as similar cases. Alternatively, the secondary search unit 190 searches the case database 100 for a predetermined number of cases having a small weighting distance as similar cases.

The weighted distance can be calculated by, for example, (Equation 3). Here, x is a vector in which a plurality of ( _NIF ) image feature amounts extracted from the image to be interpreted are all connected. u ⁱ is an image feature amount extracted from the i-th case among cases stored in the case database 100. When different types of image feature quantities are connected, canonicalization (normalized to 0 average and 1 variance) is performed so as not to be affected by the difference in scale for each image feature quantity. w _j is a weight related to the j-th image feature amount. Here, the weight calculated in step S26 is used.

In the present embodiment, weighted distance calculation is performed based on the weight determined for each disease type candidate cluster in step S26. That is, for the dominant interpretation items (that is, interpretation items having a large average interpretation item vector element value) among the disease type candidate clusters, the weight is relatively large, and at the same time, the image feature amount having a large correlation with the interpretation item. Increases the weight relatively. This makes it possible to search for similar cases that reflect the characteristics of the disease type candidate cluster. This will be described with reference to FIG. (A) of FIG. 23 shows how the weight vector W is acquired in step S26 using the average interpretation item vector corresponding to the disease type candidate cluster as an input. As described in step S26, the weight vector W is calculated from the elements of the average interpretation item vector and the correlation between the interpretation item and the image feature amount. In (b) of FIG. 23, using the weight vector _W, with respect to _{N IF} pieces of image feature amounts, performed respectively weighted. In other words, the image feature quantity that well expresses the characteristic of the dominant interpretation item in the average interpretation item vector is heavily weighted for retrieval.

<Weighted search using N _CW weight vectors>
The secondary search unit 190 performs a weighted search for each of a plurality (N _CW ) of disease type candidate clusters. Each disease type candidate cluster is generated so that combinations of interpretation items included in the disease type candidate cluster are similar. That is, when a diagnosis target case is viewed as a plurality of disease types, similar cases are searched for each disease type. FIG. 24 shows an image diagram in which a plurality of weighted searches are performed. Since the interpretation items included in each disease type candidate cluster are different, the search is performed so that the weighting to the image feature space of the _NIF dimension is different.

  Although an example in which a secondary search is performed for cases stored in the case database 100 has been described here, a secondary search may be performed by limiting the search target cases to cases of the primary search result. . By limiting the target, a high-speed search becomes possible. On the other hand, when performing a secondary search from the case database 100, by limiting the points of interest, it is possible to search for cases that have been leaked in the primary search that treats all dimensions of the image feature amount equally.

  In step S28, the similar case display unit 200 displays the searched similar cases in the similar case display area 320 of FIG. An example of the display screen of the similar case display area 320 is shown in FIG. In FIG. 25, the search result image with the highest similarity and its report are displayed for each important interpretation item. An important interpretation item is, for example, an interpretation item whose element value of the average interpretation item vector of each disease type candidate cluster is equal to or greater than a predetermined threshold. When similar cases are searched from the viewpoint of “important interpretation items” 322 with respect to the interpretation target image 300, the interpreter refers to the search result image 323 and the interpretation report 324 to be searched for, thereby interpreting the interpretation target image 300. It can be used as a reference for 300 interpretations. In particular, for a resident who is not accustomed to viewing (focusing on) an image or an interpreting doctor who handles an unspecialized organ, the interpretation skill can be improved by confirming the “important interpretation item” 322.

  In addition, the example of the display screen of the similar case display part 200 in the case of using a definite diagnosis disease name is shown in FIG. In FIG. 26, for each confirmed diagnosis disease name 321 and “important interpretation item” 322, a search result image 323 having the highest similarity and an interpretation report 324 are displayed. When an image interpreter searches similar images from the viewpoint of a definitive diagnosis disease name 321 and “important image interpretation items” 322 with respect to the image to be interpreted 300, the image interpreter refers to the retrieved search result image 323 and the image interpretation report 324. Thus, it can be used as a reference for interpretation of the interpretation target image 300.

  According to this configuration, the primary search unit 150 first acquires a plurality of past cases roughly similar as images, and the interpretation item extraction unit 160 extracts interpretation items from the acquired cases. In addition, the disease type candidate cluster generation unit 170 generates clusters having similar combinations of interpretation items, the weight determination unit 180 determines a weight for each cluster, and the secondary search unit 190 performs weighted search. This makes it possible to weight image feature amounts based on information of image groups similar to the search target image without preparing an image feature amount set for each disease type in advance. The corresponding similar case search can be efficiently performed.

(Modification of the embodiment)
In step S26 of FIG. 12 of the above-described embodiment, at least one important interpretation item may be specified among the interpretation items corresponding to the disease type candidate cluster. Since a disease type candidate cluster includes a large number of cases, using an interpretation item with a low appearance frequency may cause search noise even though the weight value for the interpretation item becomes small. In order to deal with this problem, it is possible to specify and use an interpretation item with a high appearance frequency. Since a disease type is often defined by a combination of 2 to 3 main image findings, two interpretation items are specified here.

A method for identifying related interpretation items will be described. In order to identify the interpretation items dominant in configuring the disease type candidate cluster, here, the representative vector generated for each disease type candidate cluster generated in step S25 is used. As a representative vector of the cluster, an average vector of all interpretation item vectors forming the cluster is calculated. FIG. 27 shows the calculated representative vector v of cluster 1 = (0.7, 0, 0, 0.4, 0.1, 0, 0, 0,...). Then, an interpretation item in which an angle θ _i formed by the representative vector v and each axis (unit vector u _i ) representing the interpretation item i is within a predetermined angle θTH is specified. θ _i can be calculated by (Equation 4). Note that θTH is determined in advance.

  Here, the dominant interpretation item vector is specified using the angle between the cluster representative vector and the unit vector of the axis representing the interpretation item, but the dominant interpretation item vector may be identified by other methods. good. For example, the top two interpretation items may be selected from those having a large appearance ratio (= number of appearances / number of cases constituting the cluster) of the interpretation items constituting each cluster, and the dominant interpretation item may be specified. Further, instead of fixing the number of interpretation items, it is also possible to specify interpretation items that satisfy the predetermined conditions and the appearance frequency conditions regarding the angle. In this case, an upper limit of the number of interpretation items may be set.

Next, the weight determination method will be described. In FIG. 28, in the cluster number 1, two interpretation items of “cavity lesion” and “nodule” are specified. The weight determination unit 180 refers to the correlation table between (image feature amount−interpretation item) stored in the interpretation knowledge database 110 in the format of FIG. 10, and correlates between “cavity lesion” and all image feature amounts. Relationships and correlations between “nodules” and all image feature quantities are acquired. Here, the numerical value (correlation ratio) representing the acquired correlation is used as a weight as it is, and is expressed as w1 _{, i} , w2 _{, i} , respectively. Here, i is a subscript indicating the type of the image feature amount. Using these weights, the weights W _i corresponding to the i-th image feature amount, calculated as (Equation 5). In (Expression 5), K represents the number of specified interpretation items, and K = 2 when “cavity lesion” and “nodule” are specified. In cluster numbers 3 and 4 in FIG. 28, K = 3.

  Next, weighted search using one weight vector will be described.

In this modification, weighted distance calculation is performed by similar case search based on the interpretation items specified for each disease type candidate cluster in step S26. That is, the weight is relatively large for an image feature amount having a large correlation with the specified interpretation item, and the weight is relatively small for an image feature amount having a small correlation. This makes it possible to search for similar cases that reflect the characteristics of the disease type candidate cluster. This will be described with reference to FIG. FIG. 29A shows a state in which the weight vector W determined from the interpretation items “cavity lesion” and “nodule” identified in step S26 is acquired. As described in step S26, the weight vector W is obtained by calculating the correlation between the “cavity lesion” and all the image feature amounts, and the correlation between the “nodule” and all the image feature amounts. It consists of the sum of the corresponding elements. In (b) of FIG. 29, using the weight vector _W, with respect to _{N IF} pieces of image feature amounts, performed respectively weighted. That is, the search is performed by heavily weighting the image feature amount that well expresses the features of “cavity lesion” and “nodule”. This corresponds to searching for similar cases when the diagnosis target case is viewed as a disease type having the characteristics of “cavity lesion” and “nodule”.

  Except for the points described above, this embodiment is the same as the embodiment.

  According to this configuration, the primary search unit 150 first acquires a plurality of past cases roughly similar as images, and the interpretation item extraction unit 160 extracts interpretation items from the acquired cases. In addition, the disease type candidate cluster generation unit 170 generates clusters having similar combinations of interpretation items, the weight determination unit 180 determines a weight for each cluster, and the secondary search unit 190 performs weighted search. This makes it possible to weight image feature amounts based on information of image groups similar to the search target image without preparing an image feature amount set for each disease type in advance. The corresponding similar case search can be efficiently performed. In addition, by specifying and using interpretation items with a high appearance frequency, search noise that tends to be a problem when using an interpretation item with a low appearance frequency can be suppressed.

  In the above-described embodiment, the similar case display unit 200 is presented to the user by displaying the search result of the secondary search unit 190. However, instead of the similar case display unit 200, the secondary search unit 190 You may provide the output part which prints a search result or writes in a memory | storage device.

  As described above, the embodiment and the modification of the embodiment have been described. However, the essential components for the similar case search apparatus are the image feature acquisition unit 140, the primary search unit 150, the interpretation item extraction unit 160, and the disease type candidate cluster. The generation unit 170, the weight determination unit 180, and the secondary search unit 190, and other components may not necessarily be included in the similar case search apparatus.

  INDUSTRIAL APPLICABILITY The present invention can be used for a similar case retrieval apparatus that retrieves and presents a similar case that is a reference to an interpreter, an interpretation education apparatus for a training interpreter, and the like.

100 Case Database 110 Interpretation Knowledge Database 120 Interpretation Target Image Reading Unit 130 Interpretation Target Image Display Unit 140 Image Feature Acquisition Unit 150 Primary Search Unit 160 Interpretation Item Extraction Unit 170 Disease Type Candidate Cluster Generation Unit 180 Weight Determination Unit 190 Secondary Search Unit 200 Similar Case Display Unit 210 Interpretation Terminal 300 Interpretation Target Image 320 Similar Case Display Area 321 Definitive Diagnosis Disease Name 322 Imported Interpretation Item 323 Retrieval Result Image 324 Interpretation Report 400 Interpretation Target Image 410 Surrounding Chest 420 Target Organ 430 Lesion Area 440 Point 501 Button 502 Confirmation diagnosis disease name 503 Interpretation item

Claims (11)

Includes a medical image similar to an image to be interpreted, which is a medical image to be interpreted, from a case database in which a plurality of case data including a medical image and an interpretation report, which is document data describing the result of interpretation of the medical image, is stored A similar case retrieval device for retrieving case data,
An image feature acquisition unit that acquires a plurality of image feature amounts from the image to be interpreted;
Based on the plurality of image feature amounts acquired by the image feature acquisition unit and a plurality of image feature amounts acquired from medical images included in each case data registered in the case database, A primary search unit that calculates a similarity with a medical image included in each case data, and obtains a plurality of case data including the medical image in which the calculated similarity is equal to or greater than a predetermined value;
An interpretation item extraction unit that extracts a plurality of interpretation items that are character strings obtained by verbalizing the characteristics of the medical image from the interpretation report included in each case data acquired by the primary search unit;
Using a plurality of interpretation items extracted for each case data by the interpretation item extraction unit, a disease type candidate cluster generation unit that generates a plurality of disease type candidate clusters composed of case data groups with similar combinations of interpretation items;
Extracted from each image feature amount extracted from the medical image and an interpretation report for the medical image for each of at least one disease type candidate cluster among the plurality of disease type candidate clusters generated by the disease type candidate cluster generation unit. For each image feature amount acquired by the image feature acquisition unit based on an interpretation knowledge database in which relevance of each interpretation item is determined in advance, between the image feature amount and the interpretation item included in the disease type candidate cluster A weight determination unit that determines the weight of a larger value as the relationship of
The weight determination unit determines the plurality of image feature amounts extracted by the image feature acquisition unit and the plurality of image feature amounts acquired from medical images included in case data registered in the case database. A similar case retrieval apparatus comprising: a secondary retrieval unit that retrieves case data including a medical image similar to the interpretation target image from the case database by weighting and comparing with the weight for each image feature amount. The weight determination unit, based on the interpretation knowledge database, for each of at least one disease type candidate cluster among the plurality of disease type candidate clusters generated by the disease type candidate cluster generation unit, the image feature acquisition unit For each acquired image feature amount, the relationship between the image feature amount and the interpretation item included in the disease type candidate cluster is higher, and the frequency of appearance of the interpretation item included in the disease type candidate cluster is higher. The similar case search device according to claim 1, wherein the weight of a larger value is determined as the value is higher. The secondary search unit includes the plurality of image feature amounts extracted from the image feature acquisition unit and medical data included in case data acquired by the primary search unit among case data registered in the case database. Case data including a medical image similar to the image to be interpreted is compared by weighting the plurality of image feature amounts acquired from the image with weights for each image feature amount determined by the weight determination unit. The similar case search device according to claim 1, wherein the search is performed from a plurality of case data acquired by the primary search unit. The disease type candidate cluster generation unit uses a plurality of interpretation items extracted for each case data by the interpretation item extraction unit, the combination of interpretation items is similar, and the definitive diagnosis disease name given to the case data The similar case search device according to any one of claims 1 to 3, wherein a plurality of disease type candidate clusters including the same case data group are generated. further,
The similar case search apparatus of any one of Claims 1-4 provided with the presentation part which shows a user the search result of the said secondary search part. The similar case search device according to claim 5, wherein the presentation unit presents the case data searched by the secondary search unit to a user for each disease type candidate cluster generated by the disease type candidate cluster generation unit. The weight determination unit includes, for each of at least one disease type candidate cluster among the plurality of disease type candidate clusters generated by the disease type candidate cluster generation unit, among interpretation items included in the disease type candidate cluster. Specify at least one interpretation item, and based on the interpretation knowledge database, for each image feature amount acquired by the image feature acquisition unit, the higher the relationship between the image feature amount and the specified interpretation item, the greater The similar case retrieval apparatus according to claim 1, wherein weights of values are determined. further,
8. A presentation unit that presents the case data searched by the secondary search unit and the interpretation items specified by the weight determination unit for each disease type candidate cluster generated by the disease type candidate cluster generation unit. The similar case search device described in 1. further,
Of the plurality of disease type candidate clusters generated by the disease type candidate cluster generation unit, comprising an input unit that receives selection input of any disease type candidate cluster by the user,
The weight determination unit, for each of the disease type candidate clusters indicated by the selection input received by the input unit among the plurality of disease type candidate clusters generated by the disease type candidate cluster generation unit, Identify at least one interpretation item among the interpretation items included in the cluster, and for each image feature amount acquired by the image feature acquisition unit based on the interpretation knowledge database, the image feature amount and the specified interpretation item The similar case search device according to claim 7, wherein the weight of a larger value is determined as the relationship between the higher values. Includes a medical image similar to an image to be interpreted, which is a medical image to be interpreted, from a case database in which a plurality of case data including a medical image and an interpretation report, which is document data describing the result of interpretation of the medical image, is stored A similar case search method for searching case data,
An image feature acquisition step of acquiring a plurality of image feature amounts from the image to be interpreted;
Based on the plurality of image feature amounts acquired in the image feature acquisition step and a plurality of image feature amounts acquired from medical images included in each case data registered in the case database, the image to be interpreted A primary search step of obtaining a plurality of case data including the medical image in which the calculated similarity is equal to or greater than a predetermined value;
An interpretation item extraction step for extracting a plurality of interpretation items, which are character strings obtained by verbalizing the characteristics of the medical image, from the interpretation report included in each case data acquired in the primary search step;
A disease type candidate cluster generation step for generating a plurality of disease type candidate clusters composed of case data groups with similar combinations of interpretation items using a plurality of interpretation items extracted for each case data from the interpretation item extraction step; ,
Of each of the plurality of disease type candidate clusters generated in the disease type candidate cluster generation step, for each of at least one disease type candidate cluster, from each image feature amount extracted from the medical image and an interpretation report for the medical image Based on the interpretation knowledge database in which the relevance of each extracted interpretation item is determined in advance, for each image feature amount acquired in the image feature acquisition step, the image feature amount and the interpretation item included in the disease type candidate cluster A weight determination step for determining the weight of a larger value as the relationship between
In the weight determination step, the plurality of image feature amounts extracted in the image feature acquisition step and the plurality of image feature amounts acquired from medical images included in case data registered in the case database. A similar case search method comprising: a secondary search step of searching case data including a medical image similar to the interpretation target image from the case database by weighting and comparing with the determined weight for each image feature amount.   The program for making a computer perform the similar case search method of Claim 10.

Images