JP5789791B2 - Similar case retrieval device and interpretation knowledge extraction device - Google Patents

Description

  The present invention relates to a similar case retrieval apparatus for retrieving a similar case that serves as a reference for interpretation of a medical image, and an interpretation knowledge extraction apparatus for extracting interpretation knowledge used when retrieving a similar case.

  In recent years, high-definition medical images that have been digitized can be acquired in a large capacity due to the development and spread of medical imaging devices such as CT (Computed Tomography) and MRI (Magnetic Resonance Imaging). In addition, medical images that have been interpreted by a doctor are being sequentially accumulated in a PACS (Picture Archiving and Communication Systems) together with an interpretation report. Here, in order to refer to a new interpretation, a technique for searching a past medical image similar to a medical image to be interpreted from accumulated past cases has begun to be developed.

  For the similar image search, it is important to select an image feature amount that determines the similarity between images. However, the following technique is disclosed as a conventional image search apparatus.

  Image feature values for determining the similarity between medical images should be different depending on the type of disease, the degree of disease progression, the severity of disease, etc., but the conventional medical image similarity search is related to the situation. It has been a problem to use the same image feature amount. Non-Patent Document 1 proposes a two-step search method called “customized-queries” approach (CQA) as a solution. In the first step, the query image is classified by using an image feature amount that can best classify a class such as a disease type, a disease progression degree, or a disease severity. In a second step, similar images are searched using image feature quantities optimized for further classifying cases included in the class that is the classification result. At this time, an optimal image feature amount for each class is obtained in advance by unsupervised learning. Further, in this document, CQA is applied to a lung CT image, and the retrieval recall rate is improved as compared with the conventional similar image retrieval using one type of image feature amount.

Jennifer G. Dy et al. “Unsupervised Feature Selection Applied to Content-based Retrieval of Lung Images”, IEEE Transactions on Pattern Analysis and Machine Intelligence. 25, no. 3, March 2003

  However, in the conventional configuration, an image feature amount (that is, a similarity criterion) is determined based on the identified disease type, disease progression degree, or disease severity, and the like. The image is only searched, and it is not a similar image search that reflects the point of interest of the interpreter for the medical image to be interpreted. That is, there is a problem that it is difficult to support the diagnosis made by the image interpreter or the diagnosis that the image interpreter is at a loss.

  This invention solves the said subject, and it aims at providing the similar case search apparatus which reflected the point of interest of an image interpreter in the similar image search.

  It is another object of the present invention to provide an interpretation knowledge extraction device for extracting interpretation knowledge used in a similar case search in which a point of interest of an interpreter is reflected in a similar image search.

  In order to solve the conventional problem, a similar case retrieval apparatus according to an aspect of the present invention is similar to case data including a medical image and an interpretation report that is document data describing a result of interpretation of the medical image. A similar case retrieval apparatus that retrieves case data to be retrieved from a case database, wherein the interpretation report includes (a) an interpretation item that is a character string indicating the characteristics of the medical image, and (b) a diagnostic result of the interpreter based on the medical image The similar case search device includes an image feature extraction unit that extracts a plurality of image feature amounts from an image to be interpreted, which is a medical image to be interpreted, and an image interpretation for the image to be interpreted. Report analysis unit that extracts disease names from target interpretation reports that are reports, and weight determination that determines weights for each image feature based on the interpretation knowledge database A plurality of image feature amounts extracted by the image feature extraction unit, and a plurality of image feature amounts extracted from medical images included in case data registered in a case database. A similar case retrieval unit that retrieves case data including a medical image similar to the image to be interpreted from the case database by weighting with the weight for each image feature amount determined in The database includes first binomial relationship information in which a relationship between each image feature amount extracted from a medical image and each interpretation item is predetermined, and a relationship between each interpretation item and each disease name. The second binomial relationship information, the third binomial relationship information that predetermines the relationship between each image feature quantity and each disease name, and the disease name for each disease name. Multiple combinations of highly relevant image feature quantities Presence / absence information indicating whether or not it exists, and the weight determination unit is related to a target disease name that is a disease name extracted by the report analysis unit based on the presence / absence information included in the interpretation knowledge database. When it can be determined that there are a plurality of combinations of high image feature amounts, calculation is performed from the first binomial relationship information and the second binomial relationship information included in the knowledge database for each image feature amount combination. The fourth binomial relationship information indicating the relationship between each image feature quantity via the interpretation item and the target disease name is used instead of the third binomial relationship information, For each image feature amount extracted by the image feature extraction unit, a higher weight is determined as the relationship between the image feature amount and the target disease name is higher.

  According to this configuration, each image feature amount extracted from a medical image or an image to be interpreted is weighted with a greater weight as the relevance to the target disease name described in the object interpretation report is higher. Thus, similar cases are searched by comparing the weighted image feature amounts. Thereby, a similar case search can be performed after reflecting the point of interest of the interpreter entered in the target interpretation report. At this time, when there are a plurality of combinations of image feature quantities that are highly relevant to the target disease name, for each combination of image feature quantities, the relationship between the image feature quantity and the target disease name via the interpretation item Weighting is performed using the fourth binomial relationship information indicating. For this reason, even when there are a plurality of types of image patterns for the target disease name, it is possible to perform a similar case search while distinguishing them.

  An interpretation knowledge extraction apparatus according to another aspect of the present invention is an interpretation knowledge that extracts interpretation knowledge from a plurality of case data including a medical image and an interpretation report that is document data describing a result of interpretation of the medical image. In the extraction device, the interpretation report includes (a) an interpretation item that is a character string indicating a feature of the medical image, or (b) a disease name that is a diagnosis result of the interpreter based on the medical image, and the interpretation knowledge extraction The apparatus includes an image feature extraction unit that extracts a plurality of image feature amounts from a medical image included in each case data, a report analysis unit that extracts a disease name and an interpretation item from an interpretation report included in each case data, The first binomial relationship information that defines the relationship between each image feature amount extracted by the image feature extraction unit and each interpretation item extracted by the report analysis unit, and the report analysis unit extracts Second binomial relationship information that defines the relationship between each interpretation item and each disease name extracted by the report analysis unit, each image feature amount extracted by the image feature extraction unit, and the report analysis unit Interpreting knowledge extracting unit for extracting third binomial relationship information that defines the relationship between each disease name extracted as a reading interpretation knowledge, the first binomial relationship information, and the second Interpretation knowledge for determining whether there are a plurality of combinations of image feature quantities that are highly related to a disease name for each disease name based on the binomial relationship information and the third binomial relationship information Based on the determination unit and the second binomial relationship information, for each disease name that the interpretation knowledge determination unit determines that there are multiple combinations of image feature amounts that are highly relevant, the disease name and the image feature amount For each combination of image features, a set of interpretation items to be associated with An interpretation item extraction unit that outputs, and the interpretation knowledge extraction unit further extracts, for each set of interpretation items extracted by the interpretation item extraction unit, each set of interpretation items and the image feature extraction unit. Information on the relationship between the two terms that defines the relevance between the image feature amounts is determined between the image feature amounts included in the combination of the disease name and the image feature amount that the interpretation knowledge determination unit has determined to have high relevance. It is extracted as fourth binomial relationship information that is interpretation knowledge showing relevance.

  According to this configuration, even when there are a plurality of types of image patterns for one disease name, the fourth binomial relationship information includes the disease name for each image pattern and each image feature amount. The relevance of can be extracted. For this reason, it is possible to extract the interpretation knowledge used in the similar case search in which the point of interest of the interpreter is reflected in the similar image search.

  Note that the present invention can be realized not only as an apparatus including such a characteristic processing unit, but also as a method in which processing executed by the characteristic processing unit included in the apparatus is used as a step. . Moreover, it can also be realized as a program for causing a computer to execute a characteristic step included in a program or a method for causing a computer to function as a characteristic processing unit included in the apparatus. Such a program can be distributed via a computer-readable non-transitory recording medium such as a CD-ROM (Compact Disc-Read Only Memory) or a communication network such as the Internet. .

  ADVANTAGE OF THE INVENTION According to this invention, the similar case search apparatus which reflected the imager's attention point on the similar image search can be provided. It is also possible to provide an interpretation knowledge extraction device for extracting interpretation knowledge used in a similar case search in which a point of interest of an interpreter is reflected in a similar image search.

The block diagram which shows the structure of the similar case search apparatus in embodiment of this invention The block diagram which shows the structure of the interpretation knowledge extraction apparatus in embodiment of this invention The flowchart which shows the procedure of the interpretation knowledge preparation in embodiment of this invention The flowchart which shows the procedure of the image feature-value extraction in embodiment of this invention The figure which shows the example of the interpretation report of the abdominal CT examination in embodiment of this invention The figure which shows the interpretation item and disease name extracted from the interpretation report in embodiment of this invention The figure which shows the information of the position and time phase extracted simultaneously with the interpretation item and disease name extracted from the interpretation report in embodiment of this invention and the interpretation item The figure which shows the information of the position and time phase extracted from the interpretation item and disease name extracted from the interpretation report in embodiment of this invention, and context interpretation and simultaneous with the interpretation item The figure which shows a data set acquired for the interpretation knowledge extraction in embodiment of this invention. The conceptual diagram of the correlation (binary) between the interpretation item and image feature-value in embodiment of this invention The conceptual diagram of the correlation (multi-value) between the interpretation item and image feature-value in embodiment of this invention The conceptual diagram of the correlation (binary) between the disease name and the image feature-value in embodiment of this invention Conceptual diagram of correlation (binary) between interpretation item and disease name in the embodiment of the present invention The figure which shows the storage format of the correlation between the (image feature-value-interpretation item) extracted as interpretation knowledge in embodiment of this invention. The figure which shows the storage format of the correlation between (image feature-value-disease name) extracted as interpretation knowledge in embodiment of this invention. The figure which shows the storage format of the correlation between (interpretation item-disease name) extracted as interpretation knowledge in embodiment of this invention. Conceptual diagram showing the difference in image pattern in the early lesion area of the disease name "hepatocellular carcinoma" Conceptual diagram showing the difference in the distribution of the image feature “average luminance within the tumor in the early phase” depending on the presence or absence of the text feature (disease name “hepatocellular carcinoma”) The conceptual diagram of the relative relationship between the three terms when there are a plurality of image patterns in the embodiment of the present invention The conceptual diagram of the correlation between (image feature-value-disease name) at the time of adding the interpretation knowledge in embodiment of this invention The flowchart which shows the addition procedure of the interpretation knowledge in embodiment of this invention The flowchart which shows the detail of the interpretation knowledge determination process (step S31 of FIG. 21) in embodiment of this invention. The figure which shows the storage format of the correlation between (image feature-value-disease name) after the addition of the interpretation knowledge in embodiment of this invention. The flowchart which shows the detail of the interpretation item extraction process (step S32 of FIG. 21) in embodiment of this invention. The figure which shows the storage format of the correlation between the (image feature-value-disease name) extracted beforehand by the group of the interpretation item with a high correlation with the disease name in embodiment of this invention. The flowchart which shows the addition procedure of the interpretation knowledge when performing only extraction of the interpretation item in embodiment of this invention The figure which shows the storage format of the group of the interpretation item with a high correlation with the disease name in embodiment of this invention The figure which shows the storage format of the correlation between the (image feature-value-disease name) extracted beforehand by the group of the interpretation item with a high correlation with the disease name in embodiment of this invention. The flowchart which shows the procedure of the similar case search in embodiment of this invention The figure which shows the weighting method at the time of similar case search in embodiment of this invention The figure which shows the weighting method at the time of similar case search in embodiment of this invention The figure which shows the weighting method at the time of similar case search in embodiment of this invention The figure which shows the weighting method when the correlation between (image feature-value-disease name) is added in embodiment of this invention. The figure which shows the weighting method when the correlation between (image feature-value-disease name) is added in embodiment of this invention. The figure which shows the weighting method when the correlation between (image feature-value-disease name) is added in embodiment of this invention. The figure which shows the weighting method when the correlation between (image feature-value-disease name) is added in embodiment of this invention.

  Embodiments of the present invention will be described below with reference to the drawings. Each of the embodiments described below shows a preferred specific example of the present invention. Numerical values, components, connection modes of components, steps, order of steps, and the like shown in the following embodiments are merely examples, and are not intended to limit the present invention. The invention is limited only by the claims. Therefore, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims indicating the highest concept of the present invention are not necessarily required to achieve the object of the present invention. It will be described as constituting a preferred form.

  A similar case retrieval apparatus according to an embodiment of the present invention retrieves case data similar to case data including a medical image and an interpretation report, which is document data describing a result of interpretation of the medical image, from the case database. In the similar case search apparatus, the interpretation report includes (a) an interpretation item that is a character string indicating a feature of a medical image and (b) a disease name that is a diagnosis result of an interpreter based on the medical image. The similar case retrieval apparatus includes: an image feature extraction unit that extracts a plurality of image feature amounts from an interpretation target image that is a medical image to be interpreted; and a disease name from the target interpretation report that is an interpretation report for the interpretation target image. A report analysis unit for extracting image data, a weight determination unit for determining a weight for each image feature amount based on an interpretation knowledge database, and the image feature extraction unit The plurality of image feature amounts that have been issued and a plurality of image feature amounts that are extracted from medical images included in case data registered in the case database, for each image feature amount determined by the weight determination unit. A similar case retrieval unit that retrieves case data including a medical image similar to the image to be interpreted from the case database by weighting and comparing, and the interpretation knowledge database is extracted from the medical image The first binomial relationship information that predetermines the relevance between each image feature and each interpretation item, and the second two terms that predetermine the relevance between each interpretation item and each disease name. A combination of inter-related information, third binary relation information that predetermines the relationship between each image feature amount and each disease name, and image feature amount that is highly related to the disease name for each disease name Presence / absence information indicating whether or not there are multiple The weight determination unit includes a plurality of combinations of image feature quantities that are highly relevant to the target disease name that is the disease name extracted by the report analysis unit based on the presence / absence information included in the interpretation knowledge database. If it can be determined, for each combination of image features, each calculated via the interpretation items calculated from the first binomial relationship information and the second binomial relationship information included in the knowledge database The image feature extracted by the image feature extraction unit using the fourth binomial relationship information indicating the relationship between the image feature quantity and the target disease name instead of the third binomial relationship information For each quantity, a higher weight is determined as the relationship between the image feature quantity and the target disease name is higher.

  According to this configuration, each image feature amount extracted from a medical image or an image to be interpreted is weighted with a greater weight as the relevance to the target disease name described in the object interpretation report is higher. Thus, similar cases are searched by comparing the weighted image feature amounts. Thereby, a similar case search can be performed after reflecting the point of interest of the interpreter entered in the target interpretation report. At this time, when there are a plurality of combinations of image feature quantities that are highly relevant to the target disease name, for each combination of image feature quantities, the relationship between the image feature quantity and the target disease name via the interpretation item Weighting is performed using the fourth binomial relationship information indicating. For this reason, even when there are a plurality of types of image patterns for the target disease name, it is possible to perform a similar case search while distinguishing them.

  Preferably, when the weight determination unit can determine that there is only one kind of combination of the image feature amount highly related to the target disease name based on the presence / absence information included in the interpretation knowledge database, For each image feature amount extracted by the image feature extraction unit, a higher value weight is determined as the relationship between the image feature amount and the target disease name is higher.

  An interpretation knowledge extraction device according to another embodiment of the present invention extracts an interpretation knowledge from a plurality of case data including a medical image and an interpretation report that is document data describing a result of interpretation of the medical image. The knowledge extraction device, wherein the interpretation report includes (a) an interpretation item that is a character string indicating a feature of a medical image, or (b) a disease name that is a diagnosis result of an interpreter based on the medical image, and the interpretation knowledge The extraction device includes an image feature extraction unit that extracts a plurality of image feature amounts from a medical image included in each case data, a report analysis unit that extracts a disease name and an interpretation item from an interpretation report included in each case data, A first binomial relationship information for defining a relationship between each image feature amount extracted by the image feature extraction unit and each interpretation item extracted by the report analysis unit; and the report analysis unit Second binomial relationship information that defines the relationship between each extracted interpretation item and each disease name extracted by the report analysis unit, each image feature amount extracted by the image feature extraction unit, and the report analysis The third binomial relation information that defines the relationship between each disease name extracted by the department as the interpretation knowledge, the first binomial relation information, Interpretation for determining whether there are a plurality of combinations of image feature quantities that are highly related to a disease name for each disease name based on the binary relationship information of 2 and the third binary relationship information Based on the knowledge determination unit and the second binomial relationship information, for each disease name determined by the interpretation knowledge determination unit that there are a plurality of combinations of image feature quantities that are highly relevant, the disease name and image feature The combination of image interpretation items to be associated with the combination of quantities is the combination of image feature quantities. An interpretation item extraction unit for extracting the interpretation item, and the interpretation knowledge extraction unit further extracts, for each interpretation item group extracted by the interpretation item extraction unit, the interpretation item set and the image feature extraction unit. The binomial relationship information that defines the relevance between each image feature amount is determined between the image feature amount included in the combination of the disease name and the image feature amount that the interpretation knowledge determination unit has determined to be highly relevant. It is extracted as fourth binomial relationship information which is interpretation knowledge indicating the relevance of.

  According to this configuration, even when there are a plurality of types of image patterns for one disease name, the fourth binomial relationship information includes the disease name for each image pattern and each image feature amount. The relevance of can be extracted. For this reason, it is possible to extract the interpretation knowledge used in the similar case search in which the point of interest of the interpreter is reflected in the similar image search.

  Preferably, the interpretation knowledge determination unit is configured to: (a) a first correlation that defines a relationship between a first image feature amount selected from the third binomial relationship information and a first disease name; A correlation value that defines an association between the first disease name selected from the second binomial relationship information and the first interpretation item is greater than or equal to a correlation value threshold value. Difference between the first correlation item selected from the first binomial relationship information and the second correlation value that defines the relationship between the first image feature amount A value is calculated, and when the calculated difference value is larger than a difference threshold value, it is determined that there are a plurality of combinations of image feature amounts highly related to the first disease name.

  More preferably, the interpretation item extraction unit includes the disease indicated by the second binomial relationship information for each disease name determined by the interpretation knowledge determination unit to have a plurality of combinations of image feature quantities that are highly relevant. Among all the interpretation item sets created from the interpretation items whose correlation value with the name is larger than the first threshold value, the set of interpretation items whose correlation value with the disease name is equal to or more than the second threshold value Are extracted as a set of interpretation items associating the disease name with the combination of the image feature amounts.

  FIG. 1 is a block diagram showing a configuration of a similar case retrieval apparatus according to an embodiment of the present invention.

  The similar case retrieval apparatus includes a case database 100, an interpretation knowledge database 110, an interpretation target image reading unit 120, an interpretation target image display unit 130, a report input / output unit 140, a report analysis unit 150, and an image feature extraction unit. 160, a weight determination unit 170, a similar case search unit 180, and a similar case display unit 190.

  The case database 100 includes a medical image such as CT (Computed Tomography) or MRI (Magnetic Resonance Imaging) (in this specification, “image data” is simply referred to as “image”), and a result of reading the medical image. This is a database storing a plurality of case data (hereinafter simply referred to as “cases”) composed of pairs with interpretation reports. The interpretation knowledge database 110 is a database that stores interpretation knowledge obtained by analyzing a plurality of cases. Details will be described later. The case database 100 and the interpretation knowledge database 110 are stored in a storage device such as an HDD (Hard Disk Drive).

  The interpretation target image reading unit 120 reads an image captured by a medical image capturing device such as CT or MRI from a medical image capturing device or an externally connected storage device.

  The interpretation target image display unit 130 includes a medical high-definition monitor or the like, and displays the interpretation target image read by the interpretation target image reading unit 120.

  The report input / output unit 140 includes an input device such as a keyboard and a mouse, and a display device that allows an input person to confirm by displaying a report input using such an input device. The interpreter inputs an interpretation report through the report input / output unit 140 while referring to the interpretation target image displayed on the interpretation target image display unit 130.

  The interpretation target image display unit 130, the report input / output unit 140, and a similar case display unit 190 described later constitute the interpretation terminal 200.

  The report analysis unit 150 analyzes the interpretation report input by the report input / output unit 140 and extracts text feature quantities (interpretation items and disease names).

  The image feature extraction unit 160 extracts a plurality of types of image feature amounts from the interpretation target images read by the interpretation target image reading unit 120.

  The weight determination unit 170 is used for image search from the text feature amount extracted by the report analysis unit 150, the image feature amount extracted by the image feature extraction unit 160, and the interpretation knowledge stored in the interpretation knowledge database 110. Weights for a plurality of image feature amounts are respectively determined.

  The similar case search unit 180 includes a medical image similar to the image to be interpreted from the case database 100 using the image feature amount extracted by the image feature extraction unit 160 and the weight determined by the weight determination unit 170. Search for cases.

  The similar case display unit 190 displays the similar cases searched by the similar case search unit 180. The similar case display unit 190 may be separately configured with the same model as the high-definition monitor that constitutes the interpretation target image display unit 130. The similar case display unit 190 is similar to the interpretation target image and the similar case on the high-definition monitor that constitutes the interpretation target image display unit 130. May be displayed simultaneously. Note that the similar case display unit 190 and the interpretation target image display unit 130 may be of different models.

  FIG. 2 is a block diagram showing the configuration of the image interpretation knowledge extracting apparatus in the embodiment of the present invention.

  The interpretation knowledge extraction device includes a report analysis unit 250, an image feature extraction unit 260, an interpretation knowledge extraction unit 210, an interpretation knowledge determination unit 220, an interpretation item extraction unit 230, a case database 100, and an interpretation knowledge database 110. Is provided.

  The case database 100 and the interpretation knowledge database 110 have the same configuration as that shown in FIG.

  The report analysis unit 250 analyzes an interpretation report read from the case database 100 and extracts a text feature amount. The report analysis unit 250 is the same as the report analysis unit 150 except that the interpretation report reading destination is different.

  The image feature extraction unit 260 extracts a plurality of types of image feature amounts from the interpretation target image read from the case database 100. The image feature extraction unit 260 is the same as the image feature extraction unit 160 except that the image reading target image is read from a different destination.

  The interpretation knowledge extraction unit 210, interpretation knowledge determination unit 220, and interpretation item extraction unit 230 are processing units for extracting interpretation knowledge and adding interpretation knowledge.

  The interpretation knowledge extraction unit 210 stores interpretation knowledge obtained by analyzing a plurality of cases stored in the case database 100 in the interpretation knowledge database 110.

  The interpretation knowledge determination unit 220 reads the interpretation knowledge stored in the interpretation knowledge database 110, and determines a target (disease name) to add the interpretation knowledge in the interpretation knowledge.

  The interpretation item extraction unit 230 analyzes the correlation with respect to the subject (disease name) to which interpretation knowledge is added, and extracts a set of interpretation items used for the addition of interpretation knowledge. At the time of adding interpretation knowledge, the interpretation knowledge extracting unit 210 acquires new interpretation knowledge by analyzing a plurality of cases using the set of interpretation items extracted by the interpretation item extracting unit 230, and stores it in the interpretation knowledge database 110. Store. Details of extraction and addition of these interpretation knowledge will be described later.

  Hereinafter, the operation of each part of the present invention will be described in detail.

<Preparation of interpretation knowledge database>
In performing similar case search, interpretation knowledge is obtained in advance and stored in the interpretation knowledge database 110. Interpretation knowledge is obtained from a collection of a plurality of “cases” composed of pairs of medical images and interpretation reports that are the results of interpretation of the medical images. As cases, those stored in the case database 100 for searching for similar cases from among similar cases may be used, or cases stored in other databases may be used. FIG. 2 shows a configuration for extracting interpretation knowledge from cases stored in the case database 100. At this time, the interpretation knowledge extraction unit 210 analyzes the text feature amount extracted from each case by the report analysis unit 250 and the image feature amount extracted from each case by the image feature extraction unit 260, and extracts interpretation knowledge. To do. Here, the necessary number of cases is a number sufficient to obtain some kind of law and knowledge using various data mining algorithms. Usually, hundreds to tens of thousands of data are used. In the present embodiment, as the interpretation knowledge, the correlation between two terms of (1) image feature value, (2) interpretation item, and (3) disease name is used.

(1) Image feature amount As “image feature amount”, there are those related to the shape of an organ or a lesion in a medical image, or those related to luminance distribution. For example, non-patent literature: “Nemoto, Shimizu, Sugawara, Obata, Nawano,” Improvement of mass shadow discrimination accuracy on mammograms by feature selection from a large number of features and a fast feature selection method "Proposal of the Institute of Electronics, Information and Communication Engineers, D-II, Vol. J88-D-II, No. 2, pp. 416-426, February 2005" describes that 490 types of feature values are used. Yes. Also in the present embodiment, several tens to several hundreds of image feature amounts determined in advance for each medical image capturing apparatus (modality) used for capturing a medical image or each target organ for interpretation are used.

(2) Interpretation Item The “interpretation item” is defined as “a character string in which the interpretation doctor verbalizes the characteristics of the image to be interpreted”. The terms used as interpretation items are almost limited depending on the medical imaging apparatus or target organ used. For example, lobed, spiny, irregular, clear border, unclear, low concentration, high concentration, low absorption, high absorption, ground glass, calcification, mosaic, early dark staining, low echo, high echo, fluff , Etc.

(3) Disease name The “disease name” is a disease name diagnosed by a doctor (reader) based on a medical image or other examination. Although the diagnosis disease name at the time of interpretation may differ from the disease name that has been confirmed through other examinations, the result of the definitive diagnosis is used when creating the interpretation knowledge database.

  Hereinafter, the procedure for creating interpretation knowledge will be described with reference to the flowchart of FIG. The medical imaging apparatus to be used in this embodiment, that is, the medical imaging apparatus to be used is a multi-slice CT, and the target organ and disease are a liver and a liver mass, respectively.

  In step S10, one case is acquired from a database storing cases for obtaining interpretation knowledge. Here, the total number of cases for obtaining interpretation knowledge is C. One case is composed of a pair of a medical image and an interpretation report that is a result of interpretation of the medical image. When a medical image is acquired by a multi-slice CT apparatus, one case includes a large number of slice images. In general, when a doctor interprets a multi-slice CT image, one to several important slice images are attached to the interpretation report as key images. Hereinafter, a large number of slice image sets or several key images may be simply referred to as “medical images” or “images”.

  In step S11, an image feature amount is extracted from the medical image. The process of step S11 is demonstrated in detail using the flowchart of FIG.

  In step S111, the region of the target organ is extracted. In the present embodiment, a liver region is extracted. Examples of liver region extraction methods include, for example, non-patent literature: “Tanaka, Shimizu, Obata,“ Improvement of liver region extraction method considering concentration pattern of abnormal part <second report> ”, IEICE Technical Report, Medical Image, 104 (580), pp. 7-12, January 2005 ”can be used.

In step S112, a lesion area is extracted from the organ area extracted in step S111. In the present embodiment, a tumor region is extracted from the liver region. Non-patent literature: “Nakagawa, Shimizu, Hitosugi, Obata,“ Development of automatic extraction method of liver tumor shadow from 3D abdominal CT image <second report> ”, medical image, 102 (575), pp. 89-94, January 2003 ”can be used. Here, when the number of i-th mass extracted from the image in case the M _i, tumor mass may be identified by (case number, mass number) pairs of (i, j). Here, a 1 ≦ i ≦ C, 1 ≦ j ≦ M i. In this embodiment, since a liver tumor is targeted as a lesion, it is called “tumor number”. However, it can also be called “lesion number” using a common expression in the present invention.

  In step S113, one region is selected from the lesion regions extracted in step S112.

In step S114, an image feature amount is extracted from the lesion area selected in step S113. In the present embodiment, non-patent literature: “Nemoto, Shimizu, Sugawara, Obata, Nawano,“ Improved mass shadow discrimination accuracy on a mammogram by selecting features from a large number of features and high speed. 490 types of feature quantities described in "Proposal of a Feature Selection Method", IEICE Transactions D-II, Vol. J88-D-II, No. 2, pp. 416-426, February 2005 Among them, some feature quantities applicable to the liver mass are selected and used. The feature quantity number to the N _F. The feature amount extracted in this step can be specified by a set (i, j, k) of (case number, tumor number extracted from this case (medical image), feature amount number). Here, a 1 ≦ i ≦ C, 1 ≦ j ≦ M i, 1 ≦ k ≦ N F.

  In step S115, it is checked whether or not there is an unselected lesion among the lesion areas extracted in step S112. If there is an unselected lesion, the process returns to step S113 to select an unselected lesion area. Re-execute S114. When there is no unselected lesion, that is, when the feature amount selection in step S114 is performed for all the lesion areas extracted in step S112, the process of the flowchart of FIG. 4 is terminated and the process returns to the flowchart of FIG. .

  In step S12 of FIG. 3, an interpretation report analysis process is performed. Specifically, an interpretation item and a disease name are extracted from the interpretation report. In this embodiment, morphological analysis and syntax analysis are performed using an interpretation item word dictionary storing interpretation items and a disease name word dictionary storing disease names. Through these processes, a word that matches the word stored in each word dictionary is extracted. Examples of morphological analysis techniques include MeCab (http://mecab.sourceforge.net) and ChaSen (http://chasen-legacy.sourceforge.jp), and syntactic analysis techniques include KNP (http: ///). nlp.kuee.kyoto-u.ac.jp/nl-resource/knp.html), CaboCha (http://chasen.org/˜take/software/caboca/), and the like. Interpretation reports are often described by doctors with expressions unique to interpretation reports, so it is desirable to develop morphological analysis technology, syntax analysis technology, and word dictionaries specialized for interpretation reports.

FIG. 5 is an example of an interpretation report of an abdominal CT examination, and FIG. 6 shows interpretation items and disease names extracted from the interpretation report of FIG. A plurality of interpretation items are usually extracted and one disease name is extracted. When i-th number of interpretation items extracted from the image interpretation report in case the N _i, interpretation items can be identified by (case number, image interpretation item number) pairs of (i, j). Here, 1 ≦ i ≦ C and 1 ≦ j ≦ N _i .

  In FIG. 6, only the interpretation item and the disease name word are extracted. However, a character string indicating the position of the lesion in the interpretation report and a character string indicating the time phase may be extracted at the same time. Here, the time phase will be supplemented. In order to distinguish liver lesions, a contrast examination in which a contrast medium is rapidly injected and images are taken over time is useful. In contrast examination of the liver, in general, the contrast medium flows into the hepatic artery and the bloody tumor is densely stained. The liver is imaged in the pulse phase, the equilibrium phase in which the contrast medium inside and outside the liver blood vessels reaches equilibrium, the late phase in which the contrast medium accumulates in the stroma of the liver, and the like. In many cases, the interpretation report describes the position of the lesion in the organ and the time phase information of interest in contrast imaging. For this reason, extracting not only the interpretation items but also the position and time phase information is effective in extracting interpretation knowledge to be described later. FIG. 7 shows an example in which position and time phase information is extracted simultaneously with the interpretation items. For example, the interpretation report of FIG. 5 is analyzed, and “liver S3 area” is extracted as a position attribute of “early dark staining” from the phrase “early dark staining is recognized in liver S3 area”. Similarly, “late phase” is extracted as the time phase attribute of “washout” from the phrase “washed out in late phase”.

  If the interpretation report in FIG. 5 is simply interpreted, the time phase relating to “early dark dyeing” and the position relating to washout are blank as shown in FIG. On the other hand, the prior knowledge that the interpretation item “early dark staining” is a word corresponding to the early phase is used, or the mass indicating the state of “early dark staining” is the same as the mass “washed out in the late phase” If an advanced context interpretation of pointing to a tumor mass can be performed, the extracted position and time phase information is as shown in FIG.

  In step S13, it is checked whether or not there is an unacquired case in the database storing cases for obtaining interpretation knowledge. If there is an unacquired case, the process returns to step S10 and an unacquired case is acquired. Steps S11 and S12 are executed. If there are no unacquired cases, that is, if image feature extraction in step S11 and report analysis in step S12 have been performed for all cases, the process proceeds to step S14.

  Since the results of step S11 and step S12 do not depend on each other, the execution order may be reversed.

  When the processing reaches step S14, the data set shown in FIG. 9 has been acquired. That is, an image feature amount, an interpretation item, and a disease name are acquired for each case. For the case of case number 1, M1 lesions are included in the medical image, and the number of image feature values extracted from each lesion is NF. The number of interpretation items in the interpretation report is N1. For example, the value of the first image feature value of the first lesion indicated by the lesion number (1, 1) is 0.851. In addition, the value of the first interpretation item indicated by the interpretation item number (1, 1) is “early dark dyeing”. In the example of FIG. 9, each image feature amount is a numerical value from 0 to 1, and the interpretation item and the disease name are character strings. An image feature value having a negative value or a value larger than 1 may be used. In addition, data may be stored in the form of a predetermined word ID as an interpretation item and a disease name.

  In step S14, interpretation knowledge is extracted from the image feature amount obtained in step S11, the interpretation item and disease name obtained in step S12. In this embodiment, the correlation between two of the three items of image feature, interpretation item, and disease name is taken as interpretation knowledge.

(1) Correlation between (image feature value−interpretation item) A method of obtaining the correlation between a pair of (image feature value−interpretation item) will be described. Although there are a plurality of expression forms of the correlation, the correlation ratio is used here. The correlation ratio is an index representing the correlation between qualitative data and quantitative data, and is expressed by Equation 1.

In the interpretation report, two categories of cases where an interpretation item is included and not included are considered as qualitative data. The value of a certain image feature amount itself extracted from the medical image is used as quantitative data. For example, for all cases included in the case database for extracting interpretation knowledge, the interpretation report is classified into those including or not including a certain interpretation item. Here, a method for obtaining the correlation ratio between the interpretation item “early dark staining” and the image feature amount “luminance average value inside the tumor in the early phase” will be described. In Equation 1, it is assumed that category i = 1 includes “early dark dyeing” and category i = 2 does not include “early dark dyeing”. The j-th observed value that is the “average luminance value in the tumor in the early phase” of the tumor image extracted from the case that includes “early dark staining” in the interpretation report is defined as x _1j . Further, the j-th observed value that is “the average luminance value in the tumor in the early phase” of the tumor image extracted from the case that does not include “early dark staining” in the interpretation report is assumed to be x _2j . “Early dark staining” means that the CT value increases in the early phase of contrast, and in this case, the correlation ratio is expected to be large (close to 1). Further, since early dark staining depends on the type of tumor and not on the size of the tumor, the correlation between the interpretation item “early dark staining” and the image feature “mass area” is small (close to 0). It is expected that. In this way, the correlation ratio between all interpretation items and all image feature amounts is calculated.

  FIG. 10 shows a conceptual diagram of the correlation (here, the correlation ratio) between the interpretation item and the image feature amount. A plurality of interpretation items are listed on the left side, and names of a plurality of image feature amounts are listed on the right side. An interpretation item having a correlation ratio equal to or greater than a threshold and an image feature amount are connected by a solid line. Since the correlation ratio takes a value of 0 or more and 1 or less, a value of about 0.3 to 0.7 can be used as the threshold value. When the calculated correlation ratio is finally binarized with a threshold value, information as shown in FIG. 10 is obtained. One example is supplemented. In contrast-enhanced CT examinations of liver masses, most masses are rendered at low density on CT images (called simple, simple CT, simple phase, etc.) before the use of contrast agents, and in many cases “low density” is used in the interpretation report. , “With LDA (Low Density Area)”. Therefore, the correlation between the interpretation items such as “low luminance” and “LDA” and the luminance average inside the tumor in the CT image before using the contrast agent (abbreviated as “simple phase luminance average” in FIG. 10) increases.

  FIG. 11 shows another conceptual diagram of the correlation (for example, correlation ratio) between the interpretation item and the image feature amount. In this figure, the correlation ratio is expressed in multiple values, and the thickness of the solid line between the interpretation item and the image feature amount corresponds to the magnitude of the correlation ratio. For example, “early dark staining” in which the CT value increases in the early phase of contrast and luminance average inside the tumor in the early arterial phase (abbreviated as early phase or arterial phase) (“arterial phase luminance average” in FIG. 11) (Abbreviation) is increased.

  By paying attention to the value of the correlation ratio, it is possible to specify an image feature amount having a high correlation with a certain interpretation item. Actually, one case often includes a plurality of images and a plurality of lesions (mass), and in that case, the interpretation report includes descriptions about the plurality of lesions. For example, in contrast CT examination, CT imaging is performed at a timing at a plurality of times before or after using a contrast medium. Therefore, a plurality of sets of slice images are obtained, and one set of slice images includes a plurality of lesions (tumors), and a plurality of image feature amounts are extracted from one lesion. Therefore, image feature amounts are obtained as many as (number of slice image sets) × (number of lesions detected from one patient) × (number of types of image feature amounts). It is necessary to obtain a correlation with a plurality of interpretation items and disease names extracted from the interpretation report. Of course, there is a possibility that correspondence can be obtained correctly by using a large number of cases. However, the description of the interpretation report and the corresponding image feature amount are used to some extent by using the lesion position and time phase as shown in FIG. If the correspondence can be made in advance, the correlation can be obtained more accurately.

  In the above description, the case where the qualitative data is in two categories, including and not including a certain interpretation item, has been described. However, a certain interpretation item (for example, “clear border”) and its interpretation item ( For example, there may be two categories such as “unclear boundary”). When the interpretation items are ordinal scales such as “low density”, “medium density”, and “high density”, the correlation ratio may be calculated using each of them as a category (three categories in this example).

  For synonyms such as “low density”, “low luminance”, and “low absorption”, a synonym dictionary is created in advance, and these are treated as the same interpretation item.

(2) Correlation between (image feature amount-disease name) As for the correlation between a pair of (image feature amount-disease name), the correlation ratio is used as in the case of (image feature amount-interpretation item). Can do. FIG. 12 shows a conceptual diagram of a correlation (for example, correlation ratio) between a disease name and an image feature amount. In this figure, the correlation is expressed in binary as in FIG. 10. Of course, multi-value expression as shown in FIG. 11 can also be performed.

(3) Correlation between (interpretation item-disease name) How to obtain the correlation between a pair of (interpretation item-disease name) will be described. Although there are a plurality of expression forms of the correlation, the support level is used here. The degree of support is an index representing a correlation rule between qualitative data, and is expressed by Formula 2.

  This degree of support means the probability (co-occurrence probability) that the interpretation item X and the disease name Y appear simultaneously in all cases. By using the support level, it is possible to specify a combination of an interpretation item having a strong relationship (correlation) and a disease name.

  In addition, you may use the reliability shown by Formula 3, and the lift value shown by Formula 4 instead of a support degree.

  The certainty factor is the probability that an item in the conclusion part Y (disease name Y) will appear when the condition item X (interpretation item X) appears as a condition. The lift value is Y (disease name) when the appearance of X (interpretation item X) is a condition relative to the appearance probability of Y (disease name Y) when the appearance of X (interpretation item X) is not a condition. This is an index indicating how much the appearance probability of Y) has increased. In addition, a connection and a φ coefficient may be used. The connection and φ coefficient are described in documents relating to association rule analysis (for example, non-patent document: “Data Mining and its Application”, Kato / Hamuro / Yada Co-author, Asakura Shoten).

  FIG. 13 shows a conceptual diagram of a correlation (for example, support level) between an interpretation item and a disease name. In this figure, the correlation is expressed in binary as in FIG. 10. Of course, multi-value expression as shown in FIG. 11 can also be performed.

  When the process of step S14 is performed by the above method, the correlation between (image feature amount-interpretation item) and the correlation between (image feature amount-disease name) as shown in FIGS. A correlation between the relationship and (interpretation item-disease name) is obtained. The correlation ratio takes a value between 0 and 1. The degree of support takes a value of 0 or more. The obtained correlation is stored in the interpretation knowledge database 110 in the format of FIGS.

<Addition of interpretation knowledge database in advance>
Next, a method for adding interpretation knowledge, which is the center of the present invention, will be described in detail. As in the case of creating the interpretation knowledge in FIG. 2, the medical imaging apparatus targeted in this embodiment is a multi-slice CT, and the target organ and the disease are the liver and liver tumor, respectively.

  As described above, in the present embodiment, a correlation is extracted as interpretation knowledge from the image feature amount, interpretation item, and disease name. At this time, a plurality of types of image patterns may exist for one text. For this reason, a situation is assumed in which the correlation between the text feature quantity and the image feature quantity cannot be obtained correctly, and the focus point of the interpreter cannot be reflected in the similar image search. As an example of this problem, an image pattern of hepatocellular carcinoma will be described in detail with reference to FIGS.

  The image of the lesion area of the hepatocellular carcinoma is strongly imaged in the contrast artery phase (CT value) as in the case of multi-hepatic hepatocellular carcinoma with a large blood flow in the tumor as shown in FIG. Ascending image pattern (image pattern 1) and not strongly contrasted in contrast-enhanced arterial phase like an ischemic hepatocellular carcinoma with little blood flow in the tumor as shown in FIG. No) image pattern (image pattern 2) exists. FIG. 18 is a conceptual diagram showing a difference in distribution of the image feature amount “average luminance value in the tumor in the early phase” depending on the presence or absence of the text feature amount (disease name “hepatocellular carcinoma”). The solid line shows the distribution of the image feature “average brightness within the tumor in the early phase” when there is a text feature (disease name “hepatocellular carcinoma”), and the broken line shows the text feature (disease name “hepatocyte The distribution of the image feature quantity “luminance average value inside the tumor in the early phase” in the absence of “)” is shown. Here, if the image pattern of the disease name “hepatocellular carcinoma” is only image pattern 1, in the case of “hepatocellular carcinoma” described in the interpretation report, the image feature consideration distribution is the image feature. The distribution is a biased distribution as shown in FIG. 18A with a high quantity “average luminance value inside the tumor in the early phase”. However, there are actually both image patterns 1 and 2 in the image pattern of the case described as “hepatocellular carcinoma” in the interpretation report. For this reason, the case described as “hepatocellular carcinoma” in the interpretation report includes a case where the image feature amount “luminance average value in the tumor in the early phase” is low as in the image pattern 2. For this reason, the distribution of the image feature amount is a distribution with little bias as shown in FIG. As described above, when a plurality of image patterns exist for a certain text feature amount, a correlation (for example, a correlation ratio) is extracted in a state where a plurality of distributions are mixed. For this reason, the value of the correlation between the text feature quantity and the image feature quantity decreases, and the image feature quantity that the user really wants to emphasize is not emphasized during the search. As a result, there arises a problem that the point of interest of the image interpreter cannot be reflected in the similar image search.

  In order to deal with such a problem, the inventors of the present application focused on the correlation between (image feature amount−interpretation item) and the relative relationship between the three terms of disease name, interpretation item, and image feature amount. Here, the correlation between (image feature amount−interpretation item), the correlation between (image feature amount−disease name), and (interpretation item−disease name), which are interpretation knowledge from the set of medical image and interpretation report Assume that each correlation has been calculated.

  As a result of analyzing the relationship between a medical image and an interpretation report written by a doctor, the inventors of the present application have noticed that many interpretation items directly reflect image contents. For example, although there are differences in the contents entered in the medical image depending on the doctor, with respect to FIG. 17A, the interpretation report shows “early dark staining” indicating that the CT value is high and “uniform” indicating no pattern. ”Is entered, and“ LDA (Low Density Area) ”indicating that the CT value is low and“ unevenness ”indicating that there is a pattern are not entered. On the other hand, for FIG. 17B, “LDA” and “unevenness” are entered in the interpretation report, and “early dark dyeing” is not entered.

  The inventors of the present application analyzed the correlation between (image feature amount−interpretation item), the correlation between (image feature amount−disease name), and the correlation between (interpretation item−disease name). As a result, when there is only one image pattern corresponding to the disease name, there are three terms in which the values of all binary relationships are large, while there are multiple types of image patterns for the disease name. In the case where it exists, as shown in FIG. 19, attention is focused on the fact that only the correlation between (image feature amount−disease name) is reduced among all corresponding binary relationships (ternary relationships). Possible causes are as follows. That is, when there is only one disease state for one disease name (one image pattern), the distribution of image feature amounts is a biased distribution as shown in FIG. -Correlation between disease names increases. On the other hand, when there are a plurality of medical conditions (image patterns) for one disease name, the distribution of the image feature amount is a distribution with little bias as shown in FIG. Correlation between feature amount-disease name) becomes low. On the other hand, many of the interpretation items directly reflect the image contents (image feature amount−interpretation item) and the disease name is the total value of the interpretation item (interpretation item−disease name). This is probably because the correlation between the two becomes high.

  Thus, in the relationship between the three terms when there are a plurality of medical conditions (image patterns) for one disease name, the (image feature value− The correlation between (interpretation item) and the correlation between (interpretation item-disease name) have a clearer correspondence and the value of the correlation is considered to be more reliable.

  For these reasons, when there are a plurality of image patterns for a disease name, the value of the correlation between (image feature value−disease name) decreases, and the image feature value that is really important is no longer considered important during the search. On the other hand, by using the correlation between (image feature value−interpretation item) and the correlation between (interpretation item−disease name), the correlation between (image feature amount−disease name) via the interpretation item We found that we can extract relationships.

  Referring to FIG. 20, for example, when it is desired to extract a correlation between (image feature amount−disease name) for a disease name “hepatocellular carcinoma” having a plurality of image patterns, (interpretation item− By extracting the correlation between (image feature amount-interpretation item) through “early dark staining”, “LDA”, “uniform”, “non-uniform”, which has a high correlation between (name of disease) It is considered that the image feature amount highly correlated with “cell cancer” can be extracted correctly. Among these, the correlation indicated by the solid line indicates the correlation of one image pattern, and the correlation indicated by the broken line indicates the correlation of the other one image pattern.

  Hereinafter, the details of the interpretation knowledge adding procedure configured based on this idea will be described with reference to the interpretation knowledge extracting apparatus of FIG. In this embodiment, the interpretation knowledge value acquired in advance (correlation between the relations between the two terms) is referred to, and if the conditions described later are not satisfied, the interpretation knowledge is additionally registered in the interpretation knowledge database 110.

  In FIG. 2, an interpretation knowledge extraction unit 210, an interpretation knowledge determination unit 220, and an interpretation item extraction unit 230 are processing units for extracting interpretation knowledge and adding interpretation knowledge.

  The interpretation knowledge extraction unit 210 analyzes a plurality of cases stored in the case database 100 and stores the interpretation knowledge obtained as a result of the analysis in the interpretation knowledge database 110.

  The interpretation knowledge determination unit 220 reads interpretation knowledge stored in the interpretation knowledge database 110, and determines a target (disease name) to which new interpretation knowledge is added in the interpretation knowledge.

  The interpretation item extraction unit 230 analyzes the correlation with respect to the subject (disease name) to which interpretation knowledge is added, and extracts a set of interpretation items used for the addition of interpretation knowledge.

  At the time of adding interpretation knowledge, the interpretation knowledge extracting unit 210 acquires new interpretation knowledge by analyzing a plurality of cases using the set of interpretation items extracted by the interpretation item extracting unit 230, and stores it in the interpretation knowledge database 110. Store.

  Hereinafter, the procedure for adding interpretation knowledge will be described with reference to the flowchart of FIG.

  In step S30, the interpretation knowledge determination unit 220 acquires one interpretation knowledge stored in the interpretation knowledge database 110.

For example, when the interpretation knowledge is a correlation (see FIG. 15) between (image feature amount−disease name), the interpretation knowledge acquired in step S30 is the one row of the table of FIG. Hereinafter referred to as correlation values) _NIF ).

  In step S31, the interpretation knowledge determination unit 220 determines whether or not to add new interpretation knowledge to the interpretation knowledge database 110 in addition to the interpretation knowledge acquired in step S30. At this time, the interpretation knowledge determination unit 220 has a large correlation between (image feature amount−interpretation item) and correlation between (interpretation item−disease name) in the relative relationship between the three terms, and (image feature amount−disease). It is determined whether or not new interpretation knowledge is to be added by determining whether or not there is a combination that decreases the correlation between the names.

  The process of step S31 will be described in detail using the flowchart of FIG.

  In step S310, the interpretation knowledge determination unit 220 determines whether or not the interpretation knowledge acquired in step S30 is a correlation between (image feature amount−disease name). If the interpretation knowledge acquired in step S30 is a correlation between (image feature amount−disease name), the process proceeds to step S311. Otherwise, the process proceeds to step S316.

  In step S311, the interpretation knowledge determination unit 220 reads from the interpretation knowledge database 110 the correlation between (interpretation item−disease name) corresponding to the disease name of the interpretation knowledge acquired in step S30. In other words, the interpretation knowledge determination unit 220 reads from the interpretation knowledge database 110 the correlation between (interpretation item−disease name) for the same disease name as that of the interpretation knowledge acquired in step S30.

  In step S312, the interpretation knowledge determination unit 220 extracts an interpretation item having a correlation value equal to or greater than a correlation value threshold in the correlation between (interpretation item−disease name) acquired in step S311.

  In step S313, the interpretation knowledge determination unit 220 calculates the correlation between (interpretation item-image feature amount) regarding the interpretation item whose correlation value between (interpretation item-disease name) extracted in step S312 is equal to or greater than the correlation value threshold. Read. That is, the interpretation knowledge determining unit 220 correlates between (interpretation item-image feature amount) for the same interpretation item as the interpretation item whose correlation value between (interpretation item-disease name) extracted in step S312 is equal to or greater than the correlation value threshold. Read relationship.

  In step S314, the interpretation knowledge determination unit 220 (a) the correlation value indicated by the correlation between (image feature amount−disease name) acquired in step S30, and (b) (interpretation item−disease name) read in S311. ) And the correlation value indicated by the correlation between (image feature amount−disease name) via the correlation between (interpretation item−image feature amount) read in S313 is calculated.

Here, the correlation value between (interpretation item-disease name) is w _{x, j,} the correlation value between (interpretation item-image feature amount) is w _{a, i} , and the correlation value between (image feature amount-disease name). Is represented as w _i . Further, i is a number of interpretation item index, the N _x, which is step S312 extracts indicating the type of interpretation item index, j is which is step S312 extracts representing the type of image feature amount. Calculating a difference value d _w as equation 5 using these correlations.

For example, considering the case where the disease name is “hepatocellular carcinoma”, the correlation between (image feature amount−disease name) is acquired in step S311, and the correlation value indicated by the acquired correlation is denoted by w _i . Here, if the interpretation items extracted in step S312 are “early dark staining”, “LDA”, and “unevenness”, the interpretation knowledge determination unit 220 stores the interpretation knowledge database 110 in the format shown in FIG. By referring to the stored correlation table between (interpretation item-disease name), correlation value between (early dark staining-hepatocellular carcinoma), correlation value between (LDA-hepatocellular carcinoma), (non-uniformity) -Correlation value between (hepatocellular carcinoma) is obtained and set as w _{x, j} . On the other hand, the interpretation knowledge determination unit 220 acquires a correlation value (for example, a correlation value between (early dark staining-luminance average value)) between (early dark staining-image feature amount), and wa _{, i} To do. Using such a ternary relationship, the difference value d _w may be calculated by Equation 5.

In step S315, the image interpretation knowledge determination unit 220 determines whether the difference value _{d w} calculated in step S314 is larger than the difference threshold.

If the calculated difference value _{d w} is greater than the difference threshold in step S314, the process proceeds to step S318, if the difference value _{d w} is less difference threshold, the process proceeds to step S317.

  In step S316, the interpretation knowledge determination unit 220 determines not to add interpretation knowledge according to the determination result in step S310. At this time, since interpretation knowledge other than the disease name is read in step S316, 0 is added to the information related to the addition of interpretation knowledge (hereinafter referred to as “additional information”), and “No” in the branch of step S31. Proceed to

  In step S317, the interpretation knowledge determination unit 220 determines not to add interpretation knowledge in accordance with the determination result in step S315. That is, the determination result in step S315 is the correlation between (image feature amount−disease name) acquired in step S30, the correlation between (interpretation item−disease name), and (interpretation item−image feature amount). It is shown that there is no difference between the correlation between (image feature amount−disease name) via the correlation. For this reason, it is determined that there is only one type of image pattern for the disease name of interest, and it is determined not to add interpretation knowledge. At this time, 0 is added to the additional information of interpretation knowledge, and the process proceeds to “No” in the branch of step S31. In this embodiment, 1 is added to the additional information when adding interpretation knowledge, and 0 is added to the additional information when not adding.

  In step S318, the interpretation knowledge determination unit 220 determines to add interpretation knowledge according to the determination result in step S315. That is, the determination result in step S315 is the correlation between (image feature amount−disease name) acquired in step S30, the correlation between (interpretation item−disease name), and (interpretation item−image feature amount). It is shown that the correlation between (image feature amount−disease name) via the correlation is different. Therefore, it is determined that there are a plurality of types of image patterns for the disease name of interest, and it is determined to add interpretation knowledge. At this time, 1 is added to the additional information of the interpretation knowledge, and the process proceeds to “Yes” in the branch of step S31.

  The additional information given in steps S317 and S318 is added to each line of the correlation between (image feature amount−disease name) in FIG. 15, and is stored in the interpretation knowledge database 110 in the format shown in FIG. For example, if disease name 1 is entered in the interpretation report, the additional information in FIG. 23 (here, “1”: addition of correlation) is read at the time of similar case search, and (interpretation item−disease) It is determined that a correlation is added, and an added correlation table (interpretation knowledge) described later is referred to.

  In step S32 of FIG. 21, the interpretation item extraction unit 230 extracts a set of interpretation items having a high correlation with the target disease name determined to add interpretation knowledge in step S31.

  The process of step S32 will be described in detail using the flowchart of FIG.

  In step S320, the interpretation item extraction unit 230 reads from the interpretation knowledge database 110 one correlation between (interpretation item-disease name) corresponding to the disease name of the interpretation knowledge acquired in step S30. For example, when the disease name is disease name 1, in FIG. 16, the correlation of interpretation items related to disease name 1 (here, all correlations in the column of disease name 1) is read from the interpretation knowledge database 110. Hereinafter, the case of the disease name “hepatocellular carcinoma” will be described as an example.

  In step S321, the interpretation item extraction unit 230 extracts one correlation value from the correlation between (interpretation item-disease name) acquired in step S320, and determines whether the correlation value is greater than a first threshold value. judge. If the correlation value between the extracted (interpretation item-disease name) is larger than the first threshold value, the process proceeds to step S322, and if the correlation value is less than the first threshold value, the process proceeds to step S323.

  In step S322, the interpretation item extraction unit 230 acquires an interpretation item from the correlation information between (interpretation item−disease name) acquired in step S320, and adds it to the interpretation item candidate when adding interpretation knowledge. For example, in FIG. 16, when the interpretation items whose correlation value is greater than the first threshold are “early dark dyeing”, “LDA”, and “non-uniform”, all of these are stored as interpretation item candidates.

  In step S323, the interpretation item extraction unit 230 determines whether all correlation values between (interpretation item-disease name) have been compared with the first threshold value. If all the correlation values have been compared with the first threshold value, the process proceeds to step S324; otherwise, the process returns to step S321.

  In step S324, the interpretation item extraction unit 230 creates all combinations of interpretation item candidates extracted in step S322.

  For example, when the extracted interpretation items are “early dark dye”, “LDA”, and “non-uniform”, “early dark dye, LDA, non-uniform”, “early dark dye, LDA”. , “LDA, non-uniform”, “early dark dye, non-uniform”, “early dark dye”, “LDA”, “non-uniform” are created.

  In step S325, the interpretation item extraction unit 230 extracts a correlation value between the interpretation item created in step S324 and the disease name included in the interpretation knowledge acquired in step S30. For example, when the disease name is “hepatocellular carcinoma” and the set of interpretation items is “early dark staining, LDA, non-uniform”, the disease name “hepatocellular carcinoma” —interpretation items “early dark staining”, “ Correlation values between “LDA” and “non-uniform” (a set of interpretation items “early dark dyeing, LDA, non-uniform”) are extracted. For the extraction of the correlation value, the method for obtaining the correlation value between (interpretation item−disease name) described above may be used.

  In step S326, the interpretation item extraction unit 230 determines whether the correlation value extracted in step S325 is greater than the second threshold value. If the correlation value extracted in step S325 is larger than the second threshold value, the process proceeds to step S327. If the correlation value is equal to or smaller than the second threshold value, the extraction of the interpretation item is terminated. For example, when the correlation value of the disease name “hepatocellular carcinoma” —interpretation items “early dark staining”, “LDA” (interpretation item set “early dark staining, LDA”) is extracted, that the contrast is CT (CT "Large staining" indicating high) and "LDA" indicating non-contrast (low CT) are low in correlation, so the disease name "hepatocellular carcinoma"-interpretation item "early" The correlation values of “dark dye” and “LDA” are equal to or lower than the second threshold value. For this reason, “early dark dyeing, LDA” is determined to be inappropriate as a set of interpretation items.

  In step S327, the interpretation item extraction unit 230 extracts a set of interpretation items having a correlation value greater than the second threshold as a set of interpretation items used for adding interpretation knowledge. For example, for the disease name “hepatocellular carcinoma”, a set of two interpretation items “early dark stain” and “LDA, non-uniform” corresponding to the image pattern of the lesion area is extracted.

  In step S33 of FIG. 21, the interpretation knowledge extraction unit 210 extracts interpretation knowledge using the set of interpretation items extracted in step S32.

  For example, when a set of two interpretation items “early dark staining” and “LDA, non-uniform” are extracted for the disease name “hepatocellular carcinoma”, the interpretation knowledge extraction unit 210 reads the interpretation item “ For “early dark staining”, a correlation between (image feature amount−interpretation item) is extracted and stored as a correlation between (image feature amount−disease name) for the disease name “hepatocellular carcinoma”. Here, the method for extracting interpretation knowledge is the same as the method for obtaining the correlation between (image feature amount−interpretation item) described above. Similarly, the interpretation knowledge extraction unit 210 also extracts the correlation between (image feature quantity-interpretation item) for the interpretation item set “LDA, non-uniformity” and sets (image feature for the disease name “hepatocellular carcinoma”. It is stored as a correlation between dose-disease name).

  FIG. 25 shows a storage format of interpretation knowledge extracted in step S33 and stored in the interpretation knowledge database 110. When there are two sets of interpretation items such as disease name 1, the interpretation knowledge is extracted for each, and all of the values are stored. For example, if disease name 1 is entered in the interpretation report, the additional information in FIG. 23 (here, “1”: addition of correlation) is read at the time of similar case search, and (interpretation item−disease) The correlation table (interpretation knowledge) in FIG. 25 is referred to.

  In step S <b> 34, the interpretation knowledge determination unit 220 checks whether or not there is undetermined interpretation knowledge in the interpretation knowledge database 110 as to whether or not to add interpretation knowledge. If there is undetermined interpretation knowledge, the process returns to step S30, and after undetermined interpretation knowledge is acquired, step S31 is executed. If there is no undetermined interpretation knowledge, that is, if all the interpretation knowledge has been subjected to the interpretation knowledge addition determination in step S31, the interpretation item extraction in step S32, and the interpretation knowledge extraction in S33, the interpretation knowledge is added. Exit.

  By the above method, the correlation between the additional information of the interpretation knowledge for each disease name shown in FIG. 23 and the (image feature amount−disease name) corresponding to the additional information shown in FIG. 25 is obtained. Further, the obtained additional information and correlation are stored in the interpretation knowledge database 110 in the format of FIGS. On the other hand, the correlation between (image feature amount−interpretation item) shown in FIG. 14 and the correlation between (interpretation item−disease name) shown in FIG. 16 obtained by the process of step S14 in FIG. The new interpretation is not added and is stored in the interpretation knowledge database 110 in the format of FIGS. 14 and 16. In the similar case search described below, this interpretation knowledge database 110 is used.

Note that even using only the information of the set of interpretation items extracted in step S32 is effective for similar case retrieval. FIG. 26 shows a flowchart of the interpretation knowledge adding process in which only interpretation item extraction is performed. In the flowchart shown in FIG. 26, there is no processing (interpretation knowledge extraction processing) in step S33 shown in the flowchart shown in FIG. That is, when it is determined that the interpretation knowledge determination unit 220 in FIG. 2 adds the interpretation knowledge (Yes in S31), the interpretation knowledge extraction unit 210 newly sets the interpretation item set information shown in FIG. 27 as the interpretation knowledge. Store (S32). For example, when there are two sets of interpretation items corresponding to the disease name 1, information of “1” and “0” is stored for each of the two sets of interpretation items. In the present embodiment, “1” is assigned to an interpretation item with high correlation (interpretation item extracted in S32), and “0” is assigned to an interpretation item with low correlation (interpretation item not extracted in S32). grant. "for example, the set number 1 disease name 1 of Figure 27. using a set of interpretation knowledge at the time of the similar case retrieval indicating that interpretation items 1 and interpretation item N _II is higher interpretation item correlated The method will be described later in detail.

  As for the extracted interpretation knowledge, as shown in FIG. 28, the added interpretation knowledge and the non-added interpretation knowledge may be stored in the same table. For example, when disease name 1 is entered in the interpretation report, a plurality of correlations for disease name 1 are obtained directly from the correlation table of disease name 1 without reading additional information when searching for similar cases. can do.

<Similar case search>
Hereinafter, the procedure of similar case search will be described using the block diagram showing the configuration of the similar case search apparatus shown in FIG. 1 and the flowchart of the similar case search processing shown in FIG.

  First, a similar case search procedure using a correlation in which interpretation knowledge is not added (additional information of disease name is “0”) will be described.

  In step S20, the interpretation target image reading unit 120 acquires an interpretation target image from the medical image photographing apparatus. As in the case of the interpretation knowledge creation shown in FIG. 3, the medical imaging apparatus targeted in this embodiment is a multi-slice CT, and the target organ and disease are a liver and a liver mass, respectively. The read image is displayed on the interpretation target image display unit 130.

  In step S <b> 21, the interpreter inputs an interpretation report through the report input / output unit 140 while referring to the interpretation target image displayed on the interpretation target image display unit 130. In the case of a multi-slice CT apparatus, a plurality of slice images of a plane perpendicular to the body axis are usually obtained by reconstruction. The radiogram interpreter checks the presence or absence of a lesion (hepatic mass in this embodiment) while changing the slice position, and inputs an interpretation report for these slice images. When inputting an interpretation report, the interpreter may specify the position of the lesion detected in the interpretation target image (slice number and coordinates or area information on the slice image) with an input device such as a mouse. When designating the coordinates, for example, the center position of the tumor is clicked with the mouse. When an area is designated, there are a method of surrounding with a rectangle, a circle or an ellipse, or a method of surrounding a boundary between a lesion and a normal tissue with a free curve. When only the center coordinates are specified, or when an area is specified by surrounding it with a rectangle, circle, ellipse, or the like, there is an advantage that the burden on the image reader is small. It is necessary to automatically extract a region of only a tumor from an image processing algorithm. For the extraction of the tumor region, the same technique as in step S112 in FIG. 4 can be used. If the radiogram interpreter does not specify the position and area of the lesion, the target organ area extraction in step S111 and the lesion area extraction in step S112 may be performed.

  In step S22, a request for similar case retrieval from an interpreter is accepted.

  In the case of a mass having typical symptoms or an expert in the interpretation, the interpretation report is usually completed without hesitation. In the case of a mass having an atypical symptom or when the skill level of the interpreter is low, the interpreter makes a similar case search request at the interpretation terminal 200. When there are a plurality of lesions in the image to be interpreted, a similar case search request is made after designating a lesion that is lost in diagnosis.

  The designation of this lesion will be described. If a plurality of lesion positions and regions have already been specified including the lesion that is confused for diagnosis when the interpretation report is entered in step S21, it is only necessary to select one of them. In step S21, if a lesion that is not suitable for diagnosis is not designated, a new lesion is designated here. As a designation method, one point near the center of the lesion may be designated, or a lesion area may be designated. When one point near the center is designated, an area of a predetermined size is set with reference to the designated point, and a detailed lesion area is set from this area using the same method as in step S112. . When a lesion area is roughly specified, a detailed lesion area is set from this area using the same method as in step S112.

  If there is a similar case search request from the radiogram interpreter, the process proceeds to step S23. At this time, the interpretation report may be in a completed state or in the middle of entry. Even in a completely blank state, the similar case search in the subsequent step S25 can be executed, but in that case, the similar case search according to the point of interest of the interpreter, which is a feature of the present embodiment, is not executed. A similar case search is executed using a standard image feature amount set in advance.

  If the interpretation report entry time has elapsed for a certain period or when there is an input related to the end of interpretation by the interpreter, the processing in FIG. 29 is terminated. The processing unit for accepting a similar case search request and an interpretation end input is not shown in FIG. 1, but may be a physical switch built in the keyboard of the interpretation terminal 200 or the like, or a medical high definition A GUI menu or the like displayed on the image interpretation target image display unit 130 constituted by a monitor or the like may be used.

In step S23, the image feature extraction unit 160 extracts an image feature amount from the lesion area specified or extracted in step S22 for the image to be interpreted. Designated or extracted lesion area was that for the all if more, to extract the N _F characteristic amounts are determined in advance. The image feature extraction method is the same as that in step S114.

  In step S24, the report analysis unit 150 analyzes the interpretation report entered in step S21. Here, the same processing as the extraction of interpretation items and disease names in step S12 when creating the interpretation knowledge database is executed.

  In step S25, the weight determination unit 170 and the similar case search unit 180 search for similar cases from the case database 100 based on the image to be interpreted and the interpretation report entered by the interpreter. Here, the interpreter has entered the interpretation report of FIG. 5 in step S21, the interpretation items and disease names in FIG. 6 have been extracted in step S24, and the image feature quantity has been extracted from the interpretation target image in step S23. think of. In addition, it is assumed that the interpretation knowledge database 110 stores the correlation between two terms among the three items of the image feature amount, the interpretation item, and the disease name as shown in FIGS. Furthermore, it is assumed that no interpretation knowledge is added to the disease name extracted in step S24 (additional information on the disease name is “0”). In the present embodiment, the weighted distance calculation is performed in the similar case search based on at least one of the interpretation item and disease name extracted from the interpretation report. That is, the weight is relatively increased for the image feature amount related to at least one of the extracted interpretation item and disease name, and the weight is relatively decreased for the unrelated image feature amount. Thereby, the similar case search reflecting the doctor's point of interest entered in the interpretation report can be performed. That is, the similar case search unit 180 calculates a weighted distance between the medical image included in the case stored in the case database 100 and the image to be interpreted. The similar case search unit 180 searches the case database 100 as a similar case for a case including a medical image from which a weighted distance smaller than a predetermined threshold is calculated. Alternatively, the similar case search unit 180 searches the case database 100 as a similar case for a case including a medical image from which a predetermined number of weighted distances are calculated.

  The weighted distance can be calculated by, for example, Expression 6.

Here, x is a vector obtained by concatenating a plurality of (N _F ) image feature amounts extracted from the image to be interpreted. u ⁱ is an image feature amount extracted from the i-th case among cases stored in the case database 100. When different types of image feature quantities are connected, canonicalization (normalized to 0 average and 1 variance) is performed so as not to be affected by the difference in scale for each feature quantity.

  A specific example of the weighting method is shown below.

(1) When both the interpretation item and the disease name can be extracted from the interpretation report In this case, the interpreter has almost completed the interpretation report, and the result of similar case search is trying to increase the confidence of the entry contents. Equivalent to.

  Here, an example will be described in which weighting is performed using a correlation between (interpretation item−image feature amount) and a correlation between (interpretation item−disease name). Although the correlation between (disease name-image feature amount) is not unusable, weighting is performed using only the above two correlations. This is because the interpreter's thought process is considered to have finally determined the disease name based on the determination result of the interpretation item after determining the interpretation item focusing on the medical image.

At this point, “early dark stain” and “washout” are extracted as interpretation items and “hepatocellular carcinoma” is extracted as a disease name from the interpretation report of FIG. 5 as shown in FIG. The weight determination unit 170 refers to the correlation table between (interpretation item−disease name) stored in the interpretation knowledge database 110 in the format of FIG. The correlation between (washout-hepatocellular carcinoma) is acquired. Here, the numerical values (correlation values) representing the acquired correlation are used as weights as they are, and are expressed as w _x and w _y , respectively. Further, the weight determination unit 170 refers to the correlation table between (image feature amount−interpretation item) stored in the interpretation knowledge database 110 in the format of FIG. And the correlation between “washout” and all image feature quantities. Here, the numerical values (correlation values) representing the acquired correlations are used as they are as weights, and are represented as w _{a, i} , w _{b and i} , respectively. Here, i is a suffix indicating the type of image feature amount. Weight determining unit 170, using these weights, the weights W _i corresponding to the i-th image feature quantity is calculated as Equation 7.

  About the above weighting method, the outline | summary was shown in FIG.

For example, the weight for the fourth image feature amount “edge strength” is obtained as the sum of w _x w _{a, 4} and w _y w _{b, 4} . Here, w _x w _{a, 4} is a value w _{a, 4} representing a correlation between (early dark staining-edge intensity) and a correlation between (early dark staining-hepatocellular carcinoma) including the same interpretation item. It is a value weighted by the value w _x representing the relationship. w _y w _{b, 4} is a value w _{b, 4} representing a correlation between (washout-edge strength), and a value w _y representing a correlation between (washout-hepatocellular carcinoma) including the same interpretation item. It is a weighted value.

  Even when the number of interpretation items is other than 2, it can be dealt with by adding the correlation between (interpretation item-image feature amount) after weighting with the correlation between (interpretation item-disease name). By this formula, it is possible to calculate the weight that takes into account the interpretation item and disease name focused by the image interpreter, the relationship between the interpretation item and the disease name, and the relationship between the interpretation item and the image feature amount, Similar case search can be performed with emphasis on them.

  In the flowchart of FIG. 29, the similar case search is executed only when there is a similar case search request. However, the similar case search may be executed at another timing during the interpretation report entry. Another timing is when an interpretation report is not entered for a certain period of time after at least one interpretation item or disease name is entered. This is an operation method that is intended to advance interpretation by interpreting that the interpreter is confused about interpretation and proactively presenting similar cases as hints. In this embodiment, when at least one interpretation item or disease name is entered, a similar case search can be executed based on the viewpoint of the interpreter. Hereinafter, a case where only an interpretation item can be extracted from an interpretation report and a case where only a disease name can be extracted will be described.

(2) When only the interpretation items can be extracted from the interpretation report In this case, although the interpretation items that the interpreter should focus on have been judged, the final diagnosis of the disease name is lost, and the results of similar case search Corresponds to the situation where you are trying to get a hint for diagnosis of the disease name. Here, weighting is performed using only the correlation between (interpretation item−image feature amount).

At this time, it is assumed that “early dark dyeing” and “washout” are extracted as interpretation items from an interpretation report (not shown). The weight determination unit 170 refers to the correlation table between (image feature amount-interpretation item) stored in the interpretation knowledge database 110 in the format of FIG. 14, and correlates between “early dark staining” and all image feature amounts. The relationship and the correlation between “washout” and all image feature quantities are acquired. Here, the numerical values (correlation values) representing the acquired correlations are used as they are as weights, and are represented as w _{a, i} , w _{b and i} , respectively. Here, i is a suffix indicating the type of image feature amount. Using these weights, a weight W _i corresponding to the i-th image feature amount is calculated as in Expression 8.

  About the above weighting method, the outline | summary was shown in FIG.

For example, the weight for the fourth image feature quantity “edge strength” is a value w _{a, 4} representing the correlation between (early darkening−edge strength) and a value representing the correlation between (washout−edge strength). It is a value obtained by adding w _{b, 4} .

  A case where the number of interpretation items is other than 2 can be dealt with by adding the correlation between (interpretation item−image feature amount). With this formula, it is possible to calculate the weight considering the interpretation item focused on by the interpreter and the relationship between the interpretation item and the image feature amount at the same time. Become.

(3) When only the disease name can be extracted from the interpretation report In this case, the interpreter estimates the disease name by intuition, but is uncertain about the interpretation item that is the basis, and is diagnosed by the results of similar case search. This corresponds to a situation where a hint of the basis (interpretation item) is being obtained. Here, weighting is performed using only the correlation between (disease name−image feature amount).

At this point, it is assumed that “hepatocellular carcinoma” is extracted as a disease name from the interpretation report. The weight determination unit 170 refers to the correlation table between (image feature amount−disease name) stored in the interpretation knowledge database 110 in the format of FIG. 23, and determines between “hepatocellular carcinoma” and all image feature amounts. Get correlation. Directly used as a weight to numeric (correlation value) representing the acquired correlation here, expressed as w _i. Here, i is a suffix indicating the type of image feature amount. Using these weights, a weight W _i corresponding to the i-th image feature amount is calculated as in Expression 9.

  About the above weighting method, the outline | summary was shown in FIG.

For example, the weight for the fourth image feature amount “edge strength” is the value w ₄ representing the correlation between (hepatocellular carcinoma−edge strength).

  Since there is usually only one disease name, the above-described process may be performed. However, when two or more disease names are entered, it is possible to deal with them by adding them. By adding, a similar case search can be performed with an average image feature amount for two or more diseases. With this formula, it is possible to calculate the weight considering the disease name focused on by the image interpreter and the relationship between the disease name and the image feature amount at the same time, and as a result, it is possible to search for similar cases with emphasis on them.

  In the present embodiment, the similarity determination between images is performed using the weighted distance of Expression 6. However, when the total number of dimensions of the feature amount to be used increases, the correlation value (weight) is calculated at the calculated distance. ) May have a large correlation value (weight) due to a large number of small (or medium) feature values. In that case, a method such as using only image feature quantities whose corresponding correlation values are equal to or greater than a predetermined threshold value for distance calculation, or using only the top few image feature quantities of correlation values for distance calculation is adopted. May be. The number in that case may be determined in advance.

  Next, a similar case search procedure using a correlation (additional information of disease name is “1”) added with interpretation knowledge will be described. That is, the similar case search procedure when it is determined in step S25 in FIG. 29 that interpretation knowledge has been added to the disease name extracted in step S24 (additional information of disease name is “0”) will be described.

  The similar case search procedure using the correlation to which the interpretation knowledge is added (the additional information of the disease name is “1”) uses the correlation (the additional information of the disease name is “0”) to which the interpretation knowledge is not added. This is the same as the similar case search procedure. However, the weighting of the image feature amount by the weight determination unit 170 is different. Therefore, a specific example of the weighting method will be described below.

(1) Weighting method in the case where a correlation between (image feature amount−disease name) is added, 1
A method for weighting image features when interpretation knowledge is added to a disease name extracted from an interpretation report will be described with reference to FIGS. 33 and 34. FIG.

  Here, a case where only the disease name can be extracted from the interpretation report will be described.

  At this time, weighting is performed using only the correlation between the disease name and the image feature amount as before.

  It is assumed that “hepatocellular carcinoma” is extracted as a disease name from the interpretation report at the end of step S24 in FIG.

In step S25, the weight determination unit 170 refers to the additional information in the correlation table between (image feature amount−disease name) stored in the interpretation knowledge database 110 in the format of FIG. When the correlation between (image feature amount−disease name) is added, “1” is stored in the additional information. In this case, the correlation shown in FIG. 23 (FIG. 33A) is stored. Correlation between “hepatocellular carcinoma” and all image feature amounts included in the relationship table is not acquired, and (image feature amount−disease name) stored in the format of FIG. 25 (FIG. 33B) The correlation between “hepatocellular carcinoma” and all image feature amounts included in the correlation table is acquired for each set number. Directly used as a weight to numeric (correlation value) representing the acquired correlation here, expressed as w _i. Here, i is a suffix indicating the type of image feature amount. Using these weights, a weight W _i corresponding to the i-th image feature amount is calculated as in Expression 9.

  The above weighting method is outlined in FIG. 33 and FIG.

  Here, when the correlation between (image feature amount−disease name) is added as in the case of the disease name 1 in FIG. 33A (when the additional information is 1), There are a plurality of image patterns. For this reason, there are a plurality of correlations as shown in FIG. When a plurality of correlations are acquired, a weight is calculated for each correlation and used for similar case search. For example, when two sets of correlations can be acquired for “hepatocellular carcinoma”, a weight is determined for each correlation and a similar case search is performed for both.

An overview of the weighting method for each correlation is shown in FIG. For example, the weight for the third image feature quantity “equilibrium phase luminance average” is the value w ₃ representing the correlation between (hepatocellular carcinoma−equilibrium phase luminance average).

  In this manner, by determining the weight for each correlation and executing the similar case search, two similar case search results with different points of interest can be obtained.

(2) Weighting method 2 in the case where a correlation between (image feature amount−disease name) is added
Next, as in the procedure shown in FIG. 26, the process of extracting interpretation knowledge shown in step S33 of FIG. 21 is not performed, and only a set of interpretation items is extracted and stored in the interpretation knowledge database 110 (FIG. 27). (Refer to FIG. 35 and FIG. 36). As before, the case where only the disease name can be extracted from the interpretation report will be described.

  At this time, weighting is performed using a correlation between (interpretation item−image feature amount) unlike the previous case.

  It is assumed that “hepatocellular carcinoma” is extracted as a disease name from the interpretation report at the end of step S24 in FIG.

In step S25, the weight determination unit 170 refers to the additional information in the correlation table between (image feature amount−disease name) stored in the interpretation knowledge database 110 in the format of FIG. When the correlation between (image feature amount−disease name) is added, “1” is stored in the additional information. In this case, the correlation shown in FIG. 23 (FIG. 35A) is stored. The correlation between the “hepatocellular carcinoma” included in the relationship table and all the image feature amounts is not acquired, and information on the set of interpretation items stored in the format of FIG. 27 (FIG. 35B) is set. Get by number. As in the case where a plurality of interpretation items are extracted from the interpretation report using the obtained interpretation item pairs for each group number, the interpretation item-image is stored in the format of FIG. 14 (FIG. 35C). Refer to the correlation table between (features). For example, it is assumed that “early dark stain” and “washout” are extracted as interpretation items having high correlation with “hepatocellular carcinoma”. At this time, the correlation between “early dark staining” and all image feature amounts and the correlation between “washout” and all image feature amounts are acquired. Here, the numerical values (correlation values) representing the acquired correlations are used as they are as weights, and are represented as w _{a, i} , w _{b and i} , respectively. Here, i is a suffix indicating the type of image feature amount. Using these weights, a weight W _i corresponding to the i-th image feature amount is calculated as in Expression 8.

  The above weighting method is outlined in FIG. 35 and FIG.

  As before, when a correlation between (image feature amount−disease name) is added as in the case of disease name 1 in FIG. 35A (when the additional information is 1), On the other hand, there are a plurality of image patterns. For this reason, a plurality of correlations exist as shown in FIGS. 35 (b) and 35 (c). When a plurality of correlations are acquired, a weight is calculated for each correlation acquired in the same manner as described above and used for similar case search.

  An overview of the weighting method for each correlation is shown in FIG.

For example, the weight for the third image feature amount “balanced phase brightness average” is a value w _{a, 3} representing the correlation between (early dark dye−balanced phase brightness average) and (washout−balanced phase brightness average). This is a value obtained by adding the value w _{b, 3} representing the correlation.

  In this manner, by determining the weight for each correlation and executing the similar case search, two similar case search results with different points of interest can be obtained.

  As described above, in the similar case search in the present embodiment, the weighted distance calculation is performed in the similar case search based on at least one of the interpretation items and disease names extracted from the interpretation report. That is, the weight is relatively large for the image feature quantity related to at least one of the extracted interpretation item and disease name, and the weight is relatively small for the image feature quantity not related. At this time, even when there are a plurality of image patterns corresponding to the disease name, the weight added with the correlation by the ternary relationship can be used. In other words, when there are a plurality of combinations of image feature quantities that are highly related to the disease name, the relationship between the image feature quantity and the target disease name via the interpretation item is indicated for each combination of image feature quantities. Weighting is performed using the binomial relationship information. For this reason, even when there are a plurality of types of image patterns for the target disease name, it is possible to perform a similar case search while distinguishing them.

  Thereby, the similar case search reflecting the doctor's point of interest entered in the interpretation report can be performed.

  1 is configured to include the case database 100 and the interpretation knowledge database 110, the interpretation knowledge database 110 and the interpretation knowledge database 110 may not necessarily be included in the similar case search apparatus. Of course, it may be provided in a place different from the place where the similar case retrieval apparatus exists. In this case, the weight determination unit 170 and the similar case search unit 180 of the similar case search device are connected to the interpretation knowledge database 110 and the case database 100 via a network, respectively.

  Similarly, the interpretation database shown in FIG. 2 may not necessarily include the case database 100 and the interpretation knowledge database 110, and may be provided in a location different from the location where the interpretation knowledge extraction device exists. good. In this case, the interpretation knowledge extraction unit 210, the interpretation knowledge determination unit 220, and the interpretation item extraction unit 230 are connected to the interpretation knowledge database 110 via a network, and the report analysis unit 250 and the image feature extraction unit 260 are connected to the network. Via the case database 100.

  Further, in the present embodiment, the weighting method in the case where the correlation between (image feature amount−disease name) has been added has been described focusing on the case where only the disease name can be extracted from the interpretation report. Similarly, when both the interpretation item and the disease name can be extracted from the interpretation report described in the above, or when only the interpretation item can be extracted from the interpretation report, weighting can be performed in the same manner.

  In addition, in the case where a correlation between (image feature amount−disease name) has been added, when a plurality of correlations can be acquired, a plurality of similar case searches with different points of interest are obtained as similar case search results. Although results are obtained, these may all be displayed as search results. Furthermore, a set of interpretation items adopted as a focus point may be displayed simultaneously with the medical image and the interpretation report.

  The essential components of the similar case search device of the present invention are the image feature extraction unit 160, the report analysis unit 150, the weight determination unit 170, and the similar case search unit 180. It is not always necessary to achieve the object of the present invention.

  The essential components of the image interpretation knowledge extraction apparatus of the present invention are the image feature extraction unit 260, the report analysis unit 250, the image interpretation knowledge extraction unit 210, the image interpretation knowledge determination unit 220, and the image interpretation item extraction unit 230. There are other components that are not necessarily required to achieve the object of the present invention.

  Further, each of the above devices may be specifically configured as a computer system including a microprocessor, ROM, RAM, hard disk drive, display unit, keyboard, mouse, and the like. A computer program is stored in the RAM or hard disk drive. Each device achieves its functions by the microprocessor operating according to the computer program. Here, the computer program is configured by combining a plurality of instruction codes indicating instructions for the computer in order to achieve a predetermined function.

  Furthermore, some or all of the constituent elements constituting each of the above-described devices may be configured by one system LSI (Large Scale Integration). The system LSI is an ultra-multifunctional LSI manufactured by integrating a plurality of components on a single chip, and specifically, a computer system including a microprocessor, ROM, RAM, and the like. . A computer program is stored in the RAM. The system LSI achieves its functions by the microprocessor operating according to the computer program.

  Furthermore, some or all of the constituent elements constituting each of the above-described devices may be configured from an IC card that can be attached to and detached from each device or a single module. The IC card or module is a computer system that includes a microprocessor, ROM, RAM, and the like. The IC card or the module may include the super multifunctional LSI described above. The IC card or the module achieves its function by the microprocessor operating according to the computer program. This IC card or this module may have tamper resistance.

  Further, the present invention may be the method described above. Further, the present invention may be a computer program that realizes these methods by a computer, or may be a digital signal composed of the computer program.

  Furthermore, the present invention provides a non-transitory recording medium capable of reading the computer program or the digital signal, such as a flexible disk, a hard disk, a CD-ROM, an MO, a DVD, a DVD-ROM, a DVD-RAM, a BD ( It may be recorded on a Blu-ray Disc (registered trademark)), a semiconductor memory, or the like. The digital signal may be recorded on these non-temporary recording media.

  In the present invention, the computer program or the digital signal may be transmitted via an electric communication line, a wireless or wired communication line, a network represented by the Internet, a data broadcast, or the like.

  The present invention may be a computer system including a microprocessor and a memory, wherein the memory stores the computer program, and the microprocessor operates according to the computer program.

  Further, by recording the program or the digital signal on the non-temporary recording medium and transferring it, or transferring the program or the digital signal via the network or the like, another independent computer It may be implemented by the system.

  Furthermore, the above embodiment and the above modification examples may be combined.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  The present invention relates to a similar case retrieval apparatus that retrieves and presents a similar case to be used as a reference for an interpreter, an interpretation knowledge extraction apparatus that extracts interpretation knowledge used in the similar case retrieval apparatus, and an interpretation education apparatus for a training interpreter Etc. are available.

100 case database 110 interpretation knowledge database 120 interpretation target image reading unit 130 interpretation target image display unit 140 report input / output unit 150, 250 report analysis unit 160, 260 image feature extraction unit 170 weight determination unit 180 similar case search unit 190 similar case display Unit 200 Interpretation Terminal 210 Interpretation Knowledge Extraction Unit 220 Interpretation Knowledge Determination Unit 230 Interpretation Item Extraction Unit

Claims (9)

A similar case retrieval device for retrieving case data similar to case data including a medical image and an interpretation report, which is document data describing a result of interpretation of the medical image, from a case database,
The interpretation report includes (a) an interpretation item that is a character string indicating the characteristics of a medical image and (b) a disease name that is at least a disease name that is a diagnostic result of an interpreter based on the medical image,
The similar case search device includes:
An image feature extraction unit that extracts a plurality of image feature amounts from an image to be interpreted that is a medical image to be interpreted;
A report analysis unit that extracts a disease name from the target interpretation report that is an interpretation report for the interpretation target image;
A weight determining unit for determining a weight for each image feature based on the interpretation knowledge database;
The weight determination unit determines the plurality of image feature amounts extracted by the image feature extraction unit and a plurality of image feature amounts extracted from medical images included in case data registered in a case database. A similar case retrieval unit that retrieves case data including a medical image similar to the image to be interpreted from the case database by weighting and comparing with a weight for each image feature amount,
The interpretation knowledge database is
First binomial relationship information that predetermines a relationship between each image feature amount extracted from the medical image and each interpretation item;
A second binomial relationship information that predetermines the association between each interpretation item and each disease name;
A third binomial relationship information that predetermines the relationship between each image feature and each disease name;
For each disease name, including presence / absence information indicating whether or not there are multiple combinations of image feature quantities highly related to the disease name,
When the weight determination unit can determine that there are a plurality of combinations of image feature quantities highly relevant to the target disease name that is the disease name extracted by the report analysis unit based on the presence / absence information included in the interpretation knowledge database In addition, for each combination of image feature amounts, each image feature amount via an interpretation item calculated from the first binomial relationship information and the second binomial relationship information included in the knowledge database; For each image feature amount extracted by the image feature extraction unit, the fourth binomial relationship information indicating the relationship with the target disease name is used instead of the third binomial relationship information. A similar case retrieval apparatus that determines a larger value weight as the relationship between the image feature quantity and the target disease name is higher. When the weight determination unit can determine that there is only one type of image feature amount combination highly relevant to the target disease name based on the presence / absence information included in the interpretation knowledge database, the third two items The weight of a larger value is determined for each image feature amount extracted by the image feature extraction unit using inter-relationship information as the relevance between the image feature amount and the target disease name is higher. Similar case search device. An interpretation knowledge extraction device that extracts interpretation knowledge from a plurality of case data including a medical image and an interpretation report that is document data describing a result of interpretation of the medical image.
The interpretation report includes (a) an interpretation item that is a character string indicating the characteristics of a medical image, or (b) a disease name that is a diagnostic result of an interpreter based on the medical image,
The image interpretation knowledge extraction device comprises:
An image feature extraction unit for extracting a plurality of image feature amounts from a medical image included in each case data;
A report analysis unit that extracts disease names and interpretation items from interpretation reports included in each case data;
The first binomial relationship information that defines the relationship between each image feature amount extracted by the image feature extraction unit and each interpretation item extracted by the report analysis unit, and each interpretation extracted by the report analysis unit Second binomial relationship information that defines the relationship between the item and each disease name extracted by the report analysis unit, each image feature amount extracted by the image feature extraction unit, and the report analysis unit An interpretation knowledge extraction unit that extracts third binomial relationship information that defines the relationship between each disease name as interpretation knowledge;
Based on the first binomial relationship information, the second binomial relationship information, and the third binomial relationship information, an image feature amount that is highly related to a disease name for each disease name An interpretation knowledge determination unit that determines whether there are a plurality of combinations of
Based on the second binomial relationship information, for each disease name that the interpretation knowledge determination unit determines that there are a plurality of highly relevant image feature amount combinations, a combination of the disease name and the image feature amount is obtained. An interpretation item extraction unit that extracts a set of associated interpretation items for each combination of image features,
With
The image interpretation knowledge extraction unit further determines, for each set of image interpretation items extracted by the image interpretation item extraction unit, a relationship between the set of image interpretation items and each image feature amount extracted by the image feature extraction unit. The binomial relationship information is the interpretation knowledge indicating the association between the disease name determined by the interpretation knowledge determination unit as being highly relevant and each image feature amount included in the combination of the image feature amounts. Interpretation knowledge extraction device that extracts as binary relational information. The interpretation knowledge determination unit (a) a first correlation value that defines a relationship between a first image feature amount selected from the third binomial relationship information and a first disease name; (B) The first interpretation item in which a correlation value that defines the relationship between the first disease name selected from the second binomial relationship information and the first interpretation item is a correlation value threshold value or more. A difference value between a first correlation item selected from the first binomial relationship information and a second correlation value defining a relationship between the first image feature quantity The interpretation knowledge extraction device according to claim 3, wherein when the calculated difference value is larger than a difference threshold, it is determined that there are a plurality of combinations of image feature amounts highly related to the first disease name. The image interpretation item extraction unit is configured to obtain, for each disease name determined by the image interpretation knowledge determination unit that there are a plurality of highly relevant image feature amount combinations, between the disease name indicated by the second binomial relationship information. Among all the interpretation item sets created from the interpretation items whose correlation value is greater than the first threshold value, the interpretation item group whose correlation value with the disease name is greater than or equal to the second threshold value is the disease The interpretation knowledge extraction device according to claim 3 or 4, wherein a name and a combination of image feature values are extracted as a combination of interpretation items. A similar case retrieval method for retrieving case data similar to case data including a medical image and an interpretation report, which is document data describing a result of interpretation of the medical image, from a case database,
The interpretation report includes (a) an interpretation item that is a character string indicating the characteristics of a medical image and (b) a disease name that is at least a disease name that is a diagnostic result of an interpreter based on the medical image,
The similar case search method includes:
An image feature extraction step for extracting a plurality of image feature amounts from an image to be interpreted which is a medical image to be interpreted;
A report analysis step of extracting a disease name from a target interpretation report that is an interpretation report for the interpretation target image;
A weight determination step for determining a weight for each image feature based on the interpretation knowledge database;
The plurality of image feature amounts extracted in the image feature extraction step and a plurality of image feature amounts extracted from medical images included in case data registered in a case database are determined in the weight determination step. A similar case search step of searching case data including a medical image similar to the image to be interpreted from the case database by weighting and comparing with a weight for each image feature amount,
The interpretation knowledge database is
First binomial relationship information that predetermines a relationship between each image feature amount extracted from the medical image and each interpretation item;
A second binomial relationship information that predetermines the association between each interpretation item and each disease name;
A third binomial relationship information that predetermines the relationship between each image feature and each disease name;
For each disease name, including presence / absence information indicating whether or not there are multiple combinations of image feature quantities highly related to the disease name,
In the weight determination step, it can be determined that there are a plurality of combinations of image feature quantities highly relevant to the target disease name, which is the disease name extracted in the report analysis step, based on the presence / absence information included in the interpretation knowledge database. In this case, for each combination of image feature amounts, each image feature amount via an interpretation item calculated from the first binomial relationship information and the second binomial relationship information included in the knowledge database. For each image feature amount extracted in the image feature extraction step, the fourth binomial relationship information indicating the relationship between the target disease name and the fourth binomial relationship information is used instead of the third binomial relationship information. A similar case retrieval method for determining a weight of a larger value as the relationship between the image feature amount and the target disease name is higher.   The program for making a computer perform the similar case search method of Claim 6. An interpretation knowledge extraction method for extracting interpretation knowledge from a plurality of case data including a medical image and an interpretation report that is document data describing a result of interpretation of the medical image.
The interpretation report includes (a) an interpretation item that is a character string indicating the characteristics of a medical image, or (b) a disease name that is a diagnostic result of an interpreter based on the medical image,
The interpretation knowledge extraction method is:
An image feature extraction step of extracting a plurality of image feature amounts from the medical image included in each case data;
A report analysis step of extracting disease names and interpretation items from interpretation reports included in each case data;
First binomial relationship information defining a relationship between each image feature amount extracted in the image feature extraction step and each interpretation item extracted in the report analysis step, and extracted in the report analysis step The second binomial relationship information for determining the relationship between each interpretation item and each disease name extracted in the report analysis step, each image feature amount extracted in the image feature extraction step, and the report An interpretation knowledge extraction step for extracting, as interpretation knowledge, third binomial relationship information that defines the relationship between each disease name extracted in the analysis step;
Based on the first binomial relationship information, the second binomial relationship information, and the third binomial relationship information, an image feature amount that is highly related to a disease name for each disease name Interpretation knowledge determination step for determining whether or not a plurality of combinations exist,
Based on the second binomial relation information, for each disease name determined that there are a plurality of image feature amount combinations having high relevance in the interpretation knowledge determination step, a combination of the disease name and the image feature amount An interpretation item extraction step for extracting a set of interpretation items to associate with each image feature amount combination,
Including
In the interpretation knowledge extraction step, for each interpretation item set extracted in the interpretation item extraction step, a relationship between the interpretation item set and each image feature amount extracted in the image feature extraction step Is the interpretation knowledge indicating the relationship between the disease name determined to be highly relevant in the interpretation knowledge determination step and each image feature amount included in the combination of image feature amounts. Interpretation knowledge extraction method that extracts as fourth binomial relation information.   A program for causing a computer to execute the interpretation knowledge extraction method according to claim 8.