JP2023057036A - Device and method for processing medical image - Google Patents

Abstract

To improve accuracy for quantitatively analyzing a medical image.SOLUTION: A device for processing a medical image includes generation means, display means and differentiation means. The generation means generates a first feature representing an image feature from a medical image including a region of interest. The display means displays the first feature. The differentiation means differentiates the region of interest of the medical image on the basis of the first feature and outputs a differentiation result relating to the region of interest.SELECTED DRAWING: Figure 1

Description

The embodiments disclosed in this specification and drawings relate to a medical image processing apparatus and a medical image processing method.

2. Description of the Related Art Conventionally, many medical image acquisition devices such as X-ray imaging devices, CT devices, and ultrasonic detection devices have been used for early detection of lesions such as cancer and abnormal anatomical structures. 2. Description of the Related Art After imaging a subject using a medical image acquisition device to obtain a medical image, a doctor interprets the medical image to find clinical features of a lesion site in the medical image.

Determining the nature of the lesion site is particularly important in such diagnosis. For example, in the case of benign/malignant differentiation for tumor sites, it is currently found that the risk or likelihood of malignancy is usually associated with some pathological feature. The interpretation allows the physician to empirically develop descriptions of the pathological features for the pseudo-sites and record those features as clinical features. Among them, some clinical characteristics can be quantified and used to evaluate and judge the comprehensive properties of the lesion, such as whether the lesion is malignant or its degree. Such clinical features are called clinically interpretable features. This clinically interpretable feature is traditionally often defined by standards such as clinical practice guidelines. For example, a breast imaging-reporting and data system BI-RADS (Breast Imaging-Reporting and Data System) is known as a clinical practice guideline. In such clinical practice guidelines and various research reports, lesion judgment criteria based on clinically interpretable features are described. can judge and evaluate the nature of

In addition to conventional methods of human interpretation, there are technologies that use artificial intelligence and machine learning systems to help doctors make decisions about interpretation in order to improve the accuracy and standardization of screening for diseases such as cancer. . For example, artificial intelligence/machine learning systems are used to assist decision-making by human experts.

In addition, using an image processing device, image basic features such as standard morphological/textural features and dynamic features of areas corresponding to specific regions such as tumors are extracted from medical images, and support is provided based on the extracted image basic features. There are methods to distinguish between benign and malignant tumors using machine learning algorithms such as vector machines and random forests. Alternatively, there is a method of distinguishing benign/malignant tumors by combining deep features extracted by a convolutional neural network with standard features. In this method, for example, learning data on the presence or absence of lesions or the degree of lesions is used to learn a classifier for determining the presence or absence of lesions or the degree of lesions based on various image feature amounts. There is also a method of learning feature amounts of clinically interpretable features using human visual judgment of image features as learning data. A simple combination of these methods is also conceivable.

However, while some clinically interpretable features of tumors, e.g., margins, margin enhancement, heterogeneity, etc., are useful for the physician's diagnosis, conventional human interpretation does not account for inter-interpreter differences. The problem is that is large. In addition, when the evaluation results of clinically interpretable features by doctors are used as training data for learning, the differences between radiologists limit the accuracy of the resulting inference, making it difficult to achieve high accuracy. There is

On the other hand, it is difficult to obtain accurate results in the method of determining a lesion using image basic features in a medical image, because of limitations on image basic features.

In addition, in the method of extracting deep features using deep learning, the extracted features cannot be interpreted, which is disadvantageous for quantitative analysis of lesions such as tumors. Further, when learning the presence or absence of lesions or the degree of lesions as learning data, the intermediate feature amount of the discriminator cannot be interpreted clinically. Can not. In addition, in a method that combines the above methods, if the feature amount learned by using the visual judgment of the image feature as learning data is displayed, the user can check the validity of the feature amount and correct it. Since the clinically interpretable features obtained depend on the interpreter, there is a problem that the learning results are not always optimal for judging the presence or absence of lesions or the extent of lesions.

U.S. Pat. No. 9,208,556 U.S. Pat. No. 8,345,940 WO2017/165801 JP 2018-061771 A Japanese Patent Publication No. 2001-511372 JP 2016-007270 A

One of the problems to be solved by the embodiments disclosed in the specification and drawings is to improve the accuracy of quantitative analysis of medical images. However, the problems to be solved by the embodiments disclosed in this specification and drawings are not limited to the above problems. A problem corresponding to each effect of each configuration shown in the embodiments described later can be positioned as another problem.

A medical image processing apparatus according to an embodiment includes generation means, display means, and discrimination means. A generator generates a first feature representing an image feature from a medical image including the region of interest. A display means displays the first characteristic. The discriminating means discriminates the region of interest in the medical image based on the first feature, and outputs a discrimination result regarding the region of interest.

FIG. 1 is a block diagram showing an example of the functional configuration of the medical image processing apparatus according to the first embodiment. FIG. 2 is a schematic diagram showing a learning process by the model generating means in the medical image processing apparatus according to the first embodiment. FIG. 3 is a schematic diagram showing a model application process in the first embodiment. FIG. 4 is a diagram showing an example of a screen displayed on display means in the medical image processing apparatus according to the first embodiment. FIG. 5 is a flowchart for explaining the model generation process in the medical image processing apparatus according to the first embodiment. FIG. 6 is a flowchart for explaining an image discrimination processing method in the medical image processing apparatus according to the first embodiment. FIG. 7 is a block diagram showing an example of the functional configuration of a medical image processing apparatus according to the second embodiment; FIG. 8 is a schematic diagram showing the process of generating clinically interpretable features in the second embodiment. FIG. 9 is a diagram showing an example of a screen displayed on display means in the medical image processing apparatus according to the second embodiment. FIG. 10 is a flowchart for explaining the image discrimination processing method in the medical image processing apparatus according to the second embodiment. FIG. 11 is a block diagram showing an example of the functional configuration of a medical image processing apparatus according to the third embodiment; FIG. 12 is a schematic diagram showing the optimization process based on the iterative method in the third embodiment. FIG. 13 is a flowchart for explaining the image discrimination processing method in the medical image processing apparatus according to the third embodiment.

A medical image processing apparatus according to an embodiment includes generating means for generating a first feature representing an image feature from a medical image including a region of interest, display means for displaying the first feature, and discrimination means for discriminating the region of interest in the medical image based on the information and outputting a discrimination result regarding the region of interest.

Further, in the medical image processing method in the medical image processing apparatus according to the embodiment, the generation means generates a first feature representing an image feature from a medical image including a region of interest; displaying a feature; and discriminating means discriminating the region of interest in the medical image based on the first feature and outputting a discrimination result for the region of interest.

According to the medical image processing apparatus and the medical image processing method according to the embodiments, the accuracy of quantitative analysis of medical images can be improved without subjective influence.

Further, in the medical image processing apparatus, the generation means acquires the medical image and performs predetermined division processing on the medical image to determine the region of interest in the medical image; clinically interpretable to generate the clinically interpretable features of the region of interest from the medical image containing the region of interest using a second discriminant model that takes as input a medical image containing the region of interest and outputs clinically interpretable features. and a feature generating means, wherein the discriminating means generates the clinically interpretable features generated by the clinically interpretable feature generating means using a first discriminant model that inputs image features and outputs a discrimination result for the region of interest. may be input to the first discrimination model as the image feature to output a discrimination result relating to the region of interest.

When training the medical image processing apparatus, as a first step, a first discriminant model for calculating a discrimination result from the image feature and the clinically interpretable feature is learned, and as a second step, the clinically interpretable feature Learn a second discriminant model that outputs . In the first stage, the image features calculated from the image and the visual judgment image features judged by the doctor are input to the first judgment model. the model is learned. In the second step, the image features calculated from the image and the clinically interpretable features output from the second discriminant model are input to the first discriminant model, and the definitive diagnosis result of benign or malignant is used as teacher data. , a second discriminant model is learned.

According to the medical image processing apparatus described above, in the learning of the second discriminant model, which is the final learning, since the feature amount of visual judgment is not used for learning the clinically interpretable feature, The accuracy of determining the presence or absence of lesions or the degree of lesions can be optimized. The on-screen display of the clinically interpretable features also allows the user to judge validity and make corrections if necessary.

Further, in the medical image processing apparatus, the generation means acquires the medical image and performs predetermined division processing on the medical image to determine the region of interest in the medical image; Extracted by the image basic feature extraction means using an image basic feature extraction means for extracting image basic features from a medical image including the region of interest, and a conversion model that associates the image basic features with clinically interpretable features clinically interpretable feature generating means for generating said clinically interpretable features of said region of interest from said image basis features, said discriminating means being adapted to said clinically interpretable features generated by said clinically interpretable feature generating means. Based on this, the region of interest may be classified, and a classification result relating to the region of interest may be output as a discrimination result.

According to the medical image processing apparatus described above, it is possible to automatically extract clinically interpretable features from a medical image, reduce subjective factors, and improve the accuracy of the extracted clinically interpretable features. Therefore, the accuracy of the quantitative analysis can be improved and the interpretation work can be reproducibly standardized compared to the subjective judgment by the doctor.

In addition, the classification results according to the present embodiment are obtained based on clinically interpretable features defined by clinical practice guidelines, and are therefore more intuitive to understand than methods of analyzing images based on deep learning methods. It is targeted and has advantages in interpreting the results.

Preferred embodiments of a medical image processing apparatus and a medical image processing method will be described in detail below with reference to the drawings.

The medical image processing apparatus according to the present embodiment is composed of, for example, a plurality of functional modules, and each functional module is installed as software in a device having a processor and memory such as an independent computer, or distributed to a plurality of devices. It is implemented by executing each functional module of the medical image processing apparatus stored in memory by a specific processor. Further, the medical image processing apparatus according to this embodiment may be implemented in the form of hardware as a circuit capable of executing each function of the medical image processing apparatus. In this case, the circuit that implements the medical image processing apparatus can transmit and receive data and collect data via a network such as the Internet. Also, the medical image processing apparatus according to the present embodiment may be directly mounted in a medical image acquisition apparatus such as a CT apparatus or a magnetic resonance imaging apparatus as part of the medical image acquisition apparatus.

Further, in the following description, a medical image captured of a mammary gland is taken as an example, and the medical image processing apparatus according to the present embodiment classifies (presence or absence of lesion) or A description will be given of the operation in the case of performing discrimination processing such as the degree of lesion. However, the present embodiment is not limited to this, and can also be applied to classification of specific regions in other medical images and discrimination processing such as classification.

(First embodiment)
First, a first embodiment will be described with reference to FIGS. 1 to 6. FIG.

FIG. 1 is a block diagram showing an example of the functional configuration of the medical image processing apparatus according to the first embodiment.

As shown in FIG. 1, the medical image processing apparatus 100 according to the first embodiment includes generation means 10, display means 20, discrimination means 30, correction means 40, and model generation means 50. .

A model generating means 50 generates a processing model used in the medical image processing apparatus 100 .

In the present embodiment, the model generating means 50 generates sample image data for which discrimination results such as the presence or absence of lesions or the degree of lesions in medical images have been verified, and image features including visual judgment image features obtained for the sample image data. Acquired as first learning data and trained by a deep learning method such as a neural network, a discriminant model 1 is generated that outputs a discrimination result relating to a region of interest with image features as input.

In addition, the model generation means 50 performs training by a deep learning method such as a neural network using the clinically interpretable features corresponding to the sample image data and the discrimination result of the first learning data as the second learning data. , generates a discriminant model 2 that takes as input a medical image containing a region of interest and outputs clinically interpretable features.

Here, the visual judgment image feature is a clinical image feature that a person with clinical experience such as a doctor finds or estimates from a medical image through interpretation of the medical image. That is, it can be said that the visually determined image feature is an image feature quantified based on human determination. For example, visual judgment image features are image features such as judgments about the shape and marginal state of a tumor.

In the learning of the discriminant model 1 by the model generating means 50, training is performed using learning data including such visual judgment image features.

FIG. 2 is a schematic diagram showing a learning process by the model generating means in the medical image processing apparatus according to the first embodiment.

Here, (a) of FIG. 2 shows a schematic diagram when the discriminant model 1 is learned.

As shown in (a) of FIG. 2, when learning the discriminant model 1, sample image data for which the determination result of the presence or absence of a lesion or the extent of a lesion has already been obtained is obtained, and a medical image that is the sample image data is acquired. By acquiring image features such as visual judgment image features obtained visually from medical images and the features of the image, and using the feature values indicating these image features and the judgment results of the presence or absence of lesions or the extent of lesions as learning data, neural A discriminant model 1 is learned using a deep learning method such as a network. As a result, a discriminant model 1 is obtained that takes image features as input and outputs discrimination results relating to the region of interest.

In addition, FIG. 2(b) shows a schematic diagram when learning the discriminant model 2. As shown in FIG.

In the learning of the discriminant model 2, after learning the discriminant model 1, the portion of the visual judgment image feature in (a) of FIG. The discriminant model 2 is learned using a deep learning method such as a neural network using the image data and the determination result of the presence/absence of a lesion or the degree of a lesion as learning data. Here, when learning the discriminant model 2, the model generating means 50 corresponds to the sample image data and the determination result of the presence or absence of a lesion or the degree of lesion based on the sample image data and the determination result of the presence or absence of a lesion or the degree of a lesion. It is possible to obtain clinically interpretable features that In this way, by using the sample image data and the determination result of the presence or absence of a lesion or the degree of a lesion as learning data, it is possible to learn the discriminant model 2 that outputs a clinically interpretable feature from a medical image including a region of interest. Alternatively, the model generating means 50 trains the visual judgment image features generated when learning the discriminant model 1 as clinically interpretable features output by the discriminant model 2, thereby enabling clinical interpretation using a medical image including the region of interest as an input. A discriminant model 2 that outputs features may be learned. As shown in FIG. 2(b), by inputting the clinically interpretable features output by the discriminant model 2 into the discriminant model 1 in place of the visual judgment image features, accurate lesion presence/absence or lesion degree determination results can be obtained. Obtainable.

Here, a clinically interpretable feature means a quantifiable feature that clinically describes a specific site. Because such quantifiable features are easy to interpret, they are often applied to interpretation-related processes such as lesion quality assessment and scoring. Also, clinically interpretable features are descriptive features whose specific content can be determined and interpreted according to clinical practice guidelines or standards within the industry.

For example, clinically interpretable features are clinically interpretable features defined by clinical practice guidelines. Also, the clinically interpretable features include clinically interpretable morphological features, which are clinically interpretable morphological features, and clinically interpretable kinetic features, which are clinically interpretable kinetic features. .

For example, in the case of mammary gland-related fields, refer to breast imaging-reporting and data system BI-RADS (Breast Imaging-Reporting and Data System), etc. to describe tumor shapes such as round, oval, lobed, and irregular. It is possible to define clinically interpretable features such as features that are smooth, features that describe boundaries such as smooth, acicular, irregular, etc., features that describe enhancement models such as Homogeneous, Heterogeneous, Annular enhancement, etc. . By quantifying each of these features, a clinically interpretable feature can be expressed.

In the first embodiment, the model generation means 50 generates the discriminant model 1 and the discriminant model 2 as described above. Thereby, the generating means 10 and the discriminating means 30 can process medical images using the models generated by the model generating means 50, respectively.

Generating means 10 generates various features representing image features from a medical image including a region of interest. Here, the various features generated by the generating means 10 correspond to the "first feature". Specifically, generating means 10 includes dividing means 11 and clinically interpretable feature generating means 12 .

The dividing unit 11 acquires a medical image and performs a dividing process on the acquired medical image to specify a region of interest. In the present embodiment, the medical image is a medical image such as a perspective view of a mammary gland, and any image to be segmented includes shadowed portions of nodules and tumors. The dividing means 11 performs normal dividing processing on the obtained medical image to divide the mammary gland part and the shadow part where the nodule or tumor exists in the mammary gland, and divide the shadow part into the region of interest. do. Since any existing dividing method can be used as a specific dividing method, description thereof is omitted here.

A clinically interpretable feature generating means 12 uses a discriminant model 2 which receives a medical image including a region of interest as an input and outputs a clinically interpretable feature, and uses the discrimination model 2 to output a clinically interpretable feature from the medical image including the region of interest divided by the dividing means 11. Generate interpretable features. Specifically, the clinically interpretable feature generation means 12 inputs a medical image including a region of interest to the discriminant model 2, and generates a clinically interpretable feature that interpretably describes the clinical features of the region of interest in the medical image. from the discriminant model 2 to generate clinically interpretable features of the region of interest from the medical image.

Display means 20 displays the clinically interpretable features generated by clinically interpretable feature generating means 12 . For example, the display means 20 may display a screen in which quantified data representing the clinically interpretable features of the medical image and related information are arranged side by side, or may display a plurality of clinically interpretable features output from the discriminant model 2 as numerical values. The screen may be displayed sequentially according to the level of the probability or the level of the probability. Note that the quantified display of the clinically interpretable features may be displayed as specific data or as a graph such as a histogram corresponding to the data.

The display means 20 may also display the clinically interpretable features in a user interactive interface. This allows the user to enter changes to descriptions and numerical values of clinically interpretable features in an interactive interface by viewing the image.

Modification means 40 receives user input and modifies the clinically interpretable features displayed by display means 20 based on the received user input.

The discriminating means 30 discriminates a region of interest in the medical image based on the clinically interpretable features and outputs a discrimination result regarding the region of interest.

Specifically, the discriminating means 30 uses the discriminant model 1 which receives image features as input and outputs discrimination results related to the region of interest, and converts the clinically interpretable features generated by the clinically interpretable feature generating means 12 into the discriminant model 1. By inputting the information, determination results such as the presence or absence of a lesion in the region of interest or the extent of the lesion are obtained from the discrimination model 1, and the discrimination result regarding the region of interest is output.

FIG. 3 is a schematic diagram showing a model application process in the first embodiment.

As shown in FIG. 3, when the medical image processing apparatus 100 determines the presence or absence of a lesion or the degree of a lesion in a region of interest in a medical image, the clinically interpretable feature generating means 12 inputs the medical image to the discrimination model 2 and Outputting the clinically interpretable features, the display means 20 and the modifying means 40 display and modify the clinically interpretable features. Further, the discriminating means 30 outputs a determination result regarding the presence/absence of a lesion or the degree of a lesion by inputting clinically interpretable features into the discriminant model 1 as image features. Thus, in the application process as shown in FIG. 3, by using two models, the discriminant model 1 and the discriminant model 2, subjective features such as visual judgment image features do not appear, and subjective influences do not appear. less. Specifically, since the discriminant model 2 is finally learned using the discrimination results as teacher data, the clinically interpretable features output by the discriminant model 2 are influenced by the reader-dependent included in the visual judgment image features. relatively less. Therefore, according to this configuration, there is a feature that the discrimination result is less likely to be influenced by subjectivity.

The display means 20 may simultaneously display the clinically interpretable features and the output result based on the discriminant model 1 of the discrimination means 30 on the display screen. In this case, when the modifying means 40 changes the size of the clinically interpretable feature, the determination result of the presence or absence of lesion or the degree of lesion by the discriminant model 1 of the discriminating means 30 is also updated.

For example, when the discriminant model 1 is a model in which a classification scale vector obtained using a machine learning algorithm is constructed from individual image features, the discrimination model 1 corresponds to a classification scale vector consisting of image features in a medical image. You may output the lesion prediction positive rate. In this case, even if the display means 20 preliminarily stores reference information related to clinically interpretable features as image features, such as names of clinically interpretable features, probability information of the clinically interpretable features in an image group, and the like. good. Then, the display means 20 may display this reference information together with the generated clinically interpretable feature data and lesion prediction positive rate on the display screen for easy reference by the user.

FIG. 4 is a diagram showing an example of a screen displayed on display means in the medical image processing apparatus according to the first embodiment.

A screen region R100a shown in FIG. 4 includes a plurality of regions R110, R120, R160, R170, and R180. Here, the region R110 displays a medical image in which it is necessary to determine whether or not the tumor portion therein is a lesion (benign or malignant). Patient information such as ID and name is displayed in the area R120. In the example shown in FIG. 4, descriptions (names) of clinically interpretable features and output results of the discriminant model 1 corresponding thereto are shown in regions R160, R170 and R180, respectively.

For example, in the region R160, it is shown that the predicted positive rate calculated based on the clinical technical features is 27.0% when it has the clinically interpretable feature of marginal smoothness. Further, in region R160, as reference information, the confidence interval, which is a measure of discriminative validity, is 20.4% to 34.3%, and as a reference group, the feature is a benign group consisting of medical images of benign tumors. It has further been shown that the probability of appearance in the malignant tumor is 83.0% and the probability of appearance in the malignant group consisting of medical imaging of malignant tumors is 45.1%.

In addition, in region R170, when having two clinically interpretable features of smooth margins and non-descending, it is shown that the predicted positive rate calculated based on the clinical technical features is 1.9%. ing. Further, in region R170, as reference information, the confidence interval is 0.2% to 6.5%, and as reference group, the probability that the feature of the combination appears in a benign group consisting of medical images of benign tumors is 72.1% and the probability of appearing in the malignant group consisting of medical imaging of malignant tumors is 2.5%.

In addition, in region R180, when there are three clinically interpretable features of smooth margins, non-descending and uniform, the predicted positive rate calculated based on the clinical technical features is 3.2%. It is shown. In the region R180, the confidence interval is 0.1% to 6.7% as reference information, and the probability that the feature of the combination appears in a benign group consisting of medical images of benign tumors as a reference group. is 20.4% and the probability of appearing in the malignant group consisting of medical imaging of malignant tumors is 1.2%.

With the display as described above, FIG. 4 shows what kind of pattern the target medical image shows, and also shows the degree of benign/malignant.

Note that the display means 20 may determine whether or not to display based on the identification validity scale. For example, in the example shown in FIG. 4, the upper limit of the confidence interval (prediction positive rate upper limit) in the region R180 is 16.7%, and the numerical value is between the regions R160 and R170. It can be seen that the values are below regions R160 and R170. Therefore, display means 20 may determine not to display region R180 in such a case.

Moreover, the display screen of FIG. 4 is merely an example, and the present embodiment is not limited to this. For example, the display means 20 may display the screen in another form in accordance with the features of the clinically interpretable features output from the discriminant model 2 .

FIG. 5 is a flowchart for explaining the model generation process in the medical image processing apparatus according to the first embodiment.

As shown in FIG. 5, when creating a model, first, the model generating means 50 acquires a sample image set including discrimination results in which the presence or absence of a lesion or the degree of a lesion is verified (step S501). In addition, the model generating means 50 generates a discriminant model 1 by performing training using image features including visual judgment image features generated from the sample image set and discrimination results of lesion presence/absence/degree as learning data ( step S502).

Then, the model generating means 50 uses only the discrimination result of lesion presence/absence/degree in the learning data used for training the discriminant model 1 as learning data, and converts the visual judgment image features generated when training the discriminant model 1 into the discriminant model. A discriminant model 2 is generated by training the clinically interpretable features output from 2 (step S503).

As a result, a discriminant model 1 that receives an image feature as an input and outputs a discriminative result related to the region of interest, and a discriminant model 2 that receives a medical image including the region of interest as an input and outputs a clinically interpretable feature are generated. Model 2 is placed in the medical image processing apparatus 100 (step S504).

Next, the overall flow of processing performed by the medical image processing apparatus 100 according to the first embodiment will be described.

FIG. 6 is a flowchart for explaining an image discrimination processing method in the medical image processing apparatus according to the first embodiment.

As shown in FIG. 6, first, the dividing means 11 acquires a medical image to be processed (step S601), and performs organ division and tumor division on the medical image, thereby dividing the mammary gland and the tumor portion in the mammary gland. The tumor region is divided, and the divided tumor region is set as the region of interest (step S602).

Subsequently, the clinically interpretable feature generating means 12 outputs the clinically interpretable features of the medical image by inputting the divided medical image to the discriminant model 2 (step S603).

Then, the display means 20 displays the generated clinically interpretable features (step S604), and the correction means 40 determines whether or not corrections to the clinically interpretable features have been received (step S605).

Here, if the modification of the clinically interpretable features is not received within a predetermined period of time (step S605: No), the discriminating means 30 inputs the clinically interpretable features into the discriminant model 1 to A discrimination result indicating the presence or absence of a lesion in the area or the degree of lesion is output (step S606).

On the other hand, if the correction of the clinically interpretable features is received within a predetermined period of time (step S605: Yes), the discriminating means 30 inputs the corrected clinically interpretable features into the discriminant model 1, A discrimination result indicating the presence or absence of a lesion in the region of interest or the degree of lesion is output (step S607).

According to the above-described first embodiment, the accuracy of discrimination is improved without subjective influence by using clinically interpretable features instead of visual judgment image features when discriminating medical images. can be made Also, because the clinical interpretability of clinically interpretable features is different from uninterpretable features such as intermediate features that occur in traditional neural network learning, by displaying clinically interpretable features to the user, the user can , the validity of the features can be determined and corrected more easily, further ensuring the reliability of the identification process. In addition, the efficiency of interpretation can be improved by using a discriminant model that can directly obtain easily understood clinically interpretable features from medical images.

(Modification of the first embodiment)
In the first embodiment, the medical image processing apparatus 100 includes the model generating means 50, and the discriminant model 1 and the discriminant model 2 are generated by the model generating means 50 and arranged in the medical image processing apparatus 100. However, embodiments are not limited to this. For example, the medical image processing apparatus 100 may directly acquire the generated discriminant model 1 and discriminant model 2 from the outside and process them. In this case, the medical image processing apparatus 100 does not need to include the model generating means 50. FIG.

In addition, in the first embodiment, the medical image processing apparatus 100 is provided with the correction means 40 to accept correction of the clinically interpretable features, but the embodiment is not limited to this. For example, the medical image processing apparatus 100 discriminates the region of interest directly using the clinically interpretable features generated by the discriminant model 2 without including the modifying means 40 and without allowing modifications to the clinically interpretable features. You may do so.

(Second embodiment)
Next, a second embodiment will be described with reference to FIGS. 7 to 10. FIG. The configuration of the medical image processing apparatus according to the second embodiment is different from that of the first embodiment in that the processes performed by the generation means 10 and the discrimination means 30 are different, and the model generation means 50 is not provided. ing. In the following, the second embodiment will be mainly described with respect to the differences from the first embodiment, and the same or similar configurations will be denoted by the same reference numerals, and redundant description will be omitted as appropriate.

FIG. 7 is a block diagram showing an example of the functional configuration of a medical image processing apparatus according to the second embodiment;

As shown in FIG. 7, the medical image processing apparatus 100a according to the second embodiment includes generation means 10a, display means 20, discrimination means 30a, and correction means 40. FIG.

Here, the generation means 10a includes a division means 11, an image basis feature extraction means 13, and a clinically interpretable feature generation means 12a.

The dividing unit 11 obtains a medical image and performs a dividing process on the obtained medical image to determine a region of interest. In the present embodiment, the medical image is a medical image such as a perspective view of a mammary gland, and any image to be segmented includes shadowed portions of nodules and tumors. The dividing means 11 performs normal dividing processing on the obtained medical image to divide the mammary gland part and the shadow part where the nodule or tumor exists in the mammary gland, and divide the shadow part into the region of interest. do. Since any existing dividing method can be used as a specific dividing method, description thereof is omitted here.

The image basic feature extraction means 13 extracts image basic features from the medical image including the region of interest.

Here, image basic features mean image features that can be accurately obtained from an image by measurement, and include, for example, general basic morphological features, basic texture features, and basic dynamic features. Among these, the basic morphological features indicate the morphological structure of the figure, such as the outer boundary size of the outline, the area size, and the like. Also, the basic texture feature indicates the surface features of the image or region of interest, such as the thickness and density of the image texture. In addition, the basic dynamic feature is a dynamic feature on the image. For example, when extracting the basic dynamic feature for the multiphase contrast-enhanced image, the dynamic features such as the enhancement rate by , the time difference from the first temporal image after enhancement to the brightest temporal image, and the enhancement rate of the brightest image.

The clinically interpretable feature generating means 12a uses a pre-generated conversion model that associates the image basic features with the clinically interpretable features, and converts the image basic features extracted by the image basic feature extraction means 13 into the conversion model. Substitution produces clinically interpretable features of the region of interest from the image basis features.

Here, a clinically interpretable feature means a quantifiable feature that clinically describes a specific site. Because such quantifiable features are easy to interpret, they are often applied to interpretation-related processes such as lesion quality assessment and scoring. Clinically interpretable features are descriptive features whose specific content can be determined and interpreted by clinical guidelines or standards within the industry. For example, in the case of mammary gland-related fields, refer to breast imaging-reporting and data system BI-RADS (Breast Imaging-Reporting and Data System), etc. to describe tumor shapes such as round, oval, lobed, and irregular. It is possible to determine clinically interpretable features, including features that are smooth, features that describe boundaries such as smooth, acicular, irregular, etc., features that describe enhancement models such as homogeneous, non-uniformity, and annular enhancement. can. Each of these features can be expressed by quantification. In addition, clinically interpretable features also include clinically interpretable kinetic features that describe the dynamics of specific sites. For example, when a contrast agent is used, there is a process in which the contrast agent is injected into the subject and absorbed. Also, the mode of change in "brightness" may be different. BI-RADS includes clinically interpretable kinetic features such as, for example, early bright and then gradually dark features (washout), features with fast onset enhancement (fast), and features with slow onset enhancement (Slow). characteristics are specified.

Further, the conversion model that associates the image basic features and the clinically interpretable features used by the clinically interpretable feature generation means 12a is also called a clinically interpretable feature generation model, and image cases that have been interpreted in the past are used as sample data. It is a model generated in advance by machine learning. A model generated by such machine learning assigns a weight to each basic image feature, and obtains a clinically interpretable feature by a predetermined algorithm.

Here, for example, the clinically interpretable feature generation model is a polynomial model.

For example, the clinically interpretable feature generation model may be a linear polynomial model such as the following equation (1).

In Equation (1), χ ₁ , χ ₂ to χ _n are parameters representing basic image features or mathematical expressions composed of parameters of basic image features. Also, y is a parameter representing a predefined clinically interpretable feature. Also, w ₁ , w ₂ to w _n are weights assigned to χ ₁ , χ ₂ to χ _n , respectively, and b is a polynomial weight.

Of these, χ ₁ , χ ₂ to χ _n are image basis features that are considered to influence the clinically interpretable feature y. Here, the clinically interpretable morphological features included in the clinically interpretable features are related to the basic morphological features of the image, and the clinically interpretable kinetic features included in the clinically interpretable features are related to the basic dynamics of the image. related to academic characteristics. Then, the clinically interpretable feature generation means 12a generates clinically interpretable morphological features using a conversion model that associates the basic morphological features of the image with the clinically interpretable morphological features, A plurality of clinically interpretable features are obtained by generating clinically interpretable kinetic features using a transformation model that associates the kinetic features with the clinically interpretable kinetic features.

The polynomials shown above are an example of the clinically interpretable feature generation model, but the present embodiment is not limited to this, and other algorithms may be used. That is, the clinically interpretable feature generation model may be a model generated by a machine learning method using sample data, and may be, for example, in a non-linear form as in Equation (2) below.

In Equation (2), χ ₁ , χ ₂ to χ _n are parameters representing basic image features or mathematical expressions composed of parameters of basic image features. Also, y is a parameter representing a predefined clinically interpretable feature. Also, w ₁ , w ₂ to w _n are weights given to χ ₁ ² and χ ₂ ² to χ _n ² , respectively, and b is a polynomial weight.

For example, the clinically interpretable feature generating means 12a uses the transformation model configured as in the above-described equations (1) and (2) to generate the image basic features χ ₁ , χ ₂ to χ _n in the transformation model. The substitution produces a quantified clinically interpretable feature y. Note that there may be a plurality of clinically interpretable features y, and when there are a plurality of clinically interpretable features y, a conversion model corresponding to each clinically interpretable feature y is used. In this example, as many polynomials as there are clinically interpretable features y are used.

FIG. 8 is a schematic diagram showing the process of generating clinically interpretable features in the second embodiment.

The left side of FIG. 8 shows images obtained by dividing the mammary gland tumors of the medical images of 12 subjects obtained by the dividing means 11 . These mammary gland tumor images are input to the image-based feature extraction means 13, and the image-based feature extraction means 13 extracts image-based data such as volume, circumference, major axis, and flatness of the tumor portion from the input image. Extract features. These image basis features can be obtained by direct measurements from the image. For example, long axis image basis features can be obtained by measuring the length of the long axis of the tumor segment.

Then, the image basis feature extraction means 13 inputs these image basis features to the clinically interpretable feature generation means 12a, whereby the clinically interpretable feature generation means 12a uses the clinically interpretable feature generation model to generate the clinical Generate and output interpretable features.

The discriminating means 30a classifies the region of interest based on the clinically interpretable features generated by the clinically interpretable feature generating means 12a, and outputs a classification result regarding the region of interest as a discrimination result.

Specifically, the discriminating means 30a compares the clinically interpretable features generated by the clinically interpretable feature generating means 12a with threshold values of the clinically interpretable features, and based on the results of the matching, classifies the region of interest. Output.

For example, if the region of interest is a tumor portion of the mammary gland, the clinically interpretable feature generating means 12a outputs clinically interpretable features of each tumor portion by the operation shown in FIG. Then, the discriminating means 30a compares these clinically interpretable features with judgment criteria defined by breast imaging-reporting and data system BI-RADS (Breast Imaging-Reporting and Data System) or the like, thereby clinically interpreting the tumor portion. A result of classifying the tumor as benign or malignant or a judgment result of the tumor corresponding to the possible characteristics is obtained.

Note that the discrimination means 30a stores in advance a correspondence table in which numerical ranges of clinically interpretable features are associated with classification results, and when acquiring clinically interpretable features, performs correspondence based on the clinically interpretable features. A table may be searched to obtain the corresponding classification result.

Further, the discrimination means 30a may be used not only for judging malignant tumors, but also for subdividing types of benign tumors.

The display means 20 displays the clinically interpretable features generated by the clinically interpretable feature generating means 12a.

The modifying means 40 receives user input and modifies the clinically interpretable features displayed on the display means 20 based on the received user input.

The display means 20 may simultaneously display the clinically interpretable features and the output result based on the discriminant model 1 of the discrimination means 30 on the display screen. In this case, when the modifying means 40 changes the size of the clinically interpretable feature, the determination result of the presence or absence of lesion or the degree of lesion by the discriminant model 1 of the discriminating means 30 is also updated.

FIG. 9 is a diagram showing an example of a screen displayed on display means in the medical image processing apparatus according to the second embodiment.

A screen region R100 shown in FIG. 9 includes a plurality of regions R110, R120, R130, R131, R140, and R150. A region R110 displays a medical image in which it is necessary to determine whether a tumor portion is a lesion (benign or malignant). In addition, patient information such as ID and name is shown in the region R120. In the example shown in FIG. 9, regions R130, R140, and R150 respectively show descriptions of the three clinically interpretable features generated by the clinically interpretable feature generating means 12a and the probability ranges of the associated reference groups. ing. Furthermore, specific quantified numerical values of the feature values generated by the clinically interpretable feature generating means 12a are shown on the lower left side of each of the regions R130, R140, and R150.

For example, in region R130, a smooth margin feature is shown, and as a reference group, the feature has a probability of appearing in a benign group consisting of medical images of benign tumors with a probability of 83.0% and a medical image of malignant tumors. It has been shown to have a 45.1% probability of appearing in the malignant cluster of imaging. Also shown in region R140 is a smooth-margined non-washout feature, with a probability of 72.0 for this feature appearing in a benign group of medical images of benign tumors as a reference group. %, with a 2.5% probability of appearing in the malignant group consisting of medical imaging of malignant tumors. Also shown in region R150 are smooth-margined, non-washout, and heterogeneous features, and as a reference group, the probability that the feature appears in a benign group of medical images of benign tumors is 20.0% and a 1.5% probability of appearing in the malignant group consisting of medical images of malignant tumors.

Check boxes R1, R2, and R3 are further provided above each feature amount. By selecting check boxes R1, R2, and R3, the user selects which clinically interpretable features are to be used for discrimination as image features, that is, which feature quantities are to be used for the processing of the discrimination means 30a. can be done. For example, in FIG. 9, the check box R1 is checked, thereby indicating that the discrimination result output from the discriminating means 30a is "the probability of malignant tumor is 0.3" in the region R131. .

Note that the display screen in FIG. 9 is merely an example, and the present embodiment is not limited to this. For example, the screen may be displayed in another form in accordance with the features of the clinically interpretable features generated by the clinically interpretable feature generating means 12a. For example, the display means 20 displays the plurality of clinical interpretable features generated by the clinically interpretable feature generating means 12a when all the clinically interpretable features generated by the clinically interpretable feature generating means 12a are used in the processing of the discriminating means 30a. The features of the interpretable features may be listed directly. In addition, the display means 20 can also receive user input in a form in which the feature amount of the clinically interpretable feature can be directly changed.

Further, for example, the display means 20 may display a screen in which quantified data indicating clinically interpretable features of a medical image and related information are arranged side by side. Alternatively, the display means 20 may display a screen that sequentially shows a plurality of clinically interpretable features generated by the clinically interpretable feature generating means 12a according to the numerical value or probability. Note that the quantified display of the clinically interpretable features may be displayed as specific data, or as a graph such as a histogram corresponding to the data.

Next, the overall flow of processing performed by the medical image processing apparatus 100a according to the second embodiment will be described.

FIG. 10 is a flowchart for explaining the image discrimination processing method in the medical image processing apparatus according to the second embodiment.

As shown in FIG. 10, the dividing means 11 first acquires a medical image to be processed (step S301), and performs organ division and tumor division on the medical image, thereby dividing the mammary gland and the tumor portion in the mammary gland. The tumor region is divided, and the divided tumor region is set as the region of interest (step S302).

Subsequently, the image basic feature extraction means 13 extracts basic morphological features and basic dynamic features from the divided medical image (step S303).

Subsequently, the clinically interpretable feature generation means 12a uses a transformation model generated in advance by machine learning to generate clinically interpretable morphological features from the basic morphological features of the image, and the basic dynamics of the image. Generating clinically interpretable kinetic features from the features (step S304).

Then, the display means 20 displays the generated clinically interpretable features (step S305), and the correction means 40 determines whether or not corrections to the clinically interpretable features have been accepted (step S306).

Here, if the modification of the clinically interpretable features is not accepted within a predetermined period of time (step S306: No), the discrimination means 30a sets each clinically interpretable feature within the judgment range described in the clinical practice guideline or the like. By matching, each tumor region is classified into the corresponding clinical level, and the classification result is output (step S307).

On the other hand, if the correction of the clinically interpretable features is received within a predetermined period of time (step S306: Yes), the discriminating means 30a determines each clinically interpretable feature after the correction within the judgment range described in the clinical practice guideline or the like. to classify each tumor region into the corresponding clinical level, and output the classification result (step S308).

Note that in the second embodiment, the medical image processing apparatus of the present embodiment has been described by taking an example of classifying an image containing a mammary gland tumor. It is possible to obtain an image of an arbitrary site and classify the site without being limited to what is processed.

According to the second embodiment described above, clinically interpretable features are extracted from a medical image by converting image basis features obtained from the image into clinically interpretable features and classifying regions of interest based on the clinically interpretable features. Accuracy of quantified analysis compared to subjective judgment by physicians, as it can be automatically extracted, subjective factors can be reduced, and the accuracy of extracted clinically interpretable features can be improved and reproducibly standardize the reading process.

(Third embodiment)
Next, a third embodiment will be described with reference to FIGS. 11 to 13. FIG. The configuration of the medical image processing apparatus according to the third embodiment is based on the second embodiment, and further includes a model generating means 50a and an optimizing means 60 compared to the second embodiment. The main difference is that the means 40 is not provided. In the following, the third embodiment will be mainly described for the points that are different from the first and second embodiments, and the same or similar configurations will be denoted by the same reference numerals, and duplicate description will be appropriately omitted. omitted.

FIG. 11 is a block diagram showing an example of the functional configuration of a medical image processing apparatus according to the third embodiment;

As shown in FIG. 11, a medical image processing apparatus 100b according to the third embodiment includes generation means 10a, display means 20, discrimination means 30a, optimization means 60, and model generation means 50a. there is

In this embodiment, the model generating means 50a obtains sample image data and clinically interpretable features corresponding to the sample image data, and uses machine learning to associate the image basic features with the clinically interpretable features. Generate a conversion model.

For example, the model generation means 50a generates a clinically interpretable feature generation model using machine learning when sample data such as an existing image interpretation case is input. Here, the sample data includes, for example, medical image data and clinically interpretable feature data of the medical image described by a doctor by interpreting the medical image. Then, the model generating means 50a uses such sample data to extract basic image features from the medical image data, and uses clinically interpretable features corresponding to the extracted basic image features and the medical image data to produce machine images. Through learning, a conversion model is generated that associates image basic features and clinically interpretable features, such as Equation (1) or Equation (2) described in the second embodiment.

Then, for a 3D medical image, for example, to generate a transformation model that yields clinically interpretable features about the shape of the tumor segment, χ ₁ is the ratio of the volume squared to the surface cube of the tumor segment in the image. , _χ2 may be the ratio of the short axis to the long axis, and _χ3 may be the ratio of the middle axis to the long axis. As a result, the model generation means 50a measures and calculates the combination parameters of these image basic features on the medical image in the sample data, and also determines the shape of the sample data as circular, elliptical, lobed, By quantifying the clinically interpretable morphological features that indicate irregular shapes, etc., and performing machine learning based on these set parameters and the quantified clinically interpretable features, the basic morphological features of the image and the clinical A transformation model can be generated that associates interpretable morphological features.

In addition, for example, for multi-phase contrast-enhanced images, a conversion model capable of obtaining clinically interpretable dynamic features, which are the brightness change modes of the tumor portion during the process of injection and absorption of the contrast agent into the subject, is provided. To generate it, χ ₁ is the enhancement ratio IE = (S2-S0)/S0, and χ ₂ is the peak time (the time difference from the first temporal image at 1 minute after enhancement to the brightest time). , the unit is s (seconds)), and _χ3 may be the brightest image enhancement rate MEIP=(Speak-S0)/S0. Here, S0 is the pre-contrast image, S2 is the contrast-enhanced image at, for example, 2 minutes after enhancement, and Speak is the brightest contrast-enhanced image. As a result, the model generating means 50a measures and calculates these three basic dynamic features on the medical image in the sample data, and also calculates the fast, slow, washout By quantifying clinically interpretable dynamic features such as and performing machine learning based on these set parameters and quantified clinically interpretable dynamic features, the basic dynamic features of the image and the clinical A transformation model can be generated that associates interpretable dynamic features.

Also, in order to generate a transformation model that can obtain clinically interpretable features, for example, the degree of blurring of the boundaries of the tumor site, _χ1 is the pixel gradient of the image, and _χ2 is from outside the tumor site to the tumor site. It may be a pixel change on emitted radiation. As a result, the model generating means 50a measures and calculates these two basic image features on the medical image in the sample data, and quantifies the clinically interpretable feature representing the degree of blurring of the boundary of the tumor site in the sample data. , by performing machine learning based on these set parameters and quantified clinically interpretable features, it is possible to generate a transformation model that associates the image basis features with the clinically interpretable features.

Although some examples of model generation have been given above, the conversion model generated by the model generation means 50a is not limited to the above examples. For example, the model generating means 50a may generate a conversion model according to the content of sample data.

The dividing unit 11 obtains a medical image and performs a dividing process on the obtained medical image to determine a region of interest. In the present embodiment, the medical image is a medical image such as a perspective view of the mammary gland, and the dividing means 11 performs normal division processing on the obtained medical image to divide the mammary gland into a mammary gland part and a nodule or a mammary gland. The shadowed portion where the tumor exists is divided, and the shadowed portion is defined as the region of interest.

The image basic feature extraction means 13 extracts image basic features from the medical image including the region of interest.

The clinically interpretable feature generating means 12a substitutes the image basic features extracted by the image basic feature extracting means 13 into the transformation model using the transformation model generated by the model generating means 50a, thereby obtaining the image basic features to generate clinically interpretable features that describe regions of interest in the image.

The discriminating means 30a classifies the region of interest based on the clinically interpretable features generated by the clinically interpretable feature generating means 12a, and outputs a classification result regarding the region of interest as a discrimination result.

Specifically, the discriminating means 30a compares the clinically interpretable features generated by the clinically interpretable feature generating means 12a with the clinically interpretable features, and outputs a classification result regarding the region of interest based on the results of the matching. .

For example, the discriminating means 30a outputs a classification result using preset classification thresholds for clinically interpretable features. For example, the differentiating means 30a may classify the quantified clinically interpretable feature as malignant if the quantified clinically interpretable feature reaches or exceeds a classification threshold for a mammary tumor, and the quantified clinical interpretation Classify the clinically interpretable feature as a benign tumor if the possible feature is below the classification threshold. This classification threshold may be set, for example, based on descriptions in clinical guidelines such as BI-RADS, or descriptions in research reports.

The optimization means 60 optimizes the classification process of the region of interest so as to obtain a more accurate classification result.

Specifically, the optimizing means 60 acquires clinical verification data on the classification result of the region of interest, and uses the clinical verification data to perform the clinical interpretation used when the region of interest is classified by the discriminating means 30a. Optimizing the possible feature thresholds or weights of the transformation model.

More specifically, the optimizing means 60 acquires clinical verification data about the classification result of the region of interest, and when the region of interest is classified by the discrimination means 30a so that the classification result approaches the clinical verification data, Iteratively adjusts the thresholds of the clinically interpretable features and the weights of the transformation model used in , until the classification results converge.

For example, the optimization means 60 acquires the biopsy results of the subject corresponding to the medical image as clinical verification data for the classification results output from the discrimination means 30a, and adjusts the classification results to approximate the clinical verification data. , the classification thresholds of the clinically interpretable features and the weights of the transformation model are iteratively adjusted until the finally obtained classification results converge.

Here, as the clinical verification data, for example, other test data such as judgment by a doctor may be used. considered better.

Note that the convergence of the classification results is not limited to a complete match between the classification results and the clinical verification data, and may be attainment of a preset accuracy target.

FIG. 12 is a schematic diagram showing the optimization process based on the iterative method in the third embodiment.

As shown from left to right in FIG. 12, first, in step S51, image basic features extracted from a plurality of medical images (for example, 12 medical images) are input to the clinically interpretable feature generating means 12a. Thus, the clinically interpretable feature generating means 12a uses the transformation model to generate clinically interpretable features for each medical image. Here, the transformation model includes model weights such as w ₁ , w ₂ to w _n , b in equation (1) described above. Subsequently, in step S52, the clinically interpretable features are input to the discrimination means 30a, and the discrimination means 30a compares the clinically interpretable features with classification thresholds to make a benign/malignant prediction of the tumor, thereby Output benign/malignant prediction results for medical images.

Subsequently, the benign/malignant prediction results are input to the optimization means 60, and the optimization means 60 determines the accuracy target value of the classification results, for example, 80%, and the biopsy results of the subject for each medical image. etc. to obtain benign/malignant verification results.

The optimization means 60 then compares the prediction result and the verification result to calculate the accuracy of the benign/malignant prediction result for each medical image. For example, if the prediction results for 9 out of 12 medical images are the same as the verification results, the accuracy of the classification results is 75%, which is less than the target value of 80%. In this case, the optimization means 60 adjusts the model weights used in step S51 and the classification thresholds used in step S52, respectively, and executes the classification process again with each value after adjustment, resulting in a new prediction result again.

After that, the optimization means 60 compares the newly obtained prediction result with the verification result again to calculate the accuracy of the classification result. is less than the target value, the model weights and classification thresholds are adjusted again. In this way, the optimizer 60 will continue until the newly obtained accuracy is greater than or equal to the target value, or until the newly obtained accuracy can no longer approach the target value (accuracy converges). Iteratively perform adjustments to the model weights and thresholds until

By performing such iterative adjustments, the weights used in the transformation model and the classification thresholds used by the discriminator 30a can be optimized, thereby improving the accuracy of the classification results.

The display means 20 displays the clinically interpretable features generated by the clinically interpretable feature generating means 12a. In addition, the display means 20 may display a prediction result.

The overall flow of processing performed by the medical image processing apparatus 100b according to the third embodiment will be described below.

FIG. 13 is a flowchart for explaining the image discrimination processing method in the medical image processing apparatus according to the third embodiment.

As shown in FIG. 13, first, the model generating means 50a acquires sample data and uses machine learning to generate a clinically interpretable feature generation model (step S901).

Subsequently, the dividing means 11 acquires a medical image to be processed (step S902), performs organ division and tumor division on the medical image, thereby dividing the mammary gland and the tumor portion in the mammary gland, The tumor region is set as the region of interest (step S903).

Subsequently, the image basic feature extraction means 13 extracts image basic features from the divided medical image (step S904).

Subsequently, the clinically interpretable feature generation means 12a uses the transformation model generated by the model generation means 50a to generate clinically interpretable features from the image basic features, and the display means 20 displays the clinically interpretable features. (Step S905). Thereafter, the discriminating means 30a classifies the tumor region by comparing the clinically interpretable features and the classification threshold, and outputs the classification result (step S906).

Subsequently, the optimization means 60 acquires the verification result corresponding to the medical image and calculates the accuracy of the conversion model (step S907). After that, the optimization means 60 determines whether or not the accuracy of the conversion model has exceeded a preset target value, or whether or not the accuracy has converged (step S908). When the accuracy of the transformation model exceeds a preset target value, or when it is determined that the accuracy cannot be further optimized, that is, when the accuracy has converged (step S908: Yes), the optimization process ends.

On the other hand, if the accuracy of the conversion model does not exceed the preset target value and the accuracy of the conversion model exceeds the previously calculated accuracy, that is, if the accuracy has not converged (step S908: No ), the optimization means 60 adjusts the weights of the transformation model generated in step S901 and the classification thresholds of the clinically interpretable features used in step S906 (step S909). Then, the medical image processing apparatus 100b repeats steps S903 to S908 to re-execute classification processing on the same group of medical images, and makes determination again in step S908. In this manner, the medical image processing apparatus 100b performs iterative processing until the accuracy of the calculated transformation model exceeds the target value or until the accuracy converges.

According to the third embodiment described above, the image basis features obtained from the image are transformed into clinically interpretable features, and the regions of interest are classified based on the clinically interpretable features, thereby performing the same steps as the second embodiment. can produce the technical effect of

Further, according to the third embodiment, accurate verification results, such as biopsy results, are used to sequentially iteratively optimize the weights of the transformation model and the classification thresholds of the clinically interpretable features, so that allogeneic image In the classification process of , a better model and classification threshold can be obtained, and the accuracy of classification can be improved. Here, the identification performance is superior to subjective judgment by a doctor. Accordingly, the purpose of diagnostic support by computer can be achieved.

(Modification of the third embodiment)
In the third embodiment, a 2D medical image has been described as an example, but the medical image to be processed may be a 3D medical image.

Further, in the third embodiment, the model generating means 50a is omitted, the process is performed directly using the already generated transformation model, and the optimization means 60 directly optimizes the weights of the already generated transformation model. may be changed.

Further, in the third embodiment, the optimization means 60 simultaneously adjusts the classification threshold of the clinically interpretable feature and the weight of the conversion model so that the classification accuracy converges to the target value. is not limited to this. For example, the optimizer 60 may fix the weights and adjust only the classification thresholds for the clinically interpretable features, or fix the classification thresholds for the clinically interpretable features and adjust only the transform model weights. good too. Alternatively, these forms of regulation may be combined during iteration.

Further, the medical image processing apparatus described in each embodiment may further include report generating means for automatically generating a report describing the classification result of the medical image based on the classification result of the discriminating means. This makes it possible to check the classification results more easily.

Further, in the third embodiment, the optimization means 60 determines whether or not to repeat using the comparison between the calculated accuracy and the target value, but the embodiment is not limited to this. . For example, the optimization means 60 creates a loss function as shown in the following equation (3), presets a target value of the loss function, and compares the result of the loss function with the target value. It may be determined whether or not to repeat processing.

In the formula (3), y represents the biopsy result (true value) and takes 0 or 1 as its value, 0 representing benign and 1 representing malignant. In addition, y represents a prediction result, and takes 0 or 1 as its value, 0 representing benign and 1 representing malignant. Here it is proved that the smaller the value of L, the closer y is to y. In this case, the iterative process is performed until the value of L is less than or equal to the target value.

The embodiments described above are given as examples, and the configurations of the medical image processing apparatus and the medical image processing method are not limited to those described in each embodiment.

For example, in the above-described embodiments, each component of each device illustrated is functionally conceptual, and does not necessarily need to be physically configured as illustrated. That is, the specific form of distribution or integration of each device is not limited to the illustrated one, and all or part of them can be functionally or physically distributed or distributed in arbitrary units according to various loads, usage conditions, etc. Can be integrated and configured. Further, each processing function performed by each device may be implemented in whole or in part by a processor and a program analyzed and executed by the processor, or may be implemented as hardware by wired logic.

Moreover, the medical image processing apparatus and the medical image processing method described in the above-described embodiments can be realized, for example, by executing a program prepared in advance on a computer such as a personal computer or a workstation. This program can be distributed via a network such as the Internet. In addition, this program can be read from a hard disk, flexible disk (FD), CD (Compact Disk)-ROM (Read Only Memory), MO (Magneto-Optical disk), DVD (Digital Versatile Disk), and other computer-readable media. It can be recorded in a possible non-temporary recording medium, read out from the recording medium by a computer, and executed.

Further, the generation means, division means, clinically interpretable feature generation means, display means, discrimination means, correction means, model generation means, image basic feature extraction means, and optimization means described in the above-described embodiments can be performed by, for example, a processor. Realized. In that case, the processing functions possessed by each means are stored in a memory circuit in the form of a computer-executable program, for example. Each means implements a processing function corresponding to each program by reading and executing each program stored in the storage circuit. In other words, the medical image processing apparatus in each embodiment has the functional configuration shown in FIGS. 1, 7, and 11 in a state where each means reads each program. In that case, the processing of each step shown in FIGS. 5, 6, 10, and 13 is implemented by, for example, each means reading out a program corresponding to each processing function from a storage circuit and executing it.

In addition, in each embodiment, each processing unit is not limited to being implemented by a single processor, but is configured by combining a plurality of independent processors, and each processor implements each processing function by executing a program. It may be something to do. Also, the processing functions of each processing unit may be appropriately distributed or integrated in a single or a plurality of processing circuits and implemented. Moreover, the processing function of each processing unit may be realized by only hardware such as circuits, only software, or a mixture of hardware and software. Also, although the example in which the programs corresponding to each processing function are stored in a single storage circuit has been described here, the embodiments are not limited to this. For example, programs corresponding to each processing function may be distributed and stored in a plurality of storage circuits, and each processing unit may read and execute each program from each storage circuit.

In addition, the term "processor" used in this specification is, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or an application specific integrated circuit (ASIC), a programmable Means circuits such as logic devices (for example, Simple Programmable Logic Device (SPLD), Complex Programmable Logic Device (CPLD), and Field Programmable Gate Array (FPGA)) do. Here, instead of storing the program in the memory circuit, the program may be configured to be directly embedded in the circuit of the processor. In this case, the processor implements its functions by reading and executing the program embedded in the circuit. Further, each processor of the present embodiment is not limited to being configured as a single circuit for each processor, but may be configured as one processor by combining a plurality of independent circuits to realize its function. good.

Further, among the processes described in the above embodiments, all or part of the processes described as being automatically performed can be manually performed, or the processes described as being manually performed can be performed manually. can also be performed automatically by known methods. In addition, information including processing procedures, control procedures, specific names, and various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified.

Various data handled in this specification are typically digital data.

According to at least one embodiment described above, the accuracy of quantitative analysis of medical images can be improved.

While several embodiments have been described, these embodiments are provided by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, changes, and combinations of embodiments can be made without departing from the scope of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, as well as the scope of the invention described in the claims and equivalents thereof.

100, 100a, 100b medical image processing apparatus 10, 10a generation means 11 division means 12, 12a clinically interpretable feature generation means 20 display means 30, 30a discrimination means 40 correction means 50, 50a model generation means 13 image basic feature extraction means 60 optimization measures

Claims (15)

generating means for generating a first feature representing an image feature from a medical image containing a region of interest;
display means for displaying the first characteristic;
A medical image processing apparatus, comprising: discrimination means for discriminating the region of interest in the medical image based on the first characteristic and outputting a discrimination result regarding the region of interest. further comprising modifying means for modifying said first characteristic displayed by said display means based on user input;
The medical image processing apparatus according to claim 1. The generating means extracts image basic features from a medical image containing the region of interest, and generates the first features based on the image basic features.
The medical image processing apparatus according to claim 1 or 2. The differential result of the region of interest is the presence or absence of a lesion or the degree of a lesion.
The medical image processing apparatus according to claim 1 or 2. said first characteristic is a clinically interpretable characteristic defined by a clinical practice guideline;
The medical image processing apparatus according to claim 1 or 2. The generating means is
a dividing unit that acquires the medical image and performs predetermined dividing processing on the medical image to determine the region of interest in the medical image;
clinically interpretable to generate the clinically interpretable features of the region of interest from the medical image containing the region of interest using a second discriminant model that takes as input a medical image containing the region of interest and outputs clinically interpretable features. and
The discriminating means uses a first discriminant model that outputs a discrimination result regarding the region of interest with image features as input, and uses the clinically interpretable features generated by the clinically interpretable feature generating means as the image features. By inputting to the discrimination model of 1, outputting the discrimination result for the region of interest,
The medical image processing apparatus according to claim 5. By acquiring sample image data for which the discrimination result is verified and image features including visual judgment image features obtained for the sample image data as first learning data and performing training, the first discrimination A model generating means for generating a model,
Model generation means for generating the second discriminant model by performing training using the sample image data of the first learning data and the clinically interpretable features corresponding to the discrimination result as second learning data. further comprising
The medical image processing apparatus according to claim 6. The generating means is
a dividing unit that acquires the medical image and performs predetermined dividing processing on the medical image to determine the region of interest in the medical image;
image basic feature extraction means for extracting image basic features from a medical image including the region of interest;
clinically interpretable to generate the clinically interpretable features of the region of interest from the image basis features extracted by the image basis feature extracting means using a transform model that associates the image basis features and clinically interpretable features. and
The discriminating means classifies the region of interest based on the clinically interpretable features generated by the clinically interpretable feature generating means, and outputs a classification result regarding the region of interest as a discrimination result.
The medical image processing apparatus according to claim 5. further comprising model generation means for obtaining sample image data and clinically interpretable features corresponding to the sample image data and using machine learning to generate the transformation model;
The medical image processing apparatus according to claim 8. The clinically interpretable features include a clinically interpretable morphological feature that is a clinically interpretable morphological feature and a clinically interpretable kinetic feature that is a clinically interpretable kinetic feature,
The medical image processing apparatus according to claim 5. wherein the transformation model is a polynomial model,
The medical image processing apparatus according to claim 8. The discriminating means compares the clinically interpretable features generated by the clinically interpretable feature generating means with a threshold of the clinically interpretable features, and outputs a classification result regarding the region of interest based on the matching result. ,
The medical image processing apparatus according to claim 8. Obtaining clinical validation data for the classification results, and using the clinical validation data to optimize clinically interpretable feature thresholds or weights of the transformation model used when the region of interest is classified by the discriminating means. further comprising an optimization means for optimizing
The medical image processing apparatus according to claim 8. clinically interpretable feature thresholds and the transformation used when the region of interest is classified by the discriminating means to obtain clinical validation data for the classification result, and to bring the classification result closer to the clinical validation data; further comprising an optimizer that iteratively adjusts model weights until the classification results converge;
The medical image processing apparatus according to claim 8. generating means for generating a first feature representing an image feature from a medical image including a region of interest;
display means displaying said first characteristic;
A medical image processing method, comprising: discriminating means discriminating the region of interest in the medical image based on the first feature, and outputting a discrimination result regarding the region of interest.

Images