JP6927211B2 - Image diagnostic learning device, diagnostic imaging device, method and program - Google Patents

Description

The present invention relates to a technique for detecting an abnormal region in a diagnosis target image in which a diagnosis target is captured.

There is a field in which it is required to detect a region where the diagnosis target may have an abnormality in the image in which the diagnosis target is captured. Hereinafter, the image in which the diagnosis target is copied is also described as the diagnosis target image. In addition, the area where the diagnosis target may have an abnormality is also described as an abnormal area. For example, in a medical examination to detect lesions such as cancer and polyps in the digestive system, a specialist visually confirms an endoscopic image of the visceral wall surface taken with a gastrointestinal endoscopic camera. I came. The endoscopic camera has a tube shape, and the internal organ wall surface is photographed by inserting it from outside the body and inserting / removing it or rotating it. In addition, since the inside of the body is dark, photography is performed while illuminating the internal organ wall surface with the illumination at the tip of the endoscopic camera.

However, due to various factors, the image to be diagnosed may have a region in which the original diagnosis target is not sufficiently captured. For example, in the case of taking a picture with the above-mentioned gastrointestinal endoscopic camera, if an obstacle such as a digestive substance remaining in the internal organs interferes, work such as watering the internal organ wall surface is performed. However, in this case, specular reflection of light is likely to occur, so that the internal organ wall surface is frequently shiny. Searching for lesions from endoscopic images that include shiny spots is a difficult task that requires specialized knowledge and high attention. In this way, even a specialist may miss an abnormal region from a diagnostic target image including a region in which the original diagnostic target is not sufficiently captured, unless the specialist observes it carefully over time.

Therefore, there is a method of obtaining an image to be diagnosed by using a special photographing method in which the image to be diagnosed is photographed more clearly. For example, there is an imaging method that emphasizes blood vessels and glandular structures in an endoscopic image by irradiating the internal organ wall surface with light of a plurality of specific frequencies when photographing with an endoscopic camera. This imaging method is called Narrow Band Imaging (registered trademark) (see, for example, Non-Patent Document 1). Hereinafter, Narrow Band Imaging will also be referred to as NBI. Endoscopic images by NBI make it easier for doctors to observe lesions. However, in this case, since the amount of light is smaller than that of a normal image, it is not realistic to perform normal imaging by NBI. As another example, an endoscopic camera called a magnifying endoscope may be used to zoom in and take a picture after finding a part that seems to be a lesion.

On the other hand, related technologies that support the detection of abnormal regions by recognizing the contents of the image to be diagnosed by a computer are known. For example, the related technique of Non-Patent Document 2 uses an endoscopic image obtained by NBI as an input to determine the stage (adenoma, advanced cancer, etc.) of a lesion area previously found in the endoscopic image. Diagnose. By making good use of this result, it is expected that it will be easier for doctors to judge the presence or absence of lesions. As described above, it is known that relatively high identification accuracy can be obtained when the recognition process is performed by inputting the endoscopic image by NBI. In the related technique of Non-Patent Document 2, a disease is identified by using a feature amount related to the shape of a blood vessel. Information designed by the technology developer is used as the feature quantity.

Further, Non-Patent Document 3 describes a method of learning the extraction of a feature amount used when detecting an abnormal region in a diagnosis target image by a computer using a large amount of actual data. This is a method using Convolutional Neural Networks (hereinafter, also referred to as CNN), which is a type of neural network (hereinafter, also referred to as NN). According to Non-Patent Document 3, it is reported that a highly accurate identification rate can be achieved as compared with the case where the feature quantity manually designed by the technical developer is used.

Here, the NN has a learning process and an identification process. These processes will be outlined with reference to FIG. FIG. 14 is a diagram showing the network structure of NN. In FIG. 14, circles represent NN units. Further, the straight line indicates that the units are connected. The unit calculates the output value x'according to the equation x'= f (w · x + b) with respect to the input value x using the weight w, the bias b, and the activation function f. The w and b can take different values for each bond between the units. Here, "・" represents a product. Further, the function f () is an arbitrary monotonically increasing function. The number of units and which unit is to be combined with which unit are designed in advance by the technical developer. The CNN is configured to be able to perform a process equivalent to a general convolution process for an image by adding a restriction that some of these parameters are shared in the layer. There are several known types of CNN intermediate layers. For example, there are a Convolution layer, a Pooling layer, a fully combined layer (linear combination layer), and the like.

The example of NN shown in FIG. 14 has a multi-layered network structure. For example, the pixel value of the image is input to each unit 1 to P of the input layer (P is an integer of 1 or more), and each unit 1 to N of the output layer (N is an integer of 1 or more) passes through the units of the intermediate layer. The identification result is output from. Class information, which is a classification of the abnormal area for which the identification result is to be obtained, is assigned to each unit of the output layer. Examples of such class information include classifications such as non-lesion, polyps, and cancer. In FIG. 14, this class information may also be referred to as a label, category or teacher signal in the field of image recognition. In the learning process, a large amount of data prepared in advance is repeatedly input to the input layer, and calculations based on network parameters are performed in order. This calculation is called the Forward calculation. In addition, the parameter group is updated while calculating in the reverse direction so as to minimize the difference (called loss) between the value of the output layer and the target value by the Forward calculation. This calculation is called the Backward calculation. In the identification process, the Forward calculation is executed using the parameter group obtained in the learning process, and the identification result is output. The CNN performs a convolution operation in the Convolution layer and a sampling process in the Pooling layer. These processes can be regarded as processes for extracting features from the input image. Further, the processing in the subsequent layers can be regarded as an identification processing. That is, it can be said that by learning one CNN, it is possible to obtain the feature amount extraction design (feature amount extraction parameter) and the identification parameter at the same time.

Olympus Corporation of the Americas, “Narrow Band Imaging”, [online], [Searched April 19, 2016], Internet, <http://medical.olympusamerica.com/technology/narrow-band-imaging> Thomas Stehle, Roland Auer, Alexander Behrens, Jonas Wulff, TilAach, Ron Winograd, Christian Trautwein, Jens Tischendorf, “Classification of Colon Polyps in NBI Endoscopy Using Vascularization Features”, Proc. Of Medical Imaging 2009, Computer—Aided Diagnosis. Vol. 7260, 72602S, 2009 Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton, “ImageNet classification with deep convolutional neural networks” Advances in neural information processing systems, 2012.

However, the above-mentioned related technology has the following problems.

The related technology of Non-Patent Document 2 uses the endoscopic image obtained by NBI of Non-Patent Document 1 for identification. Further, the related technique of Non-Patent Document 3 extracts a feature amount capable of identifying an object by using an endoscopic image by NBI in which blood vessels and glandular structures are large and clearly photographed.

However, as described above, it is not realistic to perform imaging by NBI of Non-Patent Document 1 in a normal manner. As described above, when the image to be diagnosed obtained by a normal photographing method that does not depend on a special function is input, it is difficult to accurately detect an abnormal region by using the related technique of Non-Patent Document 2. In addition, when a diagnostic target image obtained by a normal imaging method having no special function is input, it is not possible to extract a feature amount capable of identifying the target even by using the related technology of Non-Patent Document 3. difficult. Furthermore, the diagnostic target image often includes a region in which the original diagnostic target is not sufficiently captured, such as the above-mentioned shiny spot. When such a diagnosis target image is input, it is more difficult to extract a feature amount capable of identifying the target by using the related technique of Non-Patent Document 3. As a result, it is difficult to accurately detect the abnormal region even by using the related technology of Non-Patent Document 3.

The present invention has been made to solve the above-mentioned problems. That is, we provide a technology to detect abnormal areas more accurately even if the diagnosis target image obtained by a normal imaging method that does not rely on special functions includes a region where the original diagnosis target is not sufficiently captured. The purpose is to do.

The diagnostic imaging learning device of the present invention includes a CNN configuration storage unit that stores a network configuration of a convolutional neural network (CNN), a parameter storage unit that stores parameters of a learning target in the CNN, and a learning target in which a diagnosis target is copied. In the image, an inappropriate region detection unit that detects an inappropriate region, which is an region that is not suitable for identifying an abnormal region in which the diagnosis target may have an abnormality, based on a predetermined criterion, and the learning image. In the network configuration of the CNN in which the above is input, the inappropriate area invalidation unit that invalidates the unit corresponding to the inappropriate area among the units of the input layer and the unit of the input layer corresponding to the inappropriate area are invalid. The calculation of the CNN is executed using the parameters in the converted state, and the loss value is calculated based on the calculation result and the information representing the abnormality of the diagnosis target given in advance to the learning image. A loss value calculation unit for calculation and a parameter update unit for updating the parameters of the parameter storage unit based on the loss value are provided.

Further, the diagnostic imaging apparatus of the present invention includes a parameter storage unit that stores the parameters of the CNN that have been updated by applying the diagnostic imaging learning apparatus described above to one or a plurality of learning images, and a parameter storage unit that stores the parameters. By inputting the CNN configuration storage unit that stores the network configuration of the CNN used by the diagnostic imaging learning device and the information based on the diagnostic target image on which the diagnostic target is copied into the CNN to execute the calculation. A CNN identification unit that identifies an abnormal region in which the diagnosis target may have an abnormality in the diagnosis target image is provided.

Further, in the method of the present invention, the computer device determines an inappropriate region, which is a region that is not suitable for identifying an abnormal region in which the diagnostic target may have an abnormality, in a learning image in which the diagnostic target is captured. In the network configuration of the Convolutional Neural Network (CNN), which is detected based on a predetermined criterion and the learning image is input, the unit corresponding to the inappropriate region among the units of the input layer is invalidated, and the inappropriate region is invalidated. The calculation of the CNN is executed in a state where the unit of the input layer corresponding to the above is invalidated, and the calculation result is based on the calculation result and the information representing the abnormality of the diagnosis target given in advance to the learning image. Then, the loss value is calculated, and the learning target parameter in the CNN is updated based on the loss value.

In addition, the program of the present invention sets an inappropriate region, which is an unsuitable region for identifying an abnormal region in which the diagnostic target may have an abnormality, as a predetermined reference in a learning image in which the diagnostic target is captured. In the network configuration of the Convolutional Neural Network (CNN) that detects based on the above and inputs the learning image, the unit corresponding to the inappropriate region among the units of the input layer is invalidated, and the unit corresponding to the inappropriate region is described. The calculation of the CNN is executed with the unit of the input layer disabled, and the loss value is based on the calculation result and the information representing the abnormality of the diagnosis target given in advance to the learning image. Is calculated, and the computer device is made to update the parameter to be learned in the CNN based on the loss value.

In addition, another method of the present invention includes the parameters of the CNN updated by the computer device performing the above method on one or more learning images, and when the parameters are updated. By using the network configuration of the CNN used and inputting information based on the diagnosis target image on which the diagnosis target is copied into the CNN and executing the calculation, the diagnosis target is abnormal in the diagnosis target image. Identify areas of anomaly that may have.

Further, another program of the present invention is used for updating the parameters of the CNN updated by causing a computer device to execute the above program for one or more learning images, and updating the parameters. By inputting information based on the diagnosis target image on which the diagnosis target is copied into the CNN and executing the calculation using the network configuration of the CNN, the diagnosis target becomes abnormal in the diagnosis target image. Have the computer device perform to identify the anomalous areas that it may have.

The present invention is a technique for more accurately detecting an abnormal region even if the diagnostic target image obtained by a normal imaging method that does not rely on a special function includes a region in which the original diagnostic target is not sufficiently captured. Can be provided.

It is a block diagram which shows the structure of the image diagnosis learning apparatus as 1st Embodiment. It is a figure which shows an example of the hardware configuration of the image diagnosis learning apparatus as 1st Embodiment. It is a figure which schematically explains the invalidation of an inappropriate region in 1st Embodiment. It is a flowchart explaining the operation of the image diagnosis learning apparatus as the 1st Embodiment. It is a block diagram which shows the structure of the image diagnosis learning apparatus as a 2nd Embodiment. It is a figure which shows typically an example of the endoscopic image which the shiny occurred in the 2nd Embodiment. It is a figure which shows an example of the luminance histogram of the endoscopic image in which the shine occurred in the 2nd Embodiment. It is a flowchart explaining the operation of the image diagnosis learning apparatus as a 2nd Embodiment. It is a block diagram which shows the structure of the diagnostic imaging apparatus as a 3rd Embodiment. It is a figure which shows an example of the hardware configuration of the diagnostic imaging apparatus as a 3rd Embodiment. It is a flowchart explaining the operation of the diagnostic imaging apparatus as a 3rd Embodiment. It is a block diagram which shows the structure of the diagnostic imaging apparatus as a 4th Embodiment. It is a flowchart explaining the operation of the diagnostic imaging apparatus as a 4th Embodiment. It is a figure which shows typically the network structure of the neural network in the related technique.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(First Embodiment)
FIG. 1 shows a functional block configuration of the diagnostic imaging learning device 1 as the first embodiment. In FIG. 1, the image diagnosis learning device 1 includes a CNN configuration storage unit 11, a parameter storage unit 12, an inappropriate area detection unit 13, an inappropriate area invalidation unit 14, a loss value calculation unit 15, and parameter update. A unit 16 is provided. The image diagnosis learning device 1 is a device that learns parameters used in a CNN that identifies an abnormal region in an image in which a diagnosis target is captured. Here, the abnormal region means an region in which the diagnostic target may have an abnormality in the image on which the diagnostic target is captured.

Here, the diagnostic imaging learning device 1 can be configured by hardware elements as shown in FIG. In FIG. 2, the image diagnosis learning device 1 is composed of a computer device including a CPU (Central Processing Unit) 1001 and a memory 1002. The memory 1002 is composed of a RAM (Random Access Memory), a ROM (Read Only Memory), an auxiliary storage device (hard disk, etc.) and the like. In this case, the CNN configuration storage unit 11 and the parameter storage unit 12 are configured by the memory 1002. Further, the inappropriate area detection unit 13, the inappropriate area invalidation unit 14, the loss value calculation unit 15, and the parameter update unit 16 are composed of a CPU 1001 that reads and executes a computer program stored in the memory 1002. The hardware configuration of the diagnostic imaging learning device 1 and its functional blocks is not limited to the above configuration.

The CNN configuration storage unit 11 stores information representing the network configuration of the Convolutional Neural Network (CNN). The information representing the network configuration may include, for example, information such as the image size and the number of channels of the input layer, the type and size and processing parameters of each of the plurality of intermediate layers, and the number of units of the output layer. As the type of the intermediate layer, as described above, there are a Convolution layer, a Pooling layer, a fully combined layer (linear combination layer) and the like. Further, as an example of the processing parameter, there are the following. For example, there are parameters such as the size of the Convolution kernel (width, height, number of channels), the width of the stride when calculating the Convolution, and whether or not there is a process of filling the image edge with a specific value in order to adjust the size when calculating the Convolution.

The parameter storage unit 12 stores the parameter group to be learned in the CNN described above. The parameters to be learned include, for example, the weights and biases used in the calculation of each unit. These parameters are regarded as parameters for extracting the feature amount of the abnormal region or for identifying the abnormal region using the feature amount.

The initial value of the parameter stored in the parameter storage unit 12 may be a random number. Further, the initial value of the parameter may be a value learned for other image recognition applications. For example, if the initial value of the parameter is a parameter for recognizing a general object such as an animal or a vehicle, or a parameter learned for identifying a pattern in an image, a high learning effect may be obtained, which is useful. Is.

The inappropriate region detection unit 13 detects an inappropriate region, which is an inappropriate region for identifying an abnormal region, in a learning image in which a diagnosis target is captured, based on a predetermined criterion. As an inappropriate region, for example, a region in which the original diagnosis target is not sufficiently captured is applied. In this case, the inappropriate region detection unit 13 may detect as an inappropriate region a region that satisfies the condition that it can be determined that the original diagnosis target is not sufficiently captured.

The learning image may be, for example, a partial image of the image taken by the diagnosis target. In addition, it is assumed that the learning image is provided with information indicating anomalies related to the diagnostic target being copied as correct answer information. The information indicating the anomaly may be, for example, information indicating whether or not there is a possibility of having the anomaly. Further, the information indicating the anomaly may be, for example, information indicating the type of the information when there is a possibility of having the anomaly. Further, the information indicating the anomaly may be information indicating the high possibility of the anomaly. However, the content of the information indicating the abnormality is not limited to these.

The inappropriate area invalidation unit 14 invalidates the unit corresponding to the inappropriate area among the units of the input layer in the network configuration of the CNN in which the learning image is input. The network configuration of the CNN for inputting the learning image is stored in the CNN configuration storage unit 11 described above. Here, the invalidation of the input layer unit corresponding to the inappropriate region will be described with reference to the schematic diagram of FIG. Here, it is assumed that the network configuration of CNN has the structure shown in FIG. Further, it is assumed that each pixel value of the learning image is input to the input layer of the CNN. In this case, the unit into which the pixels included in the inappropriate region are input is invalidated as shown in FIG. The triangular mark in the figure represents the invalidated unit. In this example, pixels 1 and 3 are included in the inappropriate region. Therefore, each unit in which the pixel 1 and the pixel 3 are input is invalidated. In the CNN calculation, the connection from the invalidated unit to the unit in the next layer is treated as nonexistent. The dashed line in the figure represents a connection that is treated as non-existent. The invalidation by the inappropriate area invalidation unit 14 is temporary. That is, even if some input layer units are invalidated for a certain learning image, the inappropriate area invalidation unit 14 first invalidates the invalidated units when processing another different learning image. unlock. Then, the inappropriate region invalidation unit 14 may invalidate the unit of the input layer corresponding to the inappropriate region in the learning image again.

The loss value calculation unit 15 calculates the loss value for learning the CNN by using the parameters in the state where the unit of the input layer corresponding to the inappropriate region is invalidated. Specifically, the loss value calculation unit 15 applies the parameters stored in the parameter storage unit 12 in a state where the unit of the input layer corresponding to the inappropriate area is invalidated, and performs the Forward calculation. Then, the loss value calculation unit 15 calculates the loss value based on the calculation result and the information indicating the abnormality given in advance to the learning image.

Here, the loss value is an index for optimization calculated at the time of learning the neural network, and any index can be applied. As an example of a loss value that is often used, the value group (calculation result) output by the Forward calculation is converted into a real value that can take a value in the range of 0 to 1 using the SoftMax function, and then the correct answer class. Is 1.0 and the difference is used as the loss value. The above-mentioned "information indicating the anomaly given in advance to the learning image" is information indicating the anomaly of the correct answer, and corresponds to 1.0 of the correct answer class. However, the calculation method of the loss value is not limited to this.

The parameter update unit 16 updates the parameters to be learned stored in the parameter storage unit 12 based on the loss value. The parameters to be updated are, for example, weights and biases, as described above. Further, for example, the parameter updating unit 16 may update the parameters so as to minimize the loss by using the cumulative value of the loss values calculated for each of the plurality of learning images. Here, the plurality of learning images may be a plurality of partial images of the images taken by the diagnosis target. For example, a stochastic descent method can be applied to the minimization (optimization) method at this time. However, the minimization method at this time is not limited to this method, and any method can be applied.

The operation of the diagnostic imaging learning device 1 configured as described above will be described with reference to FIG.

In FIG. 4, first, the inappropriate region detection unit 13 detects an inappropriate region from the learning image based on a predetermined reference (step S11).

Next, the inappropriate region invalidation unit 14 invalidates the unit corresponding to the inappropriate region among the units of the input layer of the CNN (step S12).

Next, the loss value calculation unit 15 executes the calculation of the CNN in a state where the unit of the input layer corresponding to the inappropriate region is invalidated. Then, the loss value calculation unit 15 calculates the loss value based on the calculation result and the information representing the abnormality given in advance to the learning image (step S13).

Next, the parameter update unit 16 updates the parameters to be learned based on the calculated loss value (step S14).

With the above, the image diagnosis learning device 1 ends the operation.

Next, the effect of the first embodiment will be described.

The diagnostic imaging learning device as the first embodiment includes an area in which the original diagnostic target is not sufficiently captured in an image in which the diagnostic target is captured by a normal imaging method that does not rely on a special function. However, the abnormal region can be detected more accurately.

The reason will be explained. In the present embodiment, the inappropriate region detection unit detects an inappropriate region, which is a region unsuitable for identifying an abnormal region, in a learning image in which a diagnosis target is captured, based on a predetermined criterion. Then, the inappropriate region invalidation unit invalidates the unit corresponding to the inappropriate region among the units of the input layer of the CNN based on the network configuration of the CNN into which the learning image is input. Then, the loss value calculation unit executes the calculation of CNN in a state where the unit of the input layer corresponding to the inappropriate region is invalidated. Then, the loss value calculation unit calculates the loss value based on the calculation result of CNN and the information indicating the abnormality of the correct answer given in advance to the learning image. Then, the parameter update unit updates the parameter to be learned in the CNN based on the loss value.

As described above, in this embodiment, the parameters of the learning target in the CNN are determined based on the learning image. Therefore, in the present embodiment, even a learning image obtained from an image obtained by a photographing method that does not rely on a special function, a sufficient feature amount for identifying an abnormal region is automatically extracted. Further, in the present embodiment, an inappropriate region in which the original diagnosis target is not sufficiently captured is automatically detected, the inappropriate region is invalidated, and the CNN parameter is learned. Therefore, in this embodiment, the influence of the inappropriate region can be suppressed and the CNN can be learned accurately. Further, in the present embodiment, by operating on a larger number of learning images, it is possible to automatically extract features that enable more accurate identification and learn parameters. As a result, in the present embodiment, it is possible to provide more accurate parameters by the CNN that identifies the abnormal region in the image in which the diagnosis target is captured, and it is possible to detect the abnormal region with higher accuracy.

(Second Embodiment)
Next, the second embodiment will be described in detail with reference to the drawings. In this embodiment, the internal organ wall surface is applied as a diagnostic target. Further, as a learning image, an image based on an endoscopic image taken by an endoscopic camera on the internal organ wall surface is applied. In each drawing referred to in the description of the present embodiment, the same components as those in the first embodiment and the steps operating in the same manner are designated by the same reference numerals, and detailed description in the present embodiment is omitted. do.

First, FIG. 5 shows the configuration of the diagnostic imaging learning device 2 as the second embodiment. In FIG. 5, the image diagnosis learning device 2 includes an inappropriate area detection unit 23 instead of the inappropriate area detection unit 13 for the image diagnosis learning device 1 as the first embodiment, and further, a perturbation unit. The difference is that 27 is provided. The image diagnosis learning device 2 and each functional block thereof can be configured by the same hardware elements as the image diagnosis learning device 1 as the first embodiment described with reference to FIG. However, the hardware configuration of the diagnostic imaging learning device 2 and its functional blocks is not limited to the above configuration.

The perturbation unit 27 generates a learning image by performing a perturbation process on the learning sample of the endoscopic image. The perturbation process is a process that gives minute fluctuations in position, scale, and rotation angle.

For example, a patch image in which the endoscopic image is divided is input to the perturbation unit 27. The patch image is a partial image included in the endoscopic image. Further, it is assumed that lesion class information representing the lesion class is added to each patch image as correct answer information. In this case, the perturbation unit 27 generates a learning image obtained by performing perturbation processing on such a patch image.

The lesion class information may be, for example, information representing either a lesion or a non-lesion. In this case, the patch image including the lesion may be given a numerical value of 1 as the lesion class information indicating the “lesion”. Further, a patch image that does not include a lesion may be given a numerical value of 0 as lesion class information indicating "non-lesion".

Further, for example, the lesion class information may be information indicating either a lesion type 1 to n (n is a positive integer of 2 or more) and a non-lesion. In this case, the patch image including the lesion of type i (i is an integer of 1 to n) may be given a numerical value i as the lesion class information indicating the “lesion portion (type i)”. Further, a patch image that does not include a lesion may be given a numerical value of 0 as lesion class information indicating "non-lesion".

The inappropriate region detection unit 23 detects a shiny portion as an inappropriate region from the learning image. As described above, the shiny spot is generated by performing work such as sprinkling water on the internal organ wall surface when taking a picture with an endoscopic camera. Any method may be used to detect the shiny part. For example, the inappropriate region detection unit 23 may detect the shiny portion by performing the brightness value threshold processing. The detection of the shiny spot using the luminance value will be described with reference to FIGS. 6 to 7. FIG. 6 is a diagram schematically showing an example of an endoscopic image in which shine has occurred. As shown in FIG. 6, when the internal organ wall surface is shiny, the portion in the endoscopic image becomes bright white. Further, FIG. 7 is an example of a luminance histogram of an endoscopic image in which shine has occurred. As shown in FIG. 7, in the luminance distribution histogram, the luminance value of the shiny portion becomes extremely high. That is, the inappropriate region detection unit 23 can determine that a pixel having a brightness exceeding the threshold value is a pixel at a shiny spot. Further, the inappropriate region detection unit 23 may determine, for example, a partial region in which the appearance rate of the luminance value higher than the threshold value satisfies a predetermined criterion as a shiny portion. Further, the inappropriate region detection unit 23 may detect a shiny spot by using a process of searching for an isolated point where the pixel of interest has an extremely high brightness as compared with the peripheral pixels.

The operation of the diagnostic imaging learning device 2 configured as described above will be described with reference to FIG. In the training process of the neural network, a method of selecting a subset from the training data set and updating the parameters so as to minimize the cumulative value of the loss calculated from them is to obtain the effect of improving the discrimination accuracy. Often used. Therefore, here, a learning data set consisting of patch images cut out from an endoscopic image and to which lesion class information is added, and an operation example using a subset selected from the learning data set will be described.

Information on such training datasets and subsets may be stored in memory 1002. In this case, the diagnostic imaging learning device 2 may read the information of these learning data sets and subsets from the memory 1002 and execute the following operations.

The size of the patch image may be any size. For example, when the width and height are 256 pixels and the RGB components of the color are three (= 3 channels), the number of units in the input layer of the CNN is 256 × 256 × 3. Further, when the lesion class to be identified is "non-lesion", "lesion (adenoma)" and "lesion (advanced cancer)", the number of units in the output layer is 3.

Further, the network configuration of CNN may be, for example, a set of a plurality of Convolution layers and one Pooling layer, and a plurality of these sets are connected.

However, in the following operations, the number of lesion classes to be identified, the size of the patch image, the number of units in the input layer and output layer of the CNN, the network configuration, and the like are not limited to the above examples.

In FIG. 8, first, the perturbation unit 27 takes one of the patch images included in this subset as an input and generates a learning image perturbed with respect to position, scale, rotation, and the like (step S21). For example, the perturbation unit 27 may acquire a patch image created in a large size, and cut out an image in which the perturbation is performed from the acquired patch image as a learning image.

Next, the inappropriate region detection unit 23 detects a shiny portion as an inappropriate region from the learning image (step S22).

Next, the inappropriate region invalidation unit 14 operates substantially in the same manner as in step S12 in the first embodiment, and temporarily sets the unit corresponding to the inappropriate region in the input layer of the CNN into which the learning image is input. Disable to. Here, the inappropriate area invalidation unit 14 temporarily invalidates the unit corresponding to the shiny part (step S23).

Next, the loss value calculation unit 15 operates in substantially the same manner as in step S13 in the first embodiment, and performs CNN Forward calculation. Here, the loss value calculation unit 15 performs the CNN Forward calculation in a state where the unit of the input layer corresponding to the shiny part is temporarily invalidated. Then, the loss value calculation unit 15 calculates the loss value using the calculation result and the correct lesion class information given to the learning image (step S24).

Next, if there is another patch image included in this subset (Yes in step S25), the diagnostic imaging learning device 2 repeats the process from step S21.

On the other hand, if there is no other patch image included in this subset (No in step S25), the parameter update unit 16 performs a process of updating the parameter to be learned. Specifically, the parameter update unit 16 may update the weight and bias based on the cumulative value of the loss value calculated for each learning image included in this subset (step S26).

Next, if there is another subset (Yes in step S27), the diagnostic imaging learning device 2 repeats the process from step S21.

On the other hand, if there is no other subset (No in step S27), the diagnostic imaging learning device 2 ends the operation.

The image diagnosis learning device 2 may perform the same operation for other learning data sets, and may end the operation when the updated loss value or the identification error rate does not decrease.

This completes the description of the operation of the image diagnosis learning device 2.

Next, the effect of the second embodiment will be described.

The diagnostic imaging learning device as the second embodiment includes a shiny portion in an endoscopic image obtained by a normal imaging method that does not depend on a special function such as an NBI endoscope or a magnifying endoscope. Even so, it is possible to detect lesion candidate candidates more accurately.

The reason will be explained. This is because the present embodiment has the following configurations in addition to the same configurations as those of the first embodiment. That is, the perturbation unit generates a learning image by performing a perturbation process on the patch image in which the endoscopic image is divided and the lesion class information is added. Then, the inappropriate region detection unit detects the shiny portion as an inappropriate region in the learning image.

Thereby, the present embodiment updates the weights and biases constituting the CNN as the parameters for feature extraction based on the set of learning images based on the endoscopic images. Therefore, even in a general endoscopic image that does not depend on an NBI endoscope or a magnifying endoscope, a region of a normal visceral wall surface is distinguished from an abnormal region, that is, an abnormal region that may be a lesion. It is possible to automatically extract sufficient features.

Further, in the present embodiment, since the shiny part is automatically detected and invalidated to learn the CNN parameter, the influence of the abnormal value generated in the shiny part in the endoscopic image is suppressed and the CNN is learned. Can be made to. That is, in the present embodiment, the parameters can be learned from the learning image including the shiny part in the endoscopic image without any adverse effect.

As a result, the CNN discriminator using the parameters learned according to the present embodiment can more accurately detect the candidate lesion even if it is an endoscopic image that does not depend on a special function.

In addition, this embodiment can be carried out even if the perturbation part is not included. However, since the perturbation unit perturbs the shiny part by executing the perturbation process, it has the effect of not biasing the shiny part and further suppressing the adverse effect of the shiny part to learn the parameters. Such an effect of the perturbation part is particularly remarkable when operating with a small training data set.

(Third Embodiment)
Next, the third embodiment will be described in detail with reference to the drawings. In this embodiment, an example of an image diagnostic apparatus for identifying an abnormal region in a diagnostic target image will be described using the CNN parameters updated by the first embodiment. The image to be diagnosed is an image on which the diagnosis target is captured, and refers to an image of the target for identifying an abnormal region.

First, FIG. 9 shows the configuration of the diagnostic imaging apparatus 3 as the third embodiment. In FIG. 9, the diagnostic imaging apparatus 3 includes a CNN configuration storage unit 31, a parameter storage unit 32, and a CNN identification unit 38.

Here, the diagnostic imaging apparatus 3 can be configured by hardware elements as shown in FIG. In FIG. 10, the diagnostic imaging apparatus 3 is composed of a computer apparatus including a CPU 3001 and a memory 3002. The memory 3002 is composed of a RAM, a ROM, an auxiliary storage device, and the like. In this case, the CNN configuration storage unit 31 and the parameter storage unit 32 are configured by the memory 3002. Further, the CNN identification unit 38 is composed of a CPU 3001 that reads and executes a computer program stored in the memory 3002. The hardware configuration of the diagnostic imaging apparatus 3 and its functional blocks is not limited to the above configuration. Further, the diagnostic imaging device 3 may be configured by the same computer device as the diagnostic imaging learning device 1, or may be configured by a different computer device.

The CNN configuration storage unit 31 stores information representing the same network configuration as the CNN used by the diagnostic imaging learning device 1 as the first embodiment.

The parameter storage unit 32 stores the updated CNN parameters by applying the diagnostic imaging learning device 1 as the first embodiment to one or a plurality of learning images. For example, such parameters are weights and biases in the CNN. It is desirable that the CNN parameters stored in the parameter storage unit 32 are updated by applying the diagnostic imaging learning device 1 to the learning images obtained from a larger number of endoscopic images.

The CNN identification unit 38 uses the parameters of the CNN stored in the parameter storage unit 32 to input information based on the image to be diagnosed into the CNN and execute the calculation. Then, the CNN identification unit 38 identifies the abnormal region in the image to be diagnosed based on the calculation result of CNN.

The operation of the diagnostic imaging apparatus 3 configured as described above is shown in FIG.

In FIG. 11, first, the CNN identification unit 38 inputs the image to be diagnosed into the CNN of the network configuration used by the image diagnosis learning device 1 as the first embodiment (step S31).

Next, the CNN identification unit 38 executes the calculation of the CNN using the parameters of the CNN updated by the diagnostic imaging learning device 1 of the first embodiment (step S32).

Next, the CNN identification unit 38 identifies the abnormal region in the image to be diagnosed based on the calculation result of CNN (step S33).

With the above, the diagnostic imaging apparatus 3 ends the operation.

Next, the effect of the third embodiment will be described.

The diagnostic imaging apparatus as the third embodiment includes a region in which the original diagnostic target is not sufficiently captured in the diagnostic target image in which the diagnostic target is captured by a normal imaging method that does not rely on a special function. However, the abnormal region can be detected more accurately.

The reason will be explained. In the present embodiment, the CNN configuration storage unit stores the network configuration of the CNN used by the diagnostic imaging learning device as the first embodiment. In addition, the parameter storage unit stores the parameters of the CNN updated by the diagnostic imaging learning device as the first embodiment. Then, the CNN identification unit inputs the image to be diagnosed into the CNN of the network configuration used by the diagnostic imaging learning device as the first embodiment. Then, the CNN identification unit calculates the CNN using the CNN parameters updated by the diagnostic imaging learning device as the first embodiment, and identifies the abnormal region in the image to be diagnosed based on the calculation result. Because it does.

Here, the parameters of the CNN updated by the diagnostic imaging learning apparatus as the first embodiment are learned by suppressing the influence of the inappropriate region in the learning image in which the diagnostic target is copied. Therefore, in the present embodiment, since the CNN using such a parameter is applied to the image to be diagnosed, the abnormal region can be accurately identified.

(Fourth Embodiment)
Next, the fourth embodiment will be described in detail with reference to the drawings. In this embodiment, the internal organ wall surface is applied as a diagnostic target. Further, in the present embodiment, an example of an diagnostic imaging apparatus for identifying a candidate for a lesion in an endoscopic image will be described using the CNN parameters updated by the second embodiment. In each drawing referred to in the description of the present embodiment, the same reference numerals are given to the steps having the same configuration and the same operation as those of the third embodiment, and the detailed description in the present embodiment is omitted. do.

First, FIG. 12 shows the configuration of the diagnostic imaging apparatus 4 as the fourth embodiment. In FIG. 12, the image diagnosis device 4 includes a CNN identification unit 48 instead of the CNN identification unit 38 for the image diagnosis device 3 as the third embodiment, and further includes an inappropriate region detection unit 43 and an inappropriate region detection unit 43. The difference is that it includes an inappropriate area invalidation unit 44. It is assumed that the CNN configuration storage unit 31 and the parameter storage unit 32 store the network configuration of the CNN used by the diagnostic imaging learning device as the second embodiment and the updated parameters.

Here, the diagnostic imaging apparatus 4 and each functional block thereof can be configured by the same hardware elements as the diagnostic imaging apparatus 3 as the third embodiment described with reference to FIG. However, the hardware configuration of the diagnostic imaging apparatus 4 and its functional blocks is not limited to the above configuration.

The inappropriate region detection unit 43 detects a shiny portion as an inappropriate region in the image to be diagnosed. Since the details of the process of detecting the shiny portion are the same as those of the inappropriate region detection unit 23 in the second embodiment, the description in the present embodiment will be omitted. For example, the inappropriate region detection unit 43 may perform a process of detecting a shiny portion of the patch image which is a partial region of the image to be diagnosed.

The inappropriate area invalidation unit 44 invalidates the unit corresponding to the shiny part among the units of the input layer of the CNN in which the image to be diagnosed is input. For example, the inappropriate area invalidation unit 44 may perform invalidation processing on the patch image which is a partial area of the image to be diagnosed. In this case, the inappropriate area invalidation unit 44 may invalidate the unit corresponding to the shiny part among the units of the input layer of the CNN into which the patch image is input. The network configuration of the CNN into which the image to be diagnosed or the patch image thereof is input is stored in the CNN configuration storage unit 31. Since the details of the process of invalidating the unit corresponding to the shiny part are the same as those of the inappropriate area invalidation unit 14 in the first embodiment, the description in the present embodiment will be omitted.

The CNN identification unit 48 identifies the non-lesioned portion or the type of the lesioned portion of the image to be diagnosed. Specifically, the CNN identification unit 48 applies the parameters of the CNN stored in the parameter storage unit 32 to the CNN in the state where the image to be diagnosed is input and the shiny part is invalidated, and the CNN's Forward Make a calculation. For example, the CNN identification unit 48 may identify the patch image, which is a partial region of the image to be diagnosed. In this case, the CNN identification unit 48 applies the CNN parameters stored in the parameter storage unit 32 to the CNN in the state where the patch image is input and the shiny part is invalidated, and performs the CNN Forward calculation. Just do it. Then, the CNN identification unit 48 outputs the lesion class information of the visceral wall surface copied on the patch image as the identification result of the CNN.

Here, by performing the Forward calculation of CNN, the identification score for each lesion class is output from CNN. Therefore, for example, the CNN identification unit 48 may convert the identification score for each lesion class into a real value score having a range of 0 to 1 by executing the SoftMax function. As a result, the CNN identification unit 48 can calculate a score representing each lesion class-likeness for the visceral wall surface imaged in this patch image.

Further, when the CNN identification unit 48 identifies the patch image which is each partial region of the image to be diagnosed, the CNN identification unit 48 may output the score for each patch image. For example, the CNN identification unit 48 may display the score for each patch image on a display device (not shown) as a score map showing the lesion class-likeness in the image to be diagnosed. Here, the score map is an image including the score of the patch image as an image at the position of each patch image in the diagnosis target image. Further, the CNN identification unit 48 may superimpose and output such a score map on the image to be diagnosed.

The operation of the diagnostic imaging apparatus 4 configured as described above will be described with reference to FIG. Here, it is assumed that the diagnostic imaging apparatus 4 operates for each patch image cut out from the endoscopic image to be diagnosed.

In FIG. 13, first, the inappropriate region detection unit 43 detects a shiny portion in the patch image cut out from the endoscopic image (step S41).

Next, the inappropriate area invalidation unit 44 temporarily invalidates the unit corresponding to the shiny part among the units of the input layer of the CNN into which the patch image is input (step S42).

Next, the CNN identification unit 48 performs the Forward calculation of the CNN in a state where the unit of the input layer corresponding to the shiny part is temporarily invalidated. As a result, the score for each lesion class is output from CNN (step S43).

Here, if there is another patch image included in the endoscopic image (Yes in step S44), the diagnostic imaging apparatus 4 repeats the operation from step S41.

On the other hand, if there is no other patch image included in the endoscopic image (No in step S44), the CNN identification unit 48 outputs the lesion class-likeness for each patch image (step S45). As described above, for example, the CNN identification unit 48 superimposes the lesion class-likeness of each patch image on the endoscopic image as a score map for displaying the patch image at the position in the endoscopic image. It may be displayed on the display device. The CNN identification unit 48 may include the display device. This has the advantage that the location of the lesion can be easily visually confirmed by a human.

Next, the effect of the fourth embodiment will be described.

The diagnostic imaging apparatus as the present embodiment includes a shiny part in an endoscopic image in which the internal organ wall surface is photographed by a normal imaging method that does not depend on a special function such as an NBI endoscope or a magnifying endoscope. Even if this is the case, the area that may be a lesion can be detected more accurately.

The reason will be explained. In this embodiment, the CNN configuration storage unit and the parameter storage unit store the CNN network configuration and updated parameters used by the diagnostic imaging learning apparatus as the second embodiment. Then, the inappropriate region detection unit detects the shiny part in the endoscopic image. Further, the inappropriate region invalidation unit inputs a patch image of the endoscopic image into the CNN of the network configuration used by the diagnostic imaging learning device as the second embodiment, and the patch image of the endoscopic image is input to the shiny part in the input layer. Temporarily disable the corresponding unit. Then, the CNN identification unit uses the parameters updated by the diagnostic imaging learning device as the second embodiment to perform the Forward calculation of the CNN in the state where the unit corresponding to the shiny part is temporarily disabled. This is because the identification is performed by executing.

As described above, the present embodiment uses the parameters learned by the second embodiment to perform identification by the CNN having the same network configuration as the CNN used in the second embodiment. Therefore, in the present embodiment, even a general endoscopic image other than an NBI endoscope or a magnifying endoscope can correctly detect lesion-likeness in the endoscopic image.

Further, in the present embodiment, identification is performed using the parameters learned in the state where the shiny portion is removed. Therefore, in the present embodiment, even if the shiny portion is removed from the image to be identified, the identification can be performed with high accuracy. As described above, in the present embodiment, since the shiny spot is automatically detected and the corresponding unit is invalidated in the input layer of the CNN for identification, the influence of the abnormal value generated in the shiny spot in the image to be diagnosed is affected. Can be suppressed and identification can be performed. That is, in the present embodiment, it is possible to perform identification by suppressing the influence of the shiny part as much as possible even from the data including the shiny in the image to be diagnosed.

As a network configuration of the CNN used in the present embodiment, an example in which the same network configuration as the CNN used in the second embodiment is stored in the CNN configuration storage unit has been described. Not limited to this, the CNN configuration storage unit may store the configuration from the input layer to the intermediate layer from which the feature amount is extracted in the CNN network configuration used in the second embodiment. For example, among the network configurations of CNN used in the second embodiment, the configuration including the Convolution layer and the Pooling layer excluding the fully connected layer may be stored. In that case, at least the parameters necessary for the calculation from the input layer to the intermediate layer may be stored in the CNN configuration storage unit as parameters. In this case, in the present embodiment, the information output from the CNN identification unit can be regarded as a numerical string representing a feature amount effective for identifying the lesion class. In this case, the lesion class can be identified by inputting the numerical string which is the output of the present embodiment to an arbitrary classifier such as a support vector machine.

Further, when using the configuration from the input layer to the above-mentioned intermediate layer among the CNN network configurations used in the second embodiment, the CNN identification unit is not a patch image cut out from the endoscopic image. , The entire area of the endoscopic image may be input to the CNN. In this case, the information output from the CNN identification unit is a feature amount effective for identifying each lesion class in the endoscopic image. Therefore, the CNN identification unit may output the calculation result up to the middle layer as a score map showing the possibility of each lesion class in the endoscopic image by expressing it as an image according to the position in the endoscopic image. .. In this way, when inputting the entire area of the diagnosis target image, the present embodiment reduces the amount of calculation and is faster than the case where the identification process is repeated for each patch image while scanning the diagnosis target image. It is possible to generate a score map showing the possibility of lesion class.

In each of the above-described embodiments, an example in which each pixel of the learning image or the diagnosis target image is assigned to each unit of the input layer of the CNN has been mainly described. Not limited to this, information in which the learning image or the image to be diagnosed has been processed in advance may be input to each unit of the input layer. For example, even if each pixel of the first-order differential image or the second-order differential image of the brightness is input to each unit of the input layer by processing the learning image or the image to be diagnosed by the Sobel operator, the Laplacian operator, or the like. good. In this case, in the first, second, and fourth embodiments, the inappropriate region invalidation unit is an inappropriate region detected in the learning image or the diagnosis target image before the processing is performed in each unit of the input layer. The unit in which the pixels of the differential image corresponding to the region are input may be invalidated.

Further, in the second and fourth embodiments described above, an example in which the internal organ wall surface is applied as a diagnosis target and an image based on an endoscopic image is applied as a diagnosis target image has been mainly described. Not limited to this, in each of these embodiments, a diagnostic target image to which another target is applied may be used as the diagnostic target. In addition, in each of these embodiments, an example of applying a shiny portion as an inappropriate region has been mainly described. Not limited to this, in each of these embodiments, other regions that are not suitable for identifying abnormal regions may be applied as inappropriate regions.

Further, in each of the above-described embodiments, the loss value calculation unit and the parameter update unit apply various known calculation methods applicable to CNNs in which some units are invalidated, in addition to the above-mentioned calculation methods. It is possible.

Further, in each of the above-described embodiments, an example in which each functional block of the diagnostic imaging learning device and the diagnostic imaging device is realized by a CPU that executes a computer program stored in a memory has been mainly described. Not limited to this, a part, all, or a combination thereof of each functional block may be realized by dedicated hardware.

Further, in each of the above-described embodiments, the functional blocks of the diagnostic imaging learning device or the diagnostic imaging device may be distributed and realized in a plurality of devices.

Further, in each of the above-described embodiments, the operation of the diagnostic imaging learning device or the diagnostic imaging device described with reference to each flowchart is stored in the storage device (storage medium) of the computer device as a computer program. Then, the CPU may read and execute the computer program. Then, in such a case, it is composed of a computer program code or a storage medium.

In addition, each of the above-described embodiments can be implemented in combination as appropriate.

Further, the present invention is not limited to the above-described embodiments, and can be implemented in various embodiments.

This application claims priority on the basis of Japanese application Japanese Patent Application No. 2016-132695 filed on July 4, 2016, and incorporates all of its disclosures herein.

1, 2 Image diagnostic learning device 3, 4 Image diagnostic device 11, 31 CNN configuration storage unit 12, 32 Parameter storage unit 13, 23, 43 Inappropriate area detection unit 14, 44 Inappropriate area invalidation unit 15 Loss value calculation unit 16 Parameter update unit 27 Perturbation unit 38, 48 CNN identification unit 1001, 3001 CPU
1002, 3002 memory

Claims (10)

A CNN configuration storage means for storing a network configuration of a Convolutional Neural Network (CNN), and a CNN configuration storage means.
A parameter storage means for storing parameters to be learned in the CNN, and
In the learning image in which the diagnosis target is captured, an inappropriate region that is not suitable for identifying an abnormal region in which the diagnosis target may have anomalies is detected based on a predetermined criterion. Detection means and
In the network configuration of the CNN to which the learning image is input, the inappropriate area invalidating means for invalidating the unit corresponding to the inappropriate area among the units of the input layer, and the inappropriate area invalidating means.
In a state where the unit of the input layer corresponding to the inappropriate region is invalidated, the calculation of the CNN is executed using the parameters, the calculation result, and the diagnosis given in advance to the learning image. Loss value calculation means that calculates the loss value based on the information that represents the anomaly of the object,
A parameter updating means for updating the parameter of the parameter storage means based on the loss value, and a parameter updating means.
Image diagnosis learning device equipped with. The image diagnosis learning apparatus according to claim 1, further comprising a perturbation means for generating the learning image by performing a perturbation process on a learning sample of an image on which the diagnosis target is copied. A parameter storage means for storing the parameters of the CNN updated by applying the diagnostic imaging learning device according to claim 1 or 2 to one or a plurality of learning images.
A CNN configuration storage means for storing the CNN network configuration used by the diagnostic imaging learning device when updating the parameters, and a CNN configuration storage means.
By inputting information based on the diagnosis target image on which the diagnosis target is copied into the CNN and executing the calculation, the CNN that identifies the abnormal region in which the diagnosis target may have an abnormality in the diagnosis target image. Identification means and
Diagnostic imaging device equipped with. An inappropriate region detecting means for detecting the inappropriate region in the image to be diagnosed based on a predetermined criterion,
Inappropriate region invalidation means for invalidating the unit corresponding to the inappropriate region among the units of the input layer of the CNN into which the image to be diagnosed is input.
With more
The CNN identification means is characterized in that the abnormal region in the image to be diagnosed is identified by executing the calculation of the CNN in a state where the unit of the input layer corresponding to the inappropriate region is invalidated. The diagnostic imaging apparatus according to claim 3. 3. The claim 3 or claim is characterized in that the CNN identification means outputs an identification result for each partial region by performing the identification using the CNN for each partial region included in the image to be diagnosed. The diagnostic imaging apparatus according to 4. The CNN identification means inputs the entire area of the image to be diagnosed into the CNN, and calculates the CNN using the configuration up to the layer from which the feature amount is extracted from the network configuration of the CNN. The diagnostic imaging apparatus according to claim 3 or 4, wherein a feature amount related to the abnormal region in the target image is output. The computer device
In the learning image in which the diagnosis target is captured, an inappropriate region, which is a region that is not suitable for identifying the abnormal region in which the diagnosis target may have an abnormality, is detected based on a predetermined criterion.
In the network configuration of the Convolutional Nuclear Network (CNN) into which the learning image is input, the unit corresponding to the inappropriate region among the units of the input layer is invalidated.
The calculation of the CNN is executed in a state where the unit of the input layer corresponding to the inappropriate region is invalidated, and the calculation result and the abnormality of the diagnosis target previously given to the learning image are displayed. Calculate the loss value based on the information you represent
A method of updating the parameters to be learned in the CNN based on the loss value. In the learning image in which the diagnosis target is captured, an inappropriate region, which is a region in which the diagnosis target is not suitable for identifying an abnormal region that may have anomalies, is detected based on a predetermined criterion.
In the network configuration of the Convolutional Neural Network (CNN) into which the learning image is input, the unit corresponding to the inappropriate region among the units of the input layer is invalidated.
The calculation of the CNN is executed in a state where the unit of the input layer corresponding to the inappropriate region is invalidated, and the calculation result and the abnormality of the diagnosis target previously given to the learning image are displayed. Calculate the loss value based on the information you represent
The parameters to be learned in the CNN are updated based on the loss value.
A program that lets a computer device do that. The computer device
Using the parameter of the CNN updated by performing the method according to claim 7 for one or more learning images, and the network configuration of the CNN used when updating the parameter. hand,
A method of identifying an abnormal region in the diagnosis target image in which the diagnosis target may have an abnormality by inputting information based on the diagnosis target image on which the diagnosis target is copied into the CNN and executing a calculation. .. The parameters of the CNN updated by causing a computer device to execute the program according to claim 8 for one or more learning images, and the network configuration of the CNN used when updating the parameters. With and
By inputting information based on the diagnosis target image on which the diagnosis target is copied into the CNN and executing the calculation, the abnormal region where the diagnosis target may have an abnormality is identified in the diagnosis target image. A program that causes a computer device to execute.

Images