JP6590386B1 - Image processing apparatus, image processing system, and image processing program - Google Patents

Abstract

An image processing apparatus for generating a diagnosis result image obtained by extracting an abnormal part from a medical image is provided. An image processing apparatus generates an abnormal part extraction image obtained by extracting a difference part from a medical image obtained by photographing a healthy person as an abnormal part from an input target medical image, and the image processing apparatus is included in the target medical image. A special part estimation image is generated by estimating a difference part peculiar to an image that is different from a medical image obtained by photographing a healthy person but is not medically abnormal, and is generated by an abnormal part extraction image generation unit. A difference between the abnormal portion extracted image and the specific portion estimated image generated by the specific portion estimated image generating means is calculated, and a diagnosis result image is generated by removing the specific portion from the abnormal portion extracted image. [Selection] Figure 9

Description

  The present invention relates to an image processing apparatus, an image processing system, and an image processing program.

  The following biological image processing apparatuses are known. In this biological image processing apparatus, an input training image that is a biological image including a target object, an output training image that is a biological image not including the target object, and an input image that is a biological image including the target object are received and input Using a training image and an output training image, learn a neural network that performs image processing to generate a biological image that does not include an object from a biological image that includes the object, and use the image processing of the neural network that is the learning result Then, an output image is generated by removing the object from the input image (for example, Patent Document 1).

JP-A-2018-89301

  In a conventional biological image processing apparatus, a neural network that performs image processing for generating a biological image that does not include a target object from a biological image that includes a target object is learned, and an input image is obtained using the neural network image processing that is a learning result. An output image in which the target object is removed from is generated. However, in diagnosis using an image, it is often required to find a part that may be abnormal from a medical image, and the accuracy is also required. However, in the conventional technique, no investigation has been made on a mechanism for extracting an abnormal part with high accuracy from a medical image and supporting diagnosis by a diagnostician.

An image processing apparatus according to the present invention includes an abnormal part extraction image generation unit that generates an abnormal part extraction image obtained by extracting a difference part from a medical image obtained by photographing a healthy person as an abnormal part from an input target medical image; A unique part estimation that generates a unique part estimation image that is a difference part from a medical image obtained by photographing a healthy person, but that is not medically abnormal and that is estimated as a unique part. Calculating a difference between the abnormal part extracted image generated by the image generating means and the abnormal part extracted image generating means and the specific part estimated image generated by the specific part estimated image generating means, and a diagnosis result image generation means for generating a diagnosis result image rejection, abnormal portion extracted image generating means, to learn the image pattern there is no abnormality medical image, abnormality I on the input medical image An abnormal part extraction image is obtained based on a difference image obtained by inputting a target medical image to an auto encoder configured to restore and output a medical image and calculating a difference between the input image and the output image to the auto encoder. When the medical image including the specific part and having no abnormality is input, the specific part estimation image generating unit inputs the medical image including the specific part and having no abnormality to the abnormal part extraction image generating unit. A target medical image is input to a neural network learned to restore and output an extracted image, and an output image from the neural network is used as the characteristic portion estimation image .
An image processing system according to the present invention includes an abnormal part extraction image generation unit that generates an abnormal part extraction image obtained by extracting a difference part from a medical image obtained by photographing a healthy person as an abnormal part from an input target medical image; A unique part estimation that generates a unique part estimation image that is a difference part from a medical image obtained by photographing a healthy person, but that is not medically abnormal and that is estimated as a unique part. Calculating a difference between the abnormal part extracted image generated by the image generating means and the abnormal part extracted image generating means and the specific part estimated image generated by the specific part estimated image generating means, and a diagnosis result image generation means for generating a diagnosis result image rejection, abnormal portion extracted image generating means, to learn the image pattern there is no abnormality medical images, abnormal on the input medical image Extract the abnormal part based on the difference image obtained by calculating the difference between the input image and the output image to the auto encoder by inputting the target medical image to the auto encoder configured to restore and output the non-medical image An image is generated, and the unique part estimation image generation unit inputs a medical image including the specific part and having no abnormality to the abnormal part extraction image generation unit when a medical image including the specific part and having no abnormality is input. The target medical image is input to the neural network learned to restore and output the image extracted in this manner, and the output image from the neural network is used as the characteristic part estimation image .
An image processing program according to the present invention includes an abnormal part extraction image generation procedure for generating an abnormal part extraction image obtained by extracting a difference part from a medical image obtained by photographing a healthy person as an abnormal part from an input target medical image, and a target A unique part estimation that generates a unique part estimation image that is a difference part from a medical image obtained by photographing a healthy person, but that is not medically abnormal and that is estimated as a unique part. The difference between the abnormal part extraction image generated in the image generation procedure and the abnormal part extraction image generation procedure and the special part estimation image generated in the special part estimation image generation procedure is calculated, and the special part is excluded from the abnormal part extraction image a diagnosis image generation step of generating a diagnostic result image a program for causing a computer to execute the abnormal portion extracted image generation procedures, train the image pattern there is no abnormality medical image, Obtained by calculating the difference between the input image and the output image to the auto encoder by inputting the target medical image to the auto encoder configured to restore and output the medical image that is normal to the input medical image The abnormal part extraction image is generated based on the difference image, and the unique part estimation image generation procedure is performed when the medical image including the specific part is input and the medical image including the specific part is not included. The target medical image is input to the neural network learned to restore and output the extracted image by inputting to the abnormal part extracted image generating means, and the output image from the neural network is used as the characteristic part estimation image It is characterized by.

  According to the present invention, it is possible to generate an abnormal part extracted image obtained by extracting an abnormal part from the target medical image and to generate a diagnosis result image excluding the specific part from the abnormal part extracted image. It is possible to provide an image that can identify an abnormal portion with high accuracy.

1 is a block diagram illustrating a configuration of an embodiment of an image processing apparatus 100. FIG. It is the 1st figure showing typically the flow of processing by an abnormal part detector. It is the 2nd figure which showed typically the flow of processing by an abnormal part detector. It is the 1st figure showing typically the flow of processing by the peculiar part estimator. It is the 2nd figure which showed typically the flow of the process by the specific part estimator. It is the figure which showed typically the method of producing | generating a diagnostic result image using an abnormal part extraction image and a characteristic part estimation image. It is the figure which put together the flow of a training step and the step which produces | generates a diagnostic result image together, and was shown typically. It is a flowchart figure which shows the flow of a training step. It is a flowchart figure which shows the flow of a diagnostic result image generation process.

  FIG. 1 is a block diagram illustrating a configuration of an embodiment of an image processing apparatus 100 according to the present embodiment. The image processing apparatus 100 uses a medical image taken in advance, for example, an MRI image, a CT image, or an X-ray image, to extract a part that is medically abnormal from the medical image and visualize it. Execute. As the image processing apparatus 100, for example, a server apparatus or a personal computer is used. FIG. 1 is a block diagram showing a configuration of an embodiment when a personal computer is used as the image processing apparatus 100 in the present embodiment.

  The image processing apparatus 100 includes an operation member 101, a control device 102, a storage medium 103, and a display device 104.

  The operation member 101 includes various devices that are operated by an operator of the image processing apparatus 100, such as a keyboard and a mouse.

  The control device 102 includes a CPU, a memory, and other peripheral circuits, and controls the entire image processing device 100. In addition, the memory which comprises the control apparatus 102 is volatile memories, such as SDRAM, for example. This memory is used as a work memory for the CPU to expand the program when the program is executed and a buffer memory for temporarily recording data. For example, data read via the connection interface 102 is temporarily recorded in the buffer memory.

  The storage medium 103 is a storage medium for recording various data stored by the image processing apparatus 100, data of a program to be executed by the control apparatus 102, and the like, for example, HDD (Hard Disk Drive) or SSD (Solid State). Drive) or the like is used. The program data recorded in the storage medium 103 is provided by being recorded on a recording medium such as a CD-ROM or DVD-ROM, or provided via a network, and stores the program data acquired by the operator. By installing in the medium 103, the control device 102 can execute the program. In the present embodiment, programs and various data used in the processing described below are recorded in the storage medium 103.

  The display device 104 is a liquid crystal monitor, for example, and displays various display data output from the control device 102.

  In the present embodiment, the control device 102 generates an abnormal part extraction image obtained by extracting, as an abnormal part, a difference part from a medical image obtained by photographing a healthy person from the input target medical image, and includes the abnormal medical image in the target medical image. Generates a unique part estimation image that is a difference part from a medical image obtained by photographing a healthy person, but is estimated as a unique part that is unique to an image that is not medically abnormal. A process for calculating a difference between the part estimation images and generating a diagnosis result image in which the unique part is excluded from the abnormal part extraction image is executed.

  First, a process for generating an abnormal part extracted image obtained by extracting an abnormal part from the target medical image will be described. In the present embodiment, an abnormal part detector capable of outputting an abnormal part extracted image obtained by extracting an abnormal part from the target medical image is prepared in advance, and the control device 102 includes the target medical part in the target medical image. By inputting an image, an abnormal part extraction image is generated by extracting a difference part from the medical image obtained by photographing a healthy person from the target medical image as an abnormal part.

  As shown in FIG. 2, when the abnormal part detector D (Xh) receives the target medical image Xh, the abnormal part is defined as an abnormal part from a normal patient in which no medical abnormality is observed in the target medical image Xh. The extracted abnormal part extracted image Ah is configured to be output.

  The configuration of the abnormal part detector is not particularly limited, but in this embodiment, the image pattern of a medical image having no abnormality taken by a healthy person is learned, and the medical image having no abnormality with respect to the input medical image is restored. When the target medical image is input to the auto encoder and the difference image obtained by calculating the difference between the input image and the output image to the auto encoder is used as the abnormal part extracted image It is configured to output. Note that the auto encoder used in the abnormal part detector may be a known deep auto encoder that can learn the neural network so that the input and output are the same.

  As a result, when a medically abnormal portion appears in the target medical image, the input image to the auto encoder includes an abnormal portion, while the output image is an image having no abnormality, that is, an abnormal portion. Therefore, if the difference between the input image to the auto encoder and the output image is taken, an abnormal part extracted image in which the abnormal part is extracted from the target medical image can be output.

  The above-described medical image having no abnormality is an image that does not include a part that is recognized as being medically abnormal in the image, and is an image obtained by photographing a part to be diagnosed of a person who is not medically abnormal, for example, That is, an image obtained by photographing a specific part of a healthy patient may be used.

  As described above, the abnormal part extracted image output from the abnormal part detector is an image obtained by extracting a different part from the medical image having no abnormality learned by the abnormal part detector from the target medical image. For this reason, in the abnormal part extraction image output from the abnormal part detector, the difference between the target medical image and the image pattern learned by the auto encoder may be extracted in addition to the part that is recognized as medically abnormal. There is sex. In this case, the abnormal part extraction image includes a part that is recognized as medically abnormal and a part that is not recognized as medically abnormal but is specific to the target medical image caused by individual differences or image characteristics. Will be.

  For example, FIG. 2 shows a result of inputting a medical image having no abnormal portion of a healthy patient as the target medical image Xh. In this case, in the abnormal part extraction image Ah output from the abnormal part detector D (Xh), a part that is recognized as medically abnormal does not appear, but a part that is unique to the target medical image that is not medically abnormal Has appeared.

  Further, as shown in FIG. 3, when a brain image including an abnormal portion 3a that is regarded as abnormal medically is input to the abnormal portion detector D (Xa) as the target medical image Xa, an abnormality that is output In the part-extracted image Aa, in addition to the medically abnormal part 3a, a part peculiar to the target medical image that is not medically abnormal appears on the image.

  In this embodiment, in order to delete a specific part that appears on the abnormal part extracted image even though it is not an abnormal part in this way, the specific part estimation that estimates the specific part included in the target medical image is performed. An image is generated, and a difference between the abnormal part extracted image and the unique part estimated image is obtained. As a result, it is possible to generate a diagnostic result image in which the original medically abnormal portion is extracted with high accuracy by eliminating the unique portion that appears on the abnormal portion extracted image even though it is not an abnormal portion. .

  For this reason, in the present embodiment, a unique part estimator capable of outputting a unique part estimation image obtained by estimating a unique part from the target medical image is prepared in advance, and the control device 102 estimates the unique part estimation. By inputting the target medical image into the device, a specific part estimation image in which the specific part is estimated from the target medical image is generated. Hereinafter, the unique part estimator used in the present embodiment will be described.

  In the present embodiment, a training step for capturing a medical image obtained by capturing a specific part of a patient having no abnormality in advance and learning an image pattern of a specific part in the medical image having no abnormality is executed.

  In the training step, as described above, a medical image having no abnormality is used as the input image. When the medical image Xh is input, the control device 102 inputs the medical image Xh to the above-described abnormal part detector D (Xh) and generates an abnormal part extracted image Ah.

  Further, as shown in FIG. 4, the control device 102 inputs the medical image Xh having no abnormality to the specific part estimator Q (Xh) that is a neural network, and detects the abnormal part medical image Xh described above. A unique part image A′h that approximates the abnormal part extracted image Ah output when the image is input to the device is predicted. At this time, the control device 102 repeats the process while adjusting the network weight until the loss L = d (A′h, Ah) is minimized. The control device 102 sets the weight value set when the loss L is minimized as the weight of the specific part estimator. Thus, the unique part estimator can be trained to output the unique part image A′h that approximates the abnormal part extracted image Ah.

  Thus, when the target medical image is input, the unique part estimator outputs an image pattern that approximates the abnormal part extracted image Ah that is output when the medical image Xh having no abnormality is input to the abnormal part detector. Will be trained to be. The medical image Xh having no abnormality used as an input image in the training step does not include a medical abnormal part, but includes a specific part unique to a medical image that is not medically abnormal. Therefore, by using the unique part estimator that has completed the training step, it is possible to output a unique part estimated image in which the unique part is estimated when the target medical image is input to the unique part estimator.

  For example, as shown in FIG. 5, when a brain image including the abnormal part 3a that is regarded as medically abnormal as shown in FIG. 3 is input to the specific part estimator Q (Xa) as the target medical image Xa, A unique part estimation image A′a in which only the unique part is estimated without including 3a is output.

As shown in FIG. 6, the control device 102 receives the target medical image Xa according to the following equation (1), and as a result, the abnormal part extracted image Aa output from the abnormal part detector D (Xa) and the specific part estimator. By calculating the difference from the specific part estimation image A′a output from Q (Xa), the abnormal part 3a is extracted from the target medical image Xa, and the special part that is not a medical abnormal part is excluded. A diagnostic result image can be generated.
Diagnosis result image = Aa−A′a (1)

  FIG. 7 is a diagram schematically showing the flow of the above-described training step and the step of generating a diagnosis result image in one. 7 is the same as that described with reference to FIGS. 2 to 6, and thus the description thereof is omitted.

  FIG. 8 is a flowchart showing a flow of training steps in the present embodiment. The processing shown in FIG. 8 is executed by the control device 102 as a program that starts when a normal medical image without an abnormal part, for example, a brain image of a healthy person, is input.

  In step S10, the control apparatus 102 inputs the medical image Xh having no abnormality to the abnormal part detector D (Xh) and generates an abnormal part extraction image Ah. Then, it progresses to step S20.

  In step S20, the abnormal part extracted image Ah output when the medical image Xh having no abnormality is input to the specific part estimator Q (Xh) and the medical image Xh having no abnormality is input to the above-described abnormal part detector. A special part image A′h approximating to is predicted. Then, it progresses to step S30.

  In step S30, the control device 102 calculates the above-described loss L = d (A′h, Ah). Thereafter, the process proceeds to step S40.

  In step S40, the control device 102 determines whether or not the loss L calculated in step S40 has been minimized. If an affirmative decision is made in step S40, the weight at that time is adopted and the process is terminated. On the other hand, if a negative determination is made in step S40, the process proceeds to step S50.

  In step S50, as described above, the control apparatus 102 adjusts the network weight of the unique part estimator and returns to step S10.

  FIG. 9 is a flowchart showing the flow of diagnostic result image generation processing in the present embodiment. The process shown in FIG. 9 is executed by the control device 102 as a program that is activated when a medical image of a patient to be diagnosed, that is, the above-described target medical image is input.

  In step S110, the control device 102 inputs the target medical image Xa to the abnormal part detector D (Xa) and generates an abnormal part extracted image Aa. Then, it progresses to step S120.

  In step S120, the control device 102 inputs the target medical image Xa to the unique part estimator Q (Xa) to generate a unique part estimated image A′a. Thereafter, the process proceeds to step S130.

  In step S130, the control device 102 obtains a difference image between the abnormal part extracted image Aa and the unique part estimated image A′a using the equation (1). Thereafter, the process proceeds to step S140.

  In step S140, the control device 102 displays the diagnosis result image on the display device 104 by outputting the difference image generated in step S130 to the display device 104 as a diagnosis result image. Thereafter, the process ends.

According to the embodiment described above, the following operational effects can be obtained.
(1) The control device 102 generates an abnormal part extraction image obtained by extracting a difference part from a medical image obtained by photographing a healthy person as an abnormal part from the input target medical image, and is included in the target medical image. Generates a unique part estimation image that is a difference part from a medical image obtained by photographing a healthy person but estimates a unique part of the image that is not medically abnormal as a unique part, and extracts the abnormal part extracted image and the unique part estimation image. The diagnosis result image is generated by excluding the specific part from the abnormal part extracted image. Thereby, it is possible to generate a diagnosis result image in which only a medically abnormal part is extracted from the target medical image, and to support diagnosis using the image.

(2) The control device 102 displays the generated diagnosis result image on the display device 104. Accordingly, an operator such as a doctor can check the diagnosis result image on the display device 104.

(3) The control device 102 learns the image pattern of the medical image having no abnormality, and restores and outputs the medical image having no abnormality with respect to the input medical image, and outputs the target medical image to the auto encoder. And an abnormal part extraction image is generated based on the difference image obtained by calculating the difference between the input image to the auto encoder and the output image. Thus, if an image pattern of a medical image having no abnormality is learned in advance in the auto encoder, an abnormal part extracted image can be generated only by inputting the target medical image.

(4) The control device 102 learns the image pattern of the unique portion in the medical image having no abnormality, and restores and outputs the medical image including the unique portion in the medical image having no abnormality with respect to the input medical image. The target medical image is input to the neural network configured as described above, and the output image from the neural network is used as the characteristic part estimation image. As a result, if the image pattern of the specific part in the medical image having no abnormality in the neural network is learned in advance, the specific part estimation image can be generated only by inputting the target medical image.

-Modification-
The image processing apparatus according to the above-described embodiment can be modified as follows.
(1) In the above-described embodiment, the example in which the image processing apparatus 100 is a personal computer and the control apparatus 102 executes the above-described processing has been described. However, the terminal operated by the operator can be connected to the image processing apparatus 100 via a communication line such as the Internet, and the input images Xh and Xa described above are input by being transmitted from the operating terminal to the image processing apparatus 100. May be. In that case, the diagnosis result image may be transmitted to the operation terminal and displayed on the operation terminal. As a result, a client server type or cloud type image processing system can be constructed by connecting the operation terminal and the image processing apparatus 100 via a communication line, while the image processing apparatus 100 is used alone. Therefore, it can be used as a stand-alone device.

(2) In the above-described embodiment, the example in which the processing for the training step is executed by the control apparatus 102 in the image processing apparatus 100 has been described. However, the processing for the training step may be executed by another device, and the data obtained by the training step may be recorded in the storage medium 103. In this case, processing for the training step in the image processing apparatus 100 is not necessary.

  Note that the present invention is not limited to the configurations in the above-described embodiments as long as the characteristic functions of the present invention are not impaired. Moreover, it is good also as a structure which combined the above-mentioned embodiment and a some modification.

DESCRIPTION OF SYMBOLS 100 Image processing apparatus 101 Operation member 102 Control apparatus 103 Storage medium 104 Display apparatus

Claims (6)

An abnormal part extraction image generating means for generating an abnormal part extraction image obtained by extracting, as an abnormal part, a difference part from a medical image obtained by photographing a healthy person from within the input target medical image;
A unique part that generates a unique part estimated image that is included in the target medical image and that is a different part from a medical image obtained by photographing a healthy person but that is a medically abnormal image-specific different part. Part estimation image generation means;
A difference between the abnormal part extracted image generated by the abnormal part extracted image generating unit and the specific part estimated image generated by the unique part estimated image generating unit is calculated, and the unique part is extracted from within the abnormal part extracted image. and a diagnosis result image generation means for generating a diagnosis result image which eliminated the,
The abnormal part extracted image generation means learns an image pattern of a medical image having no abnormality, and restores and outputs a medical image having no abnormality with respect to the input medical image to the target. Based on the difference image obtained by inputting the medical image and calculating the difference between the input image and the output image to the auto encoder, the abnormal part extraction image is generated,
When the medical image including the specific part and having no abnormality is input, the specific part estimation image generating unit inputs and extracts the medical image including the specific part and having no abnormality to the abnormal part extraction image generating unit. An image processing apparatus , wherein the target medical image is input to a neural network that has been trained to restore and output the generated image, and the output image from the neural network is used as the characteristic portion estimation image . The image processing apparatus according to claim 1.
An image processing apparatus, further comprising an image display means for displaying the diagnosis result image generated by the diagnosis result image generation means on a display device. An abnormal part extraction image generating means for generating an abnormal part extraction image obtained by extracting, as an abnormal part, a difference part from a medical image obtained by photographing a healthy person from within the input target medical image;
From the target medical image, a unique part estimation that generates a unique part estimation image obtained by extracting a unique part of an image that is different from a medical image obtained by photographing a healthy person but is not medically abnormal. Image generating means;
A difference between the abnormal part extracted image generated by the abnormal part extracted image generating unit and the specific part estimated image generated by the unique part estimated image generating unit is calculated, and the unique part is extracted from within the abnormal part extracted image. and a diagnosis result image generation means for generating a diagnosis result image which eliminated the,
The abnormal part extracted image generation means learns an image pattern of a medical image having no abnormality, and restores and outputs a medical image having no abnormality with respect to the input medical image to the target. Based on the difference image obtained by inputting the medical image and calculating the difference between the input image and the output image to the auto encoder, the abnormal part extraction image is generated,
When the medical image including the specific part and having no abnormality is input, the specific part estimation image generating unit inputs and extracts the medical image including the specific part and having no abnormality to the abnormal part extraction image generating unit. An image processing system , wherein the target medical image is input to a neural network that has been trained to restore and output the generated image, and the output image from the neural network is used as the characteristic portion estimation image . The image processing system according to claim 3 .
An image processing system, further comprising: an image display unit that displays the diagnostic result image generated by the diagnostic result image generation unit on a display device. An abnormal part extraction image generation procedure for generating an abnormal part extraction image obtained by extracting, as an abnormal part, a difference part from a medical image obtained by photographing a healthy person from within the input target medical image;
From the target medical image, a unique part estimation that generates a unique part estimation image obtained by extracting a unique part of an image that is different from a medical image obtained by photographing a healthy person but is not medically abnormal. Image generation procedure;
The difference between the abnormal part extracted image generated in the abnormal part extracted image generation procedure and the specific part estimated image generated in the specific part estimated image generation procedure is calculated, and the unique part is excluded from the abnormal part extracted image. An image processing program for causing a computer to execute a diagnostic result image generation procedure for generating a diagnostic result image ,
The abnormal part extraction image generation procedure includes learning the image pattern of a medical image having no abnormality, restoring the medical image having no abnormality with respect to the input medical image, and outputting the target to the auto encoder Based on the difference image obtained by inputting the medical image and calculating the difference between the input image and the output image to the auto encoder, the abnormal part extraction image is generated,
In the specific part estimated image generation procedure, when a medical image having no abnormality including the specific part is input, a medical image having no abnormality including the specific part is input to the abnormal part extraction image generation unit and extracted. An image processing program, wherein the target medical image is input to a neural network that has been trained to restore and output the generated image, and the output image from the neural network is used as the characteristic portion estimation image. The image processing program according to claim 5 ,
An image processing program further comprising an image display procedure for displaying the diagnostic result image generated in the diagnostic result image generation procedure on a display device.

Images