JP2022054808A - Image processing system, program, machine learning method, and machine learning program - Google Patents

Abstract

To provide an image processing device capable of automatically extracting a target object from a medical imager with clinically applicable accuracy.SOLUTION: An image processing device 10 is provided, having a control unit 11 comprising a three-dimensional image acquisition unit configured to acquire a three-dimensional image generated using a plurality of two-dimensional images, an image division unit configured to divide the three-dimensional image into a plurality of fragments, and an area identification unit configured to recognize an area of a target object in a fragment classified as a group containing the target object and displaying the fragment in such a way that the area of the target object is distinguished.SELECTED DRAWING: Figure 1

Description

Application for application of Article 30, Paragraph 2 of the Patent Act was submitted. The paper was published on the website (https://arxiv.org/pdf/2006.1616.1.pdf) on June 29, 2nd year of Reiwa.

The present invention relates to an image processing device, a program, a machine learning method, and a machine learning program.

In the medical field, MRA (Magnetic Resonance Angiography) images of the brain are analyzed to detect aneurysms and the like, which are useful for the detection and treatment of diseases.

For example, Patent Document 1 describes that an aneurysm is detected by performing pattern matching using a template simulating the aneurysm within the search range of the aneurysm in the nuclear magnetic resonance image data (. Paragraph 0018).

It has also been attempted to analyze MRA images using a machine learning model to automatically detect an aneurysm (for example, Non-Patent Document 1).

JP-A-2015-144725

Sichtermann et al., "Deep Learning-Based Detection of Intracranial Aneurysms in 3D TOF-MRA", American Journal of Neuroradiology 40 (1), 25-32 (2019)

However, with the conventional method, although it is possible to detect the presence or absence of an aneurysm, it is difficult to perform segmentation (definition) of the aneurysm on an image. Without segmentation, it is insufficient for clinical applications such as simulation of clipping surgery. Also, when the aneurysm was very small, it was difficult to find it.

The present invention has been made in view of the circumstances described above, and an object of the present invention is to provide an image processing apparatus capable of automatically extracting an object from a medical image with an accuracy capable of clinical application. It is one of.

The image processing apparatus according to the embodiment of the present invention includes a three-dimensional image acquisition unit that acquires a three-dimensional image created based on a plurality of two-dimensional images, and an image division that divides the three-dimensional image into a plurality of fragments. It is provided with a unit and a region specifying portion for displaying the region of the object so as to be distinguishable in the fragment.

In the program according to the embodiment of the present invention, the computer has a three-dimensional image acquisition unit that acquires a three-dimensional image created based on a plurality of two-dimensional images, and an image that divides the three-dimensional image into a plurality of fragments. It functions as a division portion and a region specifying portion for displaying the region of the object so as to be distinguishable in the fragment.

The machine learning method according to the embodiment of the present invention is a machine learning method of an image processing model that identifies and displays a region of an object on a three-dimensional image, and divides the three-dimensional image into a computer. In the first machine learning step of classifying the plurality of fragments obtained in the above into a group including the object and a group not including the object, and in a fragment classified into the group including the object by a computer, the object. It includes a second machine learning step of displaying the regions so that they can be distinguished.

In the machine learning program according to the embodiment of the present invention, a plurality of fragments obtained by dividing the three-dimensional image on a computer that identifies and displays an area of an object on the three-dimensional image are subjected to the object. The first machine learning process to classify into a group containing things and a group not containing things,
In the fragment classified into the group including the object, the second machine learning step of displaying the area of the object so as to be distinguishable is executed.

According to the present invention, it is possible to provide an image processing apparatus capable of automatically extracting an object from a medical image with an accuracy capable of clinical application.

The figure which shows the schematic structure of the image processing apparatus 10 which concerns on embodiment of this invention. The figure explaining the outline of embodiment of this invention. The flowchart of the machine learning procedure of the image processing model mounted on the image processing apparatus 10 which concerns on embodiment of this invention. The figure explaining the machine learning procedure of the image processing model mounted on the image processing apparatus 10 which concerns on embodiment of this invention. The flowchart of the processing procedure of the segmentation of an aneurysm on a three-dimensional image using the machine-learned model which concerns on embodiment of this invention. The figure explaining the processing procedure of the segmentation of an aneurysm on a three-dimensional image using the machine-learned model which concerns on embodiment of this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The same elements are designated by the same reference numerals, and duplicate description will be omitted.

Embodiment FIG. 1 is a diagram showing a schematic configuration of an image processing apparatus 10 according to the present embodiment. As shown in FIG. 1, the image processing device 10 includes a control device 11 (three-dimensional image acquisition unit, image division unit, area identification unit, fragment classification unit) and a storage device 12. The control device 11 includes, as hardware, a CPU, a memory such as a ROM or RAM, an input interface, an output interface, a communication interface, and a bus connecting them. The control device 11 realizes various functions by the CPU executing a program stored in a ROM or the like. The storage device 12 is a hard disk drive or the like, and stores various programs, image data to be analyzed, and the like.

FIG. 2 is a diagram illustrating an outline of the present embodiment. The image processing apparatus 10 according to the present embodiment detects an aneurysm (object) from an MRA image of an intracranial artery by a machine learning model, and performs segmentation of the region of the aneurysm. Specifically, first, a three-dimensional image (P2) of an intracranial artery is created based on a plurality of two-dimensional MRA images (tomographic image; P1) taken in the skull. In this embodiment, a three-dimensional image of an artery is represented using a three-dimensional surface model. The three-dimensional surface model is a kind of three-dimensional model for displaying a three-dimensional object as an image, and is a model in which the inside of the object is made a cavity and expressed only by the shape of the surface. In the present embodiment, the three-dimensional image of the artery is represented by using the three-dimensional surface model, but another three-dimensional model such as a solid model that also represents the inside of the object may be used. The three-dimensional image of the artery may be created by the image processing device 10, or the created three-dimensional image may be input to the image processing device 10.

The image processing device 10 analyzes a three-dimensional image of an artery by a machine-learned model, performs segmentation of the detected aneurysm, and displays it so that it can be distinguished on the three-dimensional image (P3). The detailed procedure for segmentation of an aneurysm will be described later. In the three-dimensional image P3 of FIG. 2, the region indicated by S is a region segmented as an aneurysm.

At the stage of machine learning, a plurality of images P4 (fragments) obtained by dividing a three-dimensional image of an artery into fragments are used as learning data. The fragment image P4 includes an image in which an aneurysm is present and an image in which an aneurysm is not present.

Next, using the flowchart of FIG. 3 and FIG. 4, a machine learning procedure of an image processing model for segmenting an aneurysm, which is mounted on the image processing apparatus 10 according to the present embodiment, will be described. Machine learning is the first machine learning to classify fragment images into those with and without aneurysms, and in fragments with aneurysms, aneurysm fragmentation (so that the area of the aneurysm can be distinguished). It is performed in two stages of the second machine learning to perform (display).

(First machine learning)
First, as training data, a plurality of fragment images P4 obtained by dividing a three-dimensional image of an artery created from an MRA image are input to a neural network (first machine learning model) N1 (step S101). As a method for dividing the three-dimensional image, for example, the three-dimensional image may be divided so that the length of the included arteries is constant.

The neural network N1 classifies the input fragment image P4 into a group in which an aneurysm is present (G1) and a group in which no aneurysm is present (G2) (step S102). The above steps S101 to S102 are repeated until the fragment images P4 are correctly classified.

(Second machine learning)
Next, as training data, fragment images classified into the group (G1) in which the aneurysm exists are input to the neural network (second machine learning model) N2 (step S103).

The neural network N2 discriminates the aneurysm region (S) of the input fragment image P4 and displays it distinguishably as shown in the image P5 (step S104). The above steps S103 to S104 are repeated until the area of the aneurysm is correctly displayed.

Next, the procedure for processing the segmentation of the aneurysm on the three-dimensional image using the machine-learned model according to the present embodiment will be described with reference to the flowchart of FIG. 5 and FIG.

First, the image processing device 10 divides a three-dimensional image of an artery created from a two-dimensional MRA image to generate a plurality of fragment images (P4) (step S201). As a method for dividing the three-dimensional image, for example, the three-dimensional image may be divided so that the length of the included arteries is constant.

Next, the image processing device 10 analyzes the generated fragment image (P4) by the trained neural network N1 and classifies it into a group (G1) in which an aneurysm is present and a group (G2) in which an aneurysm is not present. (Step S202).

Next, the image processing device 10 analyzes the fragment images classified into the group (G1) in which the aneurysm exists by the trained neural network N2, and determines the region (S) of the aneurysm of the fragment image. , Displayed distinguishably as shown in image P5 (step S203).

Next, the image processing apparatus 10 performs image processing for increasing the resolution of the periphery of the aneurysm region for each image P5, and outputs the image P6 (step S204).

Further, the image processing apparatus 10 reconstructs a three-dimensional image (P7) of the artery in a state before being divided into fragments by using the fragment image P6 and the fragment image not including the aneurysm (step S205).

As described above, according to the present embodiment, a three-dimensional image of an artery created from a two-dimensional MRA image is divided into fragments, and each fragment is analyzed using a machine learning model to obtain an aneurysm. Changed to perform segmentation. By subdividing the entire three-dimensional image into fragments in this way, the region of the aneurysm on the image becomes relatively large, so that the boundary line of the region can be specified more accurately. In addition, small aneurysms can be detected without being overlooked. In this way, it can also be used for applications that require accurate shape information of aneurysms, such as simulation of clipping surgery.

In addition, after dividing the entire 3D image into fragments, the fragments were classified into fragments containing and without aneurysms, and segmentation processing was performed only on the fragments containing aneurysms. As a result, the load of the segmentation process can be reduced.

In the learning of the machine learning model, the fragment is divided into a group with and without an aneurysm, and a fragment with an aneurysm is divided into a stage of discriminating the region of the aneurysm and performing segmentation. I did it. As a result, the process of performing segmentation is performed only on the image known to have an aneurysm, so that the learning efficiency and accuracy can be improved.

The present invention is not limited to the above-described embodiment, and can be implemented in various other forms without departing from the gist of the present invention. For this reason, the above embodiments are merely examples in all respects and are not to be construed in a limited manner. For example, the above-mentioned processing steps can be arbitrarily changed in order or executed in parallel within a range that does not cause a contradiction in the processing contents.

10 ... Image processing device 11 ... Control device 12 ... Storage device

Claims (9)

A 3D image acquisition unit that acquires 3D images created based on multiple 2D images,
An image segmentation unit that divides the three-dimensional image into a plurality of fragments,
An image processing apparatus including a region specifying unit for displaying the region of the object so as to be distinguishable in the fragment. A fragment classification unit for classifying a plurality of the fragments into a group including an object and a group not including the object is provided.
The area specifying part is
The image processing apparatus according to claim 1, wherein in a fragment classified into a group including the object, a region of the object is discriminated and the discriminated region is displayed so as to be distinguishable. The fragment classification unit
The image processing apparatus according to claim 2, wherein the first machine learning model that has been trained is used to classify a group that includes an object and a group that does not include an object. The first machine learning model is created according to claim 3, wherein a plurality of fragments obtained by dividing the three-dimensional image are classified into a group including the object and a group not including the object as learning data. The image processing device described. The area identification part is
A second machine learning model that has been trained is used to discriminate regions of the object and display the discriminated regions so that they can be distinguished in fragments classified into groups containing the object. 2. The image processing apparatus according to 2. The second machine learning model is
The image processing apparatus according to claim 5, wherein the fragment classified into the group including the object is used as learning data, the area of the object is discriminated, and the discriminated region is displayed so as to be distinguishable. .. Computer,
A 3D image acquisition unit that acquires 3D images created based on multiple 2D images,
An image segmentation unit that divides the three-dimensional image into a plurality of fragments,
In the fragment, a program that functions as an area specifying unit that displays the area of the object so as to be distinguishable. It is a machine learning method of an image processing model that identifies and displays the area of an object on a three-dimensional image.
A first machine learning step of causing a computer to classify a plurality of fragments obtained by dividing the three-dimensional image into a group including the object and a group not including the object.
A machine learning method comprising a second machine learning step of causing a computer to display a region of the object so that it can be distinguished in fragments classified into a group including the object. On a computer that identifies and displays the area of an object on a three-dimensional image,
A first machine learning step of classifying a plurality of fragments obtained by dividing the three-dimensional image into a group including the object and a group not including the object.
A machine learning program for executing a second machine learning step of displaying a region of the object so as to be distinguishable in fragments classified into a group including the object.

Images