JP2020075063A - Image interpolation/organ extraction device and program thereof - Google Patents

Abstract

To provide a device and a program capable of interpolating a medical tomographic image with high accuracy and performing high-precision contour extraction with a convolution computing device for deep learning to automatically extract a 3D area of an organ specified in a medical image.SOLUTION: When interpolating an intermediate tomographic image from consecutive medical image tomographic images, input pixels are normalized, a deep learning model is configured, and a loss function is selected, and interpolation effect is improved. Especially, when a 3D area is automatically extracted, the interpolation is performed in advance, thereby displaying the volume and contour of the organ that is the extracted target area with high accuracy.SELECTED DRAWING: Figure 1

Description

The present invention relates to a medical image 3D area automatic interpolation / extraction method and program for automatically interpolating / extracting a specified area from a three-dimensional medical volume image using machine learning.

In recent years, in order to perform appropriate radiation treatment, an apparatus that precisely irradiates a tumor with a radiation dose after defining an organ region in detail on an X-ray CT image or an MRI image has been utilized. A conventional radiation irradiation support apparatus includes, for example, a tomographic image information acquisition unit that accumulates tomographic image information of an image captured by an X-ray CT image or an MRI image (nuclear magnetic resonance image) imaging apparatus, and a tomographic image information acquisition unit. The memory connected to the memory, the tomographic image / 3D volume image calculation display section connected to the memory, the calculation display section, the display for displaying the calculation results, and the organ area for the display target displayed on the display are defined. It was equipped with an input unit for simulating a radiation treatment plan. The 3D volume image means a three-dimensional image that is a set of tomographic images obtained by a diagnostic device such as CT.

In addition, in order to define the area of an organ, generally, a technique of automatically extracting the organ area is used, except for the method of pointing and inputting the contour of the organ on each tomographic image with a mouse. ing.

For example, Patent Document 1 discloses a method of automatically extracting a target organ contour from each tomographic image and then performing 3D reconstruction to extract a 3D region of the organ.

JP-A-2014-64835 (Published April 17, 2014 (April 17, 2014)) JP-A-2018-117883 (Published August 2, 2018 (August 2, 2018))

In the above-mentioned conventional apparatus for extracting an organ image, the brightness level of the pixel of the tomographic image generated from the output of the image diagnostic apparatus such as CT or MRI is adjusted for each device. After that, a low-resolution image of the processing target image is separately generated, a specific area is extracted from the low-resolution image by the graph cut method, and a contour area including the contour of the specific area in the processing target image is extracted based on the extraction result of the specific area. The contour is extracted by using the contour area setting means for setting the processing target image and the second extracting means for extracting the area corresponding to the specific area from the contour area by the graph cut method. The reason for using the low-resolution image is that the graph cut method requires a long processing time depending on the resolution of the processing target image in the sequential calculation. Then, the 3D area is constructed by linearly interpolating the areas of the finally extracted tomographic images. However, in the conventional linear interpolation method, each pixel of the tomographic image newly generated is interpolated by referring to the range of the adjacent pixels of the original tomographic image by about 1 to 2, so that a 3D image composed of a sparse tomographic image is formed. The organ curve cannot be sufficiently expressed for the volume image. Further, in the conventional organ extraction method including the graph cut method, the 3D regions are reconstructed by linearly combining the regions after performing the region extraction for each tomographic image between the tomographic images. Since the information is not available at the time of area extraction, the surface of the organ that is 3D reconstructed using the extraction result has a jagged pattern called jaggies, and the accuracy cannot be improved. Desirably, the region extraction accuracy of the organ is enhanced and the organ extraction is performed on the 3D volume image composed of high-accuracy tomographic images.

The first image processing apparatus of the present invention includes a learning image reading unit that reads a continuous m (m is an integer of 3 or more) tomographic image set, and an arbitrary (mn) number (n is 1) from continuous tomographic images. Above, an integer less than or equal to (m-2)) is used as learning input data, and the remaining n sheets are used as teacher data.A machine learning unit that performs machine learning for image interpolation and a learning result that stores rules that are learned A rule storage unit, a prediction image reading unit that reads continuous (mn) prediction tomographic images, a learned rule reading unit that calls a learned rule from the learned rule storage unit, and (mn) prediction images The first image processing apparatus includes a prediction calculation output unit that newly generates and outputs n tomographic images based on the tomographic image and the learned rule.

With such a configuration, by learning the interpolation rule by machine learning such as convolution calculation used for deep learning, each pixel of the newly generated tomographic image is interpolated by referring to a part or all pixels of the neighboring tomographic image. Therefore, it is possible to reconstruct a high-definition 3D volume image with less jaggies, which is composed of all the tomographic images including the original tomographic image and the intermediate tomographic image.

The second image processing apparatus of the present invention is configured such that x (x is an integer of 1 or more) learning tomographic images and y (y is an integer of x or less) targets from a 3D volume image with region information of a target organ. A learning image reading unit that reads a label image indicating organ region information, a machine learning unit that performs machine learning for region extraction using the tomographic image as learning input data, and the label image as teacher data, and a learning result. A learned rule storage unit that stores rules, a prediction image reading unit that reads x prediction tomographic images, a learned rule reading unit that calls a learned rule from the learned rule storage unit, and the x The image processing apparatus includes a prediction calculation output unit that newly generates and outputs y label images indicating an organ region based on the prediction tomographic images and the learned rule.

With such a configuration, it is possible to divide the 3D volume image into a set of tomographic groups without performing the machine learning calculation of the image interpolation for the entire 3D volume image at a time, and perform the sequential prediction calculation while suppressing the calculation resources. , 3D area of the whole organ can be extracted.

Further, the third image processing apparatus of the present invention is different from the second invention in that the learning image reading unit newly generates the 3D volume image A by interpolation processing using the image processing apparatus of the first invention. From the 3D volume image B, it is characterized by reading x (where x is an integer of 1 or more) learning tomographic images and y (y is an integer of x or less) label images showing the region information of the target organ. , An image processing device.

With such a configuration, by performing the organ 3D region extraction using the second image processing device on the high-definition 3D volume image interpolated and output from the first image processing device, a It is possible to extract the organ shape that expresses the volume and contour with high accuracy.

Further, in the fourth image processing apparatus of the present invention, the learning image reading unit is different from the first invention in that the learning image reading unit repeatedly learns an arbitrary m continuous tomographic image sets from a tomographic image group forming a 3D volume image. The image processing apparatus is characterized in that it is read as an image for use.

With such a configuration, a continuous part or all of the tomographic image of the 3D volume image is repeatedly acquired as learning data and machine learning is performed, so that the machine learning calculation of the image interpolation for the entire 3D volume image is not performed at one time. Also, it is possible to generate an interpolation image for an arbitrary subset of the 3D volume image tomographic image by referring to the continuity information between the tomographic images while suppressing the calculation resources.

Further, the fifth image processing apparatus of the present invention is different from the second invention in that the learning image reading unit includes an arbitrary x continuous tomographic image sets from a tomographic image group forming a 3D volume image, and The image processing device is characterized in that a corresponding y label image set is repeatedly read as an image for learning.

With such a configuration, the tomographic image of the 3D volume image and a part or all of the tomographic image are repeatedly acquired as learning data and machine learning is performed, so that it is not necessary to perform the machine learning calculation of the area extraction for the entire 3D volume image at one time. By controlling the continuity information between the tomographic images while suppressing the calculation resources, it is possible to perform the region extraction processing for an arbitrary subset of the tomographic images of the 3D volume image.

The sixth image processing method of the present invention is different from the first or second invention in that the learning image reading unit and the prediction layer image reading unit read pixels of the tomographic image when reading the learning or prediction tomographic image. The values are not processed, or the contrast and density are normalized according to the image type such as X-ray CT or MRI or the pixel value distribution of the organ, and then the machine learning unit and the prediction calculation of the first or second invention. The image processing method is characterized in that the image is input to an output unit.

In general, most medical images have pixel information of 2 bytes or more, and therefore, if they are directly input to a neural network without being processed, variations in gray value are large, leading to deterioration in accuracy of deep learning results. By using this method, the variation in the distribution of the tomographic images for learning or prediction is reduced, and then input to the machine learning unit or the prediction calculation output unit, which leads to improvement in calculation accuracy and learning speed of machine learning.

Further, the seventh image processing apparatus of the present invention is different from the second invention in that the learning image reading unit calculates in advance the volume range of the extraction target organ in the 3D volume image, and the slice including the extraction target organ is calculated. An image processing apparatus characterized in that an image is repeatedly read as a learning image with an arbitrary probability.

With such a configuration, the learning data can be efficiently acquired regardless of the size of the organ or the range of existence of the organ, which leads to improvement in the calculation accuracy and learning speed of machine learning.

Further, the eighth image processing method of the present invention is different from the first invention or the second invention in that the learning image reading unit performs a shape registration technique from the 3D volume image A to the 3D volume image B, The image processing method is characterized in that the learning pattern is changed by three-dimensionally deforming the image A, and then is input to the machine learning unit of the first invention or the second invention.

With such a method, it is possible to expand a small number of learning data to a large number of high-quality learning data while maintaining the rationality of organ deformation, which leads to an improvement in calculation accuracy of machine learning.

Further, in the ninth image processing apparatus of the present invention, when the cross-sectional axis of the 3D volume image is set to three directions of X, Y, and Z, the predicted image reading unit is different from the second invention in that the tomographic image set in each axis is set. Are input to the prediction calculation output unit according to claim 2, and the output prediction results in each direction are linearly combined to obtain a final output.

With such a configuration, the prediction result that is finally determined in a voting format based on the prediction results of a plurality of machine learning rules reduces the weakness of the machine learning rule based on a single tomographic direction, which leads to improvement in area extraction accuracy.

Further, the tenth image processing device of the present invention is different from the second invention in that the predictive image reading unit has a plurality of pixel value ranges in a plurality of pixel value ranges according to an image type such as X-ray CT and MRI and a pixel value distribution of an organ. The pixel values of the tomographic image are normalized and then input to the prediction calculation output unit according to claim 2, and the prediction results in the respective pixel value ranges that have been output are linearly combined to obtain the final output. It is an image processing device.

With such a configuration, the prediction result that is finally decided in a voting format based on the prediction results of a plurality of machine learning rules is an organ with an unclear contour by relaxing the weak points of the machine learning rule based on a single pixel value range. On the other hand, it also leads to an improvement in the accuracy of region extraction.

In addition, the eleventh image processing method of the present invention is different from the first or second invention in that the learning image reading unit reads the learning images repeatedly, from the learning 3D volume image group to the 3D volume image. The image processing method is characterized in that the process of reading the entire A and the process of reading a partial set of tomographic images from the 3D volume image A as learning images are performed asynchronously.

With this configuration, the learning image can be supplied to the machine learning unit of the first invention or the second invention regardless of the reading time of a large-capacity 3D volume image, and a decrease in the speed of machine learning can be suppressed. You can

In addition, the twelfth image processing method of the present invention provides a combination of a specific input image and a specific teacher feature image or a teacher feature vector with an arbitrary probability when reading an iterative learning image in machine learning. Thus, the image processing method is characterized in that the learned rule can be characterized and identified by fixing the combination according to the learned rule.

With this method, the learning rule can be identified by using the input image used for learning, and it is possible to prevent the learning rule from leaking or being stolen without permission.

According to the image processing device of the present invention, highly accurate organ region extraction can be performed using machine-learned rules.

Conventionally, when creating radiation treatment plan data, it took a lot of time because the organ contour was created by handwriting, but learning by including the organ data created by handwriting as a learning model The accuracy can be improved, a highly accurate organ shape can be obtained, and a highly accurate treatment plan can be planned. Furthermore, if the organ region can be easily extracted, the present invention can be utilized in the surgical operation region such as surgery planning, surgery simulation and navigation.

3 is a block diagram of the image processing apparatus 1 according to the first embodiment. FIG. 3 is a flowchart showing the operation of the image processing apparatus 1 according to the first embodiment. The figure which shows the verification result of the effect of 1st Embodiment. The block diagram of the image processing apparatus 2 in 2nd and 3rd embodiment. 9 is a flowchart showing the operation of the image processing device 2 according to the second and third embodiments. The figure which shows an example of the tomographic image of the 3D volume image with area information of an organ. The figure which shows the verification result of the effect of 2nd Embodiment. The figure for demonstrating the method of reading the repeated learning data in 4th and 5th embodiment. The figure for demonstrating the effect of 6th Embodiment. The figure for demonstrating the learning data selection method according to the range of the volume of the target organ in 7th Embodiment. The figure for demonstrating the method of learning data expansion by shape alignment in 8th Embodiment. The figure for demonstrating the ensemble processing method with respect to the prediction result by the multiple tomographic directions in 9th Embodiment. The figure for demonstrating the ensemble processing method with respect to the prediction result in the range of a several pixel value in 10th Embodiment. The figure for demonstrating the operation | movement for asynchronous learning data provision in 11th Embodiment. The figure for demonstrating the combination of the specific input image and teacher image in 12th Embodiment. The figure which shows a CPU bus structure.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are designated by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

FIG. 1 is a block diagram of an image processing apparatus 1 according to this embodiment. The image processing apparatus 1 includes a learning image reading unit 101, a machine learning unit 102, a learned rule storage unit 103, a prediction image reading unit 104, a learned rule reading unit 105, and a prediction calculation output unit 106. The image processing apparatus 1 is also connected to the 3D volume image storage device 100. The image processing device 1 and the 3D volume image storage device 100 may be connected in a wired or wireless manner.

FIG. 2 is a flowchart of processing implemented by the image processing apparatus 1 when learning interpolation processing and performing prediction output.

(Step S201):
In step S201, the learning image reading unit 101 arbitrarily acquires one learning 3D volume image from the learning image group stored in the 3D volume image storage device 100.

(Step S202):
In step S202, the learning image reading unit 101 arbitrarily selects and acquires a subset (referred to as a sub-set) of tomographic images from a set of m continuous tomographic images of the learning 3D volume image. m is an integer of 3 or more.

(Step S203 to Step S204):
The machine learning unit 102 divides the m tomographic image sets into (mn) input images (input data) and n teacher images (teacher data), and from the input images, the teacher images. Machine learning for interpolating and outputting is performed.

(Steps S205 to S206):
The machine learning unit 102 refers to the rule or the number of iterations designated by the user, and when the learning is not completed, returns to step S201 and performs the iterative machine learning. When the learning is completed, the learned rule is output to the learned rule storage unit 103.

(Step S207):
In step S207, the prediction image reading unit 104 acquires one prediction 3D volume image designated by the user from the prediction image group stored in the 3D volume image storage device 100.

(Step S208):
In step S208, the prediction image reading unit 104 divides the tomographic image group forming the 3D volume image for prediction into a plurality of (m−n) tomographic image sub-sets, and the prediction calculation output unit 106 sequentially. To donate.

(Step S209):
In step S209, the learned rule reading unit 105 acquires the learned rule stored in the learned rule storage unit 103 and supplies the learned rule to the prediction calculation output unit 106.

(Steps S210 to S212):
The prediction calculation output unit 106 performs the image interpolation calculation on the provided (mn) tomographic images based on the learned rule. If an unprocessed tomographic image sub-set remains, the process returns to step S208 to additionally perform image interpolation calculation. When the image interpolation calculation for all the tomographic image sub-sets is completed, the original tomographic image group of the 3D volume image for prediction and the tomographic image group newly generated by the interpolation calculation are merged and output as an interpolated 3D volume image.

In FIG. 3, as an example, a 3D stereoscopic image (a) generated after the conventional linear interpolation method is performed on one 3D volume image, and m = 5 shows a comparative image of a 3D stereoscopic image (c) generated after performing image interpolation on the 3D volume image after performing machine learning for image interpolation by setting 5, n = 1. Compared with the conventional linear interpolation method, the image interpolated and output by the image processing device 1 has less unnatural jaggies and can generate a smooth surface.

FIG. 4 is a block diagram of the image processing device 2 according to the present embodiment. The image processing apparatus 2 includes a learning image reading unit 201, a machine learning unit 202, a learned rule storage unit 203, a prediction image reading unit 204, a learned rule reading unit 205, and a prediction calculation output unit 206. Further, the image processing device 2 is connected to the 3D volume image storage device 100 or the prediction calculation output unit 106 of the image processing device 1 in the first embodiment. The connection form of the image processing device 2 and the 3D volume image storage device 100, or the prediction calculation output unit 106 of the image processing device 1 such as wired or wireless connection does not matter.

FIG. 5 is a flowchart of processing implemented by the image processing apparatus 2 when learning and predictive output of region extraction of an organ.

(Step S501):
In step S501, the learning image reading unit 201 arbitrarily acquires one learning 3D volume image from the learning image group stored in the region information-added 3D volume image storage device 100 of the target organ.

(Step S502):
In step S502, the learning image reading unit 201 arbitrarily selects and acquires x continuous tomographic image sets of the learning 3D volume image. Further, the learning image reading unit 201 acquires y label images indicating the region information of the target organ on the tomographic image set. x is an integer of 1 or more, and y is an integer of 1 or more and x or less.

(Steps S503 to S504):
The machine learning unit 202 performs machine learning for area extraction using the x tomographic image set as an input image and the y label images as a teacher image.

(Steps S505 to S506):
The machine learning unit 202 refers to the rule or the number of repetitions designated by the user, and when the learning is not completed, returns to step S501 and repeats machine learning. When the learning is completed, the learned rule is output to the learned rule storage unit 203.

(Step S507):
In step S507, the prediction image reading unit 204 selects one 3D volume image specified by the user from the prediction image group stored in the 3D volume image storage device 100 or the image processing device 1 according to the first embodiment. The 3D volume image interpolated and output by the prediction calculation output unit is acquired and read as the 3D volume image for prediction.

(Step S508):
In step S508, the prediction image reading unit 204 divides the tomographic image group forming the 3D volume image for prediction into a plurality of tomographic image sub-sets of x sheets, and sequentially supplies the sectional images to the prediction calculation output unit 206.

(Step S509):
In step S <b> 509, the learned rule reading unit 205 acquires the learned rule stored in the learned rule storage unit 203 and gives the learned rule to the prediction calculation output unit 206.

(Steps S510 to S512):
The prediction calculation output unit 206 performs a region extraction calculation on the provided x tomographic images based on the learned rule. When the unprocessed tomographic image sub-set remains, the process returns to step S508 to additionally perform the area extraction calculation. When the area extraction calculation is completed for all the tomographic image sub-sets, the newly generated area label images are merged and output as a 3D label image.

FIG. 6 shows an example of one tomographic image of a DICOM-RT format image as a 3D volume image with organ region information. In the DICOM-RT image, region information 602 of each organ is added to each tomographic image 601 of CT or MRI. By using the image processing apparatus 2 for such a 3D volume image with region information of an organ, it is possible to perform machine learning of a region extraction rule using CT tomographic images as input data and region information as teacher data. Further, in FIG. 7, as an example, a region 702 is extracted for each tomographic image based on a rule in which the region of the right lung portion is machine-learned by the image processing device 2 for a sub-set 701 of continuous tomographic images of one 3D volume. As a result, a surface 3D image 703 of the right lung reconstructed from the output 3D label image is shown.

As described above, according to the present embodiment, when the contour of the target extracted organ can be displayed as a part of the tomographic image, the organ can be extracted easily and accurately by using machine learning. it can.

In addition, the contour of the organ image that has been extracted by drawing the contour by handwriting can be reused as a learning model image, as a matter of course, and by using a tomographic image already accumulated as an organ image for machine learning, Organ extraction can be performed accurately.

Next, the fourth and fifth embodiments of the present invention will be described with reference to FIG. In the fourth and fifth embodiments, the learning image reading units 101 and 201 of the image processing device 1 or the image processing device 2 select the 3D volume image 801 each time learning is performed, and a subset that configures the image. The image 802 is read as learning data, but this is repeated a sufficiently large number of times to randomly acquire from the 3D volume image 701 and provide it to the machine learning units 102 and 202, so that an arbitrary subset within the 3D volume image is obtained. It is possible to construct a rule for which a predictive calculation can be performed.

In techniques such as deep learning that are often used in recent years, the hardware resources required for learning and prediction increase in proportion to the number of images to be processed at once, so that 3D volume images are processed by machine learning at once. Implementation environment is limited.

According to the present embodiment, the machine learning is repeatedly performed on an arbitrary subset image, so that the continuity of information between tomographic images is not lost even when a resource more limited than the conventional one is used. Highly accurate machine learning and prediction calculation can be performed on the entire 3D volume image.

Next, a sixth embodiment will be described.

In most of the 3D volume image, each pixel is composed of 2 bytes or more, but the image actually recorded is not distributed over the entire range of pixel values. For example, in a CT image in which each pixel is composed of 2 bytes, the total range of each pixel value is -32768 to 32767, but the CT value of an organ or bone actually imaged is about -500 to 500. is there. Further, when limited to specific organs such as the liver and kidneys, the range of CT values that can be taken by the target organ is further narrowed.

Machine learning such as deep learning is an algorithm that extracts the feature amount based on the difference in brightness value between pixels in the image, so it is better to convert it to an image that excludes unnecessary pixel areas and use it for analysis. Often improves.

Therefore, in the present embodiment, the machine learning unit 102 and the prediction calculation output unit 106 of the image processing apparatus 1 are 106, and the machine learning unit 202 and the prediction calculation output unit 206 of the image processing apparatus 2 are the type and extraction of the provided tomographic image. The accuracy of interpolation calculation and region extraction can be improved by limiting the pixel value range according to the target organ, normalizing the pixel value, and then using it for machine learning and prediction calculation.

As an example, FIG. 9 shows a tomographic image (a) before normalization and a tomographic image (b) that is normalized by limiting the pixel value range to about −150 to 150, which is close to the existence range of the pixel value of the liver. Show. Since the contour of the liver region is outstanding due to the normalization, it is estimated that the extraction accuracy is improved.

Next, a seventh embodiment will be described with reference to FIG.

In the second embodiment, the learning image reading unit 201 reads an arbitrary continuous tomographic image set and a label image indicating the region information of the target organ from the 3D volume image, and uses the machine learning unit 202 as a learning image. To donate.

In the case of an organ having a small volume, since the organ region exists only in a few tomographic images in the 3D volume image, acquiring any continuous tomographic image according to the uniform distribution acquires a tomographic image that does not affect the learning accuracy. This often leads to a reduction in learning efficiency.

Therefore, in the present embodiment, as shown in FIG. 10, when the learning image reading unit 201 calculates the existence range of the volume of the extraction target organ 1001 in the 3D volume image in advance and acquires it as a repeated learning image. In addition, by performing control so as to select a tomographic image set with an arbitrary probability from the tomographic image unit 1002 including the extraction target organ, efficient machine learning can be implemented in a short time. The arbitrary probability may be a probability designated by the user or may be automatically determined according to a normal distribution or the like.

Next, an eighth embodiment will be described. In machine learning, it is indispensable to use more learning data in order to improve learning accuracy and judgment performance. When the number of learning data is limited, the learning data is expanded by using means such as contrast conversion and expansion / contraction deformation with respect to the original learning data. Although a similar method can be applied to a 3D volume image, unnatural organ deformation leads to a reduction in learning accuracy.

Therefore, in the present embodiment, the learning image reading units 101 and 201 in the first embodiment or the second embodiment, as shown in FIG. 11, include the 3D volume original image 1101 to the 3D volume reference image 1102. By performing the 3D shape registration technique on the 3D volume and adding a new deformed image 1103 of the 3D volume generated by deforming the soft body from the original image 1101 of the 3D volume to the learning image group, rational organ deformation is maintained. However, it is possible to expand the learning data as high quality learning data and improve the accuracy of machine learning.

Next, a ninth embodiment will be described. In machine learning, a technique is known in which a plurality of models having equivalent performance are learned and the prediction results are linearly combined to improve the determination accuracy. This technique is called an ensemble processing technique. In the present embodiment, by applying the ensemble processing technique, the prediction image reading unit 104 or 204 of the image processing device 1 or the image processing device 2 sets the cross-sectional axis of the 3D volume image in three directions of X, Y, and Z. At this time, the accuracy of the final prediction result can be improved by using the tomographic image set on each axis as the input of the prediction calculation output unit 106 or 206 and linearly combining the obtained prediction results in each tomographic direction.

In FIG. 12, as an example, for one 3D volume image, the prediction image reading unit 204 of the image processing apparatus 2 receives the tomographic image 1201 of each of the X, Y, and Z axes of the 3D volume image as the input of the prediction calculation unit 206. By linearly combining the obtained region extraction results 1202 on the respective axes and outputting them as the final prediction result 1203, it is possible to improve the region extraction accuracy.

Next, a tenth embodiment will be described. In the present embodiment, the ensemble technique is applied, and the predictive image reading unit 204 of the image processing apparatus 2 uses the tomographic images in a plurality of pixel value ranges according to the image type such as X-ray CT or MRI and the pixel value distribution of an organ. The pixel value normalization method according to the sixth embodiment is applied to the pixel values of 1), and the obtained prediction results are linearly combined to improve the accuracy of the final prediction result.

In FIG. 13, as an example, the original image 1301 of the read CT tomographic image group is obtained by normalizing the three types of pixel value ranges of −1000 to 1000, −400 to 400, and −100 to 100. The normalized normalized tomographic image group 1302 is supplied as an input to the prediction calculation unit 206, and the obtained region extraction results 1303 in each pixel value range are linearly combined to output the final prediction result 1304. , Realization of improvement of region extraction accuracy.

The plurality of pixel value ranges may be designated by the user or may be automatically determined according to the distribution of the target organ.

Next, an eleventh embodiment will be described with reference to FIG. The learning image reading unit 101 or 201 of the image processing device 1 or the image processing device 2 reads a 3D volume image each time learning is performed, and selects a subset image forming the image as learning data. If the size is large, it takes a long time to provide the learning data to the machine learning unit, which causes a delay in learning time and a decrease in learning efficiency.

Therefore, in the present embodiment, as shown in FIG. 14, a process 1402 of reading a 3D volume image 1401 from a group of learning 3D volume images and a subset 1403 of arbitrary tomographic images from the 3D volume image are selected as learning images. By performing the processing 1404 and the processing 1404 that are performed asynchronously, learning data can be provided to the machine learning unit without delay and efficient learning can be performed.

General-purpose machine learning software is easy to use for machine learning, but since learned rules are stored in a common format, the learned rules are easily secondarily used without permission of the creator.

Therefore, in the present embodiment, as shown in FIG. 15, when machine learning is carried out, when a learning image is read, a specific input image is input to learning input image group 1501 and learning teacher image group 1502. And a specific teacher feature image or a teacher feature vector combination 1503 is given to the machine learning unit at an arbitrary probability. Then, by learning the combination in the learned rule and fixing the output for the input, the learned rule can be characterized and identified, so that it can be determined from the obtained image that it has been used, It becomes a deterrent to secondary use. The combination 1503 may be an image unrelated to the input image group 1501 and the teacher image group 1502. The arbitrary probability may be a probability designated by the user or may be automatically determined according to a normal distribution or the like. For example, even if the number of times the input image group 1501 and the teacher image group 1502 are given to the machine learning unit and the number of times the combination 1503 is given to the machine learning unit are specified to have a probability of 100: 1, by repeating this many times, The learning unit always learns the input image in the combination 1503 so as to output the teacher feature image in the combination 1503. Further, although the present embodiment describes a specific combination as an image, the input and teacher features may be in vector format, which is a list of numerical values.

Further, in the image processing apparatus and the image processing method described in the above embodiments, each block may be individually made into one chip by a semiconductor device such as an LSI, or may be made into one chip so as to include a part or all of the blocks. May be.

The name used here is LSI, but it may also be called IC, system LSI, super LSI, or ultra LSI depending on the degree of integration.

Also, the method of circuit integration is not limited to LSI, and it may be realized by a dedicated circuit or a general-purpose processor. A programmable FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure connection and setting of circuit cells inside the LSI may be used.

Further, a part or all of the processing of each functional block of each of the above embodiments may be realized by a program. Then, a part or all of the processing of each functional block of each of the above-described embodiments is performed by a central processing unit (CPU) in a computer. A program for performing each processing is stored in a storage device such as a hard disk or a ROM, and is read out and executed in the ROM or the RAM.

Further, each process of the above-described embodiments may be realized by hardware, or may be realized by software (including a case where it is realized together with an OS (operating system), middleware, or a predetermined library). Further, it may be realized by mixed processing of software and hardware.

For example, when each functional unit of the above-described embodiment (including modified examples) is implemented by software, the hardware configuration shown in FIG. 16 (for example, CPU, ROM, RAM, input unit, output unit, etc., is implemented by a bus Bus). Each functional unit may be realized by software processing using the connected hardware configuration).

When each functional unit of the above-described embodiment is realized by software, the software may be realized by using a single computer having the hardware configuration shown in FIG. 16, or may be realized by a plurality of computers. It may be realized by distributed processing using.

Further, the execution order of the processing methods in the above embodiments is not necessarily limited to the description of the above embodiments, and the execution order can be changed without departing from the gist of the invention.

A computer program that causes a computer to execute the above-described method and a computer-readable recording medium that records the program are included in the scope of the present invention. Here, examples of the computer-readable recording medium include a flexible disk, a hard disk, a CD-ROM, an MO, a DVD, a DVD-ROM, a DVD-RAM, a large capacity DVD, a Blue-ray, and a semiconductor memory. ..

The computer program is not limited to the one recorded in the recording medium, but may be transmitted via an electric communication line, a wireless or wired communication line, a network typified by the Internet, or the like.

The specific configuration of the present invention is not limited to the above-described embodiment, and various changes and modifications can be made without departing from the spirit of the invention.

100 3D volume image storage device 101, 201 learning image reading unit 102, 202 machine learning unit 103, 203 learned rule storage unit 104, 204 prediction image reading unit 105, 205 learned rule reading unit 106, 206 prediction calculation Output section

Claims (12)

An interpolation device for generating an interpolation image between a plurality of tomographic images by machine learning,
From the 3D volume image, a learning image reading unit that reads continuous m tomographic image sets (m is an integer of 3 or more),
From any of the continuous tomographic images, arbitrary (mn) sheets (n is an integer of 1 or more and (m-2) or less) is used as learning input data, and the remaining n sheets are used as teacher data for machine learning of image interpolation. Machine learning department,
A learned rule storage unit that stores rules that are the results of learning,
A prediction image reading unit that reads continuous (mn) prediction tomographic images,
A learned rule reading unit that calls a learned rule from the learned rule storage unit;
An image processing apparatus comprising: a prediction calculation output unit that newly generates and outputs n tomographic images based on the (mn) prediction tomographic images and learned rules. A region extraction device for automatically extracting an organ region between single or multiple tomographic images by machine learning,
Learning to read x (where x is an integer of 1 or more) learning tomographic images and y (where y is an integer of x or less) label images showing the region information of the target organ from the 3D volume image of the target organ Image reading section,
A machine learning unit that performs machine learning for area extraction using the tomographic image as learning input data and the label image as teacher data,
A learned rule storage unit that stores rules that are learning results,
A prediction image reading unit that reads x prediction tomographic images,
A learned rule reading unit that calls a learned rule from the learned rule storage unit;
An image processing apparatus comprising: a prediction calculation output unit that newly generates and outputs y label images indicating an organ region based on the x prediction tomographic images and the learned rule. The learning image reading unit according to claim 2,
From the 3D volume image B newly generated by interpolating the 3D volume image A using the image processing apparatus according to claim 1, x (x is an integer of 1 or more) learning tomographic images and y (y Is an integer less than or equal to x), which reads the label image indicating the region information of the target organ,
The image processing apparatus according to claim 2. The learning image reading unit according to claim 1,
It is characterized in that a set of arbitrary m continuous tomographic images is repeatedly read as an image for learning from a group of tomographic images forming a 3D volume image,
The image processing apparatus according to claim 1. The learning image reading unit according to claim 2,
An arbitrary x number of tomographic image sets from a tomographic image group that constitutes a 3D volume image,
It is characterized by reading a set of y label images corresponding to it as an image for repeated learning,
The image processing apparatus according to claim 1. The learning image reading unit and the prediction layer image reading unit according to claim 1 or 2,
When reading a tomographic image for learning or prediction,
Normalize contrast and density according to the image type such as X-ray CT and MRI and the pixel value distribution of organs,
An image processing method, comprising: inputting to the machine learning unit and the prediction calculation output unit according to claim 1. The learning image reading unit according to claim 2,
The volume range of the extraction target organ in the 3D volume image is calculated in advance, and the tomographic image including the extraction target organ is repeatedly read as a learning image with an arbitrary probability,
The image processing apparatus according to claim 1. The learning image reading unit according to claim 1 or 2,
From 3D volume image A, 3D volume image B is subjected to shape registration technology, and after expanding the learning pattern by three-dimensionally deforming the image A,
An image processing method, wherein the machine learning unit according to claim 1 or 2 inputs the image. When the cross-sectional axis of the 3D volume image is the three directions of X, Y, Z,
The predictive image reading unit according to claim 1 or 2,
A tomographic image set on each axis is input to the prediction calculation output unit according to claim 2,
Linearly combine the output prediction results in each direction,
An image processing device which is characterized in that the final output is performed. The predictive image reading unit according to claim 2,
Depending on the image type such as X-ray CT or MRI and the pixel value distribution of the organ,
After normalizing the pixel values of the tomographic image in multiple pixel value ranges,
Each of them is used as an input of the prediction calculation output unit according to claim 2,
Linearly combine the prediction results in each output pixel value range,
An image processing device which is characterized in that the final output is performed. The learning image reading unit according to claim 1 or 2,
When loading the image for repeated learning,
Processing to read the entire 3D volume image A from the 3D volume image group for learning,
An image processing method characterized in that an arbitrary subset of tomographic images is selected from the 3D volume image A and the reading process as a learning image is performed asynchronously. A feature extraction device for extracting features of an image by machine learning,
When loading the image for repeated learning,
By giving a combination of a specific input image and a specific teacher feature image or teacher feature vector with an arbitrary probability,
By fixing the combination according to the learned rule,
An image processing method characterized in that learned rules can be characterized and identified.

Images