JP7242410B2 - MEDICAL IMAGE PROCESSING APPARATUS, X-RAY CT APPARATUS, AND LEARNING DATA GENERATION METHOD - Google Patents

Description

The embodiments of the present invention relate to a medical image processing apparatus, an X-ray CT apparatus, and a method of generating learning data.

A problem in CT (Computed Tomography) imaging of moving organs such as the heart is motion artifacts caused by the movement of the organs. There are motion compensation techniques that compensate for motion artifacts in post-processing.

However, in motion compensation techniques, motion information about phases before and after the target phase is important. If the motion vector information representing the motion information about the forward and backward phases is different from the motion vector information possessed by the target image, the target image cannot be corrected appropriately.

JP 2018-107603 A JP 2018-175227 A

The problem to be solved by the present invention is to improve the quantitativeness at the time of diagnosis.

A medical image processing apparatus according to this embodiment includes an acquisition unit and a generation unit. The acquisition unit acquires an input image of an object. The generation unit uses a trained model trained using images of a moving object photographed in different data collection periods to perform correction that reduces the influence of the input image on the image caused by the movement of the object. Generate an image.

FIG. 1 is a block diagram showing a medical image processing apparatus according to the first embodiment. FIG. 2 is a block diagram showing a learning system that generates trained models. FIG. 3 is a block diagram showing a medical image processing apparatus in which a trained model is generated by the medical image processing apparatus itself. FIG. 4 is a diagram illustrating the concept of learning a trained model according to the first embodiment. FIG. 5 is a diagram showing the concept of how to set the data collection period for learning data according to the rotation angle of the view. FIG. 6 is a diagram showing the concept of how to set the data collection period for learning data by comparison with the ECG signal. FIG. 7 is a diagram showing a method of generating learning data according to the embodiment. FIG. 8 is a diagram illustrating a usage example of a trained model according to the first embodiment. FIG. 9 is a flow chart showing processing for calculating a motion index value as an example of using a trained model. FIG. 10 is a diagram showing a specific example of the motion index value calculation processing in FIG. FIG. 11 is a diagram illustrating an example of learning of the first trained model according to the second embodiment. FIG. 12 is a diagram illustrating an example of learning of the second trained model according to the second embodiment. FIG. 13 is a diagram illustrating an example of using a trained model according to the second embodiment. FIG. 14 is a diagram illustrating an example of a learned model during learning according to the third embodiment. FIG. 15 is a diagram illustrating an example of using a trained model according to the third embodiment. FIG. 16 is a block diagram showing an X-ray CT apparatus including a medical image processing apparatus according to the fourth embodiment.

A medical image processing apparatus, system, and program according to the present embodiment will be described below with reference to the drawings. In the following embodiments, it is assumed that parts denoted by the same reference numerals perform the same operations, and overlapping descriptions will be omitted as appropriate. An embodiment will be described below with reference to the drawings.

The medical image processing apparatus according to the present embodiment described below may be included in the console of each medical image diagnostic apparatus such as an X-ray CT (Computed Tomography) apparatus or an MRI (Magnetic Resonance Imaging) apparatus, or may be included in a workstation. may be included in a server such as a cloud server. In the following fourth embodiment, the case of being included in the console of the X-ray CT apparatus will be described. In the present embodiment described below, it is assumed that projection data captured by an X-ray CT apparatus and a reconstructed image based on the projection data are processed.

(First embodiment)
A medical image processing apparatus according to the first embodiment will be described with reference to the block diagram of FIG.
A medical image processing apparatus 1 according to the first embodiment includes a memory 11 , a processing circuit 13 , an input interface 15 and a communication interface 17 . The processing circuit 13 includes an acquisition function 131 , a generation function 133 , an image processing function 135 and a display control function 137 .

The memory 11 is a storage device such as a HDD (Hard Disk Drive), an SSD (Solid State Drive), or an integrated circuit storage device that stores various information. The memory 11 can be connected to portable storage media such as CDs (Compact Discs), DVDs (Digital Versatile Discs), flash memories, semiconductor memory devices such as RAMs (Random Access Memory), etc., in addition to HDDs and SSDs. It may also be a driving device that reads and writes various information with. Moreover, the storage area of the memory may be in the medical image processing apparatus 1 or in an external storage device connected via a network. The memory 11 stores trained models, various medical data (raw data, projection data, intermediate data such as sinograms), and various medical images (reconstructed images, CT images, MR images, ultrasound images, PET (positron emission tomography) Images, etc.) are assumed to be stored, but these trained models, medical data, medical images, etc. may be stored externally.

The processing circuit 13 includes, for example, hardware resources such as processors such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), and GPU (Graphics Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), and the like. of memory. The processing circuit 13 may also be an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other complex programmable logic device (CPLD). , may be implemented by a Simple Programmable Logic Device (SPLD). The processing circuit 13 executes an acquisition function 131, a generation function 133, an image processing function 135, and a display control function 137 by means of a processor that executes programs developed in memory. Note that each function (acquisition function 131, generation function 133, image processing function 135, and display control function 137) is not limited to being realized by a single processing circuit. The processing circuit 13 may be configured by combining a plurality of independent processors, and each function may be implemented by executing a program by each processor.

The acquisition function 131 causes the processing circuit 13 to acquire an image of a moving object (also referred to as an input image). In the present embodiment, the moving object is assumed to be a periodically moving object such as the subject's heart or lung fields that move up and down due to respiration. In addition to things that move unconsciously, such as the heart and lungs, objects that spontaneously and periodically move, such as moving hands or feet at a constant speed and within a certain range, may be used as objects. Further, the motion is not limited to periodic motion, and may be constant motion.

The generation function 133 causes the processing circuit 13 to generate a corrected image from the input image by using the trained model to reduce the influence on the image caused by the movement of the object. The trained model is trained using training data that includes a set of two or more images generated by photographing a moving object in different data collection periods, and information on the movement of the object between images (direction and motion amount). For the trained model, for example, a trained model stored in the memory 11 may be referenced, or the trained model may be stored outside the medical image processing apparatus 1 and can be referenced by the generation function 133 .

The image processing function 135 causes the processing circuit 13 to perform image processing on the input image or the corrected image. Specifically, the image processing function 135 performs filter processing and artifact removal processing on the input image or the corrected image. Further, the image processing function 135 allows the processing circuit 13 to perform filter processing and artifact removal processing as preprocessing on the first image before application of the trained model.

With the display control function 137, the processing circuit 13 controls image output so that the corrected image is displayed on a screen or the like via a display or a projector.

The input interface 15 receives various input operations from the user and outputs signals based on the received input operations to the memory 11, the processing circuit 13, the communication interface 17, and the like. For example, mice, keyboards, trackballs, switches, buttons, joysticks, touch pads, touch panel displays, etc. can be used as appropriate. In addition, in the present embodiment, the input interface 15 is not limited to physical operation components such as a mouse, keyboard, trackball, switch, button, joystick, touch pad, and touch panel display. For example, the input interface 15 also includes a processing circuit that receives a signal corresponding to an input operation from an external input device provided separately from the device and outputs this signal to the processing circuit 13 .

The communication interface 17 is a wireless or wired interface for communicating with the outside, and since a general interface may be used, description thereof will be omitted here.

Next, a learning system that generates a trained model to be used by the generation function 133 of the processing circuit 13 will be described with reference to FIG.
FIG. 2 is a block diagram showing an example of a learning system that generates trained models. The medical information processing system shown in FIG. 2 includes a learning data generation device 20, a learning data storage device 22, a model learning device 24, and a medical image diagnostic device .

The learning data generating device 20 generates learning data by generating a plurality of sets of at least two types of image data with different data acquisition periods for the same object based on various medical data and medical images. An example of a method of generating learning data will be described later with reference to FIG.

The learning data storage device 22 stores learning data generated by the learning data generation device 30 and learning data including a plurality of learning samples. For example, the learning data storage device 22 is a computer with a built-in mass storage device. The learning data storage device 22 may also be a mass storage device communicatively connected to a computer via a cable or communication network. As the storage device, an HDD, an SSD, an integrated circuit storage device, or the like can be used as appropriate.

The model learning device 24 generates a trained model by causing the machine learning model to perform machine learning according to the model learning program based on the learning data stored in the learning data storage device 22 . As machine learning assumed in this embodiment, neural networks, deep learning, decision trees, kernel regression, etc. are assumed, but not limited to these, as long as it is a method that can learn some feature from learning data, other machines It may be a learning algorithm. The model learning device 24 may be a computer such as a workstation having a general-purpose processor such as a CPU or GPU, or a processor dedicated to machine learning.

The model learning device 24 and the learning data storage device 22 may be communicably connected via a cable or a communication network. Also, the learning data storage device 22 may be installed in the model learning device 24 . In these cases, learning data is supplied from the learning data storage device 22 to the model learning device 24 . Note that the model learning device 24 and the learning data storage device 22 do not have to be communicably connected. In this case, the learning data is supplied from the learning data storage device 22 to the model learning device 24 via a portable storage medium storing the learning data.

The medical image diagnostic device 26 is assumed to be an X-ray diagnostic device, an X-ray CT device, a magnetic resonance imaging device, an ultrasonic diagnostic device, a PET device, a SPECT (Single Photon Emission Computed Tomography) device, or the like. The medical image diagnostic device 26 and the model learning device 24 may be communicably connected via a cable or a communication network. A trained model generated by the model learning device 24 is supplied to the medical image diagnostic device 26 , and the trained model is stored in the memory 11 of the medical image processing device 1 . Note that the medical image diagnostic apparatus 26 and the model learning apparatus 24 do not necessarily have to be communicably connected. In this case, the trained model is supplied from the model learning device 24 to the medical image diagnostic device 26 via a portable storage medium or the like storing the trained model.

Note that the trained model according to the first embodiment is generated by machine-learning a multi-layered network (also simply referred to as a model) before machine learning using a neural network represented by a deep neural network (DNN). Note that the trained model may be realized by a lookup table (LUT) instead of a synthetic function with parameters.

Note that the medical image processing apparatus 1 may generate a trained model without separately preparing the model learning apparatus 24 as shown in FIG.

A case of generating a trained model in the medical image processing apparatus 1 will be described with reference to the block diagram of FIG.
The processing circuitry 13 associated with the medical image processing apparatus 1 further includes a learning function 139 .
The acquisition function 131 allows the processing circuit 13 to acquire learning data from the learning data storage device 22 .
With the learning function 139, the processing circuit 13 uses the learning data to train the multi-layer network and generate a trained model. As for the method of generating the learned model, the generation method described with reference to FIG. 4 onwards may be used.
Memory 11 stores the trained model generated by learning function 139 .

Next, the concept of learning a trained model according to the first embodiment will be described with reference to FIG. FIG. 4 shows the multi-layer network 40 before learning and training data input to the multi-layer network 40 .
At the time of model learning, the multi-layer network 40 may be machine-learned using learning data, for example, at the time of shipment from the factory. Also, the learned model may be updated during repair or software update. Also, during model learning, it is desirable to prepare a trained model for each different type of movement of the object, for example, for each part of the subject, such as the heart and lungs.

The multi-layer network 40 is assumed to be, for example, a multi-layer neural network or a convolutional neural network, but other network configurations such as ResNet and DenseNet derived from convolutional networks may be used. Also, the structure of the multi-layered network 40 to be learned may be configured by GAN (Generative Adversarial Network), DCGAN (Deep Convolutional GAN), or pix2pix.

The learning data is input data of a reconstructed image based on data collected in a first data collection period for a certain object, and second data shorter than the first data collection period for the same object. It is a data set prepared by preparing a plurality of sets of data in which correct data are reconstructed images based on the data collected during the collection period. An image used as learning data is also called a learning image.

In the example of FIG. 4, when the first data collection period is a data collection period corresponding to full scan collection (hereinafter referred to as a full scan period for convenience), the second data collection period is half scan, which is shorter than full scan. A data collection period corresponding to scan collection (hereinafter referred to as a half-scan period for convenience). Full scan acquisition is a method of acquiring projection data for 360 degrees, and half scan acquisition is a method of acquiring projection data for 180 degrees + fan angle.

Note that the first data collection period and the second data collection period are not limited to the correspondence relationship between the full scan collection and the half scan collection because it is sufficient if there is a difference between the collection periods when comparing the first data collection period and the second data collection period. For example, instead of full-scan acquisition, a method of acquiring projection data for 200 degrees or a method of acquiring projection data for 720 degrees may be used.
For example, since the first data collection period only needs to be longer than the second data collection period, if the second data collection period is a half scan period, the first data collection period is set to 100% full scan collection. Then, a data collection period of 70% may be set. Similarly, the second data collection period should be shorter than the first data collection period. Alternatively, the data collection period may be 70% of one revolution. By training the multi-layer network 40 with the learning data described above, it is possible to generate a trained model 42 that has learned the movement of the object included in the image.

The reason why the movement of the target object can be learned by using the learning data described above is as follows. When photographing moving objects, the reconstructed image has the effect of motion artifacts. The shorter the data acquisition period, the less the effects of motion artifacts. That is, for example, a reconstructed image based on data in a full scan period with a long data collection period stores a large amount of motion information of the object. On the other hand, a reconstructed image based on data during the half-scan period, in which the data acquisition period is short, is less affected by motion than a reconstructed image based on data during the full-scan period.
Therefore, a reconstructed image based on data in the first collection period is used as input data (first learning image), and a reconstructed image based on data in the second collection period is used as correct data (second learning image). By learning the network, motion information (difference of motion) in the reconstructed image can be learned. In the first embodiment, the motion of the object can be learned, and the effects of motion artifacts can be learned.
If the model can learn only motion information, the multi-layer network may be trained using the second learning image as the input data and the first learning image as the correct data.

Next, the concept of how to set the data collection period for learning data will be described with reference to FIGS. 5 and 6. FIG.
FIG. 5 shows the relationship between the first data acquisition period and the second data acquisition period when the subject P, which is the target object, is present in the gantry, and the acquisition period corresponding to the rotation amount of the gantry (the rotation angle of the view). ). The solid line arrow is the first data collection period 50, which is the full scan period, and the dashed line arrow is the second data collection period 52, which is the half scan period.

A set of the first data collection period 50 and the second data collection period 52 corresponding to the input in the learning data is used when generating a reconstructed image based on the data collection period from the viewpoint of association of reconstructed images. It is desirable to use data in which the central phases of the cardiac phases are aligned. Also, a data string used for reconstruction can be partially extracted from the full-scan data string and reconstructed. In that case, since the center phase is the center of the extracted data string, the center phase can be set at an arbitrary position by adjusting the range from which the data string is extracted. Therefore, the range from which the data string is extracted should be adjusted to match the center phase of the second data acquisition period, for example, the half-scan period.

Next, FIG. 6 is a conceptual diagram showing the relationship between ECG (Electrocardiogram) signals and data collection periods.
As shown in FIG. 6, when half-scan acquisition is performed for a cardiac phase that is 60% to 90% of the RR interval of the ECG signal 60, the first data acquisition period 62 is set at 75% of the RR interval as the center phase. and a second data collection period 64 are set. As a result, a set of reconstructed images suitable for learning data can be set.

Note that the range for extracting the data string used for reconstruction from the full-scan acquired data may be adjusted, and the same position as the center phase in half-scan acquisition may be designated as the center phase in full-scan acquisition. For example, when setting a data acquisition period of 70% of full scan acquisition as the first data acquisition period, if a 70% data acquisition period 66 is extracted so that the second data acquisition period and the central phase are aligned, good.

Next, a method of generating learning data according to this embodiment will be described with reference to FIG.
In the data collection period of the input data (first data collection period), the full scan period, half scan period, 80% or 70% data collection period is partially extracted, and by changing the center phase, a plurality of A training data set may be generated.

For example, in FIG. 7, a first data acquisition period 70-1, which is a data acquisition period of 70% of full scan acquisition, and a second data acquisition period 72-1, which is a half scan period, with the center phase 75 aligned. A pair may be extracted and used as learning data.

Next, in FIG. 7B, the data collection period is slightly shifted from the first data collection period 70-1 and the second data collection period 72-1 shown in FIG. In 7(b), the center phase 75 is shifted clockwise by approximately 45 degrees to generate a set of a first data acquisition period 70-2 and a second data acquisition period 72-2. Similarly, in FIG. 7(c), the center phase 75 is shifted from FIG. 7(b) to generate a set of the first data acquisition period 70-3 and the second data acquisition period 72-3. Note that the center phases of the first data collection period and the second data collection period in each piece of learning data are aligned.

In this way, by generating a plurality of sets of learning data data sets in variations while changing the central phase from the data acquired once in a full scan, the imaging and the first data in the first data acquisition period It is possible to easily prepare a large amount of data for learning with different time phases without having to separately take pictures during the acquisition period. The learning data shown in FIG. 7 may be generated by the learning data generation device 20 or by the learning function 139 of the medical image processing device 1 .

Next, a usage example of the trained model 42 according to the first embodiment will be described with reference to FIG.
When using the trained model 42, an input image containing motion artifacts is input to the trained model 42, and a corrected image with reduced motion artifacts is output. Ideally, an input image containing motion artifacts will result in a motion artefact-free stationary corrected image. Note that the data collection period of the input image when using the trained model 42 may be any period. In the trained model 42, since the motion information itself is learned, there is no restriction that the data collection period of the input image during use is the same as the data collection period of the input image during learning. It is also possible to reduce motion artifacts contained in the input image for a period of time.

As shown in FIG. 8, an example of using a trained model generated using learning data shows an example of outputting an image with reduced motion artifacts. may be output.

As an example of using the trained model, a case of calculating a motion index value will be described with reference to the flowchart of FIG.

In step S901, when the processing circuit 13 acquires data for one round using the acquisition function 131 and the image processing function 135, data of a plurality of second data acquisition periods with different central phases of image reconstruction (for example, , half-scan period data) are extracted, the center phase is shifted along the time series, and a reconstructed image is generated from the data corresponding to each center phase. For the sake of convenience, these images are hereinafter referred to as images of each time phase.
In step S902, the generation function 133 causes the processing circuit 13 to apply the learned model to the image of each time phase to generate a corrected image of each time phase with reduced motion artifacts.

In step S903, the image processing function 135 causes the processing circuit 13 to compare the reconstructed image and the corrected image at each time phase. Specifically, for example, the motion index value for each time phase is obtained by calculating the sum of the difference amounts of the pixel values.
In step S<b>904 , the display control function 137 causes the processing circuit 13 to graphically display changes in the motion index value for each time phase. Alternatively, the time phase with the smallest index value is presented. Also, the motion index value may be presented together with the presentation of the phase. Furthermore, an image based on the second data collection period centered on the reconstruction of the time phase with the small motion index value, or a corrected image of the second data collection period may be presented.

Next, a specific example of the motion index value calculation processing in FIG. 9 will be described with reference to FIG.
FIG. 10A is a diagram schematically showing a data collection period, which is the source of an input image input to a trained model, in a view period. FIG. 10A shows four pieces of data whose center phases are shifted by a predetermined period along the time series during the half-scan period.
FIG. 10(b) is an input image generated by performing image reconstruction processing based on the data acquisition period of each time phase.

In FIG. 10(c), the learned model is applied to each input image shown in FIG. 10(b).
FIG. 10(d) shows corrected images with reduced motion artifacts after the trained model has been applied to each input image.
In FIG. 10E, a motion index value for each time phase is calculated based on the input image and the corrected image. In the example of FIG. 10E, the calculated motion index values are plotted on a graph with the horizontal axis representing time and the vertical axis representing the amount of motion. Although an example of fitting the plotted points by linear regression or the like is shown, the present invention is not limited to this, and the relationship between time and motion index value may be expressed.

By calculating the motion index value as a parameter using the trained model 42 in this way, it is possible to determine the data acquisition period during which motion artifacts are most reduced. That is, by using the best data acquisition period as an input image, a reconstructed image with the most reduced motion artifacts can be generated. As a result, for example, since the phase information of the image can be selected, in the multi-slice image captured by the step-and-shoot method, while reducing motion artifacts, an image is created in which each slice has the same central phase. be able to. Therefore, it is possible to create an image with little displacement between slices, even for an image of a moving heart or the like.

In addition, you may make it learn about the tendency of a motion as data for learning. For example, since the heart and abdominal organs such as the liver have different movement tendencies, a motion-corrected trained model for the heart that has learned data specifically for the heart, and a model that has specialized for the liver data, are trained separately for each organ. model construction may be performed separately. Since the movement pattern of the heart also differs depending on the cardiac time phase, a model for diastole and a model for systole are learned for each phase, and models are constructed individually. may
As a result, data with similar tendencies are specialized for learning, so even with a small amount of learning data, the estimation accuracy of the trained model is high and convergence can be accelerated.

Also, the input state of the input image when using the trained model 42 may be matched to the state of the input image during learning. For example, it is assumed that learning is performed with images after preprocessing such as applying a low-pass filter or removing metal artifacts. When using a trained model, if the input image is a reconstructed image of an object that contains metal, apply the trained model after filtering the input image to remove metal artifacts. Also good. By doing so, motion artifacts can be reduced with higher accuracy in the input image when using the trained model.

According to the first embodiment described above, data based on the first data collection period with a long data collection period in which much time information is stored and data based on the second data collection period with a short data collection period According to the trained model using and, it is possible to generate a corrected image with reduced motion artifacts. Images with reduced motion artifacts allow more accurate determination of time and location of where a vessel is at a given time, improving accuracy in vessel analysis such as stenosis. As a result, quantification at the time of diagnosis is improved.

(Second embodiment)
In the first embodiment, a reconstructed image based on a long data acquisition period is used as an input image, and a reconstructed image based on a short data acquisition period is used as a correct image in order to reduce motion artifacts included in the image. In other words, since the reconstructed image based on half-scan acquisition is the correct image compared to the reconstructed image based on full-scan acquisition, from the viewpoint of motion artifacts, half-scan acquisition is less affected by motion artifacts, but the image quality is improved. In terms of , an image with lower resolution than the full-scan acquisition can be output.

Therefore, when a trained model that has been learned in this way is used, the corrected image generated from the input image is more degraded than the input image in terms of image brightness and amount of noise (or SN ratio, etc.) included in the image. It is thought that Also, when learning the amount of motion in motion artifacts, it is believed that by using images with motion artifacts and images without motion artifacts in the same data acquisition period, the amount of motion in motion artifacts can be learned more accurately.

Therefore, in the second embodiment, two trained models are applied to correct the image quality change caused by the difference in the data acquisition period, and to generate a corrected image with reduced motion artifacts with higher accuracy.

First, an example of learning of a trained model according to the second embodiment will be described with reference to FIGS. 11 and 12. FIG.
FIG. 11 is an example of learning of the first trained model. By the model learning device 24 or the learning function 139, the processing circuit 13 uses as an input image a reconstructed image based on the data obtained in the first acquisition period for a stationary object, and the same object is photographed in the second acquisition period. The multi-layer network 90 is trained using the reconstructed image based on the obtained data as the correct image. For example, with respect to the input reconstructed image based on the full scan period, as a result of learning that the reconstructed image based on the half scan period is the correct answer, the luminance difference between the input image and the correct image, the amount of noise (SN ratio), etc. A first trained model 92 that has learned is generated. In other words, a trained model 92 for image quality compensation is generated.

Next, FIG. 12 is an example of learning of the second trained model. At the time of generating the second trained model, the model learning device 24 or the learning function 139 causes the processing circuit 13 to convert the reconstructed image based on the projection data captured during the first collection period to the first An image to which the trained model 92 is applied (hereinafter referred to as an intermediate image) is used as an input image, and a reconstructed image based on projection data captured during the second acquisition period of the same object is used as correct data, and the multi-layer network 94 is trained. . If the first collection period and the second collection period in FIG. 12 are set to have the same length as the first collection period and the second collection period when generating the first trained model 92, good.

A second trained model 96 that has learned the direction and amount of movement in the motion artifact is generated as a result of learning that the reconstructed image based on the second acquisition period is the correct image for the input of the intermediate image. .

Next, an example of using a trained model according to the second embodiment will be described with reference to FIG.
As shown in FIG. 13, a reconstructed image based on data in an arbitrary collection period is input as an input image to the second trained model 96, and a corrected image is output. In the corrected image, motion artifacts are reduced from the input image, and image quality deterioration components such as luminance reduction are also compensated for.

According to the second embodiment described above, a trained model for image quality compensation is generated, and an intermediate image, which is an output image of the first trained model, is input to generate a second trained model for motion compensation. By generating a model, it is possible to generate a corrected image in which motion artifacts are reduced and image quality degradation components such as luminance reduction are also compensated for.

(Third Embodiment)
In the third embodiment, instead of inputting into one learning model at the time of use, trained models are prepared according to the purpose, and each trained model is applied. A desired image can be generated by correcting the image quality change caused by the difference in the data acquisition period.

An example of learning of a trained model according to the third embodiment will be described with reference to FIG. 14 .
It is assumed that two trained models are generated in the third embodiment. The first is a trained model for motion correction (also referred to as a first trained model) similar to the first embodiment. The second is a trained model for image quality correction (also referred to as a second trained model).

As shown in FIG. 14, an input image is a reconstructed image based on projection data obtained by photographing a stationary, non-moving object during a half-scan period, and a reconstruction image is obtained based on projection data obtained by photographing the same object during a full-scan period. The multi-layer network 90 is trained using the constituent images as correct images. In this learning, the resolution of the reconstructed image obtained by full-scan acquisition, which has a longer data collection period than the input data, has a higher resolution. A second trained model 98 for image quality correction can be constructed that produces an image with .

Next, an example of using a trained model according to the third embodiment will be described with reference to FIG.
As shown in FIG. 15 , the generation function 133 causes the processing circuit 13 to apply the first trained model 42 for motion compensation with the input image being a reconstructed image with motion artifacts acquired by full-scan acquisition. After that, a corrected image is generated by applying the second learned model 98 for image quality correction to the image to which the first learned model 42 has been applied.

The order in which the first trained model 42 and the second trained model 98 are applied is random, and the first trained model 42 is applied after the second trained model 98 is applied. You may do so.

According to the third embodiment described above, a trained model for motion correction and a trained model for image quality correction are generated and applied stepwise to the input image, thereby reducing motion artifacts. A corrected image can be generated that is corrected and has similar image quality to a reconstructed image based on a full-scan acquisition.

In the third embodiment, an example of generating two types of trained models has been described. It may be applied to

(Fourth embodiment)
In the fourth embodiment, as a medical image diagnostic apparatus including the medical image processing apparatus 1, an X-ray CT apparatus including the medical image processing apparatus 1 will be described with reference to the block diagram of FIG. The X-ray CT apparatus 2 shown in FIG. 1 has a gantry apparatus 100, a bed apparatus 300, and a console apparatus 400 that realizes processing of the X-ray CT apparatus. In FIG. 16, a plurality of gantry devices 100 are depicted for convenience of explanation.

In this embodiment, the rotation axis of the rotating frame 130 in the non-tilt state or the longitudinal direction of the top plate 330 of the bed apparatus 300 is the Z-axis direction, and the axial direction perpendicular to the Z-axis direction and horizontal to the floor surface is are defined as the X-axis direction, the axis direction perpendicular to the Z-axis direction, and the Y-axis direction as the axis direction perpendicular to the floor surface.

For example, the gantry device 100 and the bed device 300 are installed in a CT examination room, and the console device 400 is installed in a control room adjacent to the CT examination room. Note that the console device 400 does not necessarily have to be installed in the control room. For example, the console device 400 may be installed in the same room together with the gantry device 100 and the bed device 300 . In any case, the gantry device 100, the bed device 300, and the console device 400 are connected by wire or wirelessly so as to be able to communicate with each other.

The gantry device 100 is a scanning device having a configuration for X-ray CT imaging of the subject P. As shown in FIG. The gantry 100 includes an X-ray tube 110, an X-ray detector 120, a rotating frame 130, an X-ray high voltage device 140, a control device 150, a wedge 160, a collimator 170, and a data acquisition device 180 (hereinafter , DAS (Data Acquisition System) 180).

The X-ray tube 110 is a vacuum tube that generates X-rays by irradiating thermal electrons from a cathode (filament) to an anode (target) by applying a high voltage and supplying a filament current from an X-ray high voltage device 140. is. Specifically, X-rays are generated by thermal electrons colliding with a target. For example, the X-ray tube 110 is a rotating anode type X-ray tube that generates X-rays by irradiating a rotating anode with thermal electrons. The X-rays generated by the X-ray tube 110 are shaped into a cone beam through a collimator 170, for example, and the object P is irradiated with the cone beam.

The X-ray detector 120 detects X-rays emitted from the X-ray tube 110 and passing through the subject P, and outputs an electrical signal corresponding to the X-ray dose to the DAS 180 . The X-ray detector 120 has, for example, a plurality of X-ray detection element arrays in which a plurality of X-ray detection elements are arranged in the channel direction along one circular arc centered on the focal point of the X-ray tube 110 . The X-ray detector 120 has, for example, a row structure in which a plurality of X-ray detection element rows each having a plurality of X-ray detection elements arranged in the channel direction are arranged in the slice direction (row direction).

Specifically, the X-ray detector 120 is, for example, an indirect conversion type detector having a grid, a scintillator array, and a photosensor array. X-ray detector 120 is an example of a detection unit.

The scintillator array has a plurality of scintillators. The scintillator has a scintillator crystal that outputs a photon amount of light corresponding to the intensity of incident X-rays.

The grid has an X-ray shielding plate arranged on the surface of the scintillator array on the X-ray incident side and having a function of absorbing scattered X-rays. Note that the grid may also be called a collimator (one-dimensional collimator or two-dimensional collimator).

The photosensor array has a function of amplifying the light received from the scintillator and converting it into an electric signal, and has photosensors such as photomultiplier tubes (PMTs), for example.

Rotating frame 130 rotatably supports X-ray generator (including X-ray tube 110, wedge 160 and collimator 170) and X-ray detector 120 about a rotation axis. Specifically, the rotation frame 130 is an annular frame that supports the X-ray tube 110 and the X-ray detector 120 so as to face each other and rotates the X-ray tube 110 and the X-ray detector 120 by a control device 150 described later. is. Rotating frame 130 is rotatably supported by a fixed frame (not shown) made of metal such as aluminum. Specifically, the rotating frame 130 is connected to the edges of the stationary frame via bearings. The rotating frame 130 receives power from the drive mechanism of the control device 150 and rotates around the rotation axis Z at a constant angular velocity.

In addition to the X-ray tube 110 and the X-ray detector 120, the rotating frame 130 further includes an X-ray high voltage device 140 and a DAS 180 to support them. Such a rotating frame 130 is accommodated in a substantially cylindrical housing in which an opening (bore) 190 forming an imaging space is formed. The aperture approximately matches the FOV. The central axis of the opening coincides with the rotational axis Z of the rotating frame 130 . The detection data generated by the DAS 180 is provided in a non-rotating portion (for example, a fixed frame, not shown in FIG. 1) of the gantry by optical communication from a transmitter having a light emitting diode (LED), for example. It is transmitted to a receiver (not shown) having a photodiode and forwarded to console device 400 . The method of transmitting the detected data from the rotating frame to the non-rotating portion of the gantry is not limited to the optical communication described above, and any method of non-contact data transmission may be adopted.

The X-ray high voltage device 140 has electric circuits such as a transformer and a rectifier, and has a function of generating a high voltage to be applied to the X-ray tube 110 and a filament current to be supplied to the X-ray tube 110. It has a generator and an X-ray control device that controls an output voltage according to the X-rays emitted by the X-ray tube 110 . The high voltage generator may be of a transformer type or an inverter type. Note that the X-ray high-voltage device 140 may be provided on a rotating frame 130 to be described later, or may be provided on a fixed frame (not shown) side of the gantry device 100 .

The control device 150 has a processing circuit having a CPU, and drive mechanisms such as motors and actuators. The processing circuit has, as hardware resources, processors such as a CPU and an MPU (Micro Processing Unit) and memories such as a ROM and a RAM. Controller 150 may also be implemented with an application specific integrated circuit (ASIC), field programmable gate array (FPGA), other complex programmable logic device (CPLD), simple programmable logic device (SPLD). . The control device 150 controls the X-ray high-voltage device 140, the DAS 180, and the like according to commands from the console device 400. FIG. The processor implements the control by reading and implementing the program stored in the memory.

Further, the control device 150 has a function of receiving an input signal from an input interface 430 (described later) attached to the console device 400 or the gantry device 100 and performing operation control of the gantry device 100 and the bed device 300 . For example, the control device 150 receives an input signal and performs control to rotate the rotating frame 130 , control to tilt the gantry device 100 , and control to operate the bed device 300 and the tabletop 330 . The control for tilting the gantry device 100 is performed by the control device 150 rotating the frame about an axis parallel to the X-axis direction based on inclination angle (tilt angle) information input by the input interface 430 attached to the gantry device 100. It is realized by rotating 130 . Also, the control device 150 may be provided in the gantry device 100 or may be provided in the console device 400 . It should be noted that the control device 150 may be configured to directly incorporate the program into the circuitry of the processor instead of storing the program in the memory. In this case, the processor implements the above control by reading and executing a program incorporated in the circuit.

Wedge 160 is a filter for adjusting the dose of X-rays emitted from X-ray tube 110 . Specifically, the wedge 160 transmits and attenuates the X-rays emitted from the X-ray tube 110 so that the X-rays emitted from the X-ray tube 110 to the subject P have a predetermined distribution. It is a filter that For example, the wedge 160 (wedge filter, bow-tie filter) is a filter machined from aluminum to have a predetermined target angle and a predetermined thickness.

The collimator 170 is a lead plate or the like for narrowing down the irradiation range of the X-rays transmitted through the wedge 160, and a slit is formed by combining a plurality of lead plates or the like. Note that the collimator 170 may also be called an X-ray diaphragm.

The DAS 180 is implemented by, for example, an ASIC (Application Specific Integrated Circuit) that includes circuit elements capable of generating detection data. DAS 180 and X-ray detector 120 constitute a detector unit.

The DAS 180 reads an electrical signal from the X-ray detector 120 and generates detection data, which is digital data regarding the dose of X-rays detected by the X-ray detector 120, based on the read electrical signal. The detection data is a set of data indicating the channel number and row number of the X-ray detection element from which it was generated, the view number indicating the acquired view (also called projection angle), and the integral value of the detected X-ray dose. be.

For example, DAS 180 includes preamplifiers, variable amplifiers, integrator circuits, and A/D converters for each sensing element. The preamplifier amplifies the electrical signal from the connected detection element with a predetermined gain. A variable amplifier amplifies the electrical signal from the preamplifier with a variable gain. An integrator circuit integrates the electrical signal from the preamplifier over one view period to generate an integrated signal. The crest value of the integral signal corresponds to the dose value of the X-rays detected by the detection element of the connection source over one view period. The A/D converter analog-to-digital converts the integration signal from the integration circuit to generate detection data.

The bed device 300 is a device for placing and moving a subject P to be scanned, and includes a base 310 , a bed driving device 320 , a top plate 330 and a support frame 340 .

The base 310 is a housing that supports the support frame 340 so as to be vertically movable.

The bed driving device 320 is a motor or actuator that moves the table 330 on which the subject P is placed in the longitudinal direction of the table 330 . The bed drive device 320 moves the tabletop 330 under the control of the console device 400 or the control device 150 . For example, the bed driving device 320 moves the top plate 330 in a direction orthogonal to the subject P so that the body axis of the subject P placed on the top plate 330 coincides with the central axis of the opening of the rotation frame 130. do. In addition, the bed driving device 320 may move the top plate 330 along the body axis direction of the subject P according to the X-ray CT imaging performed using the gantry device 100 . The bed driving device 320 generates power by being driven at a rotational speed according to the duty ratio of the drive signal from the control device 150 or the like. The bed driving device 320 is realized by a motor such as a direct drive motor or a servomotor, for example.

A top plate 330 provided on the upper surface of the support frame 340 is a plate on which the subject P is placed. Note that the bed driving device 320 may move the support frame 340 in addition to the top plate 330 in the longitudinal direction of the top plate 330 .

The console device 400 has a memory 410, a display 420, an input interface 430, a processing circuit 440, and the medical image processing apparatus 1 according to the first to third embodiments described above. Data communication between memory 410, display 420, input interface 430 and processing circuit 440 is performed via a bus (BUS). Although the console device 400 is described as being separate from the gantry device 100, the gantry device 100 may include the console device 400 or a part of each component of the console device 400. FIG.

The memory 410 is a storage device such as an HDD, an SSD, or an integrated circuit storage device that stores various information. The memory 410 stores, for example, detection data and reconstructed image data. The memory 410 may be a driving device that reads and writes various information from and to portable storage media such as CDs, DVDs, and flash memories, semiconductor memory devices such as RAM, and the like, in addition to HDDs and SSDs. . Also, the storage area of the memory 410 may be in the X-ray CT apparatus 2 or in an external storage device connected via a network. For example, the memory 410 stores data of CT images and display images. The memory 410 also stores a control program according to this embodiment.

The display 420 displays various information. For example, the display 420 outputs a medical image (CT image) generated by the processing circuit 440, a GUI (Graphical User Interface) for accepting various operations from the operator, and the like. For example, the display 420 may be, for example, a liquid crystal display (LCD: Liquid Crystal Display), a CRT (Cathode Ray Tube) display, an organic EL display (OELD: Organic Electro Luminescence Display), a plasma display, or any other arbitrary display. , is enabled. Also, the display 420 may be provided on the gantry device 100 . The display 420 may be of a desktop type, or may be configured by a tablet terminal capable of wireless communication with the main body of the console device 400, or the like.

The input interface 430 receives various input operations from the operator, converts the received input operations into electrical signals, and outputs the electrical signals to the processing circuit 440 . For example, the input interface 430 receives from the operator acquisition conditions for acquiring detection data, reconstruction conditions for reconstructing CT images, image processing conditions for generating post-processed images from CT images, and the like. . As the input interface 430, for example, a mouse, keyboard, trackball, switch, button, joystick, touch pad, touch panel display, etc. can be used as appropriate. Note that in the present embodiment, the input interface 430 is not limited to having physical operation components such as a mouse, keyboard, trackball, switch, button, joystick, touch pad, and touch panel display. For example, the input interface 430 also includes an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and outputs the electrical signal to the processing circuit 440. . The input interface 430 may be provided in the gantry device 100 . Also, the input interface 430 may be composed of a tablet terminal or the like capable of wireless communication with the main body of the console device 400 .

The processing circuit 440 controls the operation of the entire X-ray CT apparatus 2 according to the electric signal of the input operation output from the input interface 430 . For example, the processing circuit 440 has processors such as a CPU, MPU, and GPU (Graphics Processing Unit) and memories such as ROM and RAM as hardware resources. The processing circuit 440 executes a system control function 4410, a preprocessing function 4420, a reconstruction processing function 4430, and a display control function 4440 by means of a processor that executes programs expanded in memory. Note that each function (system control function 4410, preprocessing function 4420, reconstruction processing function 4430, and display control function 4440) is not limited to being realized by a single processing circuit. A processing circuit may be configured by combining a plurality of independent processors, and each function may be realized by each processor executing a program.

The system control function 4410 controls each function of the processing circuit 440 based on input operations received from the operator via the input interface 430 . Specifically, the system control function 4410 reads the control program stored in the memory 410, develops it on the memory in the processing circuit 440, and controls each part of the X-ray CT apparatus 2 according to the developed control program. . For example, the processing circuitry 440 controls each function of the processing circuitry 440 based on an input operation received from the operator via the input interface 430 . For example, the system control function 4410 acquires a two-dimensional positioning image of the subject P for determining the scan range, imaging conditions, and the like. A positioning image is also called a scanogram or a scout image.

A preprocessing function 4420 generates data by performing preprocessing such as logarithmic conversion processing, offset correction processing, inter-channel sensitivity correction processing, and beam hardening correction on the detection data output from the DAS 180 . Raw data (detection data) before preprocessing and data after preprocessing may be collectively referred to as projection data.

The reconstruction processing function 4430 performs reconstruction processing on the projection data generated by the preprocessing function 4420 using a filtered back projection (FBP) method, an iterative reconstruction method, or the like. to generate CT image data.

The display control function 4440 is processing for controlling the display 420 so as to display information on the progress of processing or processing results in each function or processing of the processing circuit 440 .

The processing circuit 440 also performs scan control processing, image processing, and display control processing.
The scan control process is a process of controlling various operations related to X-ray scanning, such as causing the X-ray high-voltage device 140 to supply a high voltage and causing the X-ray tube 110 to emit X-rays.
Image processing is a process of converting CT image data generated by the reconstruction processing function 4430 into tomographic image data or three-dimensional image data of an arbitrary cross section based on an input operation received from an operator via the input interface 430. is. Note that the generation of three-dimensional image data may be directly performed by the reconstruction processing function 4430 .

The processing circuit 440 is not limited to being included in the console device 400, and may be included in an integrated server that collectively processes data acquired by a plurality of medical image diagnostic apparatuses.
Although the console device 400 has been described as executing a plurality of functions with a single console, separate consoles may execute a plurality of functions.

Note that each function according to the above-described embodiment can also be realized by installing a program for executing the processing in a computer such as a workstation and deploying them on the memory. At this time, a program that can cause a computer to execute the method is stored in a storage medium such as a magnetic disk (hard disk, etc.), an optical disk (CD-ROM, DVD, Blu-ray (registered trademark) disk, etc.), a semiconductor memory, etc. It is also possible to distribute it as

According to the medical image processing apparatuses according to the first to third embodiments and the X-ray CT apparatus according to the fourth embodiment described above, it is possible to improve the quantitativeness at the time of diagnosis.

In the present embodiment, the case of processing a reconstructed image based on projection data captured by an X-ray CT apparatus has been described as an example. A trained model can be similarly generated for an ultrasonic image generated from ultrasonic data obtained by an ultrasonic diagnostic apparatus.
Specifically, in the case of an X-ray image captured by an X-ray diagnostic apparatus, by controlling the exposure time to the object, the X-ray image based on the first data acquisition period and the second data acquisition period X-ray images based on the above may be generated and used as the above-described learning data.
In addition, in the case of ultrasonic images in an ultrasonic diagnostic apparatus, an image with a high beam density per image is an ultrasonic image based on the first data acquisition period, and an image with a low beam density is acquired during the second data acquisition period. It is sufficient to generate learning data in the form of ultrasound images based on the above.

While several embodiments of the invention have been described, these embodiments have been presented by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and modifications can be made without departing from the scope of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, as well as the scope of the invention described in the claims and equivalents thereof.

1 medical image processing device 2 X-ray CT device 11, 410 memory 13, 440 processing circuit 15 input interface 17 communication interface 20 learning data generation device 22 learning data storage device 24 model learning device 26 medical image diagnosis device 40, 90, 94 multilayer network 42, 92, 96, 98 trained model 50, 62, 70-1 to 70-3 first data collection period 52, 64, 72-1 to 72-3 second data collection period 60 ECG signal 66 70% data acquisition period 75 Center phase 100 Mounting device 110 X-ray tube 120 X-ray detector 130 Rotating frame 131 Acquisition function 133 Generation function 135 Image processing function 137,4440 Display control function 139 Learning function 140 X-ray high voltage device 150 Controller 160 Wedge 170 Collimator 180 Data Acquisition System (DAS)
190 opening 300 bed device 310 base 320 bed driving device 330 top board 340 support frame 400 console device 411 trained model 420 display 430 input interface 4410 system control function 4420 preprocessing function 4430 reconstruction processing function

Claims (8)

an acquisition unit that acquires an input image of an object;
A first learning image obtained by photographing a moving object in a first data collection period, and a second training image obtained by photographing the moving object in a second data collection period shorter than the first data collection period. a generation unit that generates a corrected image that reduces the effect on the image caused by the movement of the object from the input image, using the first trained model that has been trained using a set of training images;
A medical image processing apparatus comprising: 2. The medical image processing apparatus according to claim 1 , wherein a set of said first training image and said second training image has the same central phase in each data acquisition period. further comprising an image processing unit that performs preprocessing on the input image in accordance with the image used for learning of the first trained model;
3. The medical image processing apparatus according to claim 1 , wherein the generator generates the corrected image from the preprocessed input image using the first trained model. 4. The medical image processing according to any one of claims 1 to 3 , wherein the generation unit performs correction processing for image quality changes caused by differences in data acquisition periods on the input image or the corrected image. Device. The generating unit generates an intermediate image generated by applying the first trained model to a third learning image obtained by photographing the moving object in the first data collection period, and the moving object. Using a second trained model generated by learning a second model using a set of fourth learning images of the object captured in the second data collection period, from the input image 2. The medical image processing apparatus according to claim 1, which generates the corrected image. The generation unit converts the first trained model, the third learning image obtained by photographing the moving object in the first data collection period, and the moving object into the second data. generating the corrected image from the input image by using a second trained model generated by learning the second model using a set of fourth learning images taken during the acquisition period; The medical image processing apparatus according to claim 1. an x-ray tube;
a detector for detecting X-rays emitted from the X-ray tube and transmitted through an object;
a reconstruction unit that reconstructs an input image of an object based on the output from the detector;
A first learning image obtained by photographing a moving object in a first data collection period, and a second training image obtained by photographing the moving object in a second data collection period shorter than the first data collection period. a generation unit that generates a corrected image that reduces the effect on the image caused by the movement of the object from the input image, using the trained model that has been trained using the set of training images;
An X-ray CT apparatus comprising: Acquiring first data obtained by photographing an object for a first data collection period;
For the second data corresponding to a second data acquisition period shorter than the first data acquisition period, while changing the center phase that is the center of image reconstruction, extract the second data of each time phase,
A method of generating learning data, wherein a set of a reconstructed image based on the first data and a reconstructed image based on the second data having the same center phase is generated as learning data for each time phase. .

Images