JP2022180971A - Learning apparatus, medical image processing apparatus, learning method, and program - Google Patents

Abstract

To shorten a processing time when a high-resolution medical image is generated.SOLUTION: A learning apparatus according to an embodiment has an acquisition unit, a generation unit, and a learning unit. The acquisition unit acquires a first medical image and a second medical image. The first medical image has a first matrix size and a first resolution. The second medical image has the first matrix size and a second resolution higher than the first resolution. The generation unit generates a third medical image from the first medical image and generates a fourth medical image from the second medical image. The third medical image has a second matrix size that is smaller than the first matrix size and the first resolution. The fourth medical image has the second matrix size and the second resolution. The learning unit generates a model for improving the resolution of the medical image by machine learning based on the third medical image and the fourth medical image.SELECTED DRAWING: Figure 5

Description

The embodiments disclosed in this specification and drawings relate to a learning device, a medical image processing device, a learning method, and a program.

In order to generate ultra-high-definition medical images, it is usually necessary to collect high-definition data, which requires dedicated hardware. For example, when generating a high-definition CT (Computed Tomography) image, an X-ray CT (Computed Tomography) apparatus equipped with an X-ray tube having a very small focus and a high-precision detector is required. If it were possible to generate ultra-high-definition medical images, it would be possible to create training data for deep learning based on those medical images, and it would be possible to bring normal-resolution images closer to higher-definition images. can. On the other hand, in such deep learning, it is expected that images with a large matrix size will be used, and normal resolution images used as input must also be created with a large matrix size. In this case, the problem is that it takes time to create the input image. Such problems are not limited to X-ray CT devices, but also occur in other medical imaging devices (also called medical imaging diagnostic devices) such as MRI (Magnetic Resonance Imaging) devices, ultrasonic diagnostic imaging devices, and nuclear medicine diagnostic devices. This is a common point.

Japanese Patent Application Publication No. 2019-534763 Japanese Patent Publication No. 2020-525258 Japanese Patent Application Laid-Open No. 2020-93076

The problem to be solved by the embodiments disclosed in the specification and drawings is to reduce the processing time when generating high-resolution medical images. However, the problems to be solved by the embodiments disclosed in this specification and drawings are not limited to the above problems. A problem corresponding to each effect of each configuration shown in the embodiments described later can be positioned as another problem.

A learning device according to an embodiment has an acquisition unit, a generation unit, and a learning unit. The acquisition unit acquires a first medical image and a second medical image. The first medical image is a reconstructed image of a region of the subject, and has a first matrix size and a first resolution. The second medical image is a reconstructed image of the region and has the first matrix size and a second resolution higher than the first resolution. The generator generates a third medical image by down-sampling the first medical image, and generates a fourth medical image by down-sampling the second medical image. The third medical image is an image having a second matrix size smaller than the first matrix size and having the first resolution. The fourth medical image is an image having the second matrix size and the second resolution. The learning unit generates a model for improving the resolution of medical images by machine learning based on the third medical image and the fourth medical image.

1 is a diagram showing a configuration example of a medical image processing system 1 according to an embodiment; FIG. 1 is a diagram showing a configuration example of an X-ray CT apparatus 100 according to an embodiment; FIG. 4 is a flow chart showing a series of processes related to image generation by the X-ray CT apparatus 100 according to the embodiment. FIG. 4 is a diagram schematically showing how an SR image is obtained; 1 is a diagram showing a configuration example of a learning device 200 according to an embodiment; FIG. 4 is a flowchart showing a series of processes of the learning device 200 according to the embodiment; FIG. 4 is a diagram schematically showing how 512-matrix NR and HR images are generated;

Hereinafter, a learning device, a medical image processing device, a learning method, and a program according to embodiments will be described with reference to the drawings.

[Configuration of medical image processing system]
FIG. 1 is a diagram showing a configuration example of a medical image processing system 1 according to an embodiment. The medical image processing system 1 includes, for example, one or more medical imaging devices 100 and a learning device 200 . The medical imaging apparatus 100 and the learning apparatus 200 are communicably connected via a communication network NW.

The communication network NW means all information communication networks using telecommunication technology. The communication network NW includes a wireless/wired LAN such as a hospital backbone LAN (Local Area Network), an Internet network, a telephone communication network, an optical fiber communication network, a cable communication network, a satellite communication network, and the like.

The medical imaging apparatus 100 is an apparatus that scans a subject P to generate a medical image. The subject P is, for example, a human patient, but is not limited to this, and may be an animal or an inorganic substance. The medical imaging apparatus 100 may be, for example, an X-ray CT apparatus, an MRI apparatus, an ultrasonic diagnostic imaging apparatus, a nuclear medicine diagnostic apparatus, or the like. The medical imaging apparatus 100 is a so-called modality. As an example, the medical imaging apparatus 100 will be described below as an X-ray CT apparatus.

The learning device 200 acquires medical images from an X-ray CT apparatus (medical imaging apparatus) 100 and uses the acquired medical images to learn a machine learning model. Details of the model will be described later. The learning device 200 may be a workstation or cloud server connected to the modality, the X-ray CT apparatus 100 .

[Configuration of X-ray CT apparatus]
FIG. 2 is a diagram showing a configuration example of the X-ray CT apparatus 100 according to the embodiment. The X-ray CT apparatus 100 includes, for example, a gantry device 110, a bed device 130, and a console device 140. For convenience of explanation, FIG. 2 shows both a view of the gantry device 110 viewed from the Z-axis direction and a view of the gantry device 110 viewed from the X-axis direction. In the embodiment, the rotation axis of the rotating frame 117 in the non-tilt state or the longitudinal direction of the top plate 133 of the bed device 130 is defined as the Z-axis direction, and the axis perpendicular to the Z-axis direction and horizontal to the floor surface is defined as the Z-axis direction. The X-axis direction and the direction perpendicular to the Z-axis direction and perpendicular to the floor surface are defined as the Y-axis direction.

The gantry device 110 includes, for example, an X-ray tube 111, a wedge 112, a collimator 113, an X-ray high voltage device 114, an X-ray detector 115, and a data acquisition system (DAS: Data Acquisition System) 116. , a rotating frame 117 and a controller 118 .

The X-ray tube 111 generates X-rays by applying a high voltage from the X-ray high voltage device 114 and irradiating thermal electrons from a cathode (filament) toward an anode (target). X-ray tube 111 includes a vacuum tube. For example, the X-ray tube 111 is a rotating anode type X-ray tube that generates X-rays by irradiating a rotating anode with thermal electrons.

The wedge 112 is a filter for adjusting the dose of X-rays irradiated from the X-ray tube 111 to the subject P. FIG. The wedge 112 attenuates the X-rays passing through itself so that the X-ray dose distribution irradiated from the X-ray tube 111 to the subject P becomes a predetermined distribution. Wedge 112 is also called a wedge filter or a bow-tie filter. The wedge 112 is, for example, processed aluminum so as to have a predetermined target angle and a predetermined thickness.

The collimator 113 is a mechanism for narrowing down the irradiation range of X-rays transmitted through the wedge 112 . The collimator 113 narrows down the X-ray irradiation range by, for example, forming a slit by combining a plurality of lead plates. Collimator 113 is sometimes called an X-ray diaphragm.

The X-ray high voltage device 114 has, for example, a high voltage generator and an X-ray controller. The high voltage generator has an electric circuit including a transformer, a rectifier, etc., and generates a high voltage to be applied to the X-ray tube 111 . The X-ray controller controls the output voltage of the high voltage generator in accordance with the amount of X-rays to be generated by the X-ray tube 111 . The high voltage generator may be one that boosts the voltage with the transformer described above, or one that boosts the voltage with an inverter. The X-ray high-voltage device 114 may be provided on the rotating frame 117 or may be provided on the fixed frame (not shown) side of the gantry device 110 .

The X-ray detector 115 detects the intensity of X-rays generated by the X-ray tube 111 and incident through the subject P. FIG. The X-ray detector 115 outputs to the DAS 116 an electrical signal (which may be an optical signal or the like) corresponding to the intensity of the detected X-rays. The X-ray detector 115 has, for example, multiple X-ray detection element arrays. Each of the multiple X-ray detection element arrays has multiple X-ray detection elements arranged in the channel direction along an arc centered on the focal point of the X-ray tube 111 . A plurality of X-ray detection element arrays are arranged in a slice direction (column direction, row direction).

X-ray detector 115 is, for example, an indirect detector having a grid, a scintillator array, and a photosensor array. The scintillator array has a plurality of scintillators. Each scintillator has a scintillator crystal. The scintillator crystal emits an amount of light corresponding to the intensity of incident X-rays. The grid has an X-ray shielding plate arranged on the surface of the scintillator array on which X-rays are incident and having a function of absorbing scattered X-rays. Note that the grid may also be called a collimator (one-dimensional collimator or two-dimensional collimator). The photosensor array has photosensors such as, for example, photomultiplier tubes (photomultipliers: PMTs). The photosensor array outputs an electrical signal corresponding to the amount of light emitted by the scintillator. The X-ray detector 115 may be a direct conversion detector having a semiconductor element that converts incident X-rays into electrical signals.

DAS 116 includes, for example, amplifiers, integrators, and A/D converters. The amplifier amplifies the electrical signal output from each X-ray detection element of the X-ray detector 115 . The integrator integrates the amplified electrical signal over a view period (described later). The A/D converter converts an electrical signal representing the result of integration into a digital signal. The DAS 116 outputs detection data based on digital signals to the console device 140 . Detected data is a digital value of x-ray intensity identified by the channel number of the x-ray detector element from which it was generated, the row number, and the view number indicating the acquired view. The view number is a number that changes according to the rotation of the rotating frame 117, and is a number that is incremented according to the rotation of the rotating frame 117, for example. Therefore, the view number is information indicating the rotation angle of the X-ray tube 111 . A view period is a period that falls between the rotation angle corresponding to a certain view number and the rotation angle corresponding to the next view number. The DAS 116 may detect view switching by a timing signal input from the control device 118, by an internal timer, or by a signal obtained from a sensor (not shown). . When X-rays are continuously emitted from the X-ray tube 111 when performing a full scan, the DAS 116 collects detection data groups for the entire circumference (360 degrees). When X-rays are continuously emitted from the X-ray tube 111 when half scanning is performed, the DAS 116 collects detection data for a half circumference (180 degrees).

The rotating frame 117 is an annular rotating member that rotates the X-ray tube 111, the wedge 112, the collimator 113, and the X-ray detector 115 while holding them facing each other. The rotating frame 117 is supported by a fixed frame so as to be rotatable around the subject P introduced therein. Rotating frame 117 also supports DAS 116 . The detected data output by DAS 116 is transmitted by optical communication from a transmitter having a light emitting diode (LED) mounted on rotating frame 117 to a photodiode mounted on a non-rotating portion (e.g., fixed frame) of mounting assembly 110. It is sent to the receiver and transferred to the console device 140 by the receiver. The method of transmitting the detection data from the rotating frame 117 to the non-rotating portion is not limited to the above-described method using optical communication, and any non-contact transmission method may be employed. The rotating frame 117 is not limited to an annular member, and may be a member such as an arm as long as it can support and rotate the X-ray tube 111 or the like.

The control device 118 has, for example, a processing circuit having a processor such as a CPU (Central Processing Unit), and a driving mechanism including a motor, an actuator, and the like. The control device 118 receives input signals from the console device 140 or the input interface 143 attached to the gantry device 110 and controls the operations of the gantry device 110 and the bed device 130 .

The control device 118 rotates the rotating frame 117, tilts the gantry device 110, and moves the top plate 133 of the bed device 130, for example. When tilting the gantry device 110 , the control device 118 rotates the rotating frame 117 about an axis parallel to the Z-axis direction based on the tilt angle (tilt angle) input to the input interface 143 . The control device 118 grasps the rotation angle of the rotating frame 117 from the output of a sensor (not shown) or the like. Controller 118 also provides the rotation angle of rotating frame 117 to processing circuit 150 at any time. The control device 118 may be provided in the gantry device 110 or may be provided in the console device 140 .

The control device 118 causes the gantry device 110 to move along the moving rails to perform main scan photography, or to perform scanography, which is positioning photography performed before execution of main scan photography.

The bed device 130 is a device on which the subject P to be scanned is placed and introduced into the rotating frame 117 of the gantry device 110 . The bed device 130 has, for example, a base 131 , a bed driving device 132 , a top board 133 and a support frame 134 . The base 131 includes a housing that supports the support frame 134 so as to be movable in the vertical direction (Y-axis direction). The bed driving device 132 includes motors and actuators. The bed driving device 132 moves the tabletop 133 on which the subject P is placed in the longitudinal direction (Z-axis direction) of the tabletop 133 along the support frame 134 . The top plate 133 is a plate-like member on which the subject P is placed.

Console device 140 has, for example, memory 141 , display 142 , input interface 143 , communication interface 144 , and processing circuitry 150 . In this embodiment, the console device 140 is described as being separate from the gantry device 110 , but the gantry device 110 may include part or all of each component of the console device 140 . The console device 140 is an example of a “medical image processing device”. The display 142 is an example of the "output section", and the communication interface 144 is another example of the "output section".

The memory 141 is implemented by, for example, a RAM (Random Access Memory), a semiconductor memory device such as a flash memory, a hard disk, an optical disk, or the like. The memory 141 may also include a storage medium such as a ROM (Read Only Memory) and a register. The memory 208 may also include a storage medium such as a ROM (Read Only Memory) and a register.

The memory 141 stores, for example, detection data, projection data, reconstructed images, and the like. These data may be stored not in the memory 141 (or in addition to the memory 141) but in an external memory (for example, NAS (Network Attached Storage) or the like) with which the X-ray CT apparatus 100 can communicate.

In addition, memory 141 stores model information. Model information is information (program or data structure) that defines a learned model MDL. The trained model MDL is typically trained so that when a certain medical image is input, it outputs a medical image with higher resolution than the input medical image (high-definition or high-quality medical image). model.

A high-resolution medical image is an image obtained by reducing noise, artifacts, and the like from an input medical image. Since the trained model MDL functions to reduce noise, artifacts, etc. from medical images, it may be read as a denoising filter.

The trained model MDL may be implemented by, for example, a DNN (Deep Neural Network(s)) such as a CNN (Convolutional Neural Network). Also, the trained model MDL is not limited to DNN, and may be implemented by other models such as support vector machines, decision trees, naive Bayes classifiers, and random forests.

When the trained model MDL is realized by a DNN, the model information includes, for example, the input layer, one or more hidden layers (intermediate layers), and the units (units or nodes ) are combined with each other, and weighting information such as how many coupling coefficients are given to data input/output between the combined units. The connection information includes, for example, the number of units included in each layer, information specifying the type of unit to which each unit is connected, an activation function that realizes each unit, information such as a gate provided between units in the hidden layer. including. The activation function that realizes the unit may be, for example, a ReLU (Rectified Linear Unit) function, an ELU (Exponential Linear Units) function, a clipping function, a sigmoid function, a step function, a hyperpolic tangent function, an identity function, or the like. . A gate selectively passes or weights data communicated between units, for example, depending on the value (eg, 1 or 0) returned by an activation function. A coupling coefficient includes, for example, a weight given to output data when data is output from a unit in a certain layer to a unit in a deeper layer in a hidden layer of a neural network. The coupling coefficients may also include bias components unique to each layer, and the like.

The display 142 displays various information. For example, the display 142 displays a CT image generated by the processing circuit 150, a GUI (Graphical User Interface) for receiving various operations by an operator (for example, a doctor), and the like. The display 142 is, for example, an LCD (Liquid Crystal Display), a CRT (Cathode Ray Tube) display, an organic EL (Electro Luminescence) display, or the like. The display 142 may be provided on the gantry device 110 . The display 142 may be of a desktop type, or may be a display device capable of wireless communication with the main body of the console device 140 (for example, a tablet terminal).

The input interface 143 accepts various input operations by an operator (for example, a doctor) and outputs an electrical signal indicating the content of the accepted input operation to the processing circuit 150 . For example, the input interface 143 accepts acquisition conditions for acquiring detection data or projection data (described later), reconstruction conditions for reconstructing CT images, image processing conditions for generating post-processed images from CT images, and the like. accepts the input operation of For example, the input interface 143 is implemented by a mouse, keyboard, touch panel, drag ball, switch, button, joystick, foot pedal, camera, infrared sensor, microphone, and the like. The input interface 143 may be provided on the gantry device 110 . Also, the input interface 143 may be realized by a display device (for example, a tablet terminal) capable of wireless communication with the main body of the console device 140 . It should be noted that the input interface 143 in this specification is not limited to those having physical operation parts such as a mouse and a keyboard. For example, the input interface 143 also includes an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and outputs the electrical signal to the control circuit.

The communication interface 144 includes, for example, a NIC (Network Interface Card), a wireless communication module, and the like. Communication interface 144 communicates with learning device 200 and other external devices via communication network NW.

Processing circuit 150 controls the overall operation of X-ray CT apparatus 100 . The processing circuitry 150 executes, for example, a system control function 151, a preprocessing function 152, a reconstruction processing function 153, an image processing function 154, an output control function 155, and the like. The processing circuit 150 implements these functions by executing a program stored in the memory 141 by a hardware processor, for example. The preprocessing function 152 and the reconstruction processing function 153 are examples of the "acquisition unit", and the output control function 155 is an example of the "output control unit".

A hardware processor is, for example, a CPU, a GPU (Graphics Processing Unit), an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (Simple Programmable Logic Device; SPLD) or Circuitry such as Complex Programmable Logic Device (CPLD), Field Programmable Gate Array (FPGA)).

Instead of storing the program in the memory 141, the program may be directly embedded in the circuitry of the hardware processor. In this case, the hardware processor realizes its function by reading and executing the program embedded in the circuit. The above program may be stored in the memory 141 in advance, or may be stored in a non-temporary storage medium such as a DVD or CD-ROM. ), it may be installed in the memory 141 from a non-transitory storage medium. The hardware processor is not limited to being configured as a single circuit, and may be configured as one hardware processor by combining a plurality of independent circuits to implement each function. Also, a plurality of components may be integrated into one hardware processor to realize each function.

Each component of console device 140 or processing circuit 150 may be distributed and realized by a plurality of pieces of hardware. The processing circuit 150 may be realized by a separate processing device that can communicate with the console device 140 instead of the configuration that the console device 140 has. The processing device may be, for example, a workstation connected to the X-ray CT apparatus 100 or a cloud server capable of collectively executing processing equivalent to that of the processing circuit 150 .

The system control function 151 controls various functions of the processing circuit 150 based on input operations received by the input interface 143 .

A preprocessing function 152 performs preprocessing such as logarithmic conversion processing, offset correction processing, inter-channel sensitivity correction processing, and beam hardening correction on the detection data output from the DAS 116 to generate projection data. The projection data thus obtained are stored in the memory 141 .

A reconstruction processing function 153 performs reconstruction processing using a filtered back projection method, an iterative reconstruction method, or the like on the projection data generated by the preprocessing function 152 to obtain a CT image, which is one of medical images. and store the generated CT image in the memory 141 .

The image processing function 154 converts the CT image into a three-dimensional image or cross-sectional image data of an arbitrary cross-section by a known method based on the input operation received by the input interface 143 . Conversion to a three-dimensional image may be performed by preprocessing function 152 .

Also, the image processing function 154 may use the learned model MDL defined by the model information in the memory 141 to generate a CT image with higher resolution. For example, the image processing function 154 inputs 512-matrix CT images (CT images with a matrix size of 512×512) to the trained model MDL. In response to this, the trained model MDL outputs a CT image that is 512 matrices and has a higher resolution than the input CT image.

The output control function 155 displays the CT image generated through the reconstruction processing by the reconstruction processing function 153 and the image generated by the image processing function 154 (eg, three-dimensional image, cross-sectional image, high-resolution image) on the display 142. to display. Also, the output control function 155 may transmit these images to the learning device 200 or other external devices via the communication interface 144 .

[Processing flow of X-ray CT apparatus]
A series of processes related to image generation by the X-ray CT apparatus 100 according to the embodiment will be described below. FIG. 3 is a flow chart showing a series of processes related to image generation by the X-ray CT apparatus 100 according to the embodiment. This X-ray CT apparatus 100 is not equipped with, for example, an X-ray tube 111 having a minimal focus or a high-precision X-ray detector 115, and is an apparatus having a normal or general resolution. .

First, the preprocessing function 152 acquires projection data based on the detection data output by the DAS 116 (step S100).

Next, the reconstruction processing function 153 performs reconstruction processing on the projection data generated by the preprocessing function 152 to obtain a reconstructed image of a region or part of the subject P (for example, heart, lungs, etc.). A CT image having a matrix size of 512×512 and having normal resolution (hereinafter referred to as “NR (Normal Resolution) image”) is generated (step S102).

Next, the image processing function 154 applies an NR image (that is, a 512-matrix, normal-resolution CT image) generated by reconstruction processing to the trained model MDL defined by the model information in the memory 141. Input (step S104). In response to this, the trained model MDL outputs an image with higher resolution than the input NR image. Hereinafter, the high-resolution image output by the learned model MDL will be referred to as an "SR (Super Resolution) image". The matrix size of the SR image is the same as the NR image input to the trained model MDL.

FIG. 4 is a diagram schematically showing how an SR image is obtained. The trained model MDL is a model trained in advance so that when an NR image of 512 matrices is input, a CT image of 512 matrices and having a higher resolution than the NR image is output as an SR image.

Next, the image processing function 154 acquires an SR image of 512 matrices from the learned model MDL (step S106).

Next, the output control function 155 causes the display 142 to display the SR image (step S108). In addition to displaying the SR image on the display 142, or instead of displaying the SR image, the output control function 155 may transmit the SR image to the learning device 200 or other external device via the communication interface 144. This completes the processing of this flowchart.

[Configuration of learning device]
FIG. 5 is a diagram showing a configuration example of the learning device 200 according to the embodiment. Learning device 200 includes, for example, communication interface 202 , input interface 204 , display 206 , memory 208 , and processing circuitry 210 .

The communication interface 202 includes, for example, a NIC and a wireless communication module. The communication interface 202 communicates with the X-ray CT apparatus 100 and other external devices via the communication network NW.

The input interface 204 accepts various input operations by the operator and outputs an electrical signal indicating the content of the accepted input operation to the processing circuit 210 . For example, the input interface 204 is implemented by a mouse, keyboard, touch panel, drag ball, switch, button, joystick, foot pedal, camera, infrared sensor, microphone, and the like. It should be noted that the input interface 204 in this specification is not limited to having physical operation components such as a mouse and keyboard. For example, the input interface 204 also includes an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and outputs the electrical signal to the control circuit.

The display 206 displays various information. For example, the display 206 displays a processing result of the processing circuit 210, a GUI for receiving various operations by the operator, and the like. The display 206 is, for example, an LCD, an organic EL display, or the like.

The memory 208 is implemented by, for example, a RAM, a semiconductor memory device such as a flash memory, a hard disk, or an optical disk. These storage media may be realized by other storage devices such as NAS and external storage server devices connected via the communication network NW. The memory 208 may also include storage media such as ROMs and registers.

The processing circuitry 210 includes, for example, an acquisition function 212 , a generation function 214 , a learning function 216 and an output control function 218 . The processing circuit 210 implements these functions by executing a program stored in the memory 208 by a hardware processor (computer), for example. The acquisition function 212 is an example of an “acquisition unit”, the generation function 214 is an example of a “generation unit”, and the learning function 216 is an example of a “learning unit”.

Hardware processor means circuitry such as, for example, CPUs, GPUs, application specific integrated circuits, programmable logic devices (eg, simple or complex programmable logic devices, field programmable gate arrays). Instead of storing the program in memory 208, the program may be configured to be directly embedded within the circuitry of a hardware processor. In this case, the hardware processor implements its functions by reading and executing the program embedded in the circuit. The above program may be stored in the memory 208 in advance, or may be stored in a non-temporary storage medium such as a DVD or CD-ROM. ) may be installed into memory 208 from a non-transitory storage medium. The hardware processor is not limited to being configured as a single circuit, and may be configured as one hardware processor by combining a plurality of independent circuits to implement each function. Also, a plurality of components may be integrated into one hardware processor to realize each function.

[Overall flow of learning device]
A series of processes of the learning device 200 according to the embodiment will be described below. FIG. 6 is a flow chart showing a series of processes of the learning device 200 according to the embodiment.

First, the acquisition function 212 obtains the image of the subject P from the high-resolution X-ray CT apparatus 100 equipped with the X-ray tube 111 having a minimal focus and the high-precision X-ray detector 115 via the communication interface 202 . An NR image that is a reconstructed image of a region or site, has a matrix size of 1024×1024 and has a resolution higher than the normal resolution, and a reconstructed image of the same region or site as the NR image, 1024×1024 and having the same resolution as the NR image (step S200).

An HR image is a CT image reconstructed based on projection data of X-rays emitted from an X-ray tube 111 having a minimal focus and then detected by a high-precision X-ray detector 115 . In other words, the HR image is a CT image in which noise, artifacts, etc. are reduced compared to the CT image reconstructed by the low-resolution X-ray CT apparatus 100 .

Next, the generation function 214 down-samples the 1024-matrix NR and HR images acquired by the acquisition function 212 (step S202) to generate 512-matrix NR and HR images (step S204).

FIG. 7 is a diagram schematically showing how 512-matrix NR and HR images are generated. As shown, the generation function 214 down-samples the 1024-matrix NR image to generate a 512-matrix NR image, and down-samples the 1024-matrix HR image to generate a 512-matrix HR image.

For example, the generation function 214 may reduce the number of pixels in each of the NR image and the HR image by aggregating a predetermined number of pixels into one pixel. The 1024-matrix NR image is an example of the “first medical image”, and the 512-matrix NR image generated by downsampling the 1024-matrix NR image is an example of the “third medical image”. be. The 1024-matrix HR image is an example of the "second medical image", and the 512-matrix HR image generated by downsampling the 1024-matrix HR image is an example of the "fourth medical image". be. The 1024 matrix is an example of the "first matrix size" and the 512 matrix is an example of the "second matrix size". The resolution of the NR image (normal resolution) is an example of the "first resolution", and the resolution of the HR image is an example of the "second resolution higher than the first resolution".

Next, the learning function 216 learns an unlearned model MDL using a data set in which the 512-matrix NR and HR images generated by the generation function 214 are combined as training data (step S206). In other words, the learning function 216 uses, as training data, a data set in which HR images of 512 matrices are associated with NR images of 512 matrices as correct targets (also referred to as labels) to be output by the model MDL. to learn the unlearned model MDL.

For example, the learning function 216 inputs a 512-matrix NR image to an unlearned model MDL, and adjusts the model MDL so that the image output by the model MDL approaches the target 512-matrix HR image. Adjust parameters (weighting factors, bias components, etc.). More specifically, the learning function 216 sets the parameters of the model MDL to SGD (Stochastic Gradient Descent), Adjust using Momentum SGD, AdaGrad, RMSprop, AdaDelta, Adam (Adaptive moment estimation), etc.

The output control function 218 transmits model information defining the model MDL learned by the learning function 216, that is, the learned model MDL, to the X-ray CT apparatus 100 via the communication interface 202 (step S208). For example, the output control function 218 transfers the model information defining the learned model MDL to the normal-resolution X-ray CT apparatus 100 that is not equipped with the X-ray tube 111 having a minimal focus or the high-precision X-ray detector 115. Send. As a result, the X-ray CT apparatus 100 can obtain a CT image with higher resolution from a CT image of 512 matrices, which takes less time to reconstruct than 1024 matrices.

According to the embodiment described above, the learning device 200 is a reconstructed image of a certain region or part of the subject P, and has a matrix size of 1024×1024 and an NR image having a normal resolution (“first an example of a "medical image") and an HR image ("second an example of “medical images”).

The learning device 200 down-samples the 1024-matrix NR image to generate an NR image having a matrix size of 512×512 and normal resolution (an example of a “third medical image”).

The learning device 200 down-samples the 1024-matrix HR image to generate an HR image (an example of a “fourth medical image”) having a matrix size of 512×512 and a resolution higher than the normal resolution. Generate.

Then, the learning device 200 performs machine learning (supervised learning) on the model MDL based on training data in which the 512-matrix NR image is associated with the 512-matrix HR image, which is the target image. As a result, a trained model MDL capable of improving the resolution of medical images is generated.

By using this trained model MDL, even in the normal resolution X-ray CT apparatus 100 which is not equipped with the X-ray tube 111 having a minimal focus or the high-precision X-ray detector 115, the 512 matrix and A CT image with resolution higher than normal resolution can be generated as an SR image. In this way, when learning the model MDL used for generating SR images, instead of using CT images of 1024 matrices as training data, 512 matrices obtained by downsampling the CT images of 1024 matrices By using the CT images as training data, the processing time required to generate high-resolution medical images (SR images) can be shortened.

(Other embodiments (modifications))
Modifications of the above-described embodiment will be described below. In the above description, the X-ray CT apparatus 100 and the learning apparatus 200, which are modalities, are separate apparatuses, but the present invention is not limited to this. For example, the X-ray CT apparatus 100 may have various functions of the learning device 200 (acquisition function 212, generation function 214, learning function 216).

Also, in the above description, the training data is a data set obtained by combining 512-matrix NR images and HR images, but the present invention is not limited to this. For example, the NR images and HR images of the training data may be images obtained by enlarging and reconstructing a partial region such as the heart or lungs on a 1024-matrix CT image, or by increasing the slice thickness in the Z-axis direction. It may be an image (for example, an image enlarged from 0.25 mm to 0.5 mm). Also, the NR image and HR image of the training data may be images obtained by trimming a part of the 1024-matrix CT image.

In the above description, the matrix of the CT image before downsampling is 1024, and the matrix of the CT image after downsampling is 512. However, the matrix of the CT image before downsampling is not limited to this. Any numerical value may be used as long as it satisfies the relationship that the matrix of the CT image after downsampling is smaller than .

Further, when 1024-matrix NR images and HR images are captured for each region or region such as the heart or lungs of the subject P, the learned model MDL may be generated for each region or region of the subject P. good. For example, the learning function 216 may generate a trained model MDL dedicated to the region of the heart by learning the model MDL using, as training data, a data set combining NR and HR images of the heart. . Similarly, when generating a trained model MDL for the lung region, the learning function 216 learns the model MDL using a data set combining NR and HR images of the lung as training data. , a trained model MDL dedicated to the lung region may be generated.

In the above description, the learning function 216 inputs a 512-matrix NR image to an unlearned model MDL, and the difference between the image output by the model MDL and the target 512-matrix HR image is Although the parameter of the model MDL is adjusted so that , is not limited to this.

For example, the learning function 216 prepares two models MDL, and learns one model MDL1 using, as training data, a data set in which NR images and HR images of 1024 matrices before downsampling are combined. Furthermore, similarly to the above-described embodiment, the learning function 216 learns the other model MDL2 using a data set combining 512-matrix NR and HR images after downsampling as training data. The learning function 216 inputs an arbitrary 1024-matrix NR image to the learned model MDL1, and applies a 512-matrix NR image down-sampled from the arbitrary 1024-matrix NR image to the learned model MDL2. to enter. As a result, the learned model MDL1 outputs an SR image of 1024 matrices, and the learned model MDL2 outputs an SR image of 512 matrices. Then, the learning function 216 may learn the learned model MDL2 so that the difference between the 1024-matrix SR image and the 512-matrix SR image becomes small.

Further, in the above description, the generation function 214 of the learning device 200 down-samples the 1024-matrix NR image to generate a 512-matrix NR image, and down-samples the 1024-matrix HR image to Although the description has been given assuming that a 512-matrix HR image is generated, the present invention is not limited to this.

For example, the image processing function 154 of the X-ray CT apparatus 100 may generate a 512-matrix NR image by down-sampling a 1024-matrix NR image. In this case, the acquisition function 212 of the learning device 200 may acquire HR images of 1024 matrices and NR images of 512 matrices from the X-ray CT apparatus 100 . Since the generation function 214 of the learning device 200 has already down-sampled the NR image, it may down-sample only the 1024-matrix HR image to generate a 512-matrix HR image. When downsampling is performed by the X-ray CT apparatus 100 and a 512-matrix NR image is generated, the 512-matrix NR image is "a first matrix size having a first matrix size and a first resolution. A 512-matrix HR image generated by downsampling from a 1024-matrix HR image is an example of a "third medical image having a first matrix size and a second resolution. is an example of

Further, in the above description, the learning device 200 uses the 512-matrix HR image down-sampled from the 1024-matrix HR image as the target image for the 512-matrix NR image down-sampled from the 1024-matrix NR image. Although the model MDL is learned based on the associated training data, it is not limited to this.

For example, the learning device 200 performs training in which a 1024 matrix HR image upsampled from a 512 matrix HR image is associated as a target image with a 1024 matrix NR image upsampled from a 512 matrix NR image. A model MDL may be learned based on the data.

More specifically, the acquisition function 212 of the learning device 200 acquires a 512-matrix NR image from the X-ray CT apparatus 100 and a 512-matrix HR image. The generation function 214 of the learning device 200 upsamples the 512-matrix NR image to generate a 1024-matrix NR image, and up-samples the 512-matrix HR image to generate a 1024-matrix HR image. do. The learning function 216 of the learning device 200 generates a data set combining the 1024-matrix NR and HR images generated by the generating function 214 as training data, and learns the model MDL based on the training data. As a result, the image processing function 154 of the X-ray CT apparatus 100 can use the trained model MDL to generate a higher-resolution 1024-matrix SR image from a 1024-matrix NR image.

Further, when the acquisition function 212 of the learning device 200 is able to acquire the 1024-matrix HR image from the X-ray CT apparatus 100, the generation function 214 upsamples only the 512-matrix NR image to obtain the 1024-matrix NR image. Images may be generated. The learning function 216 generates training data in which the 1024 matrix NR image generated by the upsampling of the generation function 214 is associated with the 1024 matrix HR image acquired by the acquisition function 212 as a target image, A model MDL is learned based on the training data. As a result, the image processing function 154 of the X-ray CT apparatus 100 can use the trained model MDL to generate a higher-resolution 1024-matrix SR image from a 1024-matrix NR image.

While several embodiments have been described, these embodiments are provided by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and modifications can be made without departing from the scope of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, as well as the scope of the invention described in the claims and equivalents thereof.

DESCRIPTION OF SYMBOLS 1... Medical image processing system 100... X-ray CT apparatus 110... Mounting apparatus 130... Bed apparatus 140... Console apparatus 141... Memory 142... Display 143... Input interface 144... Communication interface 150... Processing Circuit 151 System control function 152 Preprocessing function 153 Reconstruction processing function 154 Image processing function 155 Output control function 200 Learning device 202 Communication interface 204 Input interface 206 Display 208 Memory 210 Processing circuit 212 Acquisition function 214 Generation function 216 Learning function 218 Output control function

Claims (11)

A first medical image, a reconstructed image of a region of a subject, having a first matrix size and a first resolution; and a reconstructed image of the region, the first matrix size. and acquiring a second medical image having a second resolution higher than the first resolution;
generating a third medical image having a second matrix size smaller than the first matrix size and having the first resolution by downsampling the first medical image; a generating unit that generates a fourth medical image having the second matrix size and the second resolution by downsampling the second medical image;
a learning unit that generates a model for improving the resolution of medical images by machine learning based on the third medical image and the fourth medical image;
A learning device with The first medical image and the second medical image are captured for each region,
The learning unit generates the model for each region by machine learning based on the third medical image and the fourth medical image for each region.
A learning device according to claim 1. the first matrix size is 1024 and the second matrix size is 512;
3. A learning device according to claim 1 or 2. The generating unit sets an image obtained by trimming a portion of the first medical image as the third medical image, and sets an image obtained by trimming a portion of the second medical image as the fourth medical image,
A learning device according to any one of claims 1 to 3. The generating unit sets an image obtained by enlarging and reconstructing a part of the first medical image as the third medical image, and sets an image obtained by enlarging and reconstructing a part of the second medical image as the fourth medical image. shall be a medical image of
A learning device according to any one of claims 1 to 4. an acquisition unit that acquires a medical image of a subject;
The medical image of the subject acquired by the acquiring unit is input to a trained model, and the medical image output by the trained model to which the medical image of the subject is input is sent to an output unit. and an output control unit for outputting,
The trained model is based on a third medical image generated by downsampling the first medical image and a fourth medical image generated by downsampling the second medical image. , is a model machine-learned to improve the resolution of input medical images,
wherein the first medical image is a reconstructed image of a region of the subject and has a first matrix size and a first resolution;
the second medical image is a reconstructed image of the region and has the first matrix size and a second resolution higher than the first resolution;
the third medical image is a medical image having a second matrix size smaller than the first matrix size and having the first resolution;
the fourth medical image is a medical image having the second matrix size and the second resolution;
Medical image processing equipment. The first medical image and the second medical image are medical images captured by another medical image processing device having a resolution higher than that of the medical image processing device,
The medical image processing apparatus according to claim 6. the computer
A first medical image, a reconstructed image of a region of a subject, having a first matrix size and a first resolution; and a reconstructed image of the region, the first matrix size. and acquiring a second medical image having a second resolution higher than the first resolution;
generating a third medical image having a second matrix size smaller than the first matrix size and having the first resolution by downsampling the first medical image; downsampling the second medical image to produce a fourth medical image having the second matrix size and the second resolution;
generating a model for improving the resolution of the medical image by machine learning based on the third medical image and the fourth medical image;
learning method. the computer
A first medical image, a reconstructed image of a region of a subject, having a first matrix size and a first resolution; and a reconstructed image of the region, the first matrix size. obtaining a second medical image having a second matrix size greater than and having a second resolution higher than the first resolution;
downsampling the second medical image to produce a third medical image having the first matrix size and having the second resolution;
generating a model for improving the resolution of the medical image by machine learning based on the first medical image and the third medical image;
learning method. the computer
A first medical image, a reconstructed image of a region of a subject, having a first matrix size and a first resolution; and a reconstructed image of the region, the first matrix size. obtaining a second medical image having a second matrix size greater than and having a second resolution higher than the first resolution;
upsampling the first medical image to produce a third medical image having the second matrix size and having the first resolution;
generating a model for improving the resolution of the medical image by machine learning based on the second medical image and the third medical image;
learning method. to the computer,
A first medical image, a reconstructed image of a region of a subject, having a first matrix size and a first resolution; and a reconstructed image of the region, the first matrix size. and acquiring a second medical image having a second resolution higher than the first resolution;
generating a third medical image having a second matrix size smaller than the first matrix size and having the first resolution by downsampling the first medical image; downsampling the second medical image to generate a fourth medical image having the second matrix size and having the second resolution;
generating a model for improving the resolution of medical images by machine learning based on the third medical image and the fourth medical image;
program to run the

Images