JP2020006163A - Medical information processing device, method and program - Google Patents

Abstract

To provide a medical information processing device that can segment a flood map readily and accurately, and to provide a method and a program.SOLUTION: A medical information processing device 50 includes: a first acquisition unit for acquiring medical information on a subject; a second acquisition unit for acquiring numerical data which are digitized conditions for collecting medical data; and a generation unit that applies a machine learning model to input data including the numerical data and medical data to generate output data on the basis of the medical data.SELECTED DRAWING: Figure 1

Description

  Embodiments of the present invention relate to a medical information processing apparatus, method, and program.

  In machine learning using medical data such as medical image data and its raw data, a deep neural network (DNN: Deep Neural Network) learned from a large amount of learning data in order to restore original data from medical data with a part lost ). For example, in magnetic resonance imaging (MRI), DNN is applied to k-space data undersampled by Cartesian acquisition to generate k-space data in which a defective portion is restored, and k-space data after restoration is generated. There is a method of obtaining a reconstructed image based on the image. There is also a method of directly obtaining a restored image by applying DNN to the undersampled k-space data.

U.S. Patent Application Publication No. 2015/0276953 U.S. Pat.No.9672617 U.S. Patent No. 9686699

  The problem to be solved by the present invention is to easily and accurately classify a flood map.

  The medical information processing apparatus according to the embodiment includes: a first acquisition unit that acquires medical data regarding a subject; a second acquisition unit that acquires numerical data obtained by digitizing the medical data collection conditions; A generating unit that applies a machine learning model to input data including medical data and generates output data based on the medical data.

FIG. 1 is a diagram illustrating a configuration of a magnetic resonance imaging apparatus that mounts the medical information processing apparatus according to the present embodiment. FIG. 2 is a diagram showing a typical flow of MR imaging by the processing circuit of FIG. FIG. 3 is a diagram schematically showing the mask data generation processing in steps S2 and S3 in FIG. FIG. 4 is a diagram schematically illustrating a flow of the first DNN reconfiguration according to the present embodiment. FIG. 5 is a diagram schematically illustrating a flow of the second DNN reconfiguration according to the present embodiment. FIG. 6 is a diagram schematically illustrating the flow of the third DNN reconfiguration according to the present embodiment. FIG. 7 is a diagram illustrating a configuration of the model learning device according to the present embodiment. FIG. 8 is a diagram schematically showing processing by the learning sample generation function of FIG. FIG. 9 is a diagram illustrating another configuration of the medical information processing apparatus according to the present embodiment. FIG. 10 is a diagram schematically illustrating numerical data according to the present embodiment. FIG. 11 is a diagram schematically showing the input of the numerical data of FIG. 10 to the machine learning model. FIG. 12 is a diagram schematically illustrating input of other numerical data to the machine learning model. FIG. 13 is a diagram schematically showing input of other numerical data to the machine learning model. FIG. 14 is a diagram schematically showing other numerical data. FIG. 15 is a diagram schematically illustrating a network structure of the machine learning model according to the present embodiment. FIG. 16 is a diagram schematically illustrating a network structure of another machine learning model according to the present embodiment. FIG. 17 is a diagram schematically illustrating input and output of a machine learning model having a plurality of CNNs.

  Hereinafter, the medical information processing apparatus, method, and program according to the present embodiment will be described with reference to the drawings.

  The medical information processing apparatus according to the present embodiment is an apparatus that mounts a processing circuit that processes medical information. The medical information processing apparatus according to the present embodiment is realized by, for example, a computer mounted on a medical image diagnostic apparatus. The medical image diagnostic apparatus according to the present embodiment includes a magnetic resonance imaging apparatus (MRI apparatus), an X-ray computed tomography apparatus (CT apparatus), an X-ray diagnostic apparatus, a PET (Positron Emission Tomography) apparatus, and a SPECT apparatus (Single Photon). Emission CT) and a single modality device such as an ultrasonic diagnostic device, or a composite modality device such as a PET / CT device, a SPECT / CT device, a PET / MRI device, and a SPECT / MRI device. good. As another example, the medical information processing apparatus according to the present embodiment may be a computer communicably connected to the medical image diagnostic apparatus via a cable, a network, or the like, or may be independent of the medical image diagnostic apparatus. Computer may be used. Hereinafter, the medical information processing apparatus according to the present embodiment is mounted on a magnetic resonance imaging apparatus.

  FIG. 1 is a diagram illustrating a configuration of a magnetic resonance imaging apparatus 1 on which a medical information processing apparatus 50 according to the present embodiment is mounted. As shown in FIG. 1, the magnetic resonance imaging apparatus 1 includes a gantry 11, a bed 13, a gradient magnetic field power supply 21, a transmission circuit 23, a reception circuit 25, a bed driving apparatus 27, a sequence control circuit 29, and a medical information processing apparatus 50. .

  The gantry 11 has a static magnetic field magnet 41 and a gradient magnetic field coil 43. The static magnetic field magnet 41 and the gradient magnetic field coil 43 are housed in a housing of the gantry 11. The housing of the gantry 11 has a hollow bore. A transmission coil 45 and a reception coil 47 are arranged in the bore of the gantry 11.

  The static magnetic field magnet 41 has a hollow, substantially cylindrical shape, and generates a static magnetic field inside the substantially cylindrical portion. As the static magnetic field magnet 41, for example, a permanent magnet, a superconducting magnet, a normal conducting magnet, or the like is used. Here, the center axis of the static magnetic field magnet 41 is defined as the Z axis, an axis perpendicular to the Z axis is defined as the Y axis, and an axis horizontally orthogonal to the Z axis is defined as the X axis. The X axis, the Y axis, and the Z axis constitute an orthogonal three-dimensional coordinate system.

  The gradient magnetic field coil 43 is a coil unit attached to the inside of the static magnetic field magnet 41 and formed in a hollow substantially cylindrical shape. The gradient magnetic field coil 43 receives a current supplied from the gradient magnetic field power supply 21 and generates a gradient magnetic field. More specifically, the gradient magnetic field coil 43 has three coils corresponding to the X axis, the Y axis, and the Z axis that are orthogonal to each other. The three coils form a gradient magnetic field in which the magnetic field strength changes along each of the X, Y, and Z axes. The gradient magnetic fields along the X-axis, the Y-axis, and the Z-axis are combined to form a slice selection gradient magnetic field Gs, a phase encoding gradient magnetic field Gp, and a frequency encoding gradient magnetic field Gr, which are orthogonal to each other, in desired directions. The slice selection gradient magnetic field Gs is used to arbitrarily determine an imaging section. The phase encoding gradient magnetic field Gp is used to change the phase of the MR signal according to the spatial position. The frequency encoding gradient magnetic field Gr is used to change the frequency of the MR signal according to the spatial position. In the following description, it is assumed that the gradient direction of the slice selection gradient magnetic field Gs is the Z axis, the gradient direction of the phase encoding gradient magnetic field Gp is the Y axis, and the gradient direction of the frequency encoding gradient magnetic field Gr is the X axis.

  The gradient power supply 21 supplies a current to the gradient coil 43 according to a sequence control signal from the sequence control circuit 29. The gradient magnetic field power supply 21 supplies a current to the gradient magnetic field coil 43 to cause the gradient magnetic field coil 43 to generate a gradient magnetic field along each of the X-axis, the Y-axis, and the Z-axis. The gradient magnetic field is superimposed on the static magnetic field formed by the static magnetic field magnet 41 and applied to the subject P.

  The transmission coil 45 is arranged, for example, inside the gradient magnetic field coil 43, and receives a current from the transmission circuit 23 to generate a high-frequency magnetic field pulse (hereinafter, referred to as an RF magnetic field pulse).

  The transmission circuit 23 supplies a current to the transmission coil 45 in order to apply an RF magnetic field pulse for exciting target protons present in the object P to the object P via the transmission coil 45. The RF magnetic field pulse oscillates at a resonance frequency specific to the target proton and excites the target proton. A magnetic resonance signal (hereinafter, referred to as an MR signal) is generated from the excited target protons and detected by the receiving coil 47. The transmission coil 45 is, for example, a whole-body coil (WB coil). The whole-body coil may be used as a transmitting / receiving coil.

  The receiving coil 47 receives an MR signal emitted from a target proton existing in the subject P under the action of the RF magnetic field pulse. The receiving coil 47 has a plurality of receiving coil elements that can receive the MR signal. The received MR signal is supplied to the receiving circuit 25 via a wire or wirelessly. Although not shown in FIG. 1, the receiving coil 47 has a plurality of receiving channels mounted in parallel. The receiving channel has a receiving coil element for receiving the MR signal, an amplifier for amplifying the MR signal, and the like. The MR signal is output for each reception channel. The total number of receiving channels and the total number of receiving coil elements may be the same, or the total number of receiving channels may be larger or smaller than the total number of receiving coil elements.

  The receiving circuit 25 receives the MR signal generated from the excited target proton via the receiving coil 47. The receiving circuit 25 processes the received MR signal to generate a digital MR signal. Digital MR signals can be represented in k-space defined by the spatial frequency. Therefore, the digital MR signal is hereinafter referred to as k-space data. The k-space data is a type of raw data used for image reconstruction. The k-space data is supplied to the medical information processing device 50 via a wire or wirelessly.

  Note that the transmission coil 45 and the reception coil 47 are merely examples. Instead of the transmission coil 45 and the reception coil 47, a transmission / reception coil having a transmission function and a reception function may be used. Further, the transmission coil 45, the reception coil 47, and the transmission / reception coil may be combined.

  A bed 13 is installed adjacent to the gantry 11. The bed 13 has a top plate 131 and a base 133. The subject P is placed on the top 131. The base 133 supports the top plate 131 so as to be slidable along each of the X axis, the Y axis, and the Z axis. The bed 133 accommodates the bed driving device 27. The couch driving device 27 moves the table 131 under the control of the sequence control circuit 29. The couch driving device 27 may include, for example, any motor such as a servo motor and a stepping motor.

  The sequence control circuit 29 has, as hardware resources, a processor of a CPU (Central Processing Unit) or MPU (Micro Processing Unit) and a memory such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The sequence control circuit 29 synchronously controls the gradient magnetic field power supply 21, the transmission circuit 23, and the reception circuit 25 based on the imaging protocol determined by the imaging protocol setting function 511 of the processing circuit 51, and outputs a pulse corresponding to the imaging protocol. By executing the sequence, the subject P is subjected to MR imaging, and k-space data relating to the subject P is collected.

  As shown in FIG. 1, the medical information processing apparatus 50 is a computer device having a processing circuit 51, a memory 52, a display 53, an input interface 54, and a communication interface 55.

  The processing circuit 51 has, as hardware resources, a processor such as a CPU, a GPU (Graphics Processing Unit), and an MPU, and a memory such as a ROM and a RAM. The processing circuit 51 functions as a center of the magnetic resonance imaging apparatus 1. For example, the processing circuit 51 has an imaging protocol setting function 511, a mask data acquisition function 512, a raw data acquisition function 513, an image generation function 514, an image processing function 515, and a display control function 516 by executing various programs.

  In the imaging protocol setting function 511, the processing circuit 51 sets an imaging protocol relating to MR imaging of a target by a user instruction via the input interface 54 or automatically. The imaging protocol is a set of various imaging parameters related to one MR imaging. As the imaging parameters according to the present embodiment, various imaging parameters set directly or indirectly for performing MR imaging such as imaging time, type of k-space filling method, type of pulse sequence, TR, and TE can be applied. is there.

  In the mask data acquisition function 512, the processing circuit 51 acquires mask data relating to MR imaging of the target. The mask data is data in which a plurality of data collection trajectory candidates having a finite number of elements are assigned numerical values according to the number of times of acquisition and / or the direction of acquisition in target imaging. The data collection trajectory refers to a raw data collection trajectory on the k-space, and is related to the type of the k-space filling method. For example, when the k-space filling method is the radial method, the data collection trajectory is a general line (spoke) passing through the center of the k-space. When the k-space filling method is the spiral method or the variable density spiral method, the data collection trajectory is a spiral curve from the center of k-space toward the outer periphery. Acquisition of mask data includes processes such as generation of mask data by the processing circuit 51, selection of any one of a plurality of mask data, reception, transfer or transmission of mask data from another device.

  The raw data acquisition function 513 acquires k-space data. Acquisition of k-space data includes acquisition of k-space data by MR imaging performed under the control of the processing circuit 51, selection of any one of the k-space data from a plurality of k-space data, and acquisition of k-space data from another device. It shall include processing such as reception, transfer or transmission of spatial data.

  In the image generation function 514, the processing circuit 51 performs a reconstruction process using the machine learning model 521 on the mask data obtained by the mask data obtaining function 512 and the k-space data obtained by the raw data obtaining function 513. , An MR image of the subject P is generated. The machine learning model 521 is stored in the memory 52. The machine learning model 521 is a composite function with parameters defined by a combination of a plurality of adjustable functions and parameters (weighting matrix or bias). The machine learning model 521 is realized by a multilayer network model (DNN: Deep Neural Network) having an input layer, an intermediate layer, and an output layer. Hereinafter, the reconstruction process using the machine learning model 521 will be referred to as DNN reconstruction.

  In the image processing function 515, the processing circuit 51 performs various image processing on the MR image. For example, the processing circuit 51 performs image processing such as volume rendering, surface rendering, pixel value projection processing, MPR (Multi-Planer Reconstruction) processing, and CPR (Curved MPR) processing.

  In the display control function 516, the processing circuit 51 displays various information on the display 53. For example, the processing circuit 51 displays on the display 53 an MR image generated by the image generation function 514, an MR image generated by the image processing function 515, an imaging protocol setting screen, and the like.

  The memory 52 is a storage device such as a hard disk drive (HDD), a solid state drive (SSD), or an integrated circuit storage device for storing various information. Further, the memory 52 may be a drive device that reads and writes various information from and to a portable storage medium such as a CD-ROM drive, a DVD drive, and a flash memory. For example, the memory 52 stores k-space data, a control program, a machine learning model 521, and the like.

  The display 53 displays various information. For example, the display 53 displays an MR image generated by the image generation function 514, an MR image generated by the image processing function 515, an imaging protocol setting screen, and the like. As the display 53, for example, a CRT display, a liquid crystal display, an organic EL display, an LED display, a plasma display, or any other display known in the art can be appropriately used.

  The input interface 54 includes an input device that receives various commands from a user. As an input device, a keyboard, a mouse, various switches, a touch screen, a touch pad, and the like can be used. Note that the input device is not limited to one provided with physical operation components such as a mouse and a keyboard. For example, an electric signal processing circuit that receives an electric signal corresponding to an input operation from an external input device provided separately from the magnetic resonance imaging apparatus 1 and outputs the received electric signal to various circuits is also input. It is included in the example of the interface 54.

  The communication interface 55 connects the magnetic resonance imaging apparatus 1 with a workstation, a picture archiving and communication system (PACS), a hospital information system (HIS), a radiology information system (RIS), and the like via a LAN (local area network) or the like. Interface to connect. The network IF transmits and receives various information to and from a connected workstation, PACS, HIS, and RIS.

  Note that the above configuration is an example, and the present invention is not limited to this. For example, the sequence control circuit 29 may be incorporated in the medical information processing device 50. Further, the sequence control circuit 29 and the processing circuit 51 may be mounted on the same board. The sequence control circuit 29, the gradient power supply 21, the transmission circuit 23, and the reception circuit 25 may be mounted on a single control device different from the medical information processing device 50, or may be mounted separately on a plurality of devices. Is also good.

  Hereinafter, an operation example of the magnetic resonance imaging apparatus 1 and the medical information processing apparatus 50 according to the present embodiment will be described.

  FIG. 2 is a diagram showing a typical flow of MR imaging by the processing circuit 51. The process illustrated in FIG. 2 is started from setting of an imaging protocol of a target MR imaging.

  As shown in FIG. 2, the processing circuit 51 executes an imaging protocol setting function 511 (step S1). In step S1, the processing circuit 51 sets an imaging protocol for the subject P. As an imaging parameter included in the imaging protocol, an imaging time, a type of a k-space filling method, a type of a pulse sequence, TR, TE, and the like are set. In this embodiment, the type of the k-space filling method may be any method such as a radial method, a spiral method, and a Cartesian method. However, in order to specifically describe below, the k-space filling method is assumed to be a radial method.

  When step S1 is performed, the processing circuit 51 executes the mask data acquisition function 512 (steps S2 and S3). In steps S2 and S3, the processing circuit 51 generates mask data related to the target MR imaging. The mask data is data in which a numerical value indicating that data is collected in target imaging or a numerical value indicating that data is not collected is assigned to a plurality of data collection trajectories (spokes) having a finite number of elements. In other words, the mask data is data indicating a trajectory to be collected among candidate trajectories having a finite number of elements.

  FIG. 3 is a diagram schematically illustrating the process of generating the mask data 91 in steps S2 and S3. As shown in FIG. 3, the data collection trajectory for the radial method radially passes through the approximate center of the k-space. As shown in the upper part of FIG. 3, in the MR imaging according to the present embodiment, data collection to be collected is performed from a set 70 of data collection trajectory candidates (hereinafter, referred to as candidate trajectories) having a predetermined finite number of elements. A trajectory (hereinafter, referred to as a collection target trajectory) is selected. In FIG. 3, the candidate trajectory is indicated by a dotted line. The number of elements (number) and angles of the candidate trajectory are set in advance. The number of elements of the candidate trajectory is set to such a number that sufficient image quality can be guaranteed when data collection is performed for all the candidate trajectories of the number of elements. The number of elements of the candidate trajectory is preferably finite. That is, the number of elements of the candidate trajectory may be a large number such as one million if it is not an irrational number. FIG. 3 shows eight candidate trajectories for simplicity of the drawing.

  The angle of the candidate trajectory according to the present embodiment is defined as a reference angle, for example, an angle from 0 degrees. The angle interval between adjacent candidate trajectories is set to a predetermined angle. Specifically, it is preferable that the angular intervals between adjacent candidate trajectories are set to be substantially equal. In this way, by limiting the number of elements and the angle, it is also possible to limit the number of elements and the angle of the candidate trajectory of the learning data of the machine learning.

  The collection angle of each candidate trajectory is set to a multiple of the basic angle. That is, when the collection angle of the first data collection trajectory is 0 degrees, the collection angle of the second candidate trajectory = basic angle, the collection angle of the third candidate trajectory = “basic angle × 2”, the fourth candidate The collection angle of the trajectory = “basic angle × 3”,...

  If the number of elements is finite, the basic angle may be set to a golden angle, for example, an irrational number of about 111.25 degrees. For example, when the number of elements is 1000, the collection angle of the first candidate trajectory is 0 degree, the collection angle of the second candidate trajectory is a golden angle, and the collection angle of the third candidate trajectory is “golden angle × 2”. , The collection angle of the fourth candidate trajectory is set up to the 1000th candidate trajectory, such as “golden angle × 3”.

  The collection angle may be set to a value obtained by dividing 360 degrees by the number of elements. For example, when the number of elements is 1,000, the basic angle may be set to 360/1000 degrees = 0.36 degrees. Further, the basic angle may be set to a multiple of 360/1000 degrees. For example, when the multiple is 309, the basic angle is set to 360/1000 degrees × 309 = 111.24 degrees. Thereby, the basic angle can be made substantially equal to the golden angle.

  A number (hereinafter, referred to as a locus number) is assigned to the candidate locus for identification. The trajectory number is used for generating mask data. Any rule may be used for assigning the track number. For example, numbers may be assigned in order from a candidate trajectory having a small or large collection angle, or numbers may be assigned in accordance with the setting order of candidate trajectories.

  First, the processing circuit 51 selects a collection target trajectory based on the imaging time and the candidate trajectory (step S2). For example, the processing circuit 51 selects a collection target trajectory from the candidate trajectories included in the set 70 according to a predetermined rule (hereinafter, referred to as a selection rule) based on the imaging time set in step S1. Specifically, first, the processing circuit 51 determines the number of collection target trajectories according to the imaging time. Next, the processing circuit 51 selects the determined number of trajectories to be collected from the set 70 according to the selection rule. As a selection rule, for example, it is preferable to select such that the angles between the trajectories to be collected are substantially the same. For example, as shown in the middle part of FIG. 3, three candidate trajectories having substantially equal angles, that is, first, fourth, and seventh candidate trajectories are selected as collection target trajectories from eight candidate trajectories. . In FIG. 3, the collection target trajectory is indicated by a solid line.

  The collection target trajectory is set for each slice or volume, or for each frame. That is, the collection target trajectory is set for each image. The image quality of a radial image is typically guaranteed by about 800 spokes. However, according to the present embodiment, the same image quality can be guaranteed by using about 30 to 50 spokes per image. This is partly because the collection trajectories according to the present embodiment have substantially uniform angular intervals.

  Next, the processing circuit 51 generates the mask data 91 based on the selected collection target trajectory (Step S3). As shown in the lower part of FIG. 3, the mask data 91 includes a numerical value indicating that a plurality of candidate trajectories have been selected (that is, data collection) or a numerical value that has not been selected (that is, no data collection). Is the assigned numerical data. For example, “1” is assigned as a numerical value indicating selection, and “0” is assigned as a numerical value indicating no selection. The processing circuit 51 determines whether or not each candidate trajectory has been selected, assigns “1” when selected, and assigns “0” when not selected. The numerical values of “0” or “1” are arranged in the order of the trajectory numbers of the candidate trajectories. For example, in the case of FIG. 3, since the first, fourth, and seventh candidate trajectories are selected and the second, third, fifth, sixth, and eighth candidate trajectories are not selected, the mask data 91 is selected. , (1, 0, 0, 1, 0, 0, 1, 0).

  When step S3 is performed, the processing circuit 51 executes the raw data acquisition function 513 (step S4). In step S4, the processing circuit 51 instructs the sequence control circuit 29 to execute MR imaging. The sequence control circuit 29 controls the gradient magnetic field power supply 21, the transmission circuit 23, and the reception circuit 25 synchronously, executes MR imaging on the acquisition target trajectory selected in step S2, and converts k-space data related to the acquisition target trajectory into MR space. collect. The collected k-space data is k-space data corresponding to the collection target trajectory represented by the mask data generated in step S3. Since the collection target trajectory is selected from the candidate trajectories for the number of elements, the k-space data collected in step S4 is typically sparse.

  When step S4 is performed, the processing circuit 51 executes the image generation function 514 (step S5). In step S5, the processing circuit 51 performs DNN reconstruction on the mask data generated in step S3 and the k-space data collected in step S4 to generate an MR image.

  FIG. 4, FIG. 5, and FIG. 6 are diagrams schematically showing the flow of the DNN reconfiguration according to the present embodiment. In the first DNN reconstruction shown in FIG. 4, the processing circuit 51 applies the machine learning model 521-1 to the k-space data RD and the mask data MD, and generates an MR image RI corresponding to the k-space data RD. . In the machine learning model 521-1, parameters are learned so that k-space data and mask data are input and an MR image corresponding to the k-space data is output. The structure of the machine learning model 521-1 is not particularly limited. The MR image RI generated by the machine learning model 521-1 based on the k-space data RD and the mask data MD is different from the MR image generated by an analytic reconstruction method such as Fourier transform on the k-space data RD. Thus, artifacts due to signal loss included in the k-space data RD are reduced. This is because the machine learning model 521-1 uses the mask data MD indicating the trajectory to be collected corresponding to the k-space data RD in addition to the k-space data RD.

  Note that the signal loss according to the present embodiment is a concept including any difference between actual k-space data and desired k-space data, including sparse data. For example, signal loss includes signal deterioration due to noise caused by various causes other than sparseness, information loss due to conversion from a continuous value to a discrete value generated in the process of A / D conversion, and the like.

  In the second DNN reconstruction shown in FIG. 5, the processing circuit 51 applies the machine learning model 521-2 to the k-space data RD and the mask data MD, and the signal missing part included in the k-space data RD is restored. Generate k-space data RDM. The parameters of the machine learning model 521-2 are learned so that the k-space data and the mask data are input and the denoised k-space data is output. The k-space data RDM generated by the machine learning model 521-2 based on the k-space data RD and the mask data MD has more artifacts due to signal loss included in the k-space data RD than the k-space data RD. Reduced. This is because the machine learning model 521-2 uses the mask data MD indicating the trajectory to be collected corresponding to the k-space data RD in addition to the k-space data RD.

  As shown in FIG. 5, when the k-space data RDM is generated, the processing circuit 51 performs a Fourier transform on the k-space data RDM to generate the k-space data RD or the MR image RI corresponding to the k-space data RDM. I do. Since the MR image RI is generated by the Fourier transform on the k-space data RDM in which the signal deficient portion is restored, the image quality is lower than that of the MR image generated by the Fourier transform on the k-space data RD including the signal deficiency. Good.

  In the third DNN reconstruction shown in FIG. 6, the processing circuit 51 performs a Fourier transform on the k-space data RD to generate a temporary MR image RIM corresponding to the k-space data RD. Since the provisional MR image RIM is a reconstructed image generated by performing a Fourier transform on the k-space data RD including the signal loss, the temporary MR image RIM often includes the signal loss.

  As shown in FIG. 6, when the provisional MR image RIM is generated, the processing circuit 51 applies the machine learning model 521-3 to the provisional MR image RIM and the mask data MD, and outputs a signal included in the provisional MR image RIM. An MR image RI in which the missing part is restored is generated. The parameters of the machine learning model 521-3 are learned so that the temporary MR image RIM and the mask data MD are input and a denoised MR image is output. In the MR image RI generated by the machine learning model 521-3 based on the temporary MR image RIM and the mask data MD, artifacts due to signal loss are reduced as compared to the temporary MR image RIM. This is because the machine learning model 521-3 uses, as an input, the mask data MD indicating the acquisition target trajectory corresponding to the k-space data RD used for the temporary MR image RIM, in addition to the temporary MR image RIM.

  When step S5 is performed, the processing circuit 51 executes the display control function 516 (step S6). In step S6, the processing circuit 51 displays the MR image generated in step S5 on the display 53.

  Thus, the MR imaging by the processing circuit 51 ends.

  Note that the processing flow illustrated in FIG. 2 is an example, and the present embodiment is not limited to this. For example, in step S2, the processing circuit 51 automatically selects the collection target trajectory based on the imaging time. However, the present embodiment is not limited to this. For example, the processing circuit 51 may select a candidate trajectory specified by the user via the input interface 54 as a trajectory to be collected. In this case, for example, the processing circuit 51 displays on the display 53 a schematic diagram that graphically represents a selectable trajectory with a finite number of elements in a selectable manner. As a schematic diagram, for example, an image in which candidate trajectories arranged in the k-space and their trajectory numbers are drawn as shown in the upper part of FIG. 3 is preferable. The user specifies an arbitrary trajectory from among the candidate trajectories included in the displayed schematic diagram as the collection target trajectory via the input interface 54. The processing circuit 51 may select the designated trajectory as the trajectory to be collected.

  The above embodiment is based on the premise that one or zero data acquisitions are performed for each acquisition target trajectory for MR imaging of one image. However, the present embodiment is not limited to this. For example, data collection may be performed twice or more for a certain collection target trajectory. By performing data collection two or more times, it is possible to improve the reliability of k-space data relating to the trajectory to be collected. For example, if data collection is performed twice for the first and fourth trajectories, once for the second, fifth, and seventh trajectories, and zero times for the third, sixth, and eighth trajectories, the mask data Becomes (2,1,0,2,1,0,1,0).

  In the mask data according to the above embodiment, a numerical value according to the number of acquisitions is assigned to each data acquisition trajectory. However, the present embodiment is not limited to this. For example, a numerical value “1” may be uniformly assigned to a data collection trajectory in which two or more collections are performed. Also, a numerical value “1” may be assigned in the case of 0 times and 2 or more times, and a numerical value “0” may be assigned only in the case of 1 time. An arbitrary numerical value may be assigned to the mask data according to the present embodiment according to other rules.

  In the description of the present embodiment, a non-negative value is used as mask data. However, the present invention is not limited to this, and may include a negative value, for example, -1.

  Further, the numerical value of the mask data has a value corresponding to the number of times of collection. However, the present embodiment is not limited to this. For example, consider a case in which the same trajectory is collected in a positive direction and a negative direction in radial collection. When the positive data collection trajectory and the negative data collection trajectory are treated as different trajectories, a numerical value according to the number of times of collection is assigned to each data collection trajectory. A numerical value according to the collecting direction or a numerical value according to the combination of the number of times of collection and the collecting direction is assigned to one data collecting trajectory without distinguishing between the data collecting trajectory in the positive direction and the data collecting trajectory in the negative direction. You may be. For example, when data is collected 100 times in the positive and negative directions for a 0-degree data collection trajectory, (1, 0) and the like are assigned to the data collection trajectory. Note that “1” in (1, 0) is an example of a numerical value indicating a positive direction, and “0” is an example of a numerical value indicating a negative direction.

  The design of the structure of the machine learning model 521 can be changed as appropriate. For example, in the case of the second DNN reconstruction, the machine learning model 521 is an arbitrary multilayer network (hereinafter, referred to as the present network layer) in which the k-space data RD and the mask data MD are input and the k-space data RDM is output. , An FFT (Fast Fourier Transfer) layer may be incorporated. K-space data is input to the FFT layer, FFT is applied to the input k-space data, and an MR image is output. Thus, the machine learning model 521 alone can output the MR image RI based on the k-space data RD and the mask data MD.

  Further, the machine learning model 521 may have a chain structure in which a unit network structure including an IFFT (Inverse Fast Fourier Transfer) layer subsequent to the present network layer and the FFT layer is cascaded. An MR image is input to the IFFT layer, IFFT is applied to the input MR image, and k-space data is output. With the chain structure, it is possible to improve the restoration accuracy of the signal missing portion.

  In the chain structure, a matching layer may be provided after the IFFT layer. The matching layer receives k-space data based on the MR image output from the present network layer and k-space data before processing input to the present network layer, and performs processing using the k-space data before processing. Matching processing is performed on the already-processed k-space data, and the matched k-space data is output. In the k-space data that has been processed in the matching processing, the k-space data before the processing is weighted and added for each pixel in accordance with the degree of signal loss. For example, the lower the degree of signal loss, the higher the weight assigned to the pixel value of the k-space data before processing, and the higher the degree of signal loss, the lower the weight given to the pixel value of the k-space data before processing. Thus, consistency between the processed k-space data and the k-space data before the processing can be ensured.

  Further, in the machine learning model 521, an element product calculation layer may be provided before the present network layer. The element product calculation layer calculates an element product of the k-space data and the mask data, and outputs the element product data. The present network layer receives k-space data and element product data as input, and outputs MR image data in which a signal missing portion of the k-space data is restored. The unit network structure of the present network layer and the element product calculation layer may be cascaded.

  In the above description, it is assumed that the k-space data input to the machine learning model 521 is the original k-space data collected by MR imaging. However, the present embodiment is not limited to this. The k-space data according to the present embodiment may be calculated k-space data generated by performing forward projection processing on an MR image. Further, the k-space data or the MR image according to the present embodiment may be raw data that has been subjected to any signal processing such as signal compression processing, resolution decomposition processing, signal interpolation processing, and resolution synthesis processing. Further, the raw data according to the present embodiment may be hybrid data generated by performing a Fourier transform on the k-space data in the readout direction.

  Next, learning of the machine learning model 521 will be described. The machine learning model 521 is generated by a model learning device. The model learning device causes a machine learning model to perform machine learning according to a model learning program based on learning data including a plurality of learning samples, and generates a learned machine learning model (hereinafter, referred to as a learned model). The model learning device is a computer such as a workstation having a processor such as a CPU and a GPU. The model learning device and the medical information processing device 50 may be communicably connected via a cable or a communication network, or may not be connected. Further, the model learning device and the medical information processing device 50 may be configured by an integrated computer.

  FIG. 7 is a diagram illustrating a configuration of the model learning device 6 according to the present embodiment. As shown in FIG. 7, the model learning device 6 includes a processing circuit 61, a memory 62, an input interface 63, a communication interface 64, and a display 65.

  The processing circuit 61 has a processor such as a CPU and a GPU. The processor executes the learning sample generation function 611, the learning function 612, the display control function 613, and the like by activating the model learning program installed in the memory 62 or the like. The functions 611 to 613 are not limited to being realized by a single processing circuit. A processing circuit may be configured by combining a plurality of independent processors, and each processor may execute a program to realize each of the functions 611 to 613.

  In the learning sample generation function 611, the processing circuit 61 generates a learning sample which is a combination of input data and output data. The processing circuit 61 generates a learning sample based on the k-space data collected along all the candidate trajectories.

  In the learning function 612, the processing circuit 61 causes the machine learning model to learn parameters based on learning data on a plurality of learning samples. The learned machine learning model 521 shown in FIG. 1 is generated by learning the parameters by the learning function 612.

  In the display control function 613, the processing circuit 61 displays learning data, learning results, and the like on the display 65.

  The memory 62 is a storage device such as a ROM, a RAM, an HDD, an SSD, and an integrated circuit storage device for storing various information. The memory 62 stores, for example, a model learning program for learning a multilayered network. The memory 62 may be a drive device that reads and writes various information between a portable storage medium such as a CD, a DVD, and a flash memory, and a semiconductor memory element such as a RAM, in addition to the storage device. Further, the memory 62 may be in another computer connected to the model learning device 6 via a network.

  The input interface 63 receives various input operations from the user, converts the received input operations into electric signals, and outputs the electric signals to the processing circuit 61. Specifically, the input interface 63 is connected to input devices such as a mouse, a keyboard, a trackball, a switch, a button, a joystick, a touchpad, and a touch panel display. The input interface 63 outputs an electric signal corresponding to an input operation to the input device to the processing circuit 61. Further, the input device connected to the input interface 63 may be an input device provided in another computer connected via a network or the like.

  The communication interface 64 is an interface for performing data communication with the medical information processing apparatus 50, the medical image diagnostic apparatus, and another computer.

  The display 65 displays various information according to the display control function 613 of the processing circuit 61. For example, the display 65 displays learning data, learning results, and the like. The display 65 outputs a GUI or the like for receiving various operations from the user. For example, as the display 65, a liquid crystal display, a CRT display, an organic EL display, a plasma display, or any other display can be appropriately used.

  Next, an operation example of the model learning device 6 according to the present embodiment will be described.

  As described above, the processing circuit 61 generates a learning sample based on the k-space data collected along all the candidate trajectories. The types of input data and output data differ depending on the type of the machine learning model 521 shown in FIGS. In the case of the machine learning model 521-1 in FIG. 4, mask data and k-space data including signal loss are used as input data, and an MR image with reduced signal loss is used as output data. In the case of the machine learning model 521-2 in FIG. 5, mask data and k-space data including signal loss are used as input data, and k-space data with reduced signal loss is used as output data. In the case of the machine learning model 521-3 in FIG. 6, the mask data and the MR image including the signal loss are used as input data, and the MR image with reduced signal loss is used as output data.

  Hereinafter, as an example of the machine learning model 521, a machine learning model 521-1 of FIG. 4 in which k-space data and mask data are input and parameters are learned so as to output an MR image will be described. explain.

  FIG. 8 is a diagram schematically illustrating processing by the learning sample generation function 611. As shown in FIG. 8, the full k-space data RDF is obtained by the processing circuit 61. The full k-space data is k-space data obtained by collecting data on all candidate trajectories having a finite number of elements. The full k-space data RDF is collected in advance by the magnetic resonance imaging apparatus 1 or the like.

  As shown in FIG. 8, in the learning sample generation function 611, the processing circuit 61 performs a thinning-out process on the full k-space data RDF, and performs virtual k processing on all combinations of processing target trajectories (hereinafter, referred to as trajectory combinations). The spatial data RD1 to RDn are generated. For example, the processing circuit 61 calculates the k-space data corresponding to each trajectory combination by simulation based on the full k-space data RDF. In FIG. 8, virtual k-space data RD1 to RDn show k-space data related to three processing target trajectories, but the number of processing target trajectories constituting virtual k-space data RD1 to RDn is full. Any number may be used as long as it is smaller than the number of candidate trajectories constituting the k-space data RDF. The virtual k-space data RD1 to RDn are used as input data of the machine learning model 521-1.

  When the thinning processing is performed, the processing circuit 61 performs Fourier transform on each of the virtual k-space data RD1 to RDn, and generates virtual MR images RI1 to RIn corresponding to the virtual k-space data RD1 to RDn. The virtual MR images RI1 to RIn are used as correct MR images of the machine learning model 521-1.

  Further, the processing circuit 61 performs mask data conversion processing on each of the virtual k-space data RD1 to RDn to generate mask data MD1 to MDn corresponding to the virtual k-space data RD1 to RDn. Specifically, the processing circuit 61 first specifies a trajectory combination of each of the virtual k-space data RD1 to RDn. Then, for each of the candidate trajectories, the processing circuit 61 assigns a numerical value “1” to a trajectory included in the trajectory combination and assigns a numerical value “0” to a trajectory not included in the trajectory combination. Thereby, mask data MD1 to MDn are generated. The mask data MD1 to MDn are used as input data of the machine learning model 521-1.

  The mask data MD1 to MDn, the k-space data RD1 to RDn, and the MR images RI1 to RIn are associated for each trajectory combination. Mask data MD1 to MDn, k-space data RD1 to RDn, and MR images RI1 to RIn for each trajectory combination are treated as learning samples. The learning sample for each trajectory combination is stored in the memory 62.

  Thus, the processing by the learning sample generation function 611 ends. Next, a learning process by the learning function 612 is performed. In the learning function 612, the processing circuit 61 performs forward propagation processing by applying the k-space data RD1 to RDn and the mask data MD1 to MDn to the machine learning model for each trajectory combination, and performs an MR image (hereinafter referred to as an estimated MR image). ) Is output. Next, the processing circuit 61 performs a back propagation process by applying a difference (error) between the estimated MR result and the correct MR image to the machine learning model, and calculates a gradient vector. Next, the processing circuit 61 updates parameters such as a weighted matrix and a bias of the machine learning model based on the gradient vector. By repeating the forward propagation process, the back propagation process, and the parameter update process while changing the learning sample, a learned machine learning model 521-1 is generated.

  The learned machine learning model 521-1 is stored in the memory 62. The learned machine learning model 521-1 is transmitted to the magnetic resonance imaging apparatus 1 via the communication interface 64.

  As described above, the type of the k-space filling method according to the present embodiment may be a spiral method. Also in this case, the mask data can be generated similarly to the radial method. The candidate trajectory for the spiral method draws a spiral with the center of k-space as the starting point and an arbitrary point as the ending point. Similarly to the radial method, the processing circuit 51 selects an arbitrary candidate trajectory from a set including a plurality of candidate trajectories as a collection target trajectory, and generates mask data corresponding to the selected collection target trajectory. Since the candidate trajectory that can be generated depends on the hardware performance of the gradient coil, it is necessary to prepare learning data for each target hardware. Or. A set of candidate trajectories of the spiral method may be determined according to a low gradient coil so as to be compatible with a plurality of hardware.

  In the above description, the medical information processing apparatus 50 is assumed to be incorporated in the magnetic resonance imaging apparatus 1. However, the present embodiment is not limited to this.

  FIG. 9 is a diagram illustrating another configuration of the medical information processing apparatus 50 according to the present embodiment. As shown in FIG. 9, the medical information processing device 50 is a single computer independent of a modality device such as a magnetic resonance imaging device. The medical information processing apparatus 50 in FIG. 9 includes a processing circuit 51, a memory 52, a display 53, an input interface 54, and a communication interface 55, similarly to the medical information processing apparatus 50 in FIG. The functions and the like of each component are the same as those in FIG.

  Note that the medical information processing apparatus 50 in FIG. 9 can process raw data collected by any modality device. For example, the raw data may be sinogram data collected by an X-ray computed tomography apparatus.

  As described above, the medical information processing apparatus 50 according to the present embodiment includes the processing circuit 51. The processing circuit 51 implements at least a mask data acquisition function 512, a raw data acquisition function 513, and an image generation function 514. In the mask data acquisition function 512, the processing circuit 51 acquires mask data in which a numerical value according to the number of acquisitions and / or the direction of acquisition in target imaging is assigned to a plurality of data acquisition trajectory candidates having a finite number of elements. In the raw data acquisition function 513, the processing circuit 51 acquires raw data collected in subject imaging of the subject P. In the image generation function 514, the processing circuit 51 performs a reconstruction process using the machine learning model 521 on the obtained mask data and the obtained raw data to generate a medical image of the subject P.

  According to the above configuration, the machine learning model 521 uses the mask data indicating the trajectory to be collected corresponding to the k-space data in addition to the k-space data. As a result, the quality of the output medical image can be improved. Further, since the candidate trajectory is limited to a trajectory having a predetermined finite number of elements, it is not necessary to calculate the trajectory to be acquired for each MR imaging. In this connection, the data collection trajectory used for learning data and the data collection trajectory used for MR imaging can be easily matched.

  In the above embodiment, the input of the machine learning model is performed according to raw data or medical data such as medical images, and the number of collections and / or the direction of collection in candidate imaging for a plurality of data collection trajectory candidates having a finite number of elements. And the assigned mask data. However, the present embodiment is not limited to this. It is possible to extend the concept of mask data to numerical data obtained by digitizing medical data acquisition conditions. The medical data collection condition is a concept that includes not only an imaging protocol for medical imaging but also data processing conditions for raw data and image processing conditions for medical images. The imaging protocol of the medical imaging includes not only the data collection trajectory described above, but also the type of the pulse sequence, the frame number, and the type of the k-space filling trajectory.

  In the above embodiment, the machine learning model is used for DNN reconstruction from raw data to a medical image. However, the present embodiment is not limited to this. The output of the machine learning model may be any data as long as the data is used for medical diagnosis. For example, a machine learning model performs restoration of a missing portion of raw data, restoration of a missing portion of a medical image, generation of a segmentation image in which an anatomical tissue and the like are segmented, and output of an identification result of the anatomical tissue and the like. May be.

  FIG. 10 is a diagram schematically showing the numerical data 93. Numerical data 93 shown in FIG. 10 is data in which the type of the pulse sequence is digitized. As shown in FIG. 10, the numerical data 93 includes two or more finite elements 94. The number of elements matches the number of the plurality of candidate conditions included in the category to which the collection condition to be digitized belongs. For example, as shown in FIG. 10, when the category of the acquisition condition to be digitized is the type of the pulse sequence, the candidate conditions include, for example, FSE (Fast Spin Echo), FE (Field Echo), and EPI (Echo Planar). Imaging). In this case, the numerical data 93 is expressed as (FSE, FE, EPI).

  Each element 94 corresponds to a numerical value according to whether or not the acquisition condition of the target imaging is employed for a plurality of candidate conditions. For example, as the value of the element 94, one of a first value indicating that the candidate condition is adopted and a second value indicating not being adopted are assigned. In this case, the numerical data 93 is called a one-hot vector. Any integer or natural number may be used as long as the first value and the second value are different. For example, the first value is set to “1” and the second value is set to “0”. I do.

  For example, as shown in FIG. 10, when the acquisition condition of the target imaging uses FSE, the value of the element 84 corresponding to FSE is “1”, the value of the element 84 corresponding to FE is “0”, and the value of EPI corresponds to EPI. The value of the element 84 is set to “0”. Note that the number of elements may be finite or infinite. That is, the number of candidate conditions may be any. For example, the number of elements, that is, the number of candidate conditions, can be set to 1,000,000 or more.

  FIG. 11 is a diagram schematically showing the input of the numerical data 93-1 in FIG. 10 to the machine learning model. As shown in the upper part of FIG. 11, when the FSE is adopted, the value of the element corresponding to the FSE of the numerical data 93-1 is “1”, the value of the element corresponding to the FE is “0”, and the value corresponds to the EPI. The value of the element is set to “0”. The combination of the numerical data 93-1 and the input medical image 101 is input to the machine learning model. As shown in the middle part of FIG. 11, when the FE is adopted, the value of the element corresponding to the FE of the numerical data 93-1 is “0”, the value of the element corresponding to the FE is “1”, and the value corresponding to the EPI. The value of the element is set to “0”. The combination of the numerical data 93-1 and the input medical image 101 is input to the machine learning model. As shown in the lower part of FIG. 11, when the EPI is adopted, the value of the element corresponding to the EPI of the numerical data 93-1 is “0”, the value of the element corresponding to the FE is “0”, and the value of the element corresponding to the EPI is “0”. The value of the element is set to "1". The combination of the numerical data 93-1 and the input medical image 101 is input to the machine learning model. By adding the numerical data 93-1 obtained by digitizing the type of the pulse sequence to the input of the machine learning model, output data can be generated in consideration of not only the medical image or raw data but also the type of the pulse sequence. Becomes possible. Therefore, the accuracy of the output data of the machine learning model is improved.

  FIG. 12 is a diagram schematically showing the input of other numerical data 93-2 to the machine learning model. Numerical data 93-2 shown in FIG. 12 digitizes the collection condition “golden frame number”. The golden corner frame number is a frame number of a moving image capturing using the radial method in which the collection angle of the data collection trajectory is set according to the rule of the golden corner, and associates the frame number with the golden angle of the reference data collection trajectory for the frame It is a thing. When the frame is composed of a plurality of data collection trajectories, the reference data collection trajectory is a data collection trajectory of an arbitrary order such as first, last, or middle. In this case, the numerical data 93-2 is expressed as (golden frame # 1, golden frame # 2,...). The number of elements of the numerical data 93-2 matches or corresponds to the number of frame number candidates.

  As shown in the upper part of FIG. 12, when the input medical image 101 is the golden corner frame # 1, the value of the element corresponding to the golden corner frame # 1 of the numerical data is “1”, and the value of the element corresponding to the other frame is “1”. The value is set to “0”. The combination of the numerical data 93-2 and the input medical image 101 is input to the machine learning model. As shown in the middle part of FIG. 12, when the input medical image 101 is the golden corner frame # 2, the value of the element corresponding to the golden corner frame # 2 of the numerical data is “1”, and the value of the element corresponding to the other frame is “1”. The value is set to “0”. The combination of the numerical data 93-2 and the input medical image 101 is input to the machine learning model. By adding the numerical data obtained by digitizing the golden corner frame number to the input of the machine learning model, it is possible to generate output data in consideration of not only the medical image or raw data but also the golden corner frame number. Become. Therefore, the accuracy of the output data of the machine learning model is improved.

  Note that the collection angle and the collection order of the data collection trajectory of the radial method in moving image capturing are not limited to those using the golden angle, but may be those using an inverted bit pattern or those using a sequential pattern. In this case, the number of elements of the numerical data matches or corresponds to the number of candidates for the inverted bit pattern or the sequential pattern. The value of each element is set according to whether the inverted bit pattern or the sequential pattern regarding the input medical image employs the inverted bit pattern or the sequential pattern corresponding to the element.

  FIG. 13 is a diagram schematically illustrating input of other numerical data 93-3 to the machine learning model. Numerical data 93-3 shown in FIG. 13 digitizes the collection condition “pseudo random pattern number”. The pseudo-random pattern number is the number of a random number pattern for capturing a moving image using the pseudo-random Cartesian method, and is obtained by associating a frame number with a random number pattern for the frame. The pseudo-random Cartesian method refers to a Cartesian method in which a data collection trajectory is set according to a random number pattern. In this case, the numerical data 93-3 is expressed as (pseudo-random pattern # 1, pseudo-random pattern # 2,...). The number of elements of the numerical data 93-3 matches or corresponds to the number of random number pattern candidates.

  As shown in the upper part of FIG. 13, when the input medical image 101 is the pseudo random pattern # 1, the value of the element corresponding to the pseudo random pattern # 1 of the numerical data 93-3 is “1”, and the other pseudo random patterns Is set to “0”. The combination of the numerical data 93-3 and the input medical image 101 is input to the machine learning model. As shown in the middle part of FIG. 13, when the input medical image 101 is the pseudo random pattern # 2, the value of the element corresponding to the pseudo random pattern # 2 of the numerical data 93-3 is “1”, and the other pseudo random patterns Is set to “0”. The combination of the numerical data 93-3 and the input medical image 101 is input to the machine learning model. By adding numerical data obtained by digitizing the pseudo random pattern number to the input of the machine learning model, it is possible to generate output data in consideration of not only a medical image or raw data but also a pseudo random pattern number. Become. Therefore, the accuracy of the output data of the machine learning model is improved.

  The moving image capturing conditions to be digitized are not limited to those described above, and may be, for example, a frame number defined by a cardiac phase. The frame number defined by the cardiac phase is, for example, a number that defines the cardiac phase of the frame when the period from the R wave to the next R wave is 100%.

  The acquisition condition to be digitized may be a double speed rate of parallel imaging. In this case, the number of elements of the numerical data is defined as the number of double speed rate candidates. The value of the element has a value corresponding to whether or not the speed ratio corresponding to the element is employed in the target imaging. Artifacts caused by parallel imaging have characteristics according to the speed ratio. Therefore, by adding numerical data obtained by digitizing the speed ratio of the parallel imaging to the input of the machine learning model, the feature corresponding to the speed ratio can be detected by the machine learning model. As a result, the accuracy of the output data of the machine learning model can be improved. Is improved.

  The acquisition conditions to be digitized are not limited to the above, and may be, for example, data processing conditions for raw data or image processing conditions for medical images. The data processing condition or the image processing condition includes, for example, the number of repetitions of the repetition operation. The repetition operation is an operation such as a noise removal process, an edge enhancement process, and a smoothing process repeatedly applied to raw data or a medical image, and the number of repetitions is the number of times the operation is repeated. Alternatively, when the medical image is reconstructed from the raw data, the number of times the calculation is repeated to reduce the reconstruction error between the raw data and the reconstructed image and the number of times when the calculation is alternately performed. The calculation is performed using a filter, a DNN, or the like. The number of elements of the numerical data is defined as the number of candidates for the number of repetitions, and the element value has a value corresponding to the number of repetitions corresponding to the element, depending on whether or not the target operation is repeated. As described above, by adding the medical data after the repetition operation and the numerical data obtained by digitizing the number of repetitions of the repetition operation to the input of the machine learning model, not only the medical data but also the output data in consideration of the number of repetitions. Can be generated. Therefore, the accuracy of the output data of the machine learning model is improved. Note that the number of repetitions may be a one-hot vector. Alternatively, a vector may be provided in the form of a one-hot vector in which the first few times (for example, three times) are provided and the number of repetitions exceeding the number is given.

  The numerical data relating to the data processing conditions or the image processing conditions may be numerical data indicating whether or not a plurality of types of data processing or image processing are adopted. In this case, the number of elements of the numerical data is the number of candidates for data processing or image processing, and the value of each element is set to a value according to whether individual data processing or image processing is adopted. For example, the numerical data may be expressed as (noise reduction, edge enhancement, segment processing) or the like.

  The number of elements of the above numerical data was assumed to match the number of candidate conditions. However, the present embodiment is not limited to this.

  FIG. 14 is a diagram schematically showing other numerical data 93-4. As shown in FIG. 14, the numerical data 93-4 includes an element 94 corresponding to the candidate condition and an element 95 corresponding to not satisfying any of the candidate conditions. For example, the value of the element 95 is set to “0” when one of the candidate conditions is satisfied, and is set to “1” when the one of the candidate conditions is satisfied. For example, as shown in FIG. 14, when the numerical data 93-4 is obtained by digitizing the pulse sequence, and the pulse sequence of the actual MR imaging is not any of FSE, FE, and EPI, the value of the element 95 is “ 1 ". By providing the element 95 in the numerical data 93-4, it is possible to clearly indicate to the machine learning model that the pulse sequence of the actual MR imaging does not correspond to any of the candidate conditions.

  The above numerical data is based on the premise that one type of acquisition conditions such as the type of pulse sequence and the frame number are digitized. However, the present embodiment is not limited to this. For example, in the numerical data, the type of pulse sequence and the frame number may be represented as a vector in one column. In this case, the numerical data is specifically represented by (FSE, FE, EPI, frame number 1, frame number 2, ...).

  In the above description, the element of the numerical data is a one-hot vector that expresses in binary whether or not to adopt the candidate condition. However, the element may be a vector that expresses a multi-stage concept such as the strength, direction, and number of times of the candidate condition in three or more values.

  Next, a network structure of the machine learning model according to the present embodiment will be described.

  FIG. 15 is a diagram schematically illustrating a network structure of the machine learning model 521-4 according to the present embodiment. As illustrated in FIG. 15, as an example, the machine learning model 521-4 receives, as an example, a combination of a medical image # 1 having noise and numerical data # 2 obtained by digitizing the acquisition conditions regarding the medical image, and has a noise. It is assumed that the parameter is a DNN for which a parameter has been learned so as to output a medical image that is not to be output. It is assumed that medical image # 1 and numerical data # 2 are each one set.

  The machine learning model 521-4 has a CNN layer 523. The CNN layer 523 is configured to add each of the medical image # 1 and the numerical data # 2 by multiplying the weight by a different weight for each channel. For example, a plurality of filters of the number of channels ch is applied to the medical image # 1, and a plurality of filters of the same number ch are applied to the numerical data # 2. The filtered medical image and numerical data are added by the adder 203 for each channel and converted into added data. The added data is added with a constant value for each channel by the bias 204 and is converted into bias data. Bias data for each channel is output from the CNN layer 523.

  The network structure of the CNN layer 523 is an example, and various modifications are possible. For example, the CNN layer 523 may be provided with multi-stage filters 201 and 202 instead of the single-stage filters 201 and 202. For example, the size of the filter 201 may be set to 5 × 5, the size of the filter 202 may be set to 1 × 1, etc., but the sizes of the filters 201 and 202 may be any size.

  Subsequent to the CNN layer 523, a post-layer 525 is provided. The post-layer 525 performs an operation on the bias data for a plurality of channels from the CNN layer 523 and outputs a noise-free medical image as an output of the machine learning model 521-4. The post-layer 525 has at least one or more total coupling layers and output layers, but may have any one or more layers such as one or more CNN layers, pooling layers, and normalization layers in addition to these layers.

  It is assumed that the input of the CNN layer 523 shown in FIG. 15 is one set of medical images and one set of numerical data. However, the present embodiment is not limited to this. The input of the CNN layer may be a plurality of sets of medical images and the same set of numerical data.

  FIG. 16 is a diagram schematically illustrating a network structure of another machine learning model 521-5 according to the present embodiment. As illustrated in FIG. 16, the machine learning model 521-5 includes, for example, a medical image # 1 having noise, a medical image # 2 having noise, and numerical data # 3 obtained by digitizing acquisition conditions regarding the medical image # 1. It is assumed that the DNN is a parameter that has been learned by inputting a combination with numerical data # 4 obtained by digitizing the acquisition conditions regarding medical image # 2 and outputting a medical image having no noise. The medical image # 2 is a medical image derived from the medical image # 1, such as a copy of the medical image # 1 or a medical image obtained by performing arbitrary image processing on the medical image # 1. The medical image 2 may be another medical image acquired under the same acquisition conditions as the medical image # 1, or may be a medical image acquired under another acquisition condition.

  The machine learning model 521-5 has a CNN layer 527. The CNN layer 527 is configured to add each of the medical image # 1, the medical image # 2, the numerical data # 3, and the numerical data # 4 by multiplying a different weight for each channel. For example, a plurality of filters 211 of the same number of channels are applied to the medical image # 1, a plurality of filters 212 of the same number of channels are applied to the medical image # 2, and a plurality of filters 213 of the same number of channels are applied to the numerical data # 3. Is applied, and a plurality of filters 214 of the same number of channels are applied to the numerical data # 4. The filtered medical image and the numerical data are added by the adder 215 for each channel and converted into added data. A constant value is added to the addition data for each channel by the bias 216 and is converted into bias data. Bias data for each channel is output from the CNN layer 527.

  The network structure of the CNN layer 527 is an example, and various modifications are possible. For example, the CNN layer 527 may be provided with multi-stage filters 211, 212, 213, 214 instead of the single-stage filters 211, 212, 213, 214.

  Subsequent to the CNN layer 527, a post-layer 529 is provided. The post-layer 529 performs an operation on the bias data for a plurality of channels from the CNN layer 527 to output a noise-free medical image as an output of the machine learning model 521-5. The output medical image is a medical image corresponding to the medical image # 1, such as an image obtained by removing noise from the medical image # 1. The post-layer 529 has at least one or more total coupling layers and output layers, but in addition to these layers, may have any one or more layers such as a CNN layer, a pooling layer, and a normalization layer.

  In the case of a machine learning model having a plurality of CNN layers, numerical data does not necessarily have to be input to each CNN layer.

  FIG. 17 is a diagram schematically illustrating input and output of a machine learning model 521-6 having a plurality of CNN layers 531. The machine learning model 521-6 illustrated in FIG. 17 has, for example, a series of three CNN layers 531-1, a CNN layer 531-2, and a CNN layer 531-3. Subsequent to CNN 531-3, a back layer 533 is provided. The CNN 531-1 receives the input medical image, the CNN layer 531-2 receives the output data of the CNN layer 531-1, and the CNN layer 531-1-3 receives the output data of the CNN layer 531-2. The post-layer 533 receives the output data from the CNN layer 531-3 and outputs, for example, a medical image from which noise has been removed from the input medical image # 1 as an output of the machine learning model 521-6. The network structure of the CNN layer 531-1, the CNN layer 531-2, and the CNN layer 531-3 may be the same or different, and may be arbitrarily designed.

  As shown in FIG. 17, the numerical data # 2 may not be input to all of the three CNN layers 531-1, CNN layer 531-2, and CNN layer 531-3. For example, the CNN closest to the input side may be used. The numerical data # 2 may not be input to the layer 531-1, and the numerical data # 2 may be input to the subsequent CNN layers 531-2 and 531-3. The present invention is not limited to this, and the numerical data # 2 may be input to the CNN layer 531-1 closest to the input side, and the numerical data # 2 may not be input to the subsequent CNN layers 531-2 and 531-3. . Further, even if the numerical data # 2 is input to any one of the CNN layers 531-1, 531-2 and 531-3, and the numerical data # 2 is not input to the other CNN layers. Good. Of course, the numerical data # 2 may be input to all of the CNN layer 531-1, the CNN layer 531-2, and the CNN layer 531-3.

  In the above embodiment, the output of the machine learning model is a medical image having no noise. However, the output of the machine learning model is not limited to this.

  For example, the machine learning model may output a segmentation result of the medical image from the medical image and the numerical data. For example, with respect to the input of the T1-weighted image or the T2-weighted image, an image in which the resolving tissue or the lesion area is divided is output as the segmentation result. The lesion area is, for example, an image area in which cerebral infarction, ischemia, or cancer is suspected. The identification result of the medical image may be output from the medical image and the numerical data. For example, in response to an input of a T1-weighted image, a T2-weighted image, or a combination of a T1-weighted image and a T2-weighted image, a probability of cerebral infarction is output as the identification result. From the medical image and the numerical data, the segmentation result of the medical image may be output. For example, in response to an input of a T1-weighted image, a T2-weighted image, or a combination of a T1-weighted image and a T2-weighted image, an image in which an image region having a cerebral infarction-likeness is output as a segmentation result.

  Even when learning is performed based on medical images of different image types such as T1-weighted images and T2-weighted images, some of the parameters of the machine learning model are shared. Therefore, the machine learning model can acquire versatility for input of medical images of different image types while suppressing reduction in learning efficiency.

  The machine learning model may output super-resolution data of the medical data from the medical data and the numerical data. The super-resolution data is medical data having a higher spatial resolution than the input medical data. The machine learning model can be generated by learning DNN based on input data including medical data and numerical data, and super-resolution data as teacher data.

  For example, when k-space data is collected by the half-Fourier method with a limited frequency, the k-space data is partially missing and spatial resolution is reduced. By learning DNN based on input data including k-space data and numerical data collected by the half Fourier method and k-space data (teacher data) collected from MR imaging without frequency limitation, the machine is learned. A learning model can be generated. By using the machine learning model, it is possible to output k-space data (super-resolution data) in which a missing data portion is restored from k-space data collected by the half Fourier method.

  The machine learning model described above can be generated by the model learning device 6 using supervised machine learning. The learning sample is prepared by collecting medical data under various collection conditions. Specifically, the input data of the learning sample includes input medical data collected under a certain collection condition and numerical data obtained by digitizing the collection condition. The output data of the learning sample includes output medical data corresponding to the medical data and corresponding to the purpose of the machine learning model. For example, if the purpose of the machine learning model is de-noise, the output data is medical data in which noise is reduced as compared to the input medical data.

  In the learning function 612, the processing circuit 61 performs forward propagation processing by applying the input medical data and the numerical data to the machine learning model, and outputs output medical data. Next, the processing circuit 61 performs a back propagation process by applying a difference (error) between the estimated output medical data and the correct output medical data to the machine learning model, and calculates a gradient vector. Next, the processing circuit 61 updates parameters such as a weighted matrix and a bias of the machine learning model based on the gradient vector. By repeating the forward propagation process, the back propagation process, and the parameter update process while changing the learning sample, a learned machine learning model is generated.

  As described above, the medical information processing apparatus 50 includes the processing circuit 51. The processing circuit 51 obtains medical data on the subject, obtains numerical data obtained by digitizing the medical data collection conditions, applies the machine learning model 521 to input data including the numerical data and the medical data, and Generate output data based on medical data.

  According to the above configuration, the machine learning model 521 uses, in addition to the medical data, numerical data obtained by digitizing the acquisition conditions corresponding to the medical data for input, as compared with the case where only the medical data is input. The accuracy of output data can be improved. The number of elements of the numerical data corresponds to a predetermined number of candidate conditions, and the value of each element is a value according to whether or not the candidate condition is adopted. That is, since the rules for digitizing the collection conditions are predetermined, the machine learning model can determine the collection conditions with high accuracy at the time of learning or application. This is another factor in improving the accuracy of the output data.

  According to at least one embodiment described above, the accuracy of output data using machine learning can be improved.

  The term “processor” used in the above description refers to, for example, a CPU, a GPU, or an application specific integrated circuit (ASIC), a programmable logic device (eg, a simple programmable logic device). : SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA). The processor realizes each function 511 to 516 by reading and executing the program stored in the storage circuit. Instead of storing the program in the storage circuit, the program may be directly incorporated in the circuit of the processor. In this case, the processor implements the functions 511 to 516 by reading and executing a program incorporated in the circuit. Instead of executing the program, the functions 511 to 516 corresponding to the program may be realized by a combination of logic circuits. Note that each processor of the present embodiment is not limited to the case where each processor is configured as a single circuit, but may be configured as one processor by combining a plurality of independent circuits to realize its function. Good. Further, a plurality of components in FIGS. 1, 3 and 7 may be integrated into one processor to realize its function.

  While some embodiments of the invention have been described, these embodiments have been presented by way of example only, and are not intended to limit the scope of the inventions. These embodiments can be implemented in other various forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are also included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 1 PET apparatus 10 Gantry 11 Detector ring 13 Detection processor 15 Simultaneous counting circuit 17 Gamma ray detector 19 Processing circuit 21 Memory 23 Center of gravity calculation function 25 Energy calculation function 27 Center of gravity / crystal table 30 Bed 50 Medical information processing device (console)
51 processing circuit 52 display 53 memory 54 input interface 55 communication interface 511 reconstruction function 512 image processing function 513 imaging control function 514 flood map acquisition function 515 crystal section generation function 516 crystal section correction function 517 table generation function 518 model update function 519 display Control function 531 Machine learning model

Claims (23)

A first acquisition unit for acquiring medical data on the subject,
A second acquisition unit that acquires numerical data obtained by digitizing the medical data collection conditions;
A generator that applies a machine learning model to input data including the numerical data and the medical data, and generates output data based on the medical data,
A medical information processing apparatus comprising:   The medical information processing apparatus according to claim 1, wherein the machine learning model receives the numerical data and the medical data as inputs and is learned using teacher data. The numerical data includes two or more finite elements,
The number of elements corresponds to the number of a plurality of candidate conditions corresponding to the collection condition,
The value of the element corresponds to a numerical value according to whether or not the collection condition is adopted for the plurality of candidate conditions and / or a value.
The medical information processing apparatus according to claim 1.   4. The medical information processing apparatus according to claim 3, wherein the value of the element has one of a first value indicating that the collection condition adopts and a second value indicating that the collection condition does not adopt. 5. The numerical data includes two or more finite elements,
The elements for the number of elements include a first element corresponding to the plurality of candidate conditions and a second element corresponding to a condition that does not correspond to any of the plurality of candidates.
The medical information processing apparatus according to claim 3.   The medical information processing apparatus according to claim 1, wherein the acquisition condition includes a type of a pulse sequence, a frame number, a type of a k-space filling trajectory, a number of repetitions of a repetition operation, and / or a double speed rate of parallel imaging.   The medical information processing apparatus according to claim 1, wherein the generation unit generates, as the output data, data used for a medical diagnosis on the subject.   The medical information processing apparatus according to claim 1, wherein the machine learning model is configured to add each of the numerical data and the medical data by multiplying the weight by a different weight for each channel. The second obtaining unit obtains the numerical data obtained by digitizing the acquisition conditions of medical imaging for the subject,
The first acquisition unit acquires, as the medical data, raw data collected by the medical imaging according to the collection condition,
The generation unit applies the machine learning model to input data including the numerical data and the raw data or a medical image based on the raw data to generate the output data based on the raw data or the medical image. ,
The medical information processing apparatus according to claim 1.   The medical information according to claim 9, wherein the generation unit performs a reconstruction process on the input data including the numerical data and the raw data using a machine learning model to generate the image data regarding the subject. Processing equipment. The collection conditions are the number of collections and / or the collection direction for each data collection trajectory,
The numerical data is mask data in which numerical values corresponding to the number of times of collection and / or the direction of collection in target imaging are assigned to a plurality of data collection trajectory candidates having a finite number of elements.
The medical information processing apparatus according to claim 9.   The medical information processing apparatus according to claim 11, wherein the second acquisition unit determines the number of times of collection and / or the direction of collection for the plurality of data collection trajectory candidates in accordance with a user instruction or automatically.   The medical information processing apparatus according to claim 11, wherein the second acquisition unit determines the number of times of collection and / or the direction of collection of the plurality of data collection trajectory candidates according to a predetermined rule based on an imaging time in the medical imaging. .   The medical information processing apparatus according to claim 11, wherein the numerical value includes a first numerical value indicating that data collection is to be performed and a second numerical value indicating not performing data collection.   The medical information processing apparatus according to claim 11, wherein each of the plurality of data collection trajectory candidates is a data collection trajectory in a k-space related to a radial method, and passes through a substantially center of the k-space in a straight line.   The medical information processing apparatus according to claim 15, wherein an angle interval between the plurality of data collection trajectory candidates is set to a predetermined angle.   16. The medical information processing apparatus according to claim 15, wherein the plurality of data collection trajectory candidates are arranged at substantially equal intervals at an angle based on an angle obtained by dividing 360 degrees by the number of elements.   12. The medical information processing apparatus according to claim 11, wherein each of the plurality of data collection trajectory candidates is a data collection trajectory in k-space related to spiral collection, and spirally passes through k-space.   The generator applies the machine learning model to the mask data and the raw data, generates denoised raw data as the output data, and performs a Fourier transform on the denoised raw data to produce the medical image. The medical information processing apparatus according to claim 11, wherein:   The generation unit performs a Fourier transform on the raw data to generate a temporary medical image, applies the machine learning model to the mask data and the temporary medical image, and outputs a denoised medical image to the output data. The medical information processing apparatus according to claim 11, wherein the medical information processing apparatus outputs the information.   The medical information processing apparatus according to claim 1, further comprising: an imaging unit configured to perform magnetic resonance imaging on the subject and collect k-space data as the medical data. Obtaining medical data on the subject;
A step of obtaining numerical data obtained by digitizing the medical data collection conditions,
Applying a machine learning model to the input data including the numerical data and the medical data, generating output data based on the medical data,
A medical information processing method comprising: On the computer,
A function of acquiring medical data on the subject;
A function of acquiring numerical data obtained by digitizing the medical data collection conditions,
A function of applying a machine learning model to the input data including the numerical data and the medical data, and generating output data based on the medical data,
Medical information processing program that realizes