JP7262933B2 - Medical information processing system, medical information processing device, radiological diagnostic device, ultrasonic diagnostic device, learning data production method and program - Google Patents

Description

The embodiments of the present invention relate to a medical information processing system, a medical information processing apparatus, a radiological diagnostic apparatus, an ultrasonic diagnostic apparatus, a learning data production method, and a program.

An image processing technique using machine learning is known. In this technique, for example, a set of images of the same area with different amounts of noise is used as learning data to generate a trained model that is incorporated in the process of removing noise in the image. Here, it is conceivable to apply image processing technology using machine learning to medical images, but it is not easy to collect data for learning about medical images.

JP 2016-062524 A

The problem to be solved by the present invention is to eliminate the shortage of learning data related to medical images.

A medical information processing apparatus according to an embodiment includes an acquisition unit and an identification unit. The acquisition unit acquires a medical image. The specifying unit specifies, among the medical images acquired by the acquiring unit, learning data including a set of medical images having at least one of different imaging conditions and subject conditions and having approximately the same imaging direction. do.

FIG. 1 is a block diagram showing an example of the configuration of a medical information processing system according to the first embodiment. FIG. 2 is a block diagram showing an example configuration of the X-ray diagnostic apparatus according to the first embodiment. FIG. 3 is a diagram for explaining imaging directions according to the first embodiment. FIG. 4 is a diagram for explaining imaging directions according to the first embodiment. FIG. 5 is a diagram for explaining processing by a processing circuit according to the first embodiment; FIG. 6 is a diagram for explaining a process of generating a trained model according to the first embodiment. FIG. 7 is a diagram for explaining the process of generating a trained model according to the first embodiment. FIG. 8 is a diagram for explaining a learned model generation process according to the first embodiment. FIG. 9 is a diagram for explaining a learned model generation process according to the first embodiment. FIG. 10 is a diagram for explaining conversion processing according to the first embodiment. FIG. 11 is a flowchart for explaining a series of processing flows of the medical information processing apparatus according to the first embodiment. FIG. 12 is a block diagram showing an example configuration of an X-ray diagnostic apparatus according to the second embodiment. FIG. 13 is a block diagram showing an example configuration of an ultrasonic diagnostic apparatus according to the fourth embodiment.

Hereinafter, embodiments of a medical information processing system, a medical information processing apparatus, a radiological diagnostic apparatus, an ultrasonic diagnostic apparatus, a learning data production method, and a program will be described in detail with reference to the drawings.

(First embodiment)
First, the first embodiment will be described. In the first embodiment, a medical information processing system 1 including a medical information processing apparatus 30 will be described as an example.

As shown in FIG. 1, the medical information processing system 1 according to the first embodiment includes a medical image diagnostic apparatus 10, an image storage apparatus 20, and a medical information processing apparatus 30. FIG. Note that FIG. 1 is a block diagram showing an example of the configuration of the medical information processing system 1 according to the first embodiment. As shown in FIG. 1, a medical image diagnostic apparatus 10, an image storage apparatus 20, and a medical information processing apparatus 30 are interconnected via a network NW.

A medical image diagnostic apparatus 10 is an apparatus that acquires medical images from a subject. Medical images processed as data are also referred to as medical image data. For example, the medical image diagnostic apparatus 10 collects medical image data from a subject and transmits the collected medical image data to the image storage apparatus 20 and the medical information processing apparatus 30 . For example, the medical image diagnostic apparatus 10 includes an X-ray diagnostic apparatus, an X-ray CT (Computed Tomography) apparatus, a SPECT (Single Photon Emission Computed Tomography) apparatus, and a SPECT-CT apparatus in which a SPECT apparatus and an X-ray CT apparatus are integrated. , ultrasonic diagnostic equipment, etc.

The image storage device 20 is a device that stores medical image data collected by the medical image diagnostic device 10 . For example, the image storage device 20 acquires medical image data from the medical image diagnostic device 10 via the network NW, and stores the acquired medical image data in a memory provided inside or outside the device. For example, the image storage device 20 is realized by computer equipment such as a server device.

The medical information processing apparatus 30 acquires medical image data via the network NW and executes various processes using the acquired medical image data. For example, the medical information processing apparatus 30 acquires medical image data from the medical image diagnostic apparatus 10 or the image storage apparatus 20 via the network NW. In addition, the medical information processing apparatus 30 includes a set of medical image data in which at least one of the imaging conditions and the subject conditions is different from the acquired medical image data and the imaging directions are substantially the same. identify. For example, the medical information processing apparatus 30 is implemented by computer equipment such as a workstation.

The place where the medical image diagnostic apparatus 10, the image storage apparatus 20 and the medical information processing apparatus 30 are installed is arbitrary as long as they can be connected via the network NW. For example, the medical information processing apparatus 30 may be installed in a hospital different from the medical image diagnostic apparatus 10 . That is, the network NW may be configured by a closed local network within the hospital, or may be a network via the Internet. Although one medical image diagnostic apparatus 10 is shown in FIG. 1 , the medical information processing system 1 may include a plurality of medical image diagnostic apparatuses 10 .

As shown in FIG. 1, the medical information processing apparatus 30 has an input interface 31, a display 32, a memory 33, and a processing circuit .

The input interface 31 receives various input operations from the operator, converts the received input operations into electrical signals, and outputs the electrical signals to the processing circuit 34 . For example, the input interface 31 includes a mouse, a keyboard, a trackball, a switch, a button, a joystick, a touch pad that performs input operations by touching an operation surface, a touch screen that integrates a display screen and a touch pad, and an optical sensor. It is realized by the used non-contact input circuit, voice input circuit, or the like. The input interface 31 may be composed of a tablet terminal or the like capable of wireless communication with the main body of the medical information processing apparatus 30 . Also, the input interface 31 is not limited to those having physical operation parts such as a mouse and a keyboard. For example, an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the medical information processing apparatus 30 and outputs the electrical signal to the processing circuit 34 is also included in the input interface 31. included in the example.

The display 32 displays various information. For example, the display 32 displays medical images acquired by the processing circuit 34 from the medical image diagnostic apparatus 10 or the image storage device 20 under the control of the processing circuit 34 and medical images after conversion processing by the processing circuit 34 . The display 32 also displays a GUI (Graphical User Interface) for receiving various instructions and various settings from the operator via the input interface 31 . For example, the display 32 is a liquid crystal display or a CRT (Cathode Ray Tube) display. The display 32 may be of a desktop type, or may be configured by a tablet terminal or the like capable of wireless communication with the main body of the medical information processing apparatus 30 .

The memory 33 is implemented by, for example, a RAM (Random Access Memory), a semiconductor memory device such as a flash memory, a hard disk, an optical disk, or the like. For example, the memory 33 stores medical image data acquired from the medical image diagnostic apparatus 10 or the image storage apparatus 20 . In addition, for example, the memory 33 stores learning data sets and learned models generated by the processing circuit 34 . In addition, for example, the memory 33 stores programs for the circuits included in the medical information processing apparatus 30 to implement their functions. Note that the memory 33 may be realized by a server group (cloud) connected to the medical information processing apparatus 30 via the network NW.

The processing circuit 34 executes an acquisition function 34a, a specific function 34b, a learning data set generation function 34c, a model generation function 34d, a conversion function 34e, an output function 34f, and a control function 34g, so that the entire medical information processing apparatus 30 control behavior. Here, the acquisition function 34a is an example of an acquisition unit. Also, the specific function 34b is an example of a specific unit. Also, the learning data set generation function 34c is an example of a learning data set generation unit. Also, the model generation function 34d is an example of a model generation unit. Also, the conversion function 34e is an example of a conversion unit. Also, the control function 34g is an example of a control unit.

For example, the processing circuit 34 acquires medical image data from the medical image diagnostic apparatus 10 or the image storage apparatus 20 by reading a program corresponding to the acquisition function 34a from the memory 33 and executing it. Further, for example, the processing circuit 34 reads out from the memory 33 a program corresponding to the specific function 34b and executes it, so that among the medical image data acquired by the acquisition function 34a, among the imaging conditions and the subject conditions, Learning data including a set of medical image data in which at least one is different and the imaging directions of which are substantially the same is specified.

Further, for example, the processing circuit 34 reads a program corresponding to the learning data set generation function 34c from the memory 33 and executes it, thereby generating a learning data set including a plurality of learning data specified by the specifying function 34b. Generate. Further, for example, the processing circuit 34 reads the program corresponding to the model generation function 34d from the memory 33 and executes it, thereby converting the input medical image data into imaging conditions and the subject based on the learning data set. generates a trained model to be incorporated in the process of converting to medical image data corresponding to medical image data different in at least one of the conditions of .

Further, for example, the processing circuit 34 reads the program corresponding to the conversion function 34e from the memory 33 and executes it, thereby converting the input medical image data into imaging conditions and subject conditions based on the learned model. At least one of the data is converted into medical image data corresponding to different medical image data. Further, for example, the processing circuit 34 reads a program corresponding to the output function 34f from the memory 33 and executes it, thereby displaying the medical image acquired by the acquisition function 34a or the medical image after conversion processing by the conversion function 34e on the display 32. It displays and outputs various data (learning data, learning data set, trained model, medical image data after conversion processing, etc.) to an external device. Further, for example, the processing circuit 34 reads out a program corresponding to the control function 34g from the memory 33 and executes it, thereby controlling various functions of the processing circuit 34 based on an input operation received from the operator via the input interface 31. to control.

In the medical information processing apparatus 30 shown in FIG. 1, each processing function is stored in the memory 33 in the form of a computer-executable program. The processing circuit 34 is a processor that implements a function corresponding to each program by reading the program from the memory 33 and executing the program. In other words, the processing circuit 34 in a state where each program is read has a function corresponding to the read program. In FIG. 1, the single processing circuit 34 realizes an acquisition function 34a, a specific function 34b, a learning data set generation function 34c, a model generation function 34d, a conversion function 34e, an output function 34f, and a control function 34g. However, the processing circuit 34 may be configured by combining a plurality of independent processors, and each processor may implement each processing function by executing each program. Further, each processing function of the processing circuit 34 may be appropriately distributed or integrated in a single or a plurality of processing circuits and implemented.

Next, the medical image diagnostic apparatus 10 that collects medical image data will be described. In this embodiment, an X-ray diagnostic apparatus 100 shown in FIG. 2 will be described as an example of the medical image diagnostic apparatus 10. As shown in FIG. FIG. 2 is a block diagram showing an example configuration of the X-ray diagnostic apparatus 100 according to the first embodiment. Also, in this embodiment, two-dimensional X-ray image data will be described as an example of medical image data.

As shown in FIG. 2, the X-ray diagnostic apparatus 100 includes an X-ray high voltage device 101, an X-ray tube 102, a collimator 103, a filter 104, a top plate 105, a C-arm 106, and an X-ray detector. 107 , a memory 108 , a display 109 , an input interface 110 and a processing circuit 111 .

The X-ray high voltage device 101 supplies high voltage to the X-ray tube 102 under the control of the processing circuitry 111 . For example, the X-ray high voltage device 101 has electric circuits such as a transformer and a rectifier, and includes a high voltage generator that generates a high voltage to be applied to the X-ray tube 102 and a and an X-ray controller for controlling an output voltage according to X-rays. The high voltage generator may be of a transformer type or an inverter type.

The X-ray tube 102 is a vacuum tube having a cathode (filament) that generates thermoelectrons and an anode (target) that generates X-rays upon collision with thermoelectrons. The X-ray tube 102 uses a high voltage supplied from the X-ray high-voltage device 101 to irradiate thermoelectrons from the cathode to the anode, thereby generating X-rays.

A collimator (also called an X-ray diaphragm device) 103 has, for example, four slidable diaphragm blades. The collimator 103 narrows down the X-rays generated by the X-ray tube 102 by sliding the diaphragm blades and irradiates the subject P1 with the X-rays. Here, the diaphragm blade is a plate-shaped member made of lead or the like, and is provided near the X-ray irradiation port of the X-ray tube 102 to adjust the irradiation range of X-rays. For example, the collimator 103 has a drive mechanism such as a motor and an actuator, and controls the irradiation range of X-rays under the control of the processing circuit 111, which will be described later. For example, the collimator 103 applies a drive voltage to the drive mechanism in accordance with the control signal received from the processing circuit 111, thereby adjusting the opening of the diaphragm blades and increasing the amount of X-rays irradiated to the subject P1. Control the irradiation range.

The filter 104 changes the quality of X-rays transmitted by its material and thickness for the purpose of reducing the exposure dose to the subject P1 and improving the image quality of the X-ray image data. or to reduce high-energy components that cause a reduction in the contrast of X-ray image data. In addition, the filter 104 changes the X-ray dose and irradiation range depending on its material, thickness, position, etc., so that the X-ray irradiated from the X-ray tube 102 to the subject P1 has a predetermined distribution. Attenuate the line. For example, the filter 104 has a driving mechanism such as a motor and an actuator, and is moved by operating the driving mechanism under the control of the processing circuit 111, which will be described later. For example, the filter 104 adjusts its position by applying a drive voltage to the drive mechanism in accordance with the control signal received from the processing circuit 111, and adjusts the dose distribution of the X-rays irradiated to the subject P1. Control.

The top board 105 is a bed on which the subject P1 is placed, and is arranged on a bed (not shown). Note that the subject P1 is not included in the X-ray diagnostic apparatus 100. FIG. For example, the bed has a drive mechanism such as a motor and an actuator, and controls the movement and inclination of the tabletop 105 by operating the drive mechanism under the control of the processing circuit 111, which will be described later. For example, the bed moves or inclines the table top 105 by applying a drive voltage to the drive mechanism according to the control signal received from the processing circuit 111 .

The C-arm 106 holds the X-ray tube 102, the collimator 103, the filter 104, and the X-ray detector 107 so as to face each other with the subject P1 interposed therebetween. For example, the C-arm 106 has a driving mechanism such as a motor and an actuator, and rotates or moves by operating the driving mechanism under the control of a processing circuit 111, which will be described later. For example, the C-arm 106 applies a drive voltage to the drive mechanism according to the control signal received from the processing circuit 111, thereby moving the X-ray tube 102, collimator 103, filter 104, and X-ray detector 107 to the subject. It is rotated and moved with respect to P1 to control the X-ray irradiation position and irradiation angle. In FIG. 2, the case where the X-ray diagnostic apparatus 100 is a single plane is described as an example, but the embodiment is not limited to this, and may be a biplane. Also, in FIG. 2, the case where the X-ray diagnostic apparatus 100 is an X-ray diagnostic apparatus for circulatory system diagnosis is described as an example, but the embodiment is not limited to this. For example, the X-ray diagnostic apparatus 100 may be various X-ray diagnostic apparatuses such as a general X-ray diagnostic apparatus, a gastrointestinal X-ray TV apparatus, and a mammography apparatus. The C-arm 106 is an example of a holding section, holding mechanism, and holding device that hold at least one of the X-ray tube and the X-ray detector.

The X-ray detector 107 is, for example, an X-ray flat panel detector (FPD) having detection elements arranged in a matrix. The X-ray detector 107 detects X-rays emitted from the X-ray tube 102 and transmitted through the subject P1 and outputs a detection signal corresponding to the detected X-ray dose to the processing circuit 111 . The X-ray detector 107 may be an indirect conversion type detector having a grid, a scintillator array, and a photosensor array, or a direct conversion type detector having a semiconductor element that converts incident X-rays into electrical signals. It may be a detector. Also, the X-ray detector 107 may be configured to be rotatable about an axis perpendicular to the detection surface as a rotation axis. That is, the X-ray detector 107 may be configured to be rotatable within the detection plane. Hereinafter, the rotation angle of the X-ray detector 107 within the detection plane is also referred to as an in-plane rotation angle. Note that the X-ray detector 107 is an example of a detector.

The memory 108 is implemented by, for example, a RAM, a semiconductor memory device such as a flash memory, a hard disk, an optical disk, or the like. For example, memory 108 receives and stores x-ray image data acquired by processing circuitry 111 . The memory 108 also stores programs corresponding to various functions read and executed by the processing circuit 111 . Note that the memory 108 may be realized by a server group (cloud) connected to the X-ray diagnostic apparatus 100 via the network NW.

The display 109 displays various information. For example, the display 109 displays a GUI for accepting operator's instructions and various X-ray images under the control of the processing circuit 111 . For example, display 109 is a liquid crystal display or a CRT display. The display 109 may be of a desktop type, or may be configured by a tablet terminal or the like capable of wireless communication with the processing circuit 111 .

The input interface 110 receives various input operations from an operator, converts the received input operations into electrical signals, and outputs the electrical signals to the processing circuit 111 . For example, the input interface 110 includes a mouse, a keyboard, a trackball, a switch, a button, a joystick, a touch pad that performs input operations by touching an operation surface, a touch screen that integrates a display screen and a touch pad, and an optical sensor. It is realized by the used non-contact input circuit, voice input circuit, or the like. Note that the input interface 110 may be configured by a tablet terminal or the like capable of wireless communication with the processing circuit 111 . Also, the input interface 110 is not limited to having physical operation parts such as a mouse and a keyboard. For example, an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the X-ray diagnostic apparatus 100 and outputs this electrical signal to the processing circuit 111 is also included in the input interface 110. included in the example.

The processing circuit 111 controls the overall operation of the X-ray diagnostic apparatus 100 by executing the acquisition function 111a, the output function 111b, and the control function 111c.

For example, the processing circuit 111 acquires X-ray image data by reading out a program corresponding to the acquisition function 111a from the memory 108 and executing the program. For example, the acquisition function 111a controls the X-ray high-voltage device 101 and adjusts the voltage supplied to the X-ray tube 102, thereby controlling the X-ray dose irradiated to the subject P1 and ON/OFF.

Also, the acquisition function 111a controls an imaging target area and an imaging direction. Here, in the present embodiment, the imaging target area refers to a portion of the subject P1 that is visualized by an X-ray image, and the imaging direction refers to the direction of X-ray irradiation to the subject P1. For example, the acquisition function 111a controls the irradiation range of X-rays with which the subject P1 is irradiated by controlling the collimator 103 and adjusting the aperture blade opening. The acquisition function 111a also controls the X-ray dose distribution by controlling the filter 104 and adjusting the position of the filter 104 . The collection function 111a also rotates and moves the C-arm 106 by controlling the operation of the C-arm 106 . Also, for example, the collection function 111a moves or tilts the tabletop 105 by controlling the motion of the bed.

Hereinafter, a mechanism used for imaging in the medical image diagnostic apparatus 10 is also referred to as an imaging unit. For example, the imaging unit of the X-ray diagnostic apparatus 100 includes an X-ray tube 102, a collimator 103, a filter 104, a top plate 105, a C-arm 106 and an X-ray detector 107. The acquisition function 111a controls the imaging target area and the imaging direction by controlling part or all of the imaging unit.

The acquisition function 111 a also generates X-ray image data based on the detection signal received from the X-ray detector 107 and stores the generated X-ray image data in the memory 108 . At this time, the acquisition function 111a records the imaging target area, the imaging direction, the patient ID, the examination type, the examination date and time, the in-plane rotation angle of the X-ray detector 107, and the like in the incidental information of the generated X-ray image data. may be

For example, the acquisition function 111a identifies the imaging direction according to the orientation of the X-ray tube 102 and records it in the supplementary information of the X-ray image data. For example, the acquisition function 111a first defines a direction with the subject P as a reference. For example, the acquisition function 111a defines the body axis direction of the subject P1 (the direction perpendicular to the cross section of the subject P1) as the T direction. Further, the acquisition function 111a defines the lateral direction of the subject P1 (the direction perpendicular to the sagittal plane of the subject P1) as the S direction. Also, the acquisition function 111a defines the front-rear direction of the subject P1 (the direction perpendicular to the coronal plane of the subject P1) as the C direction. In the following description, the T-direction, S-direction and C-direction are assumed to be orthogonal to each other.

For example, the subject P1 is placed face up along the longitudinal direction of the tabletop 105 . In this case, the collection function 111a uses the tilt information of the tabletop 105 to define the longitudinal direction of the tabletop 105 as the T direction and the lateral direction of the tabletop 105 as the S direction, as shown in the left diagram of FIG. A direction perpendicular to the top plate 105 is defined as a C direction. Also, the collection function 111a identifies the imaging direction as shown in the left diagram of FIG. Specifically, the acquisition function 111a specifies the orientation of the X-ray tube 102 as the imaging direction with reference to the T direction, S direction, and C direction. Note that FIG. 3 is a diagram for explaining the imaging direction according to the first embodiment.

Here, as shown in the left diagram of FIG. 3, the orientation of the X-ray tube 102 with respect to the subject P is changed by rotating the C-arm 106 by the acquisition function 111a. That is, the rotation of the C-arm 106 changes the imaging direction. In this case, the acquisition function 111a specifies the orientation of the X-ray tube 102 after the rotation operation as the imaging direction with reference to the T, S, and C directions, as shown in the right diagram of FIG.

Also, the collection function 111a may specify the imaging direction in consideration of the tilt of the tabletop 105 . That is, as shown in the left diagram of FIG. 4, the orientation of the subject P with respect to the X-ray tube 102 changes as the acquisition function 111a tilts the top plate 105. FIG. In this case, the collection function 111a redefines the T, S, and C directions using the tilt information of the tabletop 105 after tilting. Then, as shown in the right diagram of FIG. 4, the acquisition function 111a specifies the orientation of the X-ray tube 102 as the imaging direction with reference to the redefined T, S, and C directions. Note that FIG. 4 is a diagram for explaining the imaging direction according to the first embodiment.

Note that the acquisition function 111a may specify the clinical angle as the imaging direction, or may specify a value indicating the imaging direction. For example, for X-ray image data acquired in heart examination, the acquisition function 111a identifies clinical angles such as CRA, CAU, LAO, and RAO as imaging directions, and records them in supplementary information of the X-ray image data. Also, for example, the acquisition function 111a defines a TSC coordinate system consisting of the T, S, and C directions, and specifies a unit vector in this coordinate system as the imaging direction. For example, when the imaging direction coincides with the C direction, the acquisition function 111a specifies (0, 0, 1) as the imaging direction in the TSC coordinate system and records it in the incidental information of the X-ray image data.

In addition, the acquisition function 111a specifies an imaging target area according to the position of the C-arm 106, the opening degree of the diaphragm blades in the collimator 103, and the like, and records the area as supplementary information of the X-ray image data. For example, the acquisition function 111a first defines a direction and a reference position with the subject P as a reference. For example, the acquisition function 111a defines a TSC coordinate system consisting of T, S, and C directions, and uses the T, S, and C coordinates to identify the imaging target area.

For example, the acquisition function 111a first identifies the isocenter (the center of rotation of the C-arm 106) based on the position of the C-arm 106. FIG. Acquisition function 111a then identifies a straight line that passes through the isocenter and is parallel to the imaging direction. Then, the acquisition function 111a identifies the imaging target area as a columnar area or conical area having the identified straight line as an axis based on the opening degree of the aperture blades of the collimator 103 . For example, when the collimator 103 has four aperture blades, the acquisition function 111a specifies a quadrangular pyramid-shaped region having the X-ray focal point as the vertex and the specified straight line as the imaging target region.

In the following description, the +T direction is the upper side (head side) of the subject P, and the −T direction is the lower side (foot side) of the subject P. For example, when the C-arm 106 translates in the +T direction, the imaging target region translates in the +T direction. Also, when controlling the operation of the diaphragm blades in the collimator 103, the imaging target area is enlarged or reduced according to the opening degree of the diaphragm blades. For example, the acquisition function 111a can specify the imaging target area based on the position of the C-arm 106 and the opening degree of the diaphragm blades in the collimator 103 .

In addition, the collection function 111a may specify the imaging target area in consideration of parallel movement (sliding) of the tabletop 105 . That is, when the collection function 111a slides the top plate 105, the relative positional relationship between the top plate 105 and the C-arm 106 changes, and the imaging target area also changes. For example, when the tabletop 105 translates in the +T direction, the imaging target area translates in the -T direction. In this case, the acquisition function 111a identifies the imaging target area based on the relative positional relationship between the top plate 105 and the C-arm 106 and the opening degree of the diaphragm blades in the collimator 103, for example.

Note that the collection function 111a may specify an imaging target region as the imaging target region, or may specify a value indicating the imaging target region. For example, the acquisition function 111a specifies imaging target regions such as "heart", "right lung", and "head" as imaging target regions, and records them in the supplementary information of the X-ray image data. Also, for example, the acquisition function 111a defines a TSC coordinate system consisting of the T, S, and C directions, specifies a three-dimensional area in this coordinate system as an imaging target area, and uses the incidental information of the X-ray image data. Record.

The acquisition function 111a may also perform various image processing on the X-ray image data stored in the memory 108. FIG. For example, the acquisition function 111a performs noise reduction processing using an image processing filter and scattered radiation correction on X-ray image data.

In addition, the processing circuit 111 reads a program corresponding to the output function 111b from the memory 108 and executes it to display a GUI and an X-ray image on the display 109 . The output function 111b also outputs the X-ray image data to the image storage device 20 and the medical information processing device 30. FIG. Further, the processing circuit 111 reads out a program corresponding to the control function 111c from the memory 108 and executes it, thereby controlling various functions of the processing circuit 111 based on an input operation received from the operator via the input interface 110. do.

In the X-ray diagnostic apparatus 100 shown in FIG. 2, each processing function is stored in the memory 108 in the form of a computer-executable program. The processing circuit 111 is a processor that implements a function corresponding to each program by reading and executing the program from the memory 108 . In other words, the processing circuit 111 having read each program has a function corresponding to the read program. Although FIG. 2 shows a case where the processing functions of the collection function 111a, the output function 111b, and the control function 111c are realized by the single processing circuit 111, the embodiment is not limited to this. For example, the processing circuit 111 may be configured by combining a plurality of independent processors, and each processor may implement each processing function by executing each program. Further, each processing function of the processing circuit 111 may be appropriately distributed or integrated in a single or a plurality of processing circuits and implemented.

The term "processor" used in the above description includes, for example, a CPU, a GPU (Graphics Processing Unit), an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device ( Circuits such as Simple Programmable Logic Device (SPLD), Complex Programmable Logic Device (CPLD), and Field Programmable Gate Array (FPGA)). The processor implements its functions by reading and executing programs stored in the memory 33 or memory 108 .

1 and 2, it is assumed that the single memory 33 or memory 108 stores programs corresponding to each processing function. However, a configuration may be adopted in which a plurality of memories 33 are distributed and the processing circuit 34 reads corresponding programs from the individual memories 33 . Similarly, a plurality of memories 108 may be distributed and the processing circuit 111 may read corresponding programs from individual memories 108 . Also, instead of storing the program in the memory 33 and memory 108, the program may be configured to be directly embedded in the circuitry of the processor. In this case, the processor implements its functions by reading and executing the program embedded in the circuit.

Also, the processing circuit 34 and the processing circuit 111 may realize their functions using a processor of an external device connected via the network NW. For example, the processing circuit 34 reads and executes a program corresponding to each function from the memory 33, and uses a server group (cloud) connected to the medical information processing apparatus 30 via the network NW as computational resources. , implement the functions shown in FIG. Further, for example, the processing circuit 34 reads out and executes programs corresponding to each function from the memory 33, and uses the processing circuit 111 as a computational resource to realize each function shown in FIG.

An example of the configuration of the medical information processing system 1 has been described above. With such a configuration, the medical information processing apparatus 30 in the medical information processing system 1 solves the shortage of learning data related to medical images. Specifically, the medical information processing apparatus 30 acquires X-ray image data through processing by the processing circuit 34, which will be described in detail below, and among the acquired X-ray image data, By specifying learning data that includes a set of X-ray image data in which at least one is different and the imaging directions of which are substantially the same, the lack of learning data related to medical images is resolved. Processing performed by the medical information processing apparatus 30 according to the first embodiment will be described in detail below.

First, the acquisition function 34 a acquires X-ray image data from the X-ray diagnostic apparatus 100 or the image storage apparatus 20 . For example, the acquisition function 34a acquires a plurality of X-ray image data including X-ray image data I1 and X-ray image data I2 from the X-ray diagnostic apparatus 100, as shown in FIG. FIG. 5 is a diagram for explaining processing by the processing circuit 34 according to the first embodiment.

Here, the acquisition function 34a may acquire X-ray image data stored in the memory 108, or may acquire X-ray image data acquired by the acquisition function 111a without using the memory 108. . Alternatively, the acquisition function 34 a may acquire X-ray image data stored in the image storage device 20 . The acquisition function 34a may also cause the memory 33 to store the acquired X-ray image data.

Next, the specifying function 34b selects X-ray image data in which at least one of the imaging condition and the subject condition is different from the X-ray image data acquired by the acquiring function 34a and the imaging directions are substantially the same. Identify training data to include as a set.

For example, the specifying function 34b first specifies a set of X-ray image data acquired in the same examination among the X-ray image data acquired by the acquiring function 34a. To give an example, the specifying function 34b specifies a set of X-ray image data acquired on the same day and having the same imaging subject and examination type.

For example, the specifying function 34b acquires information on the subject that is the imaging target of each X-ray image data based on the incidental information of each X-ray image data, Identify a set of line image data. For example, if the X-ray image data is in DICOM (Digital Imaging and Communications in Medicine) format, the specific function 34b performs imaging based on the patient ID recorded in the tag "(0010,0020) Patient ID". A set of X-ray image data in which the object of interest is the same is identified.

Next, the specifying function 34b specifies a set of X-ray image data of the same examination type among the sets of X-ray image data of the same subject that is the imaging target. For example, the specifying function 34b acquires an examination type such as whether the examination for each X-ray image data is a normal examination or an emergency examination based on the incidental information of each X-ray image data. Then, the identifying function 34b identifies a set of X-ray image data having the same examination type by comparing the examination type of each X-ray image data.

Furthermore, the specifying function 34b specifies a set of X-ray image data acquired on the same day, among sets of X-ray image data having the same subject to be imaged and examination type. For example, the specifying function 34b acquires the examination date and time when the examination for each X-ray image data was performed based on the incidental information of each X-ray image data. Then, the specifying function 34b specifies a set of X-ray image data acquired on the same day by comparing the examination date and time of each X-ray image data. That is, the specifying function 34b selects a set of X-ray image data acquired on the same day, among sets of X-ray image data having the same subject to be imaged and the same examination type, as a set of X-ray image data acquired in the same examination. It is specified as a set of X-ray image data.

Next, the specifying function 34b specifies a set of X-ray image data whose imaging directions are substantially the same among the sets of X-ray image data acquired in the same examination. For example, the identifying function 34b identifies, among sets of X-ray image data acquired in the same examination, sets of X-ray image data in which at least a part of the imaging target region overlaps and whose imaging directions are substantially the same. do. Here, the specifying function 34b can acquire the imaging target region and the imaging direction of each X-ray image data based on the incidental information of each X-ray image data.

For example, the specifying function 34b first determines whether or not the imaging directions of sets of X-ray image data acquired in the same examination substantially match. For example, the acquisition function 111a specifies clinical angles such as CRA, CAU, LAO, and RAO for each X-ray image data acquired in a heart examination as an imaging direction, and to record. In this case, the specifying function 34b compares the clinical angles acquired from the supplementary information of each X-ray image data, and determines that the imaging directions substantially match when the clinical angles match. To give another example, the acquisition function 111a identifies a unit vector indicating the imaging direction as the imaging direction of each piece of X-ray image data. In this case, the specifying function 34b calculates the inner product of the unit vectors acquired from the incidental information of each X-ray image data, and determines that the imaging directions substantially match when the inner product is equal to or greater than the threshold.

As described above, the acquisition function 111a identifies the imaging direction based on the arrangement of the imaging unit such as the orientation of the X-ray tube 102 and the tilt information of the tabletop 105, and records the imaging direction in the incidental information of the X-ray image data. do. Then, the specifying function 34b specifies a set of X-ray image data whose imaging directions substantially match based on the imaging directions acquired from the incidental information of the X-ray image data. That is, the specifying function 34b specifies a set of X-ray image data whose imaging directions are substantially the same based on the arrangement of the imaging units.

Also, for example, the specifying function 34b determines whether or not at least a part of the imaging target region overlaps between sets of X-ray image data determined to have approximately the same imaging direction. For example, for each piece of X-ray image data, the acquisition function 111a records the three-dimensional area specified in the TSC coordinate system in the incidental information of the X-ray image data as an imaging target area. In this case, the specifying function 34b mutually compares the three-dimensional regions acquired from the incidental information of each X-ray image data, and if at least a part of the three-dimensional regions overlaps, at least a part of the imaging target region is determined to be duplicates. To give another example, the acquisition function 111a records the position information of the isocenter in the supplementary information of the X-ray image data for each X-ray image data. In this case, the specifying function 34b determines that at least a part of the imaging target regions overlap when the distance between the isocenter positions obtained from the incidental information of each X-ray image data is equal to or less than the threshold.

As described above, the acquisition function 111a acquires an imaging target region based on the positional relationship between the top plate 105 and the C-arm 106, the opening degree of the aperture blades in the collimator 103, and the like, and acquires the imaging target region based on the arrangement of the imaging unit. Record in the accompanying information of the data. Alternatively, the acquisition function 111a acquires the positional information of the isocenter based on the arrangement of the imaging unit such as the positional relationship between the top plate 105 and the C-arm 106, and records it in the incidental information of the X-ray image data. Then, the specifying function 34b specifies a set of X-ray image data in which at least a part of the imaging target region overlaps based on the imaging target region or the isocenter position information acquired from the incidental information of the X-ray image data. do That is, the specifying function 34b specifies a set of X-ray image data in which at least a part of the imaging target region overlaps based on the arrangement of the imaging units.

Here, the specifying function 34b performs alignment between X-ray image data for a set of X-ray image data in which a part of the imaging target region overlaps and the entire imaging target region does not overlap. good too. For example, the specifying function 34b extracts anatomical feature points (landmarks) from each piece of X-ray image data in a set of X-ray image data in which part of the imaging target region overlaps. Next, the specifying function 34b carries out registration such that the anatomical feature points of the X-ray image data are overlapped by processing such as translation and enlargement/reduction. Then, the identifying function 34b identifies a set of aligned X-ray image data as learning data. Note that the specifying function 34b specifies a set of X-ray image data in which the entire imaging target region overlaps as learning data without performing alignment.

Further, the specifying function 34b may perform angle correction between X-ray image data for a set of X-ray image data whose imaging directions are substantially the same and which are not the same. For example, the specific function 34b performs trapezoidal correction based on the difference in imaging direction between X-ray image data sets in a set of X-ray image data whose imaging directions are substantially the same but are not the same. Angle correction such as deformation processing is performed based on the anatomical feature points extracted from . Then, the specifying function 34b specifies a set of angle-corrected X-ray image data as learning data. Note that the identifying function 34b identifies sets of X-ray image data having the same imaging direction as learning data without executing angle correction.

Further, the specifying function 34b may correct the rotation of the X-ray image data based on the in-plane rotation angle of the X-ray detector 107 when capturing the X-ray image data. For example, the acquisition function 111a defines the reference state of the X-ray detector 107 (state in which the in-plane rotation angle is 0°), and obtains the amount of rotation of the X-ray detector 107 from the reference state as the in-plane rotation angle. and recorded in the incidental information of the X-ray image data. For example, if the X-ray image data is in DICOM format, the acquisition function 111a records the in-plane rotation angle in a DICOM private tag.

For example, the specifying function 34b performs rotation correction on at least one piece of X-ray image data in a set of X-ray image data in which at least a part of the imaging target region overlaps and the imaging directions are substantially the same. For example, the specific function 34b executes rotation correction of each X-ray image data so as to offset the in-plane rotation angle (rotation amount of the X-ray detector 107 from the reference state).
Thereby, the specific function 34b can align the orientation of each X-ray image data. Then, the specifying function 34b specifies a set of X-ray image data with aligned orientations as learning data.

Alternatively, the specification function 34b first performs rotational correction on at least one piece of X-ray image data in a set of X-ray image data acquired within the same examination. Next, the specifying function 34b specifies, from the sets of X-ray image data after rotation correction, sets of X-ray image data in which at least a part of the imaging target region overlaps and in which the imaging directions are substantially the same. As a result, the identifying function 34b identifies a set of X-ray image data with aligned orientations as learning data.

Next, the specifying function 34b selects a set of X-ray image data in which at least a part of the imaging target region overlaps and the imaging directions are substantially the same, and at least one of the imaging condition and the subject condition is different. Identify training data consisting of a set of X-ray image data.

Here, the imaging conditions for the X-ray image data are, for example, conditions related to irradiation of X-rays, conditions related to detection of X-rays, and the like. Incidentally, examples of conditions related to X-ray irradiation include a tube current value, a tube voltage value, an irradiation time, and the like. Examples of conditions related to X-ray detection include the number of detection elements included in the X-ray detector 107, the number of binnings of the X-ray detector 107, and the type of the X-ray detector 107 (indirect conversion type/direct conversion type). etc.

In addition, the condition of the subject for the X-ray image data is, for example, the contrast condition. Examples of contrast conditions include the type and amount of the contrast medium injected into the subject's blood vessels, digestive organs, and the like, and the elapsed time after injection of the contrast medium. For example, the types of contrast agents injected into the blood vessels of the subject include positive contrast agents (contrast agents mainly composed of iodine and barium sulfate, etc.) that have larger X-ray attenuation coefficients than the surrounding tissue of the subject, and There are negative contrast agents (gas contrast agents mainly composed of carbon dioxide, oxygen, nitrogen, air, etc.) that have a smaller X-ray attenuation coefficient than the surrounding tissue.

As an example, a case where the tube current value related to the X-ray image data I1 shown in FIG. 5 is smaller than the tube current value related to the X-ray image data I2 will be described below. That is, the case where the imaging conditions of the X-ray image data I1 and the X-ray image data I2 are different will be described below. Further, as shown in FIG. 5, the X-ray image data I1 and the X-ray image data I2 are a set of X-ray image data having substantially overlapping imaging target regions and substantially matching imaging directions. Therefore, the specifying function 34b specifies a set of X-ray image data I1 and X-ray image data I2 with different imaging conditions as learning data.

Here, the X-ray image data I1 imaged with a smaller tube current value (low dose) than the X-ray image data I2 contains more noise than the X-ray image data I2. That is, the set of the X-ray image data I1 and the X-ray image data I2 is an example of a set of medical images having different degrees of noise.

Note that the specifying function 34b may determine whether or not to specify a set of X-ray image data as learning data. For example, when performing angle correction between X-ray image data for a set of X-ray image data whose imaging directions are substantially the same but whose imaging directions are not the same, the specifying function 34b performs angle correction of the angle-corrected X-ray image data. It may be determined whether or not to specify a set as learning data. For example, the specification function 34b first performs an image-to-image comparison for sets of angle-corrected X-ray image data.

For example, the specifying function 34b extracts feature points from each set of angle-corrected X-ray image data, and determines whether or not the positions of the feature points match between the images. As an example, the identification function 34b acquires virtual patient image data from the memory 33 according to various parameters such as age, adult/child, male/female, weight, height, etc. of the subject P1. Here, the virtual patient image data is data in which the positions of a plurality of anatomical feature points are defined together with identification codes for the respective anatomical features. For example, the specifying function 34b extracts a plurality of feature points from the X-ray image data by image processing such as pattern recognition, and specifies an identification code for each of the extracted feature points based on the virtual patient image data. Then, the specifying function 34b determines whether or not the positions of the feature points extracted from each X-ray image data match between the images for each identification code. Here, when the positions of the feature points match, the specifying function 34b specifies a set of angle-corrected X-ray image data as learning data. On the other hand, if the positions of the feature points do not match, the identifying function 34b determines not to identify the set of angle-corrected X-ray image data as learning data.

To give another example, the specifying function 34b extracts a characteristic region from each piece of X-ray image data for a set of angle-corrected X-ray image data, and determines whether or not the characteristic regions match between the images. For example, the specifying function 34b extracts a bone, an organ, or the like as a characteristic region from each X-ray image data, and extracts a contour line of the extracted region. Here, if the contour lines match between the images, the specifying function 34b specifies a set of angle-corrected X-ray image data as learning data. On the other hand, if the contour lines do not match, the specifying function 34b determines not to specify the set of angle-corrected X-ray image data as learning data.

To give another example, the specifying function 34b calculates the degree of similarity between images for sets of angle-corrected X-ray image data. For example, the specifying function 34b calculates the pixel value difference between the images for each pixel. Next, the specific function 34b squares the differences calculated for each pixel, and then sums them up. Then, the specifying function 34b calculates the degree of similarity based on the calculated total value. Also, for example, the specifying function 34b inputs each X-ray image data to a hash function to calculate a hash value. Then, the specifying function 34b calculates the degree of similarity based on the hash value difference between the images.

Note that when the SN ratio (signal-to-noise ratio) of each X-ray image data is different, the specifying function 34b may calculate the degree of similarity after correcting the contrast. For example, the specifying function 34b performs image processing using a smoothing filter on X-ray image data having a lower SN ratio among the set of angle-corrected X-ray image data. As a result, the specifying function 34b increases the SN ratio of the X-ray image data with the lower SN ratio to the same extent as the other X-ray image data. Then, the specifying function 34b calculates the degree of similarity between images for sets of X-ray image data having similar SN ratios.

Here, if the similarity is higher than the threshold, the specifying function 34b specifies a set of angle-corrected X-ray image data as learning data. Note that when image processing is performed on the X-ray image data with a lower SN ratio, the specific function 34b sets the X-ray image data before performing the image processing and the other X-ray image data to Identify as training data. On the other hand, if the degree of similarity is lower than the threshold, the identifying function 34b determines not to identify the set of angle-corrected X-ray image data as learning data.

In addition, the specifying function 34b may determine whether or not to use each piece of X-ray image data to specify learning data prior to specifying learning data. That is, the specifying function 34b determines whether or not to use the X-ray image data acquired by the acquiring function 34a to specify learning data, and uses the X-ray image data determined to be used to acquire learning data. may be specified.

For example, if there is a plurality of similar X-ray image data among the X-ray image data acquired by the acquisition function 34a, the specifying function 34b determines to use only a part of them. Here, the case where there are a plurality of similar X-ray image data means, for example, when X-ray fluoroscopy is performed, or when the X-ray image data is acquired while finely adjusting the conditions without changing the imaging target area and imaging direction. is repeatedly performed. X-ray fluoroscopy is a method of imaging and observation used for real-time observation. This is a method of displaying on the display 109 in parallel. An X-ray image obtained by X-ray fluoroscopy is also called a fluoroscopic image.

For example, for each piece of X-ray image data acquired by the acquisition function 34a, the specifying function 34b first determines that the imaging conditions and subject conditions are substantially the same, and the imaging target region and imaging direction are substantially the same. It is determined whether or not there is other matching X-ray image data (similar X-ray image data). Then, the specifying function 34b determines that X-ray image data for which there is no similar X-ray image data is used for specifying learning data.

On the other hand, for a plurality of X-ray image data whose imaging conditions and subject conditions substantially match and whose imaging target regions and imaging directions substantially match, the specifying function 34b converts some of them into learning data. Determined to be used specifically. That is, the specifying function 34b determines that a part of a plurality of similar X-ray image data should be used for specifying learning data. For example, the specifying function 34b determines to use X-ray image data with good image quality among a plurality of similar X-ray image data to specify learning data. For example, the specifying function 34b calculates the SN ratio for each of the plurality of X-ray image data, and determines that the X-ray image data with the highest SN ratio is used for specifying learning data.

Further, for example, the specifying function 34b sets at least one of the condition regarding the stability of gradation and the condition regarding afterimage for a plurality of pieces of time-series X-ray image data among the X-ray image data obtained by the obtaining function 34a. X-ray image data that satisfies the criteria is determined to be used for specifying learning data.

For example, the specifying function 34b calculates the difference in the average value of the pixel values between successive pieces of X-ray image data among a plurality of pieces of time-series X-ray image data. Then, the specific function 34b determines that the condition regarding tone stability is met when the calculated difference is smaller than the threshold, and the condition regarding tone stability is not satisfied when the calculated difference is greater than or equal to the threshold. I judge. As a case where the condition regarding gradation stability is not satisfied, for example, X-ray image data is collected after the X-ray tube 102 starts generating X-rays until the X-ray intensity stabilizes. This applies to cases, etc.

Further, for example, the specific function 34b executes image processing such as recursive filtering and averaging processing on a plurality of time-series X-ray image data, and converts the processed X-ray image data and the unprocessed X-ray image data into Afterimage components are extracted by subtracting . Then, the specifying function 34b determines that the afterimage condition is satisfied when the afterimage component extraction amount is smaller than the threshold, and determines that the afterimage condition is not satisfied when the afterimage component extraction amount is equal to or greater than the threshold. It should be noted that, for example, the case where the condition regarding the afterimage is not satisfied corresponds to the case where there is body movement of the subject at the time of imaging.

Next, the learning data set generation function 34c generates a learning data set including the plurality of learning data specified by the specifying function 34b. For example, the learning data set generation function 34c performs learning in which different conditions (imaging conditions and subject conditions) are common among sets of X-ray image data from the learning data specified by the specifying function 34b. Extract data for

For example, the learning data set generation function 34c generates learning data (X-ray A plurality of learning data, etc., each consisting of a set of image data I1 and X-ray image data I2 are extracted. In other words, the learning data set generation function 34c extracts a plurality of learning data sets composed of X-ray image data sets with different degrees of noise. Next, the learning data set generation function 34c generates a learning data set including the plurality of extracted learning data. Then, the learning data set generation function 34 c stores the generated learning data set in the memory 33 . In the following description, a learning data set including a plurality of sets of X-ray image data with different degrees of noise is also referred to as a learning data set D1.

Here, the learning data set generation function 34c may generate a learning data set for each part of the subject. For example, the learning data set generation function 34c first extracts a plurality of learning data consisting of sets of X-ray image data with different degrees of noise from the learning data specified by the specifying function 34b. Next, the learning data set generating function 34c extracts learning data for each part of the subject from the extracted learning data.

For example, the learning data set generation function 34c extracts learning data consisting of sets of X-ray image data of "abdomen" and generates a learning data set D11 for "abdomen". Also, for example, the learning data set generation function 34c extracts learning data composed of sets of X-ray image data of "liver" and generates a learning data set D12 for "liver". Here, the learning data set generation function 34c extracts the same learning data as learning data consisting of a set of X-ray image data of "abdomen", and extracts the same data for learning from a set of X-ray image data of "liver" may be extracted as training data. Then, the learning data set generation function 34 c stores the generated learning data set D<b>11 and learning data set D<b>12 in the memory 33 . The learning data set D11 and the learning data set D12 are examples of the learning data set D1.

Next, the model generation function 34d reads the learning data set from the memory 33 and generates a trained model, as shown in FIG. Specifically, the model generating function 34d performs processing for converting input X-ray image data into X-ray image data corresponding to X-ray image data in which at least one of the imaging conditions and the subject conditions is different. Generate an embedded trained model. Here, the trained model in this embodiment can be composed of, for example, a multi-layered neural network.

A typical configuration of the multilayer neural network according to this embodiment will be described below. Here, a multilayer neural network is a network that has a structure in which only adjacent layers arranged in layers are connected, and information propagates in one direction from the input layer side to the output layer side. As shown in FIG. 6, the multi-layer neural network according to this embodiment includes an input layer (l=1), an intermediate layer (l=2, 3, . . . , L−1), an output layer (l=L ) are composed of L layers. FIG. 6 is a diagram for explaining the process of generating a trained model according to the first embodiment. The following description is an example, and the configuration of the multilayer neural network is not limited to the following description.

The number of units in the l-th layer is I, the input u ^(l) to the l-th layer is the following formula (1-1), and the output z ^(l) from the l-th layer is the following formula (l-2 ), the relationship between the input to the l-th layer and the output from the l-th layer can be expressed by the following equation (1-3).

In formulas (1-1), (1-2) and (1-3), the upper right subscript (l) indicates the layer number. In addition, f (u) in formula (1-3) is an activation function, logistic sigmoid function (logistic function), hyperbolic tangent function, normalized linear function (ReLU: Rectified Liner Unit), linear mapping, identity Various functions such as mapping, maxout function, etc. can be selected according to the purpose.

Let the number of units in the l+1-th layer be J, the weighting matrix W ^(l+1) between the l-th layer and the l+1-th layer is expressed by the following formula (2-1), and the bias b ^{(l +1)} is expressed as in the following equation (2-2), the input u ^(l+1) to the l+1th layer and the output z ^(l+1) from the l+1th layer are as follows: It can be represented by equations (2-3) and (2-4).

In the multi-layer neural network according to this embodiment, a signal represented by the following equation (3-1) is input to the input layer (l=1). Also, in the input layer, the input data x becomes the output data z ⁽¹⁾ as it is, so the following formula (3-2) holds.

Here, for the input signal x, for example, each component x _p (p=1, 2, .

In the intermediate layers (l=2, 3, . can be calculated for the outputs z ( ^{2 )} , .

The output z ^(L) of the output layer (L-th layer) is expressed as in Equation (4-1) below. The multi-layer neural network according to this embodiment is a forward propagation network in which image data x input to an input layer propagates from the input layer side to the output layer side while connecting only between adjacent layers. Such a forward propagating network can be expressed as a composite function such as the following equation (4-2).

The composite function defined by equation (4-2) is obtained from equations (2-3) and (2-4) using the weighting matrix W ^(l+1) to obtain the linear relationship between layers, the activity defined as a combination of a non-linear relationship (or linear relationship) using a function f(u ^(l+1) ) and a bias b ^(l+1) . In particular, the weighting matrix W ^(l+1) and the bias b ^(l+1) are called parameters p of the network. The composite function defined by equation (4-2) changes its form as a function depending on how the parameter p is selected. Therefore, the multi-layer neural network according to this embodiment can be defined as a function that allows the output layer to output a preferable result y by appropriately selecting the parameter p that constitutes equation (4-2). can.

In order to properly select the parameter p, learning is performed using a training data set and an error function. Here, the learning data set is the learning data (x n , _{d n} ) represented by the following equation (5-1) where d _n is the desired output (correct output) for the input x _n is a set D (n=1, . . . , S). That is, the learning data set is a set D of learning data represented by the following equation (5-2).

The error function is a function that expresses the closeness between the output from the multilayer neural network to which _xn is input and the learning data _dn . Typical examples of error functions include a squared error function, a maximum likelihood estimation function, a cross entropy function, and the like. What kind of function is selected as the error function depends on the problem (eg, regression problem, binary problem, multi-class classification problem, etc.) to be handled by the multi-layered network.

An error function is denoted by E(p), and an error function calculated using only one learning data (x _n , d _n ) is denoted by E _n (p). The current parameter p ^(t) is the formula (6-1) using the gradient vector of the error function E (p) when following the gradient descent method, and the error function E _n (p ) is updated to the new parameter p ^(t+1) by equation (6-3) using the gradient vector of .

Here, ε is a learning coefficient that determines the magnitude of the update amount of the parameter p. Determining the parameter p that minimizes the error function E(p) by moving the current p slightly in the direction of the negative gradient according to formula (6-1) or formula (6-3) and repeating this step by step can be done.

In addition, in order to calculate the formula (6-1) or formula (6-3), the gradient vector of E(p) represented by formula (6-2) or E We need to compute _{the n} (p) gradient vectors. Taking the case where the error function is a squared error function as an example, it is necessary to differentiate the error function shown in the following equation (7-1) with the weight coefficient of each layer and the bias of each unit.

On the other hand, since the final output y is a composite function represented by equation (4-2), the calculation of the gradient vector of E(p) or E _n (p) is complicated and the amount of calculation is enormous. becomes.

Such glitches in gradient computation can be overcome by error backpropagation. For example, the differentiation of the error function with respect to the weight w _ji ^(l) connecting the i-th unit of the (l−1)-th layer and the j-th unit of the l-th layer can be expressed as in the following equation (8-1). can.

The amount of change given to E _n by the input u _j ^{(l) to} the j-th unit of the l layer is _expressed by each input u _k ^{( l+1)} only through changing. From this, the first term on the right side of the equation (8-1) can be expressed as the following equation (9-1) using the chain rule of differentiation.

Here, if the left side of equation (9-1) is δ _j ^(l) , equation (9-1) can be expressed by equation (10-1) and equation (10-2) below (10-3) can be rewritten.

From equation (10-3), it can be seen that δ _j ^(l) on the left side can be calculated from δ _k ^(l+1) (k=1, 2, . . . ). That is, given δ _k ^(l+1) for the k-th unit of the l+1-th layer existing in the output layer one level higher, δ _j ^(l) for the l-th layer can be calculated. Furthermore, if δ _k ^(l+1) related to the k-th unit of the l+1-th layer is also given δ _k ^(l+2) related to the k-th unit of the l+2-th layer existing on the output layer side one level higher, , can be calculated. This process can be repeated successively to reach the output layer, which is the highest layer.

First, if δ _k ^(L) for the k-th unit of the output layer, which is the L-th layer, is obtained, the calculation is repeated successively toward the lower side (that is, the input layer side) using equation (10-3). (backpropagation), δ _k ^(l+1) at any layer can be calculated.

On the other hand, for the second term on the right side of formula (8-1), using formula (11-1) that expresses formula (2-3) with components for the l-th layer, as in formula (11-2) can be calculated.

Therefore, the differentiation of the error function with respect to the weight w _ji ^(l) connecting the i-th unit of the l−1-th layer and the j-th unit of the l-th layer is obtained by equations (8-1) and (10-3). Using δ _j ^(l) and equation (11-2), it can be expressed as in equation (12-1) below.

From equation (12-1), the differential of the error function with respect to the weight w _ji ^(l) connecting the i-th unit of the l−1-th layer and the j-th unit of the l-th layer is δ _j ^{(l )} and z _i ^(l-1) which is the output from the i-th unit. Note that the calculation of δ _j ^(l) can be obtained by backpropagation using equation (10-3) as described above, and the first value of backpropagation, that is, the Lth layer δ _j ^(L) for the output layer can be calculated as shown in Equation (13-1) below.

Through the above procedure, learning using a certain learning sample (x _n , d _n ) can be realized for the multi-layered network according to this embodiment. Regarding the gradient vector related to the sum of errors E=Σ _n E _n for a plurality of learning samples, the above-described procedure is repeated for each learning sample (x _n , d _n ) in parallel, and the following equation (14-1) can be obtained by calculating the sum shown in

For example, the model generation function 34d first reads the learning data set D1 from the memory 33 . Next, the model generation function 34d generates X-ray image data with less noise (X-ray image data I2, etc.) from X-ray image data with much noise (X-ray image data I1, etc.) based on the learning data set D1. Train the model to restore For example, the model generation function 34d adjusts model parameters using noisy X-ray image data as input data and low-noise X-ray image data as output data, thereby learning an approximation of a function that removes noise. Generate a finished model. When the trained model to be generated is a trained model by a neural network, the parameter to be adjusted is at least one of weight and bias, for example. In addition, below, the trained model generated based on the data set D1 for learning is also described as the trained model M1. Then, the model generation function 34 d stores the generated learned model M1 in the memory 33 .

Note that the model generation function 34d may use learning data as input data, or may use data based on learning data as input data. For example, as shown in FIG. 7, the model generation function 34d performs preprocessing (for example, image processing using a spatial filter or a convolution filter) for X-ray image data with much noise (such as X-ray image data I1). and the image data after preprocessing is used as the input side data. FIG. 7 is a diagram for explaining the process of generating a trained model according to the first embodiment.

Also, the model generation function 34d may use the learning data as the output side data, or may use data based on the learning data as the output side data. For example, as shown in FIG. 7, the model generating function 34d uses image data before post-processing of X-ray image data with little noise (such as X-ray image data I2) as output data. For example, the model generation function 34d adjusts the parameters of the model so that the image data after post-processing for the output side data and the X-ray image data with less noise are approximated, and has learned an approximation function that removes noise. Generate model M1. In this case, the trained model M1 is incorporated and used together with other processes (pre-processing, post-processing, etc.) for the process of converting X-ray image data with much noise into X-ray image data with less noise. . That is, the model generation function 34d can generate a trained model M1 that is incorporated in the X-ray image data conversion process together with other processes.

Also, the output of the neural network in the trained model generated by the model generation function 34d may not be the pixel values of the image data. For example, the model generation function 34d may make the output of the neural network a combination of parameters (intensity, etc.) of various image processing filters.

In this case, the model generation function 34d performs filtering on X-ray image data with a lot of noise (X-ray image data I1, etc.) in the set of X-ray image data for each combination of image processing filter parameters. . In addition, as shown in FIG. 8, the model generating function 34d calculates the degree of similarity between the image data after filtering X-ray image data with much noise and the X-ray image data with little noise (such as X-ray image data I2). is calculated for each combination of parameters, and a combination of parameters (k ₁ , k ₂ . . . , k _M ) with high similarity is specified. Then, as shown in FIG. 8, the model generation function 34d makes the neural network learn by using X-ray image data with a lot of noise as data on the input side and using the specified combination of parameters as data on the output side. FIG. 8 is a diagram for explaining the process of generating a trained model according to the first embodiment.

As a result, X-ray image data with much noise is used as an input image, and a combination of image processing filter parameters (k ₁ , k _{2 .} A neural network can be obtained. Therefore, as shown in FIG. 9, filter processing is performed on noisy X-ray image data with a combination of filter parameters (k ₁ , k ₂ . . . , k _M ) obtained by the neural network. Thus, it is possible to construct a trained model M1 that converts X-ray image data with much noise into X-ray image data with less noise. Note that FIG. 9 is a diagram for explaining the process of generating a learned model according to the first embodiment.

Note that types of image processing filters include, for example, a moving average (smoothing) filter, a Gaussian filter, a median filter, a recursive filter, a bandpass filter, and the like. In addition, as the parameters of the image processing filter, for example, the strength of each filter, the moving average (smoothing) filter, the vertical and horizontal sizes of the Gaussian filter and median filter, the number of frames used in the recursive filter, and the Weighting, frequency setting in a band-pass filter, and the like can be mentioned.

Also, a learned model may be generated by a machine learning method other than a multi-layer neural network. For example, a machine learning method for classifying data such as SVM (support vector machine) may be used to generate a trained model that selects an appropriate image processing filter for input medical image data. As a specific method, for example, the classification destination class is the type of the image processing filter, the feature amount of the input medical image data is the input data of the model, and the output result for the input side data of the learning data is used. Model learning may be performed so as to select a filter that increases the degree of similarity between the learning data and the output side data.

Also, the time during which the model generation function 34d executes the process of generating the learned model may be set based on the inspection information. For example, the model generation function 34d acquires a time (midnight, etc.) during which X-ray image data is not collected by the X-ray diagnostic apparatus 100 based on the examination information, and generates a learned model using the acquired time. Set as time. During the time set in this way, the processor (processing circuit 111, etc.) of the X-ray diagnostic apparatus 100 is often not used, and the model generation function 34d may use the processor of the X-ray diagnostic apparatus 100 as a computational resource. can. In particular, since the X-ray diagnostic apparatus 100 often has a GPU, the model generating function 34d can quickly generate a learned model in a set time.

When new X-ray image data is acquired after the model generation function 34d has generated a learned model, the acquisition function 34a acquires the newly acquired X-ray image data. For example, as shown in FIG. 10, when X-ray image data I3 is newly acquired by the X-ray diagnostic apparatus 100, the acquiring function 34a acquires the newly acquired X-ray image data I3. FIG. 10 is a diagram for explaining conversion processing according to the first embodiment.

Then, the conversion function 34e executes conversion processing on the X-ray image data I3 based on the learned model. Specifically, the conversion function 34e converts the input X-ray image data I3 into an X-ray image that differs from the X-ray image data I3 in at least one of imaging conditions and subject conditions based on the learned model. A conversion process is executed to convert the data into X-ray image data.

For example, the conversion function 34e first reads out the learned model M1 from the memory 33. FIG. Next, the conversion function 34e inputs the X-ray image data I3 to the learned model M1, and converts it into X-ray image data I3' corresponding to X-ray image data with less noise than the X-ray image data I3. to run. In other words, the conversion function 34e converts the X-ray image data I3 into X-ray image data I3' corresponding to X-ray image data captured with a tube current value (high dose) greater than the X-ray image data I3. to remove noise from the X-ray image data I3.

The output function 34f outputs the converted X-ray image data I3'. For example, the output function 34f generates an X-ray image for display based on the X-ray image data I3' and causes the display 32 to display the generated X-ray image. As a result, the output function 34f can provide the operator of the medical information processing apparatus 30 with an X-ray image with reduced noise. Also, for example, the output function 34f outputs the X-ray image data I3' to an external device connected to the medical information processing apparatus 30 via the network NW. In this case, the external device can cause the display of the external device to display the X-ray image for display based on the X-ray image data I3'. That is, the output function 34f can provide an X-ray image with reduced noise to the operator of the external device.

Here, the specific function 34b is a set of newly acquired X-ray image data, or a set of newly acquired X-ray image data and X-ray image data stored in the memory 33. and at least one of the conditions of the object are different and the imaging directions are substantially the same can be newly specified as learning data. Further, when learning data is newly specified, the learning data set generation function 34c newly generates a learning data set including the newly specified learning data, and the model generation function 34d newly generates A new trained model is generated based on the generated learning data set.

Note that the control function 34g may receive an input operation for switching whether or not the conversion function 34e executes the conversion process. For example, the control function 34g receives an input operation from the operator via the input interface 31, thereby switching between a mode in which the conversion function 34e executes conversion processing and a mode in which the conversion function 34e does not execute conversion processing.

For example, when the X-ray diagnostic apparatus 100 acquires X-ray image data in a mode in which the conversion function 34e does not perform conversion processing, the acquisition function 34a acquires the acquired X-ray image data, and the specific function 34b , to identify training data. On the other hand, when the X-ray diagnostic apparatus 100 acquires X-ray image data in a mode in which the conversion function 34e executes conversion processing, the acquisition function 34a acquires newly acquired X-ray image data and stores it in the memory. 33, and the conversion function 34e executes conversion processing on the X-ray image data read out from the memory 33. FIG. After that, when the conversion function 34e is switched to a mode in which the conversion processing is not executed, the specifying function 34b reads the X-ray image data from the memory 33 and specifies learning data.

So far, as an example of learning data, learning data composed of sets of X-ray image data with different degrees of noise has been described. However, embodiments are not so limited.

For example, the specifying function 34b specifies learning data consisting of sets of X-ray image data with different resolutions. Specifically, the specifying function 34b specifies a set of X-ray image data having substantially the same imaging direction and having different resolutions among the X-ray image data acquired by the acquiring function 34a. For example, the specifying function 34b specifies a set of X-ray image data acquired in the same examination among the X-ray image data acquired by the acquiring function 34a. Next, the specifying function 34b selects a set of X-ray image data in which at least a part of the imaging target region overlaps and the imaging directions are substantially the same among the sets of X-ray image data acquired in the same examination. Identify. Then, the specifying function 34b specifies a set of X-ray image data in which at least a part of the imaging target region overlaps, the imaging directions are substantially the same, and the resolutions are different.

Here, the sets of X-ray image data with different resolutions are imaged using the X-ray detectors 107 with different numbers (density) of detection elements, for example, so that the number of pixels (number of pixels) is different. It is a set of X-ray image data. Further, for example, a set of X-ray image data having different resolutions is an X-ray image having a different number of pixels due to bundling processing of detection signals when at least one of the X-ray image data is captured. A set of data.

Note that the specifying function 34b may determine whether or not to specify sets of X-ray image data with different resolutions as learning data. For example, the specifying function 34b performs angle correction between sets of X-ray image data having substantially the same imaging direction, different imaging directions, and different resolutions. It is determined whether or not to specify the set of corrected X-ray image data as learning data. For example, the specification function 34b first performs an image-to-image comparison for sets of angle-corrected X-ray image data.

For example, the specifying function 34b extracts feature points from each set of angle-corrected X-ray image data, and determines whether or not the positions of the feature points match between the images. For example, the specifying function 34b extracts a plurality of feature points from the X-ray image data by image processing such as pattern recognition, and generates an identification code for each of the extracted feature points based on the virtual patient image data. Identify. Then, the specifying function 34b determines whether or not the positions of the feature points extracted from each X-ray image data match for each identification code. Here, when the positions of the feature points match, the specifying function 34b specifies a set of angle-corrected X-ray image data as learning data. On the other hand, if the positions of the feature points do not match, the identifying function 34b determines not to identify the set of angle-corrected X-ray image data as learning data.

To give another example, the specifying function 34b extracts a characteristic region from each piece of X-ray image data for a set of angle-corrected X-ray image data, and determines whether or not the characteristic regions match between the images. For example, the specifying function 34b extracts a bone, an organ, or the like as a characteristic region from each X-ray image data, and extracts a contour line of the extracted region. Here, if the contour lines match between the images, the specifying function 34b specifies a set of angle-corrected X-ray image data as learning data. On the other hand, if the contour lines do not match, the specifying function 34b determines not to specify the set of angle-corrected X-ray image data as learning data.

To give another example, the specifying function 34b calculates the degree of similarity between images for sets of angle-corrected X-ray image data. For example, the specific function 34b calculates the pixel value difference between the images for each pixel, squares each of the calculated differences, and then sums them. Then, the specifying function 34b calculates the degree of similarity based on the calculated total value. Also, for example, the specifying function 34b inputs each X-ray image data to a hash function to calculate a hash value, and calculates a degree of similarity based on the difference in hash value between images.

Note that the specifying function 34b may calculate the degree of similarity after correcting the resolution of the X-ray image data. For example, the specifying function 34b executes the pixel bundling process for the X-ray image data having the higher resolution in the set of angle-corrected X-ray image data. As a result, the specifying function 34b reduces the resolution of the X-ray image data with the higher resolution to the same extent as the other X-ray image data. Then, the specifying function 34b calculates the degree of similarity between images for sets of X-ray image data having similar resolutions.

Further, when the SN ratio of each X-ray image data is different, the specifying function 34b may calculate the degree of similarity after correcting the contrast. For example, the specifying function 34b performs image processing using a smoothing filter on X-ray image data having a lower SN ratio among the set of angle-corrected X-ray image data. As a result, the specifying function 34b increases the SN ratio of the X-ray image data with the lower SN ratio to the same extent as the other X-ray image data. Then, the specifying function 34b calculates the degree of similarity between images for sets of X-ray image data having similar SN ratios.

Here, if the similarity is higher than the threshold, the specifying function 34b specifies a set of angle-corrected X-ray image data as learning data. Note that when the pixel bundling process has been executed for the X-ray image data with the higher resolution, the specific function 34b selects the X-ray image data before executing the pixel bundling process and the other X-ray image data. A set is specified as training data. Further, when image processing is performed on X-ray image data with a lower SN ratio, the specific function 34b learns a set of X-ray image data before image processing and the other X-ray image data. data for use. On the other hand, if the degree of similarity is lower than the threshold, the identifying function 34b determines not to identify the set of angle-corrected X-ray image data as learning data.

Next, the learning data set generation function 34c extracts learning data composed of sets of X-ray image data with different resolutions from the learning data specified by the specifying function 34b, and converts the extracted learning data to Generate a training dataset containing In the following description, a learning data set including a plurality of sets of X-ray image data having different resolutions is also referred to as a learning data set D2.

Next, the model generation function 34d is based on the learning data set D2, and the learning incorporated in the process of converting the input X-ray image data into X-ray image data corresponding to X-ray image data with higher resolution. Generate a finished model. In addition, below, the trained model generated based on the data set D2 for learning is also described as the trained model M2.

When X-ray image data I4 is newly acquired after the model generation function 34d has generated the learned model M2, the acquisition function 34a acquires the X-ray image data I4. Then, the conversion function 34e executes conversion processing on the X-ray image data I4 based on the learned model M2. Specifically, the conversion function 34e converts the input X-ray image data I4 into X-ray image data I4' corresponding to higher resolution X-ray image data based on the learned model M2. to run. For example, the conversion function 34e converts the newly acquired X-ray image data I4 into X-ray image data I4' corresponding to the X-ray image data captured using the higher-definition X-ray detector 107. to increase the resolution.

The output function 34f outputs the converted X-ray image data I4'. For example, the output function 34f generates an X-ray image for display based on the X-ray image data I4' and causes the display 32 to display the generated X-ray image. Thereby, the output function 34f can provide the operator of the medical information processing apparatus 30 with a high-resolution X-ray image. Also, for example, the output function 34f outputs the X-ray image data I4' to an external device connected to the medical information processing apparatus 30 via the network NW. In this case, the external device can cause the display of the external device to display the X-ray image for display based on the X-ray image data I4'. That is, the output function 34f can provide the operator of the external device with a high-resolution X-ray image.

To give another example, the specifying function 34b specifies learning data consisting of a pair of a contrast image and a mask image. Specifically, among the X-ray image data acquired by the acquisition function 34a, the specifying function 34b is a set of X-ray image data in which at least a part of the imaging target region overlaps and the imaging directions are substantially the same. to identify pairs of contrast and mask images.

Here, the contrast image is, for example, X-ray image data captured with a contrast agent injected into the blood vessel of the subject P1. In the contrast image, a contrast agent injected into blood vessels and background components (bone, soft tissue, etc. of subject P1) are depicted. That is, the contrast image depicts blood vessels and background components. Also, the mask image is, for example, a difference image between the X-ray image data (non-contrast image) captured with no contrast agent injected into the blood vessel of the subject P1 and the contrast image. That is, the mask image is a blood vessel image obtained by removing the background component from the contrast image.

Next, the learning data set generation function 34c extracts learning data consisting of a pair of a contrast image and a mask image from the learning data specified by the specifying function 34b, and includes a plurality of extracted learning data. Generate a training dataset. In the following description, a learning data set including a plurality of sets of contrast images and mask images is also referred to as a learning data set D3.

Next, the model generation function 34d is incorporated in the process of converting the input X-ray image data into X-ray image data corresponding to X-ray image data from which background components have been removed, based on the learning data set D3. Generate a trained model. That is, the model generation function 34d generates a trained model to be incorporated in the process of converting an input contrast image into X-ray image data corresponding to a mask image, based on the learning data set D3. In addition, below, the trained model generated based on the data set D3 for learning is also described as the trained model M3.

After the model generation function 34d has generated the learned model M3, when the X-ray image data I5 (contrast image) is acquired from the subject P1 in which the contrast agent is injected into the blood vessel, the acquisition function 34a Acquire the line image data I5. Then, the conversion function 34e executes conversion processing on the X-ray image data I5 based on the learned model M3. Specifically, the conversion function 34e executes conversion processing for converting the X-ray image data I5, which is a contrast image, into X-ray image data I5', which corresponds to a mask image, based on the learned model M3. That is, the conversion function 34e removes the background component in the X-ray image data I5 to generate a blood vessel image.

The output function 34f outputs the converted X-ray image data I5'. For example, the output function 34f generates an X-ray image for display based on the X-ray image data I5' and causes the display 32 to display the generated X-ray image. Also, for example, the output function 34f outputs the X-ray image data I5' to an external device connected to the medical information processing apparatus 30 via the network NW. In this case, the external device can cause the display of the external device to display the X-ray image for display based on the X-ray image data I5'.

Next, an example of the procedure of processing by the medical information processing apparatus 30 will be described with reference to FIG. 11 . FIG. 11 is a flowchart for explaining a series of processing flows of the medical information processing apparatus 30 according to the first embodiment. Steps S101 and S108 are steps corresponding to the acquisition function 34a. Steps S102 and S103 are steps corresponding to the specific function 34b. Steps S104 and S105 are steps corresponding to the learning data set generation function 34c. Steps S106 and S107 are steps corresponding to the model generation function 34d. Steps S109 and S110 are steps corresponding to the conversion function 34e. Step S111 is a step corresponding to the output function 34f.

First, the processing circuit 34 acquires X-ray image data from the X-ray diagnostic apparatus 100 or the image storage apparatus 20 (step S101). Next, the processing circuit 34 determines whether or not to use the acquired X-ray image data for specifying learning data (step S102), and specifies learning data from the X-ray image data determined to be used. (Step S103). Next, the processing circuit 34 generates a learning data set including the specified plurality of learning data (step S104), and stores the generated learning data set in the memory 33 (step S105).

Here, the processing circuit 34 determines whether or not it is the time set based on the inspection information (step S106). If it is the set time (Yes at step S106), the processing circuitry 34 generates a trained model based on the learning data set (step S107). On the other hand, if it is not the set time (No at step S106), or after step S107, the processing circuit 34 determines whether or not new X-ray image data has been acquired (step S108).

When new X-ray image data is acquired (Yes at step S108), the processing circuit 34 determines whether or not to execute conversion processing (step S109). Here, if it is determined that the conversion process is to be executed (Yes at step S109), the processing circuit 34 executes the conversion process on the X-ray image data newly acquired at step S108 (step S110). X-ray image data is output (step S111). In step S110, the processing circuit 34 executes conversion processing based on the last generated learned model.

If it is determined not to execute the conversion process (No at step S109), or after step S111, the processing circuit 34 proceeds to step S102 again. If no new X-ray image data is acquired (No at step S108), the processing circuit 34 terminates the process.

As described above, according to the first embodiment, the acquisition function 34a acquires X-ray image data. Further, the specifying function 34b provides a set of X-ray image data obtained by the obtaining function 34a, in which at least one of the imaging conditions and the subject conditions is different and the imaging directions are substantially the same. Identify training data consisting of Therefore, the medical information processing apparatus 30 according to the first embodiment can solve the shortage of learning data.

Further, as described above, the learning data set generation function 34c generates a learning data set including a plurality of learning data specified by the specifying function 34b. In addition, the model generation function 34d is incorporated in the processing of converting the input X-ray image data into X-ray image data corresponding to X-ray image data different in at least one of the imaging conditions and the subject conditions. Generate a finished model.

For example, the model generation function 34d is based on the learning data set D1, and the learned model incorporated in the process of converting the input X-ray image data into X-ray image data corresponding to X-ray image data with less noise. Generate model M1. Therefore, the medical information processing apparatus 30 according to the first embodiment can reduce noise in X-ray image data. As a result, the medical information processing apparatus 30 can reduce the dose of X-rays used for imaging, extend the life of the X-ray tube 102, and reduce the exposure dose of the subject.

Further, for example, the model generation function 34d is incorporated in a process of converting input X-ray image data into X-ray image data corresponding to higher resolution X-ray image data based on the learning data set D2. Generate a trained model M2. Therefore, the medical information processing apparatus 30 according to the first embodiment can improve the resolution of X-ray image data. As a result, the medical information processing apparatus 30 can reduce the dose of X-rays used for imaging, extend the life of the X-ray tube 102, and reduce the exposure dose of the subject.

Also, for example, the model generation function 34d generates a learned model M3 to be incorporated in the process of converting an input contrast image into X-ray image data corresponding to a mask image, based on the learning data set D3. Therefore, the medical information processing apparatus 30 according to the first embodiment can remove the background component of the contrast image and generate the blood vessel image without using the non-contrast image. As a result, the medical information processing apparatus 30 can omit the imaging of the non-contrast image and reduce the exposure dose of the subject P1.

Further, as described above, the medical information processing apparatus 30 according to the first embodiment automatically identifies learning data each time new X-ray image data is captured. Therefore, the medical information processing apparatus 30 accumulates more data for learning as imaging is performed, and can solve the shortage of data for learning. As a result, the medical information processing apparatus 30 generates a learning data set from the learning data that is gradually accumulated, and generates a trained model based on the generated learning data set, thereby gradually improving the accuracy of conversion processing. can be made

Further, as described above, the specifying function 34b according to the first embodiment determines whether or not the X-ray image data acquired by the acquiring function 34a is used for specifying learning data, and determines that it is to be used. Learning data is specified from the obtained X-ray image data. Therefore, the medical information processing apparatus 30 according to the first embodiment can improve the quality of learning data.

For example, the specifying function 34b selects a plurality of X-ray image data obtained by the obtaining function 34a that have substantially the same imaging conditions and subject conditions, and have substantially the same imaging target region and imaging direction. As for the image data, it is determined that part of it is used to identify the learning data. As a result, the specific function 34b can prevent a situation in which a large amount of substantially identical learning data are registered, and data bias can be prevented.

Further, for example, the specifying function 34b sets at least one of the condition regarding the stability of gradation and the condition regarding afterimage for a plurality of pieces of time-series X-ray image data among the X-ray image data obtained by the obtaining function 34a. X-ray image data that satisfies the criteria is determined to be used for specifying learning data. As a result, the specific function 34b prevents the registration of learning data composed of X-ray image data with unstable gradation and X-ray image data with many afterimages, thereby improving the quality of the learning data. can be done.

Further, as described above, the control function 34g according to the first embodiment receives an input operation for switching whether or not the conversion function 34e executes conversion processing. As a result, the medical information processing apparatus 30 can perform the conversion processing by the conversion function 34e and the learning data specifying processing by the specifying function 34b at different timings, thereby distributing the processing load over time. As a result, the medical information processing apparatus 30 can secure sufficient computational resources for each process and speed up each process.

In FIG. 1, the processing circuit 34 of the medical information processing apparatus 30 includes an acquisition function 34a, a specific function 34b, a learning data set generation function 34c, a model generation function 34d, a conversion function 34e, an output function 34f, and a control function. 34g has been described, but a plurality of processors may be combined to configure the processing circuit 34, and each processing function may be realized by each processor executing each program. Also, the processing circuit 34 may be configured by combining processors of different devices.

For example, the medical information processing system 1 may include a plurality of medical information processing apparatuses 30 , and the processing circuit 34 may be configured by combining the processors of the medical information processing apparatuses 30 . As an example, the medical information processing system 1 includes a medical information processing apparatus 30 (first medical information processing apparatus) having a processor that realizes an acquisition function 34a and a specification function 34b, a learning data set generation function 34c, and a model. and a medical information processing apparatus 30 (second medical information processing apparatus) having a processor that realizes the generating function 34d. Further, for example, the processing circuit 34 may be configured by combining a processor (processing circuit 34 etc.) of the medical information processing apparatus 30 and a processor (processing circuit 111 etc.) of the medical image diagnostic apparatus 10 .

As an example, the processing circuitry of the first medical information processing apparatus has an acquisition function 34a, a specification function 34b, and a transmission function 34h (not shown). The processing circuit of the second medical information processing apparatus has a learning data set generating function 34c, a model generating function 34d, and a receiving function 34i (not shown). Note that the transmission function 34h is an example of a transmission unit. Also, the receiving function 34i is an example of a receiving section. In this case, the transmitting function 34h in the first medical information processing apparatus transmits the learning data specified by the specifying function 34b to the second medical information processing apparatus. Also, the reception function 34i in the second medical information processing apparatus receives the learning data transmitted from the first medical information processing apparatus. Also, the learning data set generation function 34c generates a learning data set including the learning data received by the reception function 34i. Then, the model generation function 34d generates a trained model based on the learning data set generated by the learning data set generation function 34c.

For example, the model generating function 34d in the second medical information processing apparatus generates a trained model when the number of learning data received by the receiving function 34i exceeds a threshold. Alternatively, the model generation function 34d acquires a time during which the X-ray diagnostic apparatus 100 does not acquire X-ray image data based on the examination information, and uses the GPU or the like of the X-ray diagnostic apparatus 100 during the acquired time. to generate a trained model. Alternatively, the model generation function 34d generates a trained model as soon as the learning data set generation function 34c generates the learning data set.

Further, in the above-described embodiment, learning data made up of sets of X-ray image data has been described as an example of learning data. However, embodiments are not so limited. For example, the identification function 34b may identify training data consisting of X-ray image data sets and other data. Here, examples of other data include information on the date and time when the specifying function 34b specified the learning data. That is, the specifying function 34b specifies learning data that includes at least X-ray image data as a set.

Similarly, the learning data set generation function 34c may generate a learning data set including multiple pieces of learning data and other data. Here, examples of other data include information on the date and time when the learning data set generation function 34c generated the learning data set. That is, the learning data set generation function 34c generates a learning data set including at least a plurality of pieces of learning data.

(Second embodiment)
In the above-described first embodiment, the processing circuit 34 has the acquisition function 34a, the specific function 34b, the learning data set generation function 34c, the model generation function 34d, the conversion function 34e, the output function 34f, and the control function 34g. explained. On the other hand, in the second embodiment, the processing circuit 111 corresponds to the acquisition function 34a, the specific function 34b, the learning data set generation function 34c, the model generation function 34d, the conversion function 34e, the output function 34f, and the control function 34g. A case of having a function to

The X-ray diagnostic apparatus 100 according to the second embodiment has the same configuration as the X-ray diagnostic apparatus 100 according to the first embodiment shown in FIG. 111e, a learning data set generation function 111f, a model generation function 111g, and a conversion function 111h. Therefore, the same reference numerals as in FIG. 2 are assigned to the same configurations as those described in the first embodiment, and descriptions thereof are omitted.

An X-ray diagnostic apparatus 100 according to the second embodiment will be described below with reference to FIG. FIG. 12 is a block diagram showing an example configuration of an X-ray diagnostic apparatus 100 according to the second embodiment. As shown in FIG. 12, the X-ray diagnostic apparatus 100 includes an X-ray high voltage device 101, an X-ray tube 102, a collimator 103, a filter 104, a top board 105, a C-arm 106, and an X-ray detector. 107 , a memory 108 , a display 109 , an input interface 110 and a processing circuit 111 .

The processing circuit 111 also controls the overall operation of the X-ray diagnostic apparatus 100 by executing the collection function 111a, the output function 111b, and the control function 111c. In addition, processing circuitry 111 executes an acquisition function 111d corresponding to acquisition function 34a. The processing circuit 111 also executes a specific function 111e corresponding to the specific function 34b. The processing circuit 111 also executes a learning data set generation function 111f corresponding to the learning data set generation function 34c. The processing circuitry 111 also executes a model generation function 111g corresponding to the model generation function 34d. The processing circuit 111 also executes a conversion function 111h corresponding to the conversion function 34e.

An example of the configuration of the X-ray diagnostic apparatus 100 according to the second embodiment has been described above. With such a configuration, the X-ray diagnostic apparatus 100 solves the shortage of data for learning regarding X-ray image data.

Specifically, the acquisition function 111a first acquires X-ray image data. For example, the acquisition function 111a controls the X-ray high-voltage device 101 so that the X-ray tube 102 irradiates the subject P1 with X-rays. At this time, the X-ray detector 107 detects X-rays that have passed through the subject P1 and outputs a detection signal. The acquisition function 111 a then generates X-ray image data based on the detection signal received from the X-ray detector 107 and stores the generated X-ray image data in the memory 108 . Here, the output function 111b may output the X-ray image data to the image storage device 20 for storage.

Next, the acquisition function 111d acquires the X-ray image data acquired by the acquisition function 111a. The acquisition function 111d may acquire the X-ray image data stored in the memory 108, or may acquire the X-ray image data acquired by the acquisition function 111a without using the memory 108. Also, the acquisition function 111d may acquire X-ray image data stored in the image storage device 20 .

Next, the specifying function 111e selects X-ray image data in which at least one of the imaging condition and the subject condition is different from among the X-ray image data acquired by the acquiring function 111d and the imaging directions are substantially the same. Identify training data to include as a set. For example, the specifying function 111e specifies a set of X-ray image data acquired in the same examination among the X-ray image data acquired by the acquiring function 111d. Next, the specifying function 111e selects a set of X-ray image data in which at least a part of the imaging target region overlaps and the imaging directions are substantially the same among the sets of X-ray image data acquired in the same examination. Identify. The specific function 111e is a set of X-ray image data in which at least a part of the imaging target region overlaps and the imaging directions are substantially the same, and at least one of the imaging condition and the subject condition is different. Identify things. Next, the learning data set generation function 111f generates a learning data set including the plurality of learning data specified by the specifying function 111e.

Next, the model generation function 111g converts the input X-ray image data into an X-ray image corresponding to X-ray image data in which at least one of the imaging condition and the subject condition is different, based on the learning data set. Generate a trained model that is embedded in the process of transforming data. For example, the model generation function 111g acquires the time during which the acquisition function 111a does not acquire X-ray image data based on the examination information, and sets the acquired time as the time for creating the learned model. Then, the model generation function 111g generates a learned model at the set time.

When the acquisition function 111a acquires new X-ray image data after the model generation function 111g has generated a learned model, the acquisition function 111d acquires the newly acquired X-ray image data. Here, the conversion function 111h executes conversion processing on the newly acquired X-ray image data based on the learned model. Then, the output function 111b outputs the X-ray image data after conversion processing. For example, the output function 111b generates an X-ray image for display based on the converted X-ray image data, and causes the display 109 to display the generated X-ray image. Also, for example, the output function 111b outputs the converted X-ray image data to an external device connected to the X-ray diagnostic apparatus 100 via the network NW.

As described above, according to the second embodiment, the X-ray detector 107 detects X-rays and outputs detection signals. Also, the acquisition function 111a acquires X-ray image data based on the detection signal. Also, the acquisition function 111d acquires the X-ray image data acquired by the acquisition function 111a. Further, the specifying function 111e combines X-ray image data obtained by the obtaining function 111d in which at least one of the imaging conditions and the subject conditions is different and the imaging directions are substantially the same. Identify the training data that contains as . Therefore, the X-ray diagnostic apparatus 100 according to the second embodiment can solve the shortage of learning data.

Note that the acquisition function 111d in the X-ray diagnostic apparatus 100 acquires not only X-ray image data acquired by the acquisition function 111a, but also X-ray image data acquired by an X-ray diagnostic apparatus other than the X-ray diagnostic apparatus 100. may In this case, the X-ray diagnostic apparatus 100 can specify more learning data to resolve the shortage of learning data.

(Third embodiment)
In the first and second embodiments described above, the X-ray diagnostic apparatus 100 has been described as an example of the medical image diagnostic apparatus 10 . On the other hand, in the third embodiment, a radiation diagnostic apparatus other than the X-ray diagnostic apparatus 100 will be described as an example of the medical image diagnostic apparatus 10. FIG.

The medical information processing system 1 according to the third embodiment has the same configuration as the medical information processing system 1 according to the first embodiment shown in FIG. The difference is that in addition to the X-ray diagnostic apparatus 100, an X-ray CT apparatus and a SPECT apparatus are provided. The medical information processing apparatus 30 according to the third embodiment has the same configuration as the medical information processing apparatus 30 according to the first embodiment shown in FIG. differ. Therefore, the same reference numerals as in FIG. 1 are assigned to the same configurations as those described in the first embodiment, and the description thereof is omitted.

For example, when the medical information processing system 1 includes an X-ray CT apparatus, the X-ray CT apparatus has a gantry device, a bed device, and a console device. The couch device includes a tabletop on which the subject is placed. The tabletop moves and tilts under the control of the console device. The gantry includes a rotating frame that supports the x-ray tube and x-ray detector. Note that the gantry device may be configured to be tiltable. In this case, the relative angle between the gantry and the top plate may be changed by tilting the top plate, by tilting the gantry device, or by combining these. good. Also, the gantry device may be configured to be movable. In this case, the relative change in the positional relationship between the gantry device and the top plate may be performed by moving the top plate, by running the gantry device, or by combining these. good too. The X-ray tube, the X-ray detector, the rotating frame, the tabletop, etc. are included in the imaging unit of the X-ray CT apparatus.

For example, an X-ray CT apparatus irradiates a subject with X-rays from the X-ray tube while rotating a rotating frame that supports an X-ray tube and an X-ray detector facing each other across the subject. . At this time, the X-ray detector detects X-rays that have passed through the subject and outputs a detection signal to the data acquisition circuit. Note that the X-ray detector included in the X-ray CT apparatus is an example of a detector.

The data acquisition circuit is a DAS (Data Acquisition System) that generates detection data by performing amplification processing and A/D conversion processing on detection signals input from the detector. Furthermore, the data acquisition circuit outputs the generated detection data to the processing circuit of the X-ray CT apparatus. The processing circuit generates data obtained by performing preprocessing such as logarithmic conversion processing, offset correction processing, inter-channel sensitivity correction processing, and beam hardening correction on the detected data.

Here, the acquisition function 34a acquires data before preprocessing (detection data) generated by the X-ray CT apparatus or data after preprocessing as medical image data. Hereinafter, data before preprocessing (detection data) and data after preprocessing are also collectively referred to as projection data. Further, the specifying function 34b includes, as a set, projection data in which at least one of the imaging condition and the subject condition is different from the projection data acquired by the acquiring function 34a and the imaging directions are substantially the same. Identify your data.

For example, the identifying function 34b first identifies a set of projection data acquired in the same examination among the projection data acquired by the acquiring function 34a. Next, the specifying function 34b specifies a set of projection data having substantially the same imaging direction among the sets of projection data acquired in the same examination. For example, the specifying function 34b specifies a set of projection data in which at least a part of the imaging target region overlaps and the imaging directions are substantially the same, based on the arrangement of all or part of the imaging units.

For example, the console device of the X-ray CT apparatus stores the position of the X-ray tube (tube position) at the time of imaging and the position of the top plate with respect to the gantry device as supplementary information of each projection data. In this case, the specifying function 34b compares the tube position of each projection data based on the incidental information to determine whether or not the imaging directions substantially match between sets of projection data acquired in the same examination. It is possible to determine whether Further, the specifying function 34b compares the positions of the tabletops at the time when each piece of projection data was captured, based on the incidental information, so that between the sets of projection data determined to have approximately the same imaging direction, , it can be determined whether or not at least a part of the imaging target regions overlap. That is, the specifying function 34b specifies a set of projection data based on the arrangement of the imaging unit, such as the position of the tube and the position of the tabletop. Then, the specifying function 34b selects an X-ray image for which at least one of the imaging condition and the subject condition is different from a set of projection data in which at least a part of the imaging target region overlaps and the imaging directions are substantially the same. Identify training data that includes the data as a set.

Next, the learning data set generation function 34c generates a learning data set including the plurality of learning data specified by the specifying function 34b. For example, the learning data set generating function 34c generates learning data having different tube current value conditions (learning data including sets of projection data with different degrees of noise) from the learning data specified by the specifying function 34b. ) to generate a learning data set containing the extracted learning data. Note that, hereinafter, a learning data set that includes a plurality of learning data sets that include sets of projection data with different degrees of noise is also referred to as a learning data set D4. Next, the model generation function 34d generates a trained model M4 incorporated in the process of converting the input projection data into projection data corresponding to projection data with less noise, based on the learning data set D4. .

After the model generation function 34d has generated the learned model, when the X-ray CT apparatus acquires new projection data, the acquisition function 34a acquires the newly acquired projection data. Then, the conversion function 34e executes conversion processing on the newly acquired projection data based on the learned model. For example, the conversion function 34e performs conversion processing to convert newly acquired projection data into projection data corresponding to projection data with less noise based on the learned model M4.

Next, the output function 34f outputs projection data after conversion processing. For example, the output function 34f outputs the projection data after conversion processing to the processing circuit of the X-ray CT apparatus. In addition, the processing circuit of the X-ray CT apparatus performs reconstruction processing using the filtered back projection method, the iterative reconstruction method, etc. on the projection data after the conversion processing, thereby obtaining CT image data (volume data). ). Further, for example, the processing circuit of the X-ray CT apparatus generates a plurality of tomographic image data from the projection data after conversion processing, and converts the plurality of tomographic image data by a known method to generate CT image data. . Furthermore, the processing circuit of the X-ray CT apparatus generates a display image such as a volume rendering image based on the generated CT image data, and displays the generated display image.

Further, for example, when the medical information processing system 1 includes a SPECT device, the SPECT device is a radiopharmaceutical (tracer) nuclide (RI: Radio Isotope ) is detected by a gamma camera and a detection signal is output. A gamma camera is an example of a detector.

Specifically, the gamma camera two-dimensionally detects the intensity distribution of gamma rays emitted from radiopharmaceuticals, and performs amplification processing and A/D conversion processing on the data showing the detected intensity distribution. Generate a detection signal. For example, a gamma camera is a photon-counting radiation detector having a scintillator and a photomultiplier tube (PMT). A scintillator converts gamma rays into light peaking in the ultraviolet region. The PMT converts light emitted from the scintillator into an electric signal and multiplies the electric current value. The gamma camera is equipped with a collimator that limits the incident direction. A gamma camera detects incoming gamma rays with incident directions limited by a collimator. The gamma camera then transmits the generated detection signal to the data acquisition circuit. The data collection circuit collects detection signals input from the gamma camera, performs correction processing such as offset correction and sensitivity correction on each of the collected detection signals, and generates projection data.

Here, the acquisition function 34a acquires projection data generated by the SPECT apparatus as medical image data. Further, the specifying function 34b includes, as a set, projection data in which at least one of the imaging condition and the subject condition is different from the projection data acquired by the acquiring function 34a and the imaging directions are substantially the same. Identify your data.

For example, the identifying function 34b first identifies a set of projection data acquired in the same examination among the projection data acquired by the acquiring function 34a. Next, the specifying function 34b specifies a set of projection data whose imaging target regions at least partially overlap and whose imaging directions are substantially the same, among the sets of projection data acquired in the same examination. For example, the identification function 34b identifies projection data sets based on the placement of some or all of the imaging units of the SPECT apparatus. Note that the imaging unit of the SPECT apparatus includes, for example, a gamma camera, a collimator attached to the gamma camera, a top plate on which the subject is placed, and the like. For example, the identification function 34b can obtain the imaging direction based on the position of the gamma camera when each projection data was captured and the incident direction restricted by the collimator.

Then, the specifying function 34b selects projection data in which at least one of the imaging condition and the subject condition is different from a set of projection data in which at least a part of the imaging target region overlaps and the imaging directions are substantially the same. Identify training data to include as a set.

Here, the imaging conditions for the projection data acquired by the SPECT apparatus are, for example, conditions related to detection of gamma rays. Examples of conditions related to gamma ray detection include the number of PMTs provided in the gamma camera, the slit interval of the collimator attached to the gamma camera, and the like. Subject conditions for projection data acquired by the SPECT apparatus are, for example, conditions related to gamma ray radiation. Examples of conditions related to gamma ray emission include the type and amount of radiopharmaceuticals to be injected into the subject.

Next, the learning data set generation function 34c generates a learning data set including the plurality of learning data specified by the specifying function 34b. For example, the learning data set generation function 34c combines learning data (projection data with different degrees of noise) in which the amount of radiopharmaceutical injected into the subject differs from the learning data specified by the specifying function 34b. ) are extracted, and a learning data set including the extracted plurality of learning data is generated. In the following description, a learning data set including a plurality of sets of projection data with different noise levels is also referred to as a learning data set D5. Next, the model generation function 34d generates a trained model M5 to be incorporated in the process of converting the input projection data into projection data corresponding to projection data with less noise, based on the learning data set D5. .

After the model generation function 34d has generated the learned model, when the SPECT apparatus newly acquires projection data, the acquisition function 34a acquires the newly acquired projection data. Then, the conversion function 34e executes conversion processing on the newly acquired projection data based on the learned model. For example, the conversion function 34e performs conversion processing to convert newly acquired projection data into projection data corresponding to projection data with less noise based on the learned model M5.

Next, the output function 34f outputs projection data after conversion processing. For example, the output function 34f outputs the projection data after conversion processing to the processing circuit of the SPECT device. In addition, the processing circuit of the SPECT apparatus performs reconstruction processing using the filtered back projection method, the iterative reconstruction method, etc. on the projection data after the conversion processing, thereby obtaining the distribution of the radiopharmaceutical in the subject. generates SPECT image data indicating Furthermore, the processing circuit of the SPECT apparatus performs various image processing on the generated SPECT image data to generate a display image, and displays the generated display image.

As described above, according to the third embodiment, the acquisition function 34a acquires projection data. In addition, the specifying function 34b includes, as a set, projection data in which at least one of the imaging condition and the subject condition is different from the projection data acquired by the acquiring function 34a and the imaging directions are substantially the same. Identify your data. Therefore, the medical information processing apparatus 30 according to the third embodiment can solve the shortage of learning data for projection data.

Further, as described above, the learning data set generation function 34c generates a learning data set including a plurality of learning data specified by the specifying function 34b. The model generation function 34d also generates a learned model to be incorporated in the process of converting input projection data into projection data corresponding to projection data in which at least one of the imaging conditions and the subject conditions is different.

For example, the model generation function 34d is a trained model incorporated in the process of converting projection data acquired by an X-ray CT apparatus into projection data corresponding to projection data with less noise, based on the learning data set D4. Generate M4. Therefore, the medical information processing apparatus 30 according to the third embodiment can reduce noise in projection data. As a result, the medical information processing apparatus 30 can reduce the dose of X-rays used in CT scanning to extend the life of the X-ray tube 102 and reduce the exposure dose of the subject.

Further, for example, the model generation function 34d is a trained model incorporated in the process of converting the projection data collected by the SPECT apparatus into projection data corresponding to projection data with less noise, based on the learning data set D5. Generate M5. Therefore, the medical information processing apparatus 30 according to the third embodiment can reduce noise in projection data. As a result, the medical information processing apparatus 30 can reduce the dose of gamma rays used in the SPECT scan to reduce the exposure dose of the subject caused by the radiopharmaceutical.

The processing circuit 34 may be configured by a single processor, or may be configured by combining a plurality of processors. Also, the processing circuit 34 may be configured by combining processors of different devices. For example, the medical information processing system 1 may include a plurality of medical information processing apparatuses 30 , and the processing circuit 34 may be configured by combining the processors of the medical information processing apparatuses 30 . Further, for example, the processing circuit 34 may be configured by combining a processor included in the medical information processing apparatus 30, a processor included in the X-ray CT apparatus, or a processor included in the SPECT apparatus. Alternatively, functions corresponding to the acquisition function 34a, the specific function 34b, the learning data set generation function 34c, the model generation function 34d, the conversion function 34e, the output function 34f, and the control function 34g in the processing circuit of the X-ray CT apparatus or the SPECT apparatus. It may be a case of having

(Fourth embodiment)
In the first to third embodiments described above, the X-ray diagnostic apparatus 100, the X-ray CT apparatus, the SPECT apparatus, and other radiological diagnostic apparatuses have been described as examples of the medical image diagnostic apparatus 10. FIG. In contrast, in the fourth embodiment, an ultrasonic diagnostic apparatus will be described as an example of the medical image diagnostic apparatus 10. FIG.

The medical information processing system 1 according to the fourth embodiment has the same configuration as the medical information processing system 1 according to the first embodiment shown in FIG. It is different in that it has an ultrasonic diagnostic device in addition to the device. Therefore, the same reference numerals as in FIG. 1 are assigned to the same configurations as those described in the first embodiment, and the description thereof is omitted.

In this embodiment, the processing circuit 34 may have an acquisition function 34a, a specific function 34b, a learning data set generation function 34c, a model generation function 34d, and a conversion function 34e. , an acquisition function 34a, a specific function 34b, a learning data set generation function 34c, a model generation function 34d, and a conversion function 34e. Below, as an example, as shown in FIG. 13, the processing circuit 224 of the ultrasonic diagnostic apparatus 200 corresponds to the acquisition function 34a, the specific function 34b, the learning data set generation function 34c, the model generation function 34d, and the conversion function 34e. A case of having an acquisition function 224d, a specific function 224e, a learning data set generation function 224f, a model generation function 224g, and a conversion function 224h will be described. Note that FIG. 13 is a block diagram showing an example of the configuration of the ultrasonic diagnostic apparatus 200 according to the fourth embodiment.

As shown in FIG. 13, the ultrasonic diagnostic apparatus 200 includes an ultrasonic probe 210, an apparatus main body 220, a display 230, and an input interface 240. The ultrasonic probe 210 , the display 230 and the input interface 240 are each connected to the device body 220 .

The ultrasonic probe 210 is brought into contact with the body surface of the subject P2 to transmit and receive ultrasonic waves (ultrasonic scanning). For example, the ultrasound probe 210 has multiple piezoelectric transducers. These piezoelectric vibrators generate ultrasonic waves based on drive signals supplied from the transmission circuit 221 . The generated ultrasonic waves are reflected by an acoustic impedance mismatching surface in the subject, and received as reflected wave signals by a plurality of piezoelectric transducers. The ultrasonic probe 210 sends reflected wave signals received by the plurality of piezoelectric transducers to the transmission circuit 221 . Note that the ultrasonic probe 210 is an example of an imaging unit of the ultrasonic diagnostic apparatus 200 .

The display 230 displays various information. For example, the display 230 displays a GUI for accepting operator's instructions and various ultrasound images under the control of the processing circuit 224 . For example, display 230 is a liquid crystal display or a CRT display. The display 230 may be of a desktop type, or may be configured by a tablet terminal or the like capable of wireless communication with the processing circuit 224 .

The input interface 240 receives various input operations from the operator, converts the received input operations into electrical signals, and outputs the electrical signals to the processing circuit 224 . For example, the input interface 240 includes a mouse, a keyboard, a trackball, a switch, a button, a joystick, a touch pad that performs input operations by touching an operation surface, a touch screen that integrates a display screen and a touch pad, and an optical sensor. It is realized by the used non-contact input circuit, voice input circuit, or the like. Note that the input interface 240 may be configured by a tablet terminal or the like capable of wireless communication with the processing circuit 224 . Also, the input interface 240 is not limited to having physical operation parts such as a mouse and a keyboard. For example, an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the ultrasonic diagnostic apparatus 200 and outputs this electrical signal to the processing circuit 224 is also included in the input interface 240. included in the example.

The device main body 220 is a device that generates ultrasonic image data based on reflected wave signals received by the ultrasonic probe 210 . As shown in FIG. 13, the device body 220 has, for example, a transmission circuit 221, a reception circuit 222, a memory 223, and a processing circuit 224. FIG.

The transmission circuit 221 controls transmission of ultrasonic waves by the ultrasonic probe 210 . For example, under the control of the processing circuit 224, the transmission circuit 221 applies a driving signal (driving pulse) to the ultrasonic probe 210 at the timing when a predetermined transmission delay time is given to each transducer. Accordingly, the transmission circuit 221 transmits an ultrasonic beam in which ultrasonic waves are focused into a beam shape.

The receiving circuit 222 controls reception of a reflected wave signal that is the transmitted ultrasonic wave reflected by internal tissue. For example, under the control of the processing circuit 224, the receiving circuit 222 gives a predetermined delay time to the reflected wave signal received by the ultrasonic probe 210 and performs addition processing. Thereby, the reflection component from the direction corresponding to the reception directivity of the reflected wave signal is emphasized. Then, the receiving circuit 222 converts the reflected wave signal after addition processing into an in-phase signal (I signal, I: In-phase) and a quadrature signal (Q signal, Q: Quadrature-phase) in the baseband band. Then, the receiving circuit 222 sends the I signal and the Q signal (hereinafter referred to as IQ signal) to the processing circuit 224 as reflected wave data. Note that the receiving circuit 222 may convert the reflected wave signal after addition processing into an RF (Radio Frequency) signal and then send the RF (Radio Frequency) signal to the processing circuit 224 . The IQ signal and RF signal are signals (reflected wave data) containing phase information.

The memory 223 is implemented by, for example, a RAM, a semiconductor memory device such as a flash memory, a hard disk, an optical disk, or the like. For example, memory 223 receives and stores ultrasound image data acquired by processing circuitry 224 . The memory 223 also stores programs corresponding to various functions read and executed by the processing circuit 224 . Note that the memory 223 may be realized by a server group (cloud) connected to the ultrasonic diagnostic apparatus 200 via the network NW.

The processing circuitry 224 controls the overall operation of the ultrasonic diagnostic apparatus 200 by executing an acquisition function 224a, an output function 224b, and a control function 224c.

For example, the processing circuit 224 acquires ultrasound image data by reading a program corresponding to the acquisition function 224a from the memory 223 and executing the program. For example, the collection function 224a performs logarithmic amplification, envelope detection processing, and the like on the reflected wave data generated from the reflected wave signal by the receiving circuit 222, so that the signal strength of each of a plurality of sample points (observation points) differs from the luminance. Data (B-mode data) represented by brightness is generated.

Various acquisition modes can be selected for the acquisition mode of the ultrasonic probe 210 . For example, the ultrasonic probe 210 acquires two-dimensional data by performing data acquisition in a 2D mode in which the position with respect to the subject P2 is fixed and acquisition is performed. In this case, the acquisition function 224a can generate two-dimensional ultrasound image data based on the acquired data.

Further, for example, the ultrasonic probe 210 acquires three-dimensional data by performing data acquisition in a 3D mode in which data is acquired while moving while detecting position information of the subject P2 with a position sensor. In this case, the acquisition function 224a can perform position correction on the collected data using position information detected by the position sensor to generate three-dimensional ultrasound image data. It should be noted that if the ultrasound probe 210 does not have a position sensor, the acquisition function 224a uses the premise that the frame (ultrasound scan plane) interval is constant, or the angle between frames is constant, so that the three-dimensional ultrasound Acoustic image data can be generated.

Further, for example, the ultrasonic probe 210 acquires three-dimensional data by performing data acquisition in a mechanical 4D mode in which data is acquired while mechanically swinging. Also, for example, the ultrasonic probe 210 performs data acquisition in a two-dimensional array 3D mode in which a two-dimensional array probe is used to electronically scan three-dimensionally, and obtains three-dimensional data. In this case, the acquisition function 224a can generate three-dimensional ultrasound image data based on the acquired data.

Further, for example, the processing circuit 224 reads out and executes a program corresponding to the output function 224b from the memory 223, thereby converting (scan-converting) the ultrasonic image data into a data format for display. and display the generated ultrasound image on the display 230 . Further, for example, the processing circuit 224 reads out a program corresponding to the control function 224c from the memory 223 and executes it, thereby controlling various functions of the processing circuit 224 based on an input operation received from the operator via the input interface 240. to control.

In addition, processing circuitry 224 executes an acquisition function 224d that corresponds to acquisition function 34a. Processing circuitry 224 also performs a specific function 224e that corresponds to specific function 34b. The processing circuit 224 also executes a learning data set generation function 224f corresponding to the learning data set generation function 34c. Processing circuitry 224 also executes model generation function 224g, which corresponds to model generation function 34d. The processing circuitry 224 also executes a conversion function 224h that corresponds to the conversion function 34e.

An example of the configuration of the ultrasonic diagnostic apparatus 200 according to the fourth embodiment has been described above. With such a configuration, the ultrasonic diagnostic apparatus 200 solves the shortage of learning data related to ultrasonic image data.

Specifically, the acquisition function 224a first acquires two-dimensional or three-dimensional ultrasound image data. For example, the acquisition function 224a controls the transmission circuit 221 and the reception circuit 222, scans the subject P2 with the ultrasonic probe 210, and acquires reflected wave data. Furthermore, the acquisition function 224 a acquires ultrasound image data such as B-mode data based on the reflected wave data, and stores the acquired ultrasound image data in the memory 223 . At this time, the collection function 224a records the imaging target area, imaging direction, patient ID, examination type, examination date and time, etc. as incidental information of the collected ultrasonic image data. Here, the output function 224b may output the ultrasound image data to the image storage device 20 for storage. Two-dimensional or three-dimensional ultrasound image data is an example of medical image data.

For example, the acquisition function 224a acquires the placement of the ultrasound probe 210 at the time each ultrasound image data was captured. For example, if the ultrasound probe 210 includes sensors, the acquisition function 224a obtains the placement of the ultrasound probe 210 based on the output from the sensors. For example, the acquisition function 224a uses the body axis direction (T direction) of the subject P2, the left-right direction (S direction) of the subject P2, and the front-back direction (C direction) of the subject P2 as references for the TSC coordinates. A system is defined, and the position of a position sensor (such as a magnetic sensor) in the TSC coordinate system is obtained as the position of the ultrasonic probe 210 . Also, if the ultrasound probe 210 comprises multiple position sensors, the acquisition function 224a obtains the orientation of the ultrasound probe 210 based on the positional relationship between the position sensors in the TSC coordinate system. Alternatively, if the ultrasonic probe 210 comprises a gyro sensor, the acquisition function 224a obtains the orientation of the ultrasonic probe 210 in the TSC coordinate system based on the output of the gyro sensor.

As another example, if the ultrasound probe 210 is supported by an arm, the acquisition function 224a obtains the placement of the ultrasound probe 210 from information about the arm. For example, if the ultrasound probe 210 is supported by a robotic arm, the acquisition function 224a changes the placement of the ultrasound probe 210 relative to the subject P2 by driving the robotic arm. In this case, the acquisition function 224a acquires the placement of the ultrasonic probe 210 in the TSC coordinate system based on the drive information of the robot arm.

To give another example, when the ultrasonic probe 210 is imaged by a medical image diagnostic apparatus 10 other than the ultrasonic diagnostic apparatus 200 (hereinafter referred to as other apparatus), the acquisition function 224a performs the medical image imaged by the other apparatus. The arrangement of the ultrasonic probe 210 is acquired based on the image of the ultrasonic probe 210 depicted in the image data.

Here, as an example, another device is an X-ray diagnostic device, and the ultrasonic probe 210 is a TEE (Trans-Esophageal Echocardiograms) probe. In this case, the X-ray diagnostic apparatus first acquires X-ray image data from the subject P2 in which the ultrasonic probe 210 is inserted. Next, the X-ray diagnostic apparatus obtains the placement of the ultrasonic probe 210 in the X-ray image data based on the image of the ultrasonic probe 210 depicted in the X-ray image data. For example, the X-ray diagnostic apparatus acquires the position and orientation of the ultrasonic probe 210 in the X-ray image data by matching a 3D model showing the ultrasonic probe 210 to the X-ray image data. In addition, the X-ray diagnostic apparatus uses a coordinate system ( The imaging direction and imaging target area of the X-ray image data with respect to the TSC coordinate system, etc.) are specified. Thereby, the X-ray diagnostic apparatus aligns the coordinate system based on the subject P2 with the X-ray image data, and obtains the arrangement of the ultrasonic probe 210 with respect to the coordinate system based on the subject P2. Then, the acquisition function 224a acquires the arrangement of the ultrasonic probe 210 at the time of capturing the ultrasonic image data from the X-ray diagnostic apparatus.

Furthermore, the acquisition function 224a identifies the imaging direction and the imaging target area of each piece of ultrasonic image data based on the placement of the ultrasonic probe 210, and records them in the incidental information of the ultrasonic image data. For example, the acquisition function 224a specifies a clinical angle or a unit vector indicating the imaging direction as the imaging direction, and records them in supplementary information of the ultrasonic image data. Also, for example, the collection function 224a records the imaging target site and the three-dimensional area specified in the TSC coordinate system in the incidental information of the ultrasonic image data as the imaging target area.

Next, the acquisition function 224d acquires the ultrasound image data acquired by the acquisition function 224a. The acquisition function 224d may acquire ultrasound image data stored in the memory 223, or may acquire ultrasound image data acquired by the acquisition function 224a without using the memory 223. FIG. Alternatively, the acquisition function 224 d may acquire ultrasound image data stored in the image storage device 20 .

Next, the specifying function 224e specifies a set of ultrasound image data, among the ultrasound image data acquired by the acquisition function 224d, in which at least a part of the imaging target region overlaps and the imaging directions are substantially the same. . For example, the specifying function 224e first acquires and compares the patient ID, examination type, examination date and time, etc. from the incidental information of the ultrasonic image data, thereby identifying a set of ultrasonic image data collected in the same examination. do.

Next, the specifying function 224e acquires the imaging direction from the incidental information of the ultrasonic image data, and determines whether or not the imaging directions substantially match between sets of ultrasonic image data acquired in the same examination. judge. For example, when a clinical angle is specified as the imaging direction of each piece of ultrasound image data, the specification function 224e compares the clinical angles acquired from the incidental information of each piece of ultrasound image data, and determines if the clinical angles match. If so, it is determined that the imaging directions are substantially the same. To give another example, when a unit vector indicating the imaging direction is specified as the imaging direction of each piece of ultrasound image data, the specifying function 224e calculates the inner product of the unit vector obtained from the incidental information of each piece of ultrasound image data. If the inner product is greater than or equal to the threshold value, it is determined that the imaging directions substantially match.

As described above, the acquisition function 224a acquires the imaging direction based on the placement of the ultrasonic probe 210 (imaging unit) and records it in the supplementary information of the ultrasonic image data. Then, the identifying function 224e identifies a set of ultrasound image data whose imaging directions substantially match, based on the imaging directions acquired from the incidental information of the ultrasound image data. That is, the specifying function 224e specifies a set of ultrasound image data whose imaging directions are substantially the same based on the arrangement of the imaging units.

Also, for example, the specifying function 224e determines whether or not at least a part of the imaging target region overlaps between sets of ultrasound image data determined to have approximately the same imaging direction. For example, for each piece of ultrasonic image data, the acquisition function 224a records the three-dimensional area specified in the TSC coordinate system in the incidental information of the ultrasonic image data as an imaging target area. In this case, the specifying function 224e compares the three-dimensional regions acquired from the incidental information of each ultrasonic image data, and if at least a part of the three-dimensional regions overlaps, at least a part of the imaging target region is determined to be duplicates.

As described above, the acquisition function 224a acquires the imaging target area based on the placement of the ultrasonic probe 210 (imaging unit) and records the imaging target area in the incidental information of the ultrasonic image data. Then, the identifying function 224e identifies a set of ultrasound image data in which at least a part of the imaging target region overlaps based on the imaging target region acquired from the incidental information of the ultrasound image data. That is, the specifying function 224e specifies a set of ultrasound image data in which at least a part of the imaging target region overlaps based on the arrangement of the imaging units.

Next, the specifying function 224e selects that at least one of the imaging condition and the subject condition is different among sets of ultrasound image data in which at least a part of the imaging target region overlaps and the imaging directions are substantially the same. A set of ultrasound image data is identified as training data. Here, the imaging conditions for the ultrasonic image data are, for example, the frequency of ultrasonic waves used for scanning, the density of scanning lines, and the like.

Next, the learning data set generation function 224f generates a learning data set including the plurality of learning data specified by the specifying function 224e. For example, the learning data set generation function 224f extracts a plurality of learning data in which the ultrasonic frequencies differ between pairs of ultrasonic image data from the learning data specified by the specifying function 224e. . Here, the higher the frequency of the ultrasound used for scanning, the shallower the depth that can be scanned and the higher the spatial resolution, so the resolution of the ultrasound image data increases. That is, the learning data set generation function 224f extracts a plurality of learning data sets each including ultrasound image data having different resolutions.

To give another example, the learning data set generation function 224f selects a plurality of learning data having different scanning line densities between pairs of ultrasound image data from the learning data specified by the specifying function 224e. Extract. Here, as the frequency of ultrasound used for scanning increases, the frame rate that can be acquired decreases, but the spatial resolution improves, so the resolution of the ultrasound image data increases. That is, the learning data set generation function 224f extracts a plurality of learning data sets each including ultrasound image data having different resolutions.

Next, the learning data set generation function 224f generates a learning data set including the plurality of extracted learning data. Then, the learning data set generation function 224 f stores the generated learning data set in the memory 223 . Note that, hereinafter, a learning data set including a plurality of learning data sets including sets of ultrasound image data having different resolutions is also referred to as a learning data set D6.

Next, based on the learning data set, the model generation function 224g converts the input ultrasound image data into an ultrasound image corresponding to ultrasound image data in which at least one of the imaging conditions and the subject conditions is different. Generate a trained model that is embedded in the process of transforming data. For example, the model generation function 224g is based on the training data set D6, and the learned model is incorporated in the process of converting the input ultrasound image data into ultrasound image data corresponding to higher resolution ultrasound image data. Generate model M6. Then, the model generation function 224g stores the generated learned model M6 in the memory 223. FIG.

After the model generation function 224g has generated a trained model, when the acquisition function 224a acquires new ultrasound image data, the acquisition function 224d acquires the newly acquired ultrasound image data. Here, the conversion function 224h executes conversion processing on the newly acquired ultrasound image data based on the learned model. For example, the conversion function 224h executes conversion processing to convert newly acquired ultrasonic image data into ultrasonic image data corresponding to higher resolution ultrasonic image data based on the learned model M6.

Then, the output function 224b outputs the ultrasound image data after conversion processing. For example, the output function 224b generates an ultrasonic image for display based on the converted ultrasonic image data, and causes the display 230 to display the generated ultrasonic image. Also, for example, the output function 224b outputs the converted ultrasound image data to an external device connected to the ultrasound diagnostic apparatus 200 via the network NW.

As described above, according to the fourth embodiment, the acquisition function 224a acquires ultrasonic image data by scanning the subject with the ultrasonic probe 210 that transmits and receives ultrasonic waves. Also, the acquisition function 224d acquires the ultrasound image data acquired by the acquisition function 224a. In addition, the specifying function 224e determines that at least one of the imaging conditions and the subject conditions is different from the ultrasound image data acquired by the acquiring function 224d, and at least a part of the imaging target region overlaps, and , to specify learning data that includes a set of ultrasound image data whose imaging directions are substantially the same. Therefore, the ultrasonic diagnostic apparatus 200 according to the fourth embodiment can solve the shortage of learning data.

The acquisition function 224d in the ultrasonic diagnostic apparatus 200 acquires not only ultrasonic image data acquired by the acquisition function 224a, but also ultrasonic image data acquired by an ultrasonic diagnostic apparatus other than the ultrasonic diagnostic apparatus 200. may In this case, the ultrasonic diagnostic apparatus 200 can specify more learning data to resolve the shortage of learning data.

Also, as described above, the learning data set generation function 224f generates a learning data set including a plurality of learning data specified by the specifying function 224e. In addition, the model generation function 224g is incorporated in the processing of converting the input ultrasound image data into ultrasound image data corresponding to ultrasound image data in which at least one of the imaging conditions and the subject conditions is different. Generate a finished model. For example, the model generation function 224g is based on the training data set D6, and the learned model is incorporated in the process of converting the input ultrasound image data into ultrasound image data corresponding to higher resolution ultrasound image data. Generate model M6. Therefore, the ultrasonic diagnostic apparatus 200 according to the fourth embodiment can improve the resolution of ultrasonic image data.

Each component of each device according to the above-described embodiments is functionally conceptual, and does not necessarily need to be physically configured as illustrated. That is, the specific forms of distribution and integration of each device are not limited to those shown in the drawings, and all or part of them can be functionally or physically distributed and integrated in arbitrary units according to various loads and usage conditions. Can be integrated and configured. Furthermore, all or any part of each processing function performed by each device can be implemented by a CPU and a program analyzed and executed by the CPU, or implemented as hardware based on wired logic.

Also, the method of producing learning data described in the above embodiments can be realized by executing a program prepared in advance on a computer such as a personal computer or a work station. This program can be distributed via a network such as the Internet. In addition, this program is recorded on a computer-readable non-transitory recording medium such as a hard disk, flexible disk (FD), CD-ROM, MO, DVD, etc., and is executed by being read from the recording medium by a computer. can also

According to at least one embodiment described above, it is possible to solve the shortage of data for learning regarding medical images.

While several embodiments of the invention have been described, these embodiments have been presented by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and modifications can be made without departing from the scope of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, as well as the scope of the invention described in the claims and equivalents thereof.

1 medical information processing system 30 medical information processing apparatus 34 processing circuit 34a acquisition function 34b specific function 34c learning data set generation function 34d model generation function 34e conversion function 34f output function 34g control function

Claims (21)

an image archiving device for storing medical images;
An acquisition unit that acquires the medical images stored in the image storage device and at least one of the imaging conditions and the subject conditions of the medical images acquired by the acquisition unit are different, and the imaging direction is approximately the same. a medical information processing apparatus comprising a specifying unit that specifies learning data that includes matching medical images as a set ;
The specifying unit determines whether or not to use the medical image acquired by the acquiring unit to specify the learning data, and specifies the learning data using the medical image determined to be used. Medical information processing system. 2. The medical information processing according to claim 1, wherein said specifying unit specifies a set of medical images based on the arrangement of part or all of an imaging unit of a medical image diagnostic apparatus used to capture said medical images. system. In the medical image acquired by the acquisition unit, the specifying unit has at least one of different imaging conditions and subject conditions, at least a part of the imaging target region overlaps, and the imaging direction is different. 3. The medical information processing system according to claim 1, wherein said learning data including a set of substantially matching medical images is specified. 4. The medical information processing system according to any one of claims 1 to 3, further comprising a learning data set generation unit that generates a learning data set including the plurality of learning data specified by the specifying unit. 5. The medical information processing system according to claim 4, wherein said learning data set generation unit generates said learning data set for each part of the subject. A trained model incorporated in a process of converting an input medical image into a medical image corresponding to a medical image different from the medical image in at least one of imaging conditions and subject conditions, based on the learning data set. 6. The medical information processing system according to claim 4, further comprising a model generator that generates The specifying unit specifies the learning data including sets of medical images with different degrees of noise,
The model generation unit generates a trained model to be incorporated in a process of converting an input medical image into a medical image equivalent to a medical image with less noise than the medical image, based on the learning data set. The medical information processing system according to claim 6. The identifying unit identifies the learning data including a set of medical images with different resolutions,
The model generation unit generates a trained model to be incorporated in a process of converting an input medical image into a medical image corresponding to a medical image having a higher resolution than the medical image, based on the learning data set. The medical information processing system according to claim 6. The identifying unit identifies the learning data including a set of a contrast image and a mask image,
The model generation unit generates a trained model to be incorporated in a process of converting an input medical image into a medical image equivalent to a medical image from which a background component is removed from the medical image, based on the learning data set. 7. The medical information processing system according to claim 6. The medical information processing system according to any one of claims 6 to 9, wherein said model generation unit generates said learned model at a time set based on examination information. 1. The specifying unit determines that, of a plurality of similar medical images among the medical images acquired by the acquiring unit, a part of the plurality of medical images is used for specifying the learning data. 11. The medical information processing system according to any one of 10 . Among the medical images acquired by the acquisition unit, the specifying unit selects a plurality of time-series medical images that satisfy at least one of a condition related to tone stability and a condition related to an afterimage from the learning data. 11. The medical information processing system according to any one of claims 1 to 10 , wherein the medical information processing system is determined to be used for specifying the. In the medical image acquired by the acquiring unit, the specifying unit has at least one of imaging conditions and subject conditions that are different, part of the imaging target region overlaps, and the imaging direction is substantially the same. 13. The medical information processing system according to any one of claims 1 to 12 , wherein registration between medical images is performed for sets of matching medical images, and the set of registered medical images is specified as the learning data. In the medical image acquired by the acquisition unit, the specifying unit has at least one of different imaging conditions and subject conditions, at least a part of the imaging target region overlaps, and the imaging direction is different. 14. The method according to any one of claims 1 to 13 , wherein angle correction is performed between medical images for a set of medical images that substantially match and whose imaging directions are not the same, and the set of angle-corrected medical images is specified as the learning data. The medical information processing system according to the item. an acquisition unit for acquiring medical images stored in the image archive ;
a specifying unit that specifies learning data that includes a set of medical images that are different in at least one of an imaging condition and a subject condition and that have substantially the same imaging direction among the medical images acquired by the acquisition unit; ,
with
The specifying unit determines whether or not to use the medical image acquired by the acquiring unit to specify the learning data, and specifies the learning data using the medical image determined to be used. Medical information processing equipment. A medical information processing system comprising an image storage device for storing medical images, a first medical information processing device, and a second medical information processing device,
The first medical information processing apparatus is
an acquisition unit that acquires the medical images stored in the image storage device ;
a specifying unit that specifies learning data that includes a set of medical images that are different in at least one of an imaging condition and a subject condition and that have substantially the same imaging direction among the medical images acquired by the acquisition unit; ,
A transmission unit that transmits the learning data,
The second medical information processing apparatus is
a receiving unit that receives the learning data transmitted by the first medical information processing apparatus;
a learning data set generation unit that generates a learning data set including the received learning data;
A trained model incorporated in a process of converting an input medical image into a medical image corresponding to a medical image different from the medical image in at least one of imaging conditions and subject conditions, based on the learning data set. and a model generator that generates
The specifying unit determines whether or not to use the medical image acquired by the acquiring unit to specify the learning data, and specifies the learning data using the medical image determined to be used.
Medical information processing system. 17. The medical information processing system according to claim 16 , wherein said model generating section generates said trained model when the number of said learning data received by said receiving section exceeds a threshold. a detector that detects radiation and outputs a detection signal;
an acquisition unit that acquires a medical image based on the detection signal;
an acquisition unit that acquires the medical images collected by the collection unit and stored in an image storage device ;
a specifying unit that specifies learning data that includes a set of medical images that are different in at least one of an imaging condition and a subject condition and that have substantially the same imaging direction among the medical images acquired by the acquisition unit; ,
with
The specifying unit determines whether or not to use the medical image acquired by the acquiring unit to specify the learning data, and specifies the learning data using the medical image determined to be used. Radiological diagnostic equipment. a collection unit that collects medical images by scanning a subject with an ultrasonic probe that transmits and receives ultrasonic waves;
an acquisition unit that acquires the medical images collected by the collection unit and stored in an image storage device ;
a specifying unit that specifies learning data that includes a set of medical images that are different in at least one of an imaging condition and a subject condition and that have substantially the same imaging direction among the medical images acquired by the acquisition unit; ,
with
The specifying unit determines whether or not to use the medical image acquired by the acquiring unit to specify the learning data, and specifies the learning data using the medical image determined to be used. Ultrasound diagnostic equipment. retrieving a medical image stored in an image archive ;
Identifying learning data including a set of medical images in which at least one of imaging conditions and subject conditions is different from among the acquired medical images and in which the imaging directions are substantially the same;
including
A method of producing learning data, wherein the learning data is specified by determining whether or not an acquired medical image is to be used for specifying the learning data, and using the medical image determined to be used. retrieving a medical image stored in an image archive ;
A program that causes a computer to execute each process for identifying a set of medical images among acquired medical images that have at least one of different imaging conditions and subject conditions and that have approximately the same imaging direction and
A program, wherein the learning data is specified by determining whether or not an acquired medical image is used for specifying the learning data, and using the medical image determined to be used.

Images