JP2022012486A - Medical image processing apparatus and medical image processing system - Google Patents

Abstract

To generate image data which can support a manipulator such as an operator in an intervention treatment.SOLUTION: A medical image processing apparatus according to an embodiment comprises: a first image acquisition unit; a second image acquisition unit; a third image acquisition unit; a first positioning unit; a second positioning unit; and an image generation unit. The first image acquisition unit acquires an ultrasonic image of a subject as a first medical image. The second image acquisition unit acquires an X-ray image of the subject as a second medical image. The third image acquisition unit acquires a third medical image of the subject different from the ultrasonic image and the X-ray image. The first positioning unit positions the X-ray image with respect to the ultrasonic image. The second positioning unit positions the third medical image with respect to the ultrasonic image. The image generation unit positions the X-ray image with the third medical image on the basis of the positioning result by the first positioning unit and the positioning result by the second positioning unit and generates an image obtained with the positioning of the third medical image with respect to the X-ray image.SELECTED DRAWING: Figure 2

Description

The embodiments disclosed in the present specification and drawings relate to a medical image processing apparatus and a medical image processing system.

In the medical field, an ultrasonic diagnostic apparatus is used that images the inside of a subject by using ultrasonic waves generated by a plurality of vibrators (piezoelectric vibrators) of an ultrasonic probe. The ultrasonic diagnostic apparatus transmits ultrasonic waves into a subject from an ultrasonic probe connected to the ultrasonic diagnostic apparatus, generates an echo signal based on a reflected wave, and obtains a desired ultrasonic image by image processing.

Intervention treatment using a catheter has been performed on the coronary arteries of the heart, but intervention treatment for organic heart diseases such as valvular heart disease has attracted attention. Intervention treatment for valvular disease is called "SHD (Structural Heart Disease) intervention treatment". In recent years, when advancing SHD intervention treatment, not only fluoroscopic image data obtained from an X-ray imaging device (for example, an X-ray cardiovascular device (Angio)) but also ultrasonic image data obtained from an ultrasonic diagnostic device are used. The number of cases is increasing. Therefore, the need for multi-modality in SHD intervention treatment is increasing.

Specifically, a surgical procedure called MitraClip (registered trademark), in which a medical device such as a clip is inserted from a catheter inserted into the body of a patient with mitral regurgitation (MR) to treat the valve. Has been applied. In this procedure, under the guidance of fluoroscopic image data, the surgeon indwells the medical device while grasping the positional relationship between the device and the heart tissue and confirming the blood flow status by ultrasonic transesophageal echocardiography. I do. Therefore, this surgical procedure can be performed in consideration of the physical strength and age of the patient without opening the chest.

In addition, a surgical procedure is applied in which a medical device such as a WATCHMAN (registered trademark) device is inserted from a catheter inserted into the body of a patient with non-valvular atrial fibrillation and placed in the left atrial appendage. .. When the device is indwelled, the surgeon confirms with an ultrasound image that it is in place in the left atrial appendage and releases the medical device.

Japanese Unexamined Patent Publication No. 2014-76331

One of the problems to be solved by the embodiments disclosed in the present specification and the drawings is to generate image data that can support an operator such as an operator in intervention treatment. However, the problems to be solved by the embodiments disclosed in the present specification and the drawings are not limited to the above problems. The problem corresponding to each effect by each configuration shown in the embodiment described later can be positioned as another problem.

The medical image processing apparatus according to the embodiment includes a first image acquisition unit, a second image acquisition unit, a third image acquisition unit, a first alignment unit, a second alignment unit, and an image generation unit. Have. The first image acquisition unit acquires an ultrasonic image of the subject as a first medical image. The second image acquisition unit acquires an X-ray image of the subject as a second medical image. The third image acquisition unit acquires a third medical image of the subject, which is different from the ultrasonic image and the X-ray image. The first alignment unit aligns the X-ray image with the ultrasonic image. The second alignment unit aligns the third medical image with the ultrasound image. The image generation unit aligns the X-ray image and the third medical image based on the alignment result by the first alignment unit and the alignment result by the second alignment unit, and the image generation unit aligns the X-ray image with respect to the X-ray image. Generates an image in which the medical images of 3 are aligned.

FIG. 1 is a schematic view showing an example of the configuration of a medical image processing system provided with the medical image processing device according to the embodiment. FIG. 2 is a diagram showing the appearance of a medical image processing system provided with the medical image processing device according to the embodiment. FIG. 3 is a schematic view showing an example of the configuration of an X-ray imaging apparatus of a medical image processing system provided with the medical image processing apparatus according to the embodiment. FIG. 4 is a block diagram showing an example of the function of the medical image processing system provided with the medical image processing device according to the embodiment. FIG. 5 is a diagram showing as a flowchart an example of the operation of a medical image processing system provided with the medical image processing device according to the embodiment. FIG. 6 is a diagram showing as a flowchart an example of the operation of a medical image processing system provided with the medical image processing device according to the embodiment. FIG. 7 is a diagram showing an example of a superimposed image based on superimposed image data in which ultrasonic image data is superimposed on X-ray fluoroscopic image data in a medical image processing system provided with the medical image processing apparatus according to the embodiment. FIG. 8 shows an example of a superimposed image based on the superimposed image data in which the CT image data including the entire left atrium is superimposed on the X-ray fluoroscopic image data in the medical image processing system provided with the medical image processing apparatus according to the embodiment. The figure which shows. FIG. 9 shows, in a medical image processing system provided with a medical image processing apparatus according to an embodiment, CT image data including the entire left atrium and ultrasonic image data including a part of the heart are superimposed on the X-ray fluoroscopic image data. The figure which shows an example of the superimposition image based on the superimposition image data. FIG. 10 is a diagram showing an example of a superimposed image based on superimposed image data in which CT image data including blood vessels is superimposed on X-ray fluoroscopic image data in a medical image processing system provided with the medical image processing apparatus according to the embodiment. FIG. 11 is a block diagram showing a modified example of the medical image processing system shown in FIG. 3 in a medical image processing system provided with the medical image processing device according to the embodiment.

Hereinafter, embodiments of the medical image processing apparatus and the medical image processing system will be described in detail with reference to the drawings.

FIG. 1 is a schematic view showing an example of the configuration of a medical image processing system provided with the medical image processing device according to the embodiment. FIG. 2 is a diagram showing the appearance of a medical image processing system provided with the medical image processing device according to the embodiment.

FIG. 1 shows a medical image processing system 1. The medical image processing system 1 includes an ultrasonic diagnostic device 10 as a first medical image generation device (also referred to as “modality”), an ultrasonic probe 20, a position sensor 30, and a second medical image generation device. An X-ray imaging device (for example, an X-ray cardiovascular device (Angio)) 40 and a medical image providing device 50 are provided. A device in which at least one of the ultrasonic probe 20 and the position sensor 30 is added to the ultrasonic diagnostic device 10 may be referred to as an ultrasonic diagnostic device. In the following description, a case where the ultrasonic probe 20 and the position sensor 30 are provided outside the ultrasonic diagnostic apparatus will be described.

The ultrasonic diagnostic apparatus 10 includes a transmission / reception circuit 11, a B-mode processing circuit 12, a Doppler processing circuit 13, an image generation circuit 14, an image memory 15, a network interface 16, and a medical image processing apparatus according to the present embodiment. 17, an input interface 18, and a display 19. The circuits 11 to 14 are configured by an integrated circuit (ASIC: Application Specific Integrated Circuit) or the like for a specific application. However, the present invention is not limited to this case, and all or part of the functions of the circuits 11 to 14 may be realized by the processing circuit 171 of the medical image processing apparatus 17 executing the program. .. Further, the ultrasonic diagnostic apparatus 10 may be provided with at least one of the input interface 18 and the display 19 outside thereof. In the following description, a case where both the input interface 18 and the display 19 are provided inside the ultrasonic diagnostic apparatus will be described.

The transmission / reception circuit 11 has a transmission circuit and a reception circuit (not shown). The transmission / reception circuit 11 controls the transmission direction and the reception direction in the transmission / reception of ultrasonic waves under the control of the processing circuit 171. Although the case where the transmission / reception circuit 11 is provided in the ultrasonic diagnostic apparatus 10 will be described, the transmission / reception circuit 11 may be provided in the ultrasonic probe 20 or both the ultrasonic diagnostic apparatus 10 and the ultrasonic probe 20. It may be provided in. The transmission / reception circuit 11 is an example of a transmission / reception unit.

The transmission circuit includes a pulse generation circuit, a transmission delay circuit, a pulser circuit, and the like, and supplies a drive signal to the ultrasonic vibrator. The pulse generation circuit repeatedly generates rate pulses for forming transmitted ultrasonic waves at a predetermined rate frequency. In the transmission delay circuit, the pulse generation circuit determines the delay time for each piezoelectric vibrator required to focus the ultrasonic waves generated by the ultrasonic vibrator of the ultrasonic probe 20 in a beam shape and determine the transmission directivity. It is given for each rate pulse generated. Further, the pulser circuit applies a drive pulse to the ultrasonic vibrator at a timing based on the rate pulse. The transmission delay circuit arbitrarily adjusts the transmission direction of the ultrasonic beam transmitted from the piezoelectric vibrator surface by changing the delay time given to each rate pulse.

The receiving circuit has an amplifier circuit, an A / D (Analog to Digital) converter, an adder, etc., receives an echo signal received by the ultrasonic transducer, and performs various processing on the echo signal. Generate echo data. The amplifier circuit amplifies the echo signal for each channel and performs gain correction processing. The A / D converter A / D-converts the gain-corrected echo signal and gives the digital data the delay time required to determine the receive directivity. The adder performs addition processing of the echo signal processed by the A / D converter to generate echo data. The addition process of the adder emphasizes the reflection component from the direction corresponding to the reception directivity of the echo signal.

The B-mode processing circuit 12 receives echo data from the receiving circuit under the control of the processing circuit 171 and performs logarithmic amplification, envelope detection processing, etc., and the signal strength is expressed by the brightness of the brightness (data (). Generates 2D or 3D data). This data is generally referred to as B-mode data. The B mode processing circuit 12 is an example of the B mode processing unit.

The B mode processing circuit 12 can change the frequency band to be visualized by changing the detection frequency by the filter processing. By using the filter processing function of the B-mode processing circuit 12, it is possible to perform harmonic imaging such as contrast harmonic imaging (CHI: Contrast Harmonic Imaging) and tissue harmonic imaging (THI: Tissue Harmonic Imaging). That is, the B-mode processing circuit 12 has reflected wave data (harmonic data or frequency dividing data) of a harmonic component having a contrast agent (microbubbles, bubbles) as a reflection source from the reflected wave data of the subject into which the contrast agent has been injected. ) And the reflected wave data (fundamental wave data) of the fundamental wave component whose reflection source is the tissue in the subject. The B-mode processing circuit 12 can also generate B-mode data for generating contrast image data from the reflected wave data (received signal) of the harmonic component, and the reflected wave data (received) of the fundamental wave component. B-mode data for generating fundamental image data can be generated from the signal).

Further, in THI by using the filter processing function of the B mode processing circuit 12, it is possible to separate the harmonic data or the frequency division data which is the reflected wave data (received signal) of the harmonic component from the reflected wave data of the subject. can. Then, the B mode processing circuit 12 can generate B mode data for generating the tissue image data from which the noise component is removed from the reflected wave data (received signal) of the harmonic component.

Further, when performing harmonic imaging of CHI or THI, the B mode processing circuit 12 can extract the harmonic component by a method different from the method using the above-mentioned filter processing. In harmonic imaging, an imaging method called an AMPM method, which is a combination of an amplitude modulation (AM) method, a phase modulation (PM) method, and an AM method and a PM method, is performed. In the AM method, the PM method, and the AMPM method, ultrasonic waves having different amplitudes and phases are transmitted to the same scanning line a plurality of times. As a result, the transmission / reception circuit 11 generates and outputs a plurality of reflected wave data (received signals) at each scanning line. Then, the B mode processing circuit 12 extracts a harmonic component by performing addition / subtraction processing on a plurality of reflected wave data (received signals) of each scanning line according to the modulation method. Then, the B mode processing circuit 12 performs envelope detection processing or the like on the reflected wave data (received signal) of the harmonic component to generate B mode data.

For example, when the PM method is performed, the transmission / reception circuit 11 scans ultrasonic waves of the same amplitude with the phase polarities inverted, as in (-1, 1), according to the scan sequence set by the processing circuit 171. Have them transmit twice by line. Then, the transmission / reception circuit 11 generates a reception signal by transmission of "-1" and a reception signal by transmission of "1", and the B mode processing circuit 12 adds these two reception signals. As a result, the fundamental wave component is removed, and a signal in which the second harmonic component remains mainly is generated. Then, the B mode processing circuit 12 performs envelope detection processing or the like on this signal to generate B mode data of THI and B mode data of CHI.

Alternatively, for example, in THI, a method of visualizing using a second harmonic component and a difference tone component included in a received signal has been put into practical use. In the visualization method using the difference sound component, for example, the transmission of a composite waveform obtained by synthesizing the first fundamental wave having a center frequency of "f1" and the second fundamental wave having a center frequency greater than "f1" is "f2". Ultrasonic waves are transmitted from the ultrasonic probe 20. This composite waveform is a waveform obtained by synthesizing the waveform of the first fundamental wave and the waveform of the second fundamental wave whose phases are adjusted so that a difference sound component having the same polarity as the second harmonic component is generated. Is. The transmission / reception circuit 11 transmits the transmitted ultrasonic wave of the composite waveform, for example, twice while inverting the phase. In such a case, for example, the B mode processing circuit 12 adds the two received signals to remove the fundamental wave component, and after extracting the harmonic component in which the difference tone component and the second harmonic component mainly remain, after that, Envelope detection processing, etc. is performed.

The Doppler processing circuit 13 frequency-analyzes the velocity information from the echo data from the receiving circuit under the control of the processing circuit 171 and extracts the motion information of the moving body such as the average velocity, the dispersion, and the power at multiple points (2). Generate dimensional or 3D data). This data is commonly referred to as Doppler data. Here, examples of the moving body include blood flow, tissues such as the heart wall, a contrast medium, and the like. The Doppler processing circuit 13 is an example of the Doppler processing unit.

The image generation circuit 14 generates ultrasonic image data expressed in a predetermined luminance range based on the echo signal received by the ultrasonic probe 20 under the control of the processing circuit 171. For example, the image generation circuit 14 generates B-mode image data in which the intensity of the reflected wave is represented by brightness from the two-dimensional B-mode data generated by the B-mode processing circuit 12 as ultrasonic image data. Further, the image generation circuit 14 uses, as ultrasonic image data, average velocity image data, distributed image data, power image data, or these, which represent movement state information from the two-dimensional Doppler data generated by the Doppler processing circuit 13. Generate color Doppler image data as combined image data. The image generation circuit 14 is an example of an image generation unit.

Here, the image generation circuit 14 generally converts (scan-converts) a scanning line signal string of ultrasonic scanning into a scanning line signal string of a video format typified by a television or the like, and ultrasonic waves for display. Generate image data. Specifically, the image generation circuit 14 generates ultrasonic image data for display by performing coordinate conversion according to the scanning form of ultrasonic waves by the ultrasonic probe 20. In addition to scan conversion, the image generation circuit 14 also performs various image processes such as image processing (smoothing processing) for regenerating an average value image of brightness by using a plurality of image frames after scan conversion. , Image processing (edge enhancement processing) using a differential filter in the image is performed. Further, the image generation circuit 14 synthesizes character information, scales, body marks, and the like of various parameters with the ultrasonic image data.

That is, the B mode data and the Doppler data are ultrasonic image data before the scan conversion process, and the data generated by the image generation circuit 14 is the ultrasonic image data for display after the scan conversion process. The B mode data and the Doppler data are also referred to as raw data (Raw Data). The image generation circuit 14 generates 2D ultrasonic image data for display from the 2D ultrasonic image data before the scan conversion process.

Further, the image generation circuit 14 generates 3D B-mode image data by performing coordinate conversion on the 3D B-mode data generated by the B-mode processing circuit 12. Further, the image generation circuit 14 generates 3D Doppler image data by performing coordinate conversion on the 3D Doppler data generated by the Doppler processing circuit 13. The image generation circuit 14 generates "three-dimensional B-mode image data and three-dimensional Doppler image data" as "three-dimensional ultrasonic image data (volume data)".

Then, the image generation circuit 14 performs a rendering process on the volume data in order to generate various two-dimensional image data for displaying the volume data stored in the three-dimensional memory on the display 19. The image generation circuit 14 performs, for example, a cross-section reconstruction method (MPR: Multi Planer Reconstruction) as a rendering process to generate MPR image data from the volume data. Further, as the rendering process, the image generation circuit 14 performs, for example, a volume rendering (VR) process for generating two-dimensional image data reflecting three-dimensional information.

The image memory 15 has, for example, a magnetic recording medium, an optical recording medium, a recording medium that can be read by a processor such as a semiconductor memory, and the like. The image memory 15 stores the ultrasonic image data generated by the image generation circuit 14 under the control of the processing circuit 171. The image memory 15 is an example of a storage unit.

The network interface 16 implements various information communication protocols according to the form of the network. The network interface 16 connects the ultrasonic diagnostic apparatus 10 with other devices such as an external X-ray imaging apparatus 40 and a medical image providing apparatus 50 according to the various protocols. An electrical connection or the like via an electronic network can be applied to this connection. Here, the electronic network means a general information communication network using telecommunications technology, and in addition to a wireless / wired hospital backbone LAN (Local Area Network) and an Internet network, a telephone communication network and an optical fiber communication network. , Cable communication networks, satellite communication networks, etc.

Further, the network interface 16 may implement various protocols for non-contact wireless communication. In this case, the ultrasonic diagnostic apparatus 10 can directly transmit and receive data to and from the ultrasonic probe 20, for example, without going through a network. The network interface 16 is an example of a network connection unit.

The medical image processing device 17 includes a processing circuit 171 and a main memory 172.

The processing circuit 171 means a dedicated or general-purpose CPU (central processing unit), MPU (microprocessor unit), GPU (Graphics Processing Unit), ASIC, programmable logic device, and the like. Examples of the programmable logic device include a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), a field programmable gate array (FPGA), and the like. Can be mentioned.

Further, the processing circuit 171 may be composed of a single circuit or a combination of a plurality of independent circuit elements. In the latter case, the main memory 172 may be provided individually for each circuit element, or a single main memory 172 may store a program corresponding to the function of the plurality of circuit elements. The processing circuit 171 is an example of a processing unit.

The main memory 172 is composed of a semiconductor memory element such as a RAM (random access memory) and a flash memory, a hard disk, an optical disk, and the like. The main memory 172 may be configured by a portable medium such as a USB (universal serial bus) memory and a DVD (digital video disk). The main memory 172 stores various processing programs (including an OS (operating system) and the like in addition to the application program) used in the processing circuit 171 and data necessary for executing the program. Further, the OS may include a GUI (graphical user interface) that makes extensive use of graphics for displaying information on the display 19 to the operator and allows basic operations to be performed by the input interface 18. The main memory 172 is an example of a storage unit.

The input interface 18 includes an input device that can be operated by an operator and an input circuit that inputs a signal from the input device. Input devices include trackballs, switches, mice, keyboards, touchpads that perform input operations by touching the operation surface, touchscreens that integrate the display screen and touchpad, and non-contact input devices that use optical sensors. And it is realized by a voice input device or the like. Further, when the ultrasonic probe 20 is a TEE probe, the input device may be realized by a controller capable of operating the TEE probe. When the input device is operated by the operator, the input circuit generates a signal corresponding to the operation and outputs the signal to the processing circuit 171. The input interface 18 is an example of an input unit.

The display 19 is composed of a general display output device such as a liquid crystal display or an OLED (Organic Light Emitting Diode) display. The display 19 displays various information according to the control of the processing circuit 171. The display 19 is an example of a display unit.

The ultrasonic probe 20 is provided with a plurality of minute vibrators (piezoelectric elements) on the front surface portion, and transmits and receives ultrasonic waves to a region including an object, for example, a region including the left atrium of the heart. Each oscillator is an electroacoustic conversion element, and has a function of converting an electric pulse into an ultrasonic pulse at the time of transmission and converting a reflected wave into an electric signal (received signal) at the time of reception. The ultrasonic probe 20 is compact and lightweight, and is connected to the ultrasonic diagnostic apparatus 10 via a cable (or wireless communication).

The ultrasonic probe 20 is classified into a linear type, a convex type, a sector type and the like depending on the difference in the scanning method. Further, the ultrasonic probe 20 is a 1D array probe in which a plurality of oscillators are arranged one-dimensionally (1D) in the azimuth direction due to a difference in the array array dimension, and two dimensions (2D) in the azimuth direction and the elevation direction. ) It is divided into the types of 2D array probes in which a plurality of oscillators are arranged. The 1D array probe includes a probe in which a small number of oscillators are arranged in the elevation direction.

Here, when 3D imaging, that is, volume imaging (for example, volume scanning) is executed, a 2D array probe having a scanning method such as a linear type, a convex type, and a sector type is used as the ultrasonic probe 20. To. Alternatively, when volumetric imaging is performed, a 1D probe equipped with a linear type, convex type, sector type, or other scanning method and a mechanism that mechanically swings in the elevation direction is used as the ultrasonic probe 20. Will be done. The latter probe is also called a mechanical 4D probe.

Here, the ultrasonic probe 20 is an ultrasonic probe (that is, a body surface probe) operated on the body surface of a subject, for example, a patient P. For example, the ultrasonic probe 20 is operated on the body surface of the chest of patient P and can scan the left ventricle, left atrium, aorta, mitral valve, aortic valve, etc. Transthoracic echocardiography (TTE). ) It is a probe. Alternatively, the ultrasonic probe 20 is an ultrasonic probe (that is, an intrabody cavity probe) that is inserted into the body cavity of patient P and operated in the body cavity. For example, the ultrasonic probe 20 is a transesophageal echocardiography (TEE) probe that is operated in the esophagus and can scan the left ventricle, left atrium, aorta, mitral valve, aortic valve, and the like. The TEE probe can acquire three-dimensional ultrasonic image data of the heart or the like in substantially real time.

The position sensor 30 detects a plurality of position information of the ultrasonic probe 20 in time series and outputs the position information to the ultrasonic diagnostic apparatus 10. The position sensor 30 includes a type sensor attached to the ultrasonic probe 20 and a type sensor provided separately from the body surface probe as the ultrasonic probe 20. The latter sensor is an optical sensor, which photographs the feature points of the body surface probe to be measured from a plurality of positions and detects each position of the body surface probe by the principle of triangulation. Hereinafter, the case where the position sensor 30 is the former sensor will be described.

The position sensor 30 is attached to the ultrasonic probe 20, detects its own position information, and outputs it to the ultrasonic diagnostic apparatus 10. The position information of the position sensor 30 can also be regarded as the position information of the ultrasonic probe 20. The position information of the ultrasonic probe 20 includes the position and posture (tilt angle) of the ultrasonic probe 20. For example, the posture of the ultrasonic probe 20 can be detected by sequentially transmitting a three-axis magnetic field by a magnetic field transmitter (not shown) and sequentially receiving the magnetic field by the position sensor 30. Further, the position sensor 30 detects a 3-axis gyro sensor that detects a 3-axis angular velocity in a 3-dimensional space, a 3-axis acceleration sensor that detects a 3-axis acceleration in a 3-dimensional space, and a 3-axis geomagnetism in a 3-dimensional space. It may be a so-called 9-axis sensor including at least one of the 3-axis geomagnetic sensors.

The X-ray imaging apparatus 40 is connected to the ultrasonic diagnostic apparatus 10 and the medical image providing apparatus 50 so as to be able to communicate with each other via the network N.

FIG. 3 is a schematic view showing an example of the configuration of the X-ray imaging apparatus 40 of the medical image processing system 1.

The X-ray imaging device 40 includes a high voltage supply device 41, an X-ray irradiation device 42, an X-ray detection device 43, an input interface 44, a display 45, a network interface 46, a medical image processing device 47, and C. It includes an arm 48 (shown only in FIG. 2) and a sleeper 49 (shown only in FIG. 2).

The high voltage supply device 41 supplies high voltage power to the X-ray tube of the X-ray irradiation device 42 under the control of the processing circuit 471.

The X-ray irradiation device 42 is provided at one end of the C arm 48. The X-ray irradiation device 42 is provided with an X-ray tube (X-ray source) and a movable diaphragm device. The X-ray tube receives the supply of high voltage power from the high voltage supply device 41 and generates X-rays according to the conditions of the high voltage power. Under the control of the processing circuit 471, the movable diaphragm device movably supports the diaphragm blades made of a substance that shields X-rays at the X-ray irradiation port of the X-ray tube. A radiation quality adjusting filter (not shown) for adjusting the quality of the X-rays generated by the X-ray tube may be provided on the front surface of the X-ray tube.

The X-ray detection device 43 is provided at the other end of the C arm 48 so as to face the X-ray irradiation device 42. The X-ray detection device 43 can operate in the SID (Source-Image Distance) direction under the control of the processing circuit 471, that is, the front-back operation. Further, the X-ray detection device 43 can operate along the rotation direction centered on the SID direction, that is, the rotation operation under the control of the processing circuit 471.

The input interface 44 has a configuration equivalent to that of the input interface 18. When the input interface 44 is operated by the operator D (technical operator D1, ultrasonic engineer D2, assistant, etc.) in the treatment room, an operation signal is sent to the processing circuit 471.

The display 45 has a configuration equivalent to that of the display 19. The display 45 displays the fluoroscopic image data as an X-ray fluoroscopic image during the intervention treatment procedure. The display 45 displays the superimposed image data in which the third medical image data (for example, CT image data) is superimposed on the X-ray fluoroscopic image data during the intervention treatment procedure as the superimposed image.

The network interface 46 has a configuration equivalent to that of the network interface 16.

The medical image processing device 47 includes a processing circuit 471 and a memory 472.

The processing circuit 471 has a configuration equivalent to that of the processing circuit 171. The memory 472 has the same configuration as the main memory 172.

The C-arm 48 (shown in FIG. 2) supports the X-ray irradiation device 42 and the X-ray detection device 43 so as to be arranged so as to face each other. The C-arm 48 can rotate in the arc direction, that is, rotate in the direction of CRA (Cranial View) and rotate in the direction of CAU (Caudal View) under the control of the processing circuit 471 or according to manual operation. be. Further, the C arm 48 rotates around the fulcrum under the control of the processing circuit 471 or according to a manual operation, that is, the rotation in the direction of LAO (Left Anterior Oblique View) and the direction of RAO (Right Anterior Oblique View). Corresponds to the rotation of. The rotation of the C arm 48 in the arc direction corresponds to the rotation of the LAO direction and the rotation of the RAO direction, and the rotation of the center of the fulcrum of the C arm 48 corresponds to the rotation of the CRA direction and the rotation of the CAU direction. It may have a configuration corresponding to.

Further, in FIG. 2, the C-arm structure included in the X-ray imaging apparatus 40 shows the case where the X-ray irradiation apparatus 42 is an undertable located below the top plate of the bed 49. However, the present invention is not limited to this case, and the X-ray irradiation device 42 may be an overtable located above the top plate. Further, the C arm 48 may be replaced by an Ω arm or may be combined with an Ω arm.

The sleeper 49 includes a top plate (not shown) on which the patient P can be placed. The top plate can operate along the X-axis direction, that is, slide in the left-right direction under the control of the processing circuit 471. The top plate can operate along the Y-axis direction, that is, slide in the ascending / descending direction under the control of the processing circuit 471. The top plate can operate along the Z-axis direction, that is, slide in the cephalopod direction under the control of the processing circuit 471. Further, the top plate can also perform a rolling operation and a tilting operation under the control of the processing circuit 471.

Returning to the description of FIGS. 1 and 2, the medical image providing device 50 is connected to the ultrasonic diagnostic device 10 and the X-ray imaging device 40 so as to be able to communicate with each other via the network N. For example, the medical image providing device 50 obtains a third medical image data from a third medical image generating device that generates a third medical image data of a different type from both the ultrasonic image data and the X-ray image data. A medical image management device (also called an "image server") that collects and stores. The medical image management device as the medical image providing device 50 is, for example, a DICOM (Digital Imaging and Communications in Medicine) server, and is connected to a device such as an ultrasonic diagnostic device 10 so as to be able to communicate via a network N. The medical image management device manages the third medical image data as a DICOM file.

Alternatively, the medical image providing device 50 is the third medical image generating device itself that generates the third medical image data. For example, the medical image providing device 50 is an X-ray CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, or the like. Hereinafter, a case where the medical image providing device 50 is an X-ray CT (Computed Tomography) device will be described. However, it is not limited to that case.

Subsequently, the function of the medical image processing system 1 will be described.

FIG. 4 is a block diagram showing an example of the function of the medical image processing system 1.

The processing circuit 171 of the ultrasonic diagnostic apparatus 10 reads and executes a computer program (for example, an image processing program) stored in a non-transient recording medium such as a memory in the main memory 172 or the processing circuit 171. Therefore, the first image acquisition function F1, the second image acquisition function F2, the third image acquisition function F3, the first alignment function F4, the second alignment function F5, the image generation function F6, and the output. The control function F7 is realized. Hereinafter, the case where the functions F1 to F7 are realized by a computer program will be described as an example, but all or a part of the functions F1 to F7 are provided in the ultrasonic diagnostic apparatus 10 as functions such as a circuit such as an ASIC. It may be a thing.

The medical image processing device 17 of the ultrasonic diagnostic apparatus 10 controls the transmission / reception circuit 11, the B mode processing circuit 12, the Doppler processing circuit 13, the image generation circuit 14, and the like, and ultrasonic waves using the ultrasonic probe 20. Imaging is executed to generate ultrasonic image data as the first medical image data. The ultrasonic image data is stored in the image memory 15.

The first image acquisition function F1 includes a function of acquiring ultrasonic image data stored in the image memory 15. Specifically, the image acquisition function F1 acquires B-mode image data and the like as ultrasonic image data. The first image acquisition function F1 is an example of the first image acquisition unit.

The second image acquisition function F2 includes a function of acquiring X-ray image data as second medical image data from the X-ray photographing apparatus 40 that generates X-ray image data via the network N. The second image acquisition function F2 is an example of the second image acquisition unit.

Although multi-fusion (fusion for image output) between X-ray image data and ultrasonic image data has been realized, it is difficult to capture the entire left atrium (or the entire heart) with ultrasonic image data. Even if the ultrasonic image data and the marker are output from the ultrasonic diagnostic apparatus 10 to the X-ray imaging apparatus 40, the operator cannot grasp the entire position information of the left atrium on the display 45 of the X-ray imaging apparatus 40. difficult. For example, when performing atrial septal puncture, it is difficult to see the puncture position on the display of the ultrasonic image alone by the TEE probe as the ultrasonic probe 20, and the puncture position and angle are currently observed by preoperative CT examination. be. Also, even if you want to check where and how large the left atrium of the heart is, the left atrium is too close to the TEE probe 20 to image the entire left atrium, so the left atrium. I can't see the whole atrium.

Therefore, the medical image processing apparatus 17 is provided with the following functions F3 to F7, and aligns the other second medical image data with the third medical image data via the ultrasonic image data, and the third medical image data is aligned. The third medical image data is fused (superimposed) with the medical image data of the second to generate the superimposed image data. As an example thereof, a case where superimposed image data is generated on the ultrasonic diagnostic apparatus 10 side, output to the X-ray imaging apparatus 40, and displayed on the display 45 of the X-ray imaging apparatus 40 will be described.

The third image acquisition function F3 includes a function of acquiring a third medical image data different from the ultrasonic image data and the X-ray image data from the medical image providing device 50 via the network N. For example, the third image acquisition function F3 acquires image data of a type different from the ultrasonic image data and the X-ray image data as a third medical image data different from the ultrasonic image data and the X-ray image data. .. The image data of a type different from the ultrasonic image data and the X-ray image data includes CT image data, MRI image data, image data including predetermined anatomical features (for example, left atrium, heart, etc.), or image data. Examples include images containing information (eg, blood vessels) indicating the path of the catheter. Further, for example, the third image acquisition function F3 is the same type of ultrasonic image data as the ultrasonic image data as the third medical image data different from the ultrasonic image data and the X-ray image data, but the acquired ultrasonic waves have different time phases. Image data (for example, past ultrasonic image data) is acquired. Hereinafter, a case where the third image acquisition function F3 acquires CT image data will be described. The third image acquisition function F3 is an example of the third image acquisition unit.

The first alignment function F4 includes a function of aligning the X-ray image data acquired by the second image acquisition function F2 with the ultrasonic image data acquired by the first image acquisition function F1. Alignment means spatial synchronization of both image data. The first alignment function F4 newly generates image data after aligning the X-ray image data with respect to the ultrasonic image data, or aligns the X-ray image data with respect to the ultrasonic image data. After that, data regarding the amount of movement of each part of the X-ray image data is generated. The first alignment function F4 is an example of the first alignment unit.

The second alignment function F5 includes a function of aligning the CT image data acquired by the third image acquisition function F3 with the ultrasonic image data acquired by the first image acquisition function F1. The second alignment function F5 newly generates image data after aligning the CT image data with respect to the ultrasonic image data, or after aligning the CT image data with respect to the ultrasonic image data. Generates data regarding the amount of movement of each part of the CT image data. The second alignment function F5 is an example of the second alignment unit.

The image generation function F6 aligns the X-ray image data and the CT image data based on the alignment result by the first alignment function F4 and the alignment result by the second alignment function F5, and makes an X-ray image. It includes a function to generate image data in which CT image data is aligned with respect to the data. For example, the image generation function F6 generates superimposed image data in which the aligned CT image data is superimposed on the X-ray image data as image data in which the CT image data is aligned with the X-ray image data. .. Hereinafter, a case where the image generation function F6 generates superimposed image data will be described as an example. The image generation function F6 is an example of an image generation unit.

The output control function F7 includes a function of outputting the superimposed image data generated by the image generation function F6 to the external X-ray photographing apparatus 40 via the network N. Further, the output control function F7 may include a function of displaying the superimposed image data on the display 19. The output control function F7 is an example of an output control unit.

The processing circuit 471 of the X-ray imaging apparatus 40 reads out and executes a computer program (for example, an image processing program) stored in a non-transient recording medium such as a memory in the memory 472 or the processing circuit 471. Then, the display control function F8 is realized. Hereinafter, the case where the function F8 is realized by a computer program will be described as an example, but all or a part of the function F8 is provided in the X-ray photographing apparatus 40 as a function of a circuit or the like such as an ASIC. May be good.

The medical image processing device 47 of the X-ray imaging device 40 controls the high voltage supply device 41, the X-ray irradiation device 42, the X-ray detection device 43, and the like to execute X-ray imaging to perform a second medical image. X-ray image data is generated as image data. The X-ray image data is stored in the memory 472.

The display control function F8 includes a function of displaying the superimposed image data output from the ultrasonic diagnostic apparatus 10 on the display 45 as a superimposed image. By outputting the superimposed image data by the output control function F7 of the ultrasonic diagnostic apparatus 10, the display 45 of the X-ray imaging apparatus 40 can display the superimposed image data as a superimposed image in substantially real time. Further, the display control function F8 includes a function of acquiring X-ray image data stored in the memory 472 and displaying the acquired X-ray image data as an X-ray image on the display 45 in substantially real time. By outputting the superimposed image data by the output control function F7, the display 45 of the X-ray imaging apparatus 40 can display the superimposed image data as a superimposed image in substantially real time. In "substantially real time", in addition to the case where X-ray image data and superimposed image data are generated (or displayed) at the same time as X-ray detection, X-ray image data and superimposed image data are generated from X-ray detection (). Or, it includes the case where there is a time lag by the amount of the processing time until the display).

Details of the functions F1 to F8 will be described with reference to FIGS. 5 to 10.

Subsequently, the operation of the medical image processing system 1 will be described.

5 and 6 are diagrams showing an example of the operation of the medical image processing system 1 as a flowchart. In FIGS. 5 and 6, the reference numeral “ST” with a number indicates each step of the flowchart. In the description of FIGS. 5 and 6, the case where the ultrasonic probe 20 is a TEE probe will be described, but the present invention is not limited to that case. The ultrasonic probe 20 may be a body surface probe.

First, FIG. 5 will be described. The third image acquisition function F3 acquires CT image data (for example, volume data) as the third medical image data regarding a predetermined patient P to be treated by the intervention treatment from the medical image providing device 50 (step ST1). .. The CT image data shall include at least the entire image of the left atrium that needs to be observed in order to place the medical device in the mitral valve.

Subsequently, the medical image processing system 1 generates X-ray fluoroscopic image data by the X-ray imaging apparatus 40 in the intervention treatment of the mitral valve of the patient P using the medical device, and is inserted into the esophagus of the patient P. Ultrasound image data is acquired by the ultrasonic diagnostic apparatus 10 using the TEE probe 20. Then, the medical image processing system 1 superimposes the third medical image data on the X-ray fluoroscopic image data aligned with respect to the ultrasonic image data, and the third medical image data is added to the X-ray fluoroscopic image data. The superimposed superimposed image data is generated in substantially real time (steps ST2 to ST13 described later). An operator such as the operator D1 who performs the procedure advances the medical device while observing the superimposed image data displayed in substantially real time, and causes the medical device to reach a predetermined position.

The X-ray imaging apparatus 40 starts X-ray fluoroscopic imaging (step ST2). The X-ray imaging device 40 controls the high voltage supply device 41, the X-ray irradiation device 42, and the X-ray detection device 43 to perform X-ray imaging, thereby performing X-ray image data of a predetermined frame relating to the patient P. (For example, X-ray fluoroscopic image data with a low X-ray dose) is generated, and the X-ray fluoroscopic image data is displayed on the display 45 of the X-ray imaging apparatus 40 (step ST3).

The first image acquisition function F1 determines whether or not to start ultrasonic imaging (for example, volume imaging) by the ultrasonic diagnostic apparatus 10 using the TEE probe 20 inserted in the esophagus of patient P (step ST4). ). If the determination in step ST4 is NO, that is, if it is determined that the ultrasonic imaging by the ultrasonic diagnostic apparatus 10 is not started, the X-ray imaging apparatus 40 generates and displays X-ray fluoroscopic image data for the next frame. (Step ST3). While displaying the fluoroscopic image data in a plurality of frames, the surgeon D1 continues the procedure of advancing the catheter in the patient P's body.

On the other hand, if the judgment in step ST4 is YES, that is, if it is determined to start ultrasonic imaging by the ultrasonic diagnostic apparatus 10, the second image acquisition function F2 causes the X-ray imaging apparatus 40 to perform X-ray fluoroscopic image data. Output is requested (step ST4). The X-ray imaging apparatus 40 generates X-ray fluoroscopic image data for the next frame (step ST7), and outputs the generated X-ray fluoroscopic image data to the ultrasonic diagnostic apparatus 10 via the network N.

The second image generation function F2 acquires X-ray fluoroscopic image data from the X-ray photographing apparatus 40 (step ST8). Further, the ultrasonic diagnostic apparatus 10 relates to the patient P by controlling the transmission / reception circuit 11, the B mode processing circuit 12 (or the Doppler processing circuit 13), and the image generation circuit 14 to perform ultrasonic imaging. Generates ultrasonic image data of a predetermined frame. The first image acquisition function F1 acquires ultrasonic image data from the image memory 15 (step ST8).

The first alignment function F4 aligns the X-ray fluoroscopic image data acquired in step ST7 with the ultrasonic image data acquired in step ST8 (step ST9). For example, the first alignment function F4 detects the position of the TEE probe 20 by the position sensor 30 or detects the position of the TEE probe 20 from the X-ray fluoroscopic image data to obtain the X-ray fluoroscopic image data. , Align with the ultrasonic image data.

Further, the second alignment function F5 aligns the CT image data including the entire left atrium acquired in step ST1 with the ultrasonic image data acquired in step ST8 (step ST10). Here, the second alignment function F5 is different from the method of aligning the X-ray fluoroscopic image data by the first alignment function F4 with the ultrasonic image data, and is different from the method of aligning the CT image data with the ultrasonic image data. Align. For example, the second alignment function F5 is based on the characteristic site (for example, heart, left atrium, blood vessel, etc.) drawn in the CT image data and the ultrasonic image data, and the ultrasonic image data of the CT image data. Align.

Moving on to the description of FIG. 6, the image generation function F6 superimposes the CT image data based on the alignment result of step ST10 on the X-ray perspective image data based on the alignment result of step ST9. Generate (step ST11). This makes it possible to generate superimposed image data that can support an operator such as the operator D1 in the intervention treatment.

The output control function F7 outputs the superimposed image data generated by step ST11 to the X-ray imaging apparatus 40 via the network N (step ST12). The display control function F8 of the X-ray imaging apparatus 40 displays the superimposed image image data output in step ST12 on the display 45 as a superimposed image (step ST13). As a result, in the intervention treatment, the superimposed image data on which the aligned CT image data is superimposed is displayed on the display 45, so that the operation of the operator such as the operator D1 can be appropriately supported.

FIG. 7 is a diagram showing an example of a superimposed image based on the superimposed image data in which the ultrasonic image data is superimposed on the X-ray fluoroscopic image data.

The superimposed image shown in FIG. 7 includes an X-ray fluoroscopic image IX acquired by X-ray imaging of a region including the entire left atrium, and an ultrasonic image IU superimposed on the X-ray fluoroscopic image IX. As described above, the position of the left atrium (image LA of the left atrium) is too close to the TEE probe 20 (image Q of the TEE probe), and the entire left atrium cannot be imaged.

Therefore, it is volume data like the ultrasonic image data, and three-dimensional alignment is possible, and the CT image data including the entire left atrium is aligned with reference to the ultrasonic image data. The image generation function F6 superimposes the CT image data after alignment on the X-ray fluoroscopic image data that is also aligned with reference to the ultrasonic image data.

FIG. 8 is a diagram showing an example of a superimposed image based on superimposed image data in which CT image data including the entire left atrium is superimposed on X-ray fluoroscopic image data.

The superimposed image shown in FIG. 8 includes an X-ray fluoroscopic image IX acquired by X-ray imaging of a region including the entire left atrium, and a CT image IC including the entire left atrium superimposed on the X-ray fluoroscopic image IX. Even if the position of the left atrium is too close to the TEE probe 20 (image Q of the TEE probe) to image the entire left atrium, the left atrium is drawn on a properly aligned CT image. (Image LA of the left atrium) allows an operator such as the operator D1 to visually recognize the entire left atrium. Further, this display eliminates the need for prior confirmation of the puncture position and angle by a CT image having a large view when performing the atrial septal puncture, and can assist the operator in determining the puncture position and angle.

In the display screen, the image area IU'of the ultrasonic image used as the reference for alignment may be shown as a frame on the superimposed image.

Returning to the description of FIG. 6, the first image acquisition function F1 determines whether or not to end the ultrasonic imaging (step ST14). When the placement and release of the medical device in the predetermined position is completed, the surgeon D1 operates the input interface 44 of the X-ray imaging apparatus 40 to instruct the ultrasonic diagnostic apparatus 10 to end the ultrasonic imaging. Alternatively, the ultrasonic engineer D2 operates the input interface 18 of the ultrasonic diagnostic apparatus 10 to instruct the end of ultrasonic imaging.

If YES in step ST14, that is, if it is determined that the ultrasonic imaging is finished, the first image acquisition function F1 requests the X-ray imaging apparatus 40 to end the output of the X-ray fluoroscopic image data ( Step ST15). Upon the request, the X-ray imaging apparatus 40 ends the display of the superimposed image data on the display 45, and returns to the display of the X-ray fluoroscopic image data alone (step ST3).

On the other hand, if the determination in step ST14 is NO, that is, if it is determined not to end the ultrasonic imaging, the X-ray imaging apparatus 40 is not required to terminate the output of the fluoroscopic image data. Therefore, the X-ray imaging apparatus 40 generates X-ray fluoroscopic image data for the next frame (step ST6), and outputs the X-ray fluoroscopic image data for the next frame to the ultrasonic diagnostic apparatus 10 (step ST7). , Maintain the updated display of the superimposed image data on the display 45.

As described with reference to FIGS. 5 and 6, the image generation function F6 has all consecutive frames based on the alignment result by the first alignment function F4 and the alignment result by the second alignment function F5. Image data in which the CT image data is aligned with the X-ray image data (for example, X-ray fluoroscopic image data) and the CT image data is aligned with the X-ray fluoroscopic image data for all consecutive frames. For example, superimposing image data is generated. Then, the display control function F8 displays the superimposed image data on the display 45 as a moving image in substantially real time. It is not limited to that case. The image generation function F6 includes X-ray fluoroscopic image data regarding a predetermined frame among a plurality of consecutive frames based on the alignment result by the first alignment function F4 and the alignment result by the second alignment function F5. Image data in which the CT image data is aligned with the X-ray fluoroscopic image related to a predetermined frame, for example, superimposed image data may be generated by aligning with the CT image data. In that case, the display control function F8 displays the X-ray fluoroscopic image data as an element of the superimposed image data as a moving image and the CT image data as a still image on the display 45.

As described above, according to the medical image processing system 1, when advancing the intervention treatment, the superimposed image data on which the aligned CT image data including the entire left atrium is superimposed is displayed on the display 45. Therefore, an operator such as the operator D1 can easily grasp the entire position information of the left atrium.

(First modification)
In FIG. 8, a case where the superimposed image data is generated by superimposing the CT image data including the entire left atrium on the X-ray fluoroscopic image data has been described, but the present invention is not limited to that case. For example, the image generation function F6 can generate superimposed image data in which CT image data including the entire left atrium and ultrasonic image data used as a reference for alignment are superimposed on X-ray fluoroscopic image data. can. The ultrasound image data partially includes an image of the left atrium. As a result, the display control function F8 can display the superimposed image data as the superimposed image on the display 45 of the X-ray photographing apparatus 40.

FIG. 9 is a diagram showing an example of a superimposed image based on the superimposed image data in which the CT image data including the entire left atrium and the ultrasonic image data including a part of the heart are superimposed on the X-ray fluoroscopic image data. ..

As shown in FIG. 9, the left atrium LA on the ultrasonic image is further superimposed on the left atrium LA of the CT image shown in FIG. Thereby, the display of the part in the past CT image including the entire left atrium can be replaced with the ultrasonic image showing the current state. The display control function F8 displays the superimposed image shown in FIG. 9 from the display of the superimposed image shown in FIG. 8 based on the position of the catheter appearing on the fluoroscopic image or the operation via the input interface 44. Can be switched to. The display control function F8 is not limited to the case where the ultrasonic image data used as the alignment reference is superimposed and displayed on the superimposed image data, but is used as the superimposed image data and the alignment reference. The ultrasonic image data may be displayed on the display 45 in parallel.

According to the first modification, in addition to the above effect, the display of the part in the past CT image including the entire left atrium can be replaced with the ultrasonic image showing the current state, so that the operator such as the operator D1 can use it. , The entire position information of the left atrium can be easily grasped.

(Second modification)
FIG. 8 describes a case where the superimposed image data is generated by superimposing the CT image data including the entire left atrium on the X-ray fluoroscopic image data, and in FIG. 9, the X-ray fluoroscopic image data is superimposed on the left atrium. The case where the superimposed image data is generated by superimposing the CT image data including the whole and the ultrasonic image data including a part of the heart has been described, but the present invention is not limited to that case. For example, the third image acquisition function F3 acquires CT image data including blood vessels, and the image generation function F6 generates superimposed image data in which the CT image data including blood vessels is superimposed on the X-ray fluoroscopic image data. can do. As a result, the display control function F8 can display the superimposed image data as the superimposed image on the display 45 of the X-ray photographing apparatus 40.

FIG. 10 is a diagram showing an example of a superimposed image based on superimposed image data in which CT image data including blood vessels is superimposed on X-ray fluoroscopic image data.

The superimposed image shown in FIG. 10 is an X-ray fluoroscopic image IX obtained by X-ray imaging of the region including the periphery of the left atrium, and the catheter path to the mitral valve superimposed on the X-ray fluoroscopic image IX, that is, a blood vessel. Consists of including a CT image IC including. Even if the position of the left atrium is too close to the TEE probe 20 (image Q of the TEE probe) to image the area around the left atrium, the blood vessels drawn on the properly aligned CT image ( The image B) of the blood vessel allows an operator such as the operator D1 to visually recognize the path of the catheter around the left atrium.

According to the second modification, when the intervention treatment is advanced, the superimposed image data on which the aligned CT image data including the blood vessel is superimposed is displayed on the display 45, whereby the operator such as the operator D1 is displayed. Can easily grasp the path of the catheter.

(Third modification example)
In FIG. 4, the case where the medical image processing device 17 of the ultrasonic diagnostic apparatus 10 is provided with the functions F1 to F7 and the medical image processing device 47 of the X-ray imaging apparatus 40 is provided with the function F8 has been described, but is limited to that case. It's not something. In the medical image processing system 1, a plurality of devices can cooperate to generate superimposed image data. That is, in the medical image processing system 1, if the X-ray photographing apparatus 40 includes the function F8, the functions F1 to F7 may be provided in any of them. For example, the ultrasonic diagnostic apparatus 10 may have functions F1 to F5 and F7, and the X-ray imaging apparatus 40 may include functions F6 and F8. A case will be described with reference to FIG.

FIG. 11 is a block diagram showing a modified example of the medical image processing system 1 shown in FIG.

The processing circuit 171 of the ultrasonic diagnostic apparatus 10 reads and executes a computer program (for example, an image processing program) stored in a non-transient recording medium such as a memory in the main memory 172 or the processing circuit 171. As a result, the first image acquisition function F1, the second image acquisition function F2, the third image acquisition function F3, the first alignment function F4, the second alignment function F5, and the output control function F7 are realized. do.

As described above, the first image acquisition function F1 includes a function of acquiring ultrasonic image data as the first medical image data. As described above, the second image acquisition function F2 includes a function of acquiring X-ray image data as second medical image data from the X-ray imaging apparatus 40 that generates X-ray image data via the network N. .. As described above, the third image acquisition function F3 includes a function of acquiring third medical image data from the medical image providing device 50 via the network N.

As described above, the first alignment function F4 includes a function of aligning the X-ray image data acquired by the second image acquisition function F2 with the ultrasonic image data acquired by the first image acquisition function F1. .. As described above, the second alignment function F5 includes a function of aligning the CT image data acquired by the third image acquisition function F3 with the ultrasonic image data acquired by the first image acquisition function F1.

The output control function F7 includes the X-ray image data after alignment generated by the first alignment function F4 and the CT image data after alignment generated by the second alignment function F5 via the network N. Is included in the function of outputting the data to the external X-ray imaging apparatus 40.

The image generation function F6 of the X-ray imaging apparatus 40 uses the second alignment function F5 to align the X-ray image data based on the alignment result by the first alignment function F4 output from the output control function F7. It includes a function to generate superimposed image data on which CT image data based on the result is superimposed.

The display control function F8 includes a function of displaying the superimposed image data generated by the image generation function F6 on the display 45. By outputting the superimposed image data by the ultrasonic diagnostic apparatus 10 output control function F7, the display 45 of the X-ray imaging apparatus 40 can display the superimposed image data as a superimposed image in substantially real time.

Further, for example, the X-ray imaging apparatus 40 has the function F8, and the third medical image processing apparatus which is neither the ultrasonic diagnostic apparatus 10 nor the X-ray imaging apparatus 40, which is connected via the network N, has the functions F1 to F7. You may prepare. For example, the third medical image processing device is a workstation that performs various image processing on medical image data, a portable information processing terminal such as a tablet terminal, or the like.

According to the third modification, the above effect can be obtained without newly introducing the functions F1 to F7 into the existing devices 10 and 40.

According to at least one embodiment described above, it is possible to generate image data that can support an operator such as an operator in intervention treatment.

The first image acquisition function F1 is an example of the first image acquisition unit. The second image acquisition function F2 is an example of the second image acquisition unit. The third image acquisition function F3 is an example of the third image acquisition unit. The first alignment function F4 is an example of the first alignment unit. The second alignment function F5 is an example of the second alignment unit. The image generation function F6 is an example of an image generation unit. The output control function F7 is an example of an output control unit. The display control function F8 is an example of a display control unit.

Although some embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other embodiments, and various omissions, replacements, changes, combinations of embodiments, 1 or 1 or the same as the embodiments, as long as the gist of the invention is not deviated. It can be combined with a plurality of modification examples. These embodiments and variations thereof are included in the scope and gist of the invention, and are also included in the scope of the invention described in the claims and the equivalent scope thereof.

1 Medical image processing system 10 Ultrasonic diagnostic device 171,471 Processing circuit 40 X-ray imaging device 45 Display 50 Medical image providing device F1 First image acquisition function F2 Second image acquisition function F3 Third image acquisition function F4 First alignment Function F5 Second alignment function F6 Image generation function F7 Output control function F8 Display control function

Claims (15)

A first image acquisition unit that acquires an ultrasonic image of a subject as a first medical image,
A second image acquisition unit that acquires an X-ray image of the subject as a second medical image, and
A third image acquisition unit that acquires a third medical image of the subject, which is different from the ultrasonic image and the X-ray image.
A first alignment unit that aligns the X-ray image with the ultrasonic image,
A second alignment unit that aligns the third medical image with the ultrasonic image,
Based on the alignment result by the first alignment unit and the alignment result by the second alignment portion, the X-ray image and the third medical image are aligned, and the X-ray image is described as described above. An image generator that generates an image in which a third medical image is aligned,
Medical image processing equipment with. The second alignment unit is different from the method of aligning the X-ray image by the first alignment unit with the ultrasonic image, and is different from the method of aligning the X-ray image with the ultrasonic image of the third medical image. Align,
The medical image processing apparatus according to claim 1. The first alignment unit detects the position of the ultrasonic probe from the X-ray image to align the X-ray image with the ultrasonic image.
The second alignment unit aligns the third medical image with the ultrasonic image based on the characteristic portion depicted in the third medical image and the ultrasonic image.
The medical image processing apparatus according to claim 2. Further, it has a display control unit for displaying an image in which the third medical image is aligned with the X-ray image on the display unit.
The display control unit aligns the ultrasonic image used as a reference for alignment and further displays it on the display unit.
The medical image processing apparatus according to any one of claims 1 to 3. The display control unit displays an image in which the third medical image is aligned with the X-ray image based on the position of the catheter appearing in the X-ray image, and the display control unit displays the image with respect to the X-ray image. Switch to the display of an image in which the third medical image and the ultrasonic image are aligned.
The medical image processing apparatus according to claim 4. The third image acquisition unit acquires a CT image or an MRI image of the subject, which is different from the ultrasonic image and the X-ray image, as the third medical image.
The medical image processing apparatus according to any one of claims 1 to 5. The third image acquisition unit acquires an image including a predetermined anatomical feature as the third medical image.
The medical image processing apparatus according to any one of claims 1 to 6. The third image acquisition unit is an image including information indicating the path of the catheter as the third medical image.
The medical image processing apparatus according to any one of claims 1 to 6. A medical image processing system including an ultrasonic diagnostic device that generates an ultrasonic image and an X-ray imaging device that generates an X-ray image.
A first image acquisition unit that acquires an ultrasonic image of a subject as a first medical image,
A second image acquisition unit that acquires an X-ray image of the subject as a second medical image, and
A third image acquisition unit that acquires a third medical image of the subject, which is different from the ultrasonic image and the X-ray image.
A first alignment unit that aligns the X-ray image with the ultrasonic image,
A second alignment unit that aligns the third medical image with the ultrasonic image,
Based on the alignment result by the first alignment unit and the alignment result by the second alignment portion, the X-ray image and the third medical image are aligned, and the X-ray image is described as described above. An image generator that generates an image in which a third medical image is aligned,
Medical image processing system with. The first alignment unit is provided in the ultrasonic diagnostic apparatus.
The medical image processing system according to claim 9. The image generation unit is provided in the X-ray imaging apparatus.
The medical image processing system according to claim 9 or 10. The X-ray imaging apparatus has a display control unit for displaying an image in which the third medical image is aligned with the X-ray image on the display unit.
The display control unit aligns the ultrasonic image used as a reference for alignment and further displays it on the display unit.
The medical image processing system according to any one of claims 9 to 11. The X-ray imaging apparatus has a display control unit for displaying an image in which the third medical image is aligned with the X-ray image on the display unit.
The display control unit causes the display unit to display in parallel an image in which the third medical image is aligned with the X-ray image and the ultrasonic image used as a reference for the alignment.
The medical image processing system according to any one of claims 9 to 11. The X-ray imaging apparatus has a display control unit for displaying an image in which the third medical image is aligned with the X-ray image on the display unit.
The image generation unit positions the X-ray image and the third medical image for all consecutive frames based on the alignment result by the first alignment unit and the alignment result by the second alignment unit. Combined to generate a plurality of images in which the third medical image is aligned with respect to the X-ray images for all the consecutive frames.
The display control unit causes the display unit to display the plurality of images as moving images in substantially real time.
The medical image processing system according to any one of claims 9 to 13. The X-ray imaging apparatus has a display control unit for displaying an image in which the third medical image is aligned with the X-ray image on the display unit.
The image generation unit has an X-ray image relating to a predetermined frame among a plurality of consecutive frames and the third image based on the alignment result by the first alignment unit and the alignment result by the second alignment unit. Aligning with the medical image, an image in which the third medical image is aligned with the X-ray image relating to the predetermined frame is generated.
The display control unit uses the X-ray image as an element of the image in which the third medical image is aligned with respect to the X-ray image as a moving image, and the third medical image as the element as a still image. Display on the display
The medical image processing system according to any one of claims 9 to 13.

Images