JP6352013B2 - Ultrasonic diagnostic apparatus, image processing apparatus, and image processing program - Google Patents

Description

  Embodiments described herein relate generally to an ultrasonic diagnostic apparatus, an image processing apparatus, and an image processing program.

  Conventionally, for the purpose of assisting image diagnosis, a composite image obtained by combining a plurality of medical images subjected to alignment has been displayed. For example, such a composite image is generated by weighting and adding each medical image with a predetermined weight. Alternatively, for example, such a composite image is generated by superimposing a region designated by an operator on another medical image on a base medical image. By referring to such a composite image, for example, a doctor can obtain various information on the imaging region of the subject from one image.

  However, in the composite image generated by weighted addition, for example, an area in which information that a doctor wants to refer to in a certain medical image is merged with the same area of another medical image, so that the visibility of the area is improved. It may get worse. In such a case, for example, the doctor needs to change the weight for each medical image. Further, in the method of generating a composite image using an area designated by the operator, for example, a doctor needs to designate an area for each medical image.

JP 2006-53102 A JP 2008-284287 A

  The problem to be solved by the present invention is to provide an ultrasonic diagnostic apparatus, an image processing apparatus, and an image processing program capable of easily obtaining an image from which various pieces of information of an imaging region can be obtained.

The ultrasonic diagnostic apparatus according to the embodiment includes an extraction unit, an identification unit, a synthesis unit, and a control unit. The extraction unit extracts a specific region of the first image data that is one of the image data of the ultrasonic image data and the other types of medical image data that are aligned based on the alignment information that associates the spatial positions between the image data. and exits extracted, the position of the second corresponding region corresponding to the specific area from the second image data which is the other of the image data is extracted by the first image processing. The identification unit uses a second image process different from the first image process to position the corresponding area corresponding to the specific area in the second image data, which is the other image data, using the alignment information. Ask. The combining unit configured to change the corresponding area of the second image data based on the first image data in the specific area when the position of the corresponding area matches the position of the second corresponding area; When image data is generated and the position of the corresponding area does not match the position of the second corresponding area, the second corresponding area of the second image data is determined based on the first image data in the specific area. The modified composite image data is generated . The control unit causes the display unit to display the composite image data.

FIG. 1 is a block diagram illustrating a configuration example of an ultrasonic diagnostic apparatus according to the present embodiment. FIG. 2 is a diagram illustrating an example of the alignment process according to the present embodiment. FIG. 3 is a diagram showing an outline of image processing performed by the image processing unit according to the present embodiment. FIG. 4 is a diagram (1) illustrating a specific example of image processing according to the present embodiment. FIG. 5 is a diagram (2) illustrating a specific example of the image processing according to the present embodiment. FIG. 6 is a diagram (3) illustrating a specific example of the image processing according to the present embodiment. FIG. 7 is a diagram (4) illustrating a specific example of image processing according to the present embodiment. FIG. 8 is a diagram (1) for explaining an example of a display form by the control unit according to the present embodiment. FIG. 9 is a diagram (2) for explaining an example of a display form by the control unit according to the present embodiment. FIG. 10 is a flowchart for explaining image processing performed by the ultrasonic diagnostic apparatus according to the present embodiment. FIG. 11 is a diagram for explaining a modification according to the present embodiment.

  Hereinafter, embodiments of an ultrasonic diagnostic apparatus will be described in detail with reference to the accompanying drawings.

(Embodiment)
First, the configuration of the ultrasonic diagnostic apparatus according to the present embodiment will be described. FIG. 1 is a block diagram illustrating a configuration example of an ultrasonic diagnostic apparatus according to the present embodiment. As illustrated in FIG. 1, the ultrasonic diagnostic apparatus according to the present embodiment includes an ultrasonic probe 1, a monitor 2, an input device 3, a position sensor 4, a transmitter 5, and an apparatus main body 10. The apparatus main body 10 is connected to the external apparatus 6 via the network 100.

  The ultrasonic probe 1 has a plurality of transducers, and the plurality of transducers generate ultrasonic waves based on a drive signal supplied from a transmission / reception unit 11 included in the apparatus main body 10 to be described later. The vibrator included in the ultrasonic probe 1 is, for example, a piezoelectric vibrator. The ultrasonic probe 1 receives a reflected wave signal from the subject P and converts it into an electrical signal. The ultrasonic probe 1 includes a matching layer provided in the piezoelectric vibrator, a backing material that prevents propagation of ultrasonic waves from the piezoelectric vibrator to the rear, and the like. The ultrasonic probe 1 is detachably connected to the apparatus main body 10.

  When ultrasonic waves are transmitted from the ultrasonic probe 1 to the subject P, the transmitted ultrasonic waves are reflected one after another at the discontinuous surface of the acoustic impedance in the body tissue of the subject P, and the ultrasonic probe is used as a reflected wave signal. 1 is received by a plurality of piezoelectric vibrators. The amplitude of the received reflected wave signal depends on the difference in acoustic impedance at the discontinuous surface where the ultrasonic wave is reflected. Note that the reflected wave signal when the transmitted ultrasonic pulse is reflected by the moving blood flow or the surface of the heart wall depends on the velocity component of the moving object in the ultrasonic transmission direction due to the Doppler effect. And undergoes a frequency shift.

  For example, in the present embodiment, a 1D array probe in which a plurality of piezoelectric vibrators are arranged in a row is connected to the apparatus main body 10 as an ultrasonic probe 1 for two-dimensional scanning of the subject P. For example, the 1D array probe as the ultrasonic probe 1 is a sector probe that performs sector scanning, a convex probe that performs offset sector scanning, a linear probe that performs linear scanning, or the like.

  Alternatively, for example, in this embodiment, a mechanical 4D probe or a 2D array probe may be connected to the apparatus main body 10 as the ultrasonic probe 1 for three-dimensional scanning of the subject P. The mechanical 4D probe is capable of two-dimensional scanning using a plurality of piezoelectric vibrators arranged in a line like a 1D array probe, and swings the plurality of piezoelectric vibrators at a predetermined angle (swing angle). By doing so, three-dimensional scanning is possible. The 2D array probe can be three-dimensionally scanned by a plurality of piezoelectric vibrators arranged in a matrix and can be two-dimensionally scanned by focusing and transmitting ultrasonic waves.

  The position sensor 4 and the transmitter 5 are devices for acquiring position information of the ultrasonic probe 1. For example, the position sensor 4 is a magnetic sensor attached to the ultrasonic probe 1. Further, for example, the transmitter 5 is a device that is arranged at an arbitrary position and that forms a magnetic field outward from the device itself.

  The position sensor 4 detects a three-dimensional magnetic field formed by the transmitter 5. Then, the position sensor 4 calculates the position (coordinates and angle) of the device itself in the space with the transmitter 5 as the origin, based on the detected magnetic field information, and transmits the calculated position to the device body 10. Here, the position sensor 4 transmits the three-dimensional coordinates and angle at which the device itself is located to the apparatus body 10 as the three-dimensional position information of the ultrasonic probe 1.

  Note that the present embodiment is applicable even when the position information of the ultrasonic probe 1 is acquired by a system other than the position detection system using the position sensor 4 and the transmitter 5. For example, this embodiment may be a case where the position information of the ultrasonic probe 1 is acquired using a gyro sensor, an acceleration sensor, or the like.

  The input device 3 is connected to the device main body 10 via an interface unit 19 described later. The input device 3 includes a mouse, a keyboard, a button, a panel switch, a touch command screen, a foot switch, a trackball, and the like. The input device 3 accepts various setting requests from the operator of the ultrasonic diagnostic apparatus and transfers the accepted various setting requests to the apparatus main body 10.

  The monitor 2 displays a GUI (Graphical User Interface) for an operator of the ultrasonic diagnostic apparatus to input various setting requests using the input device 3, and displays ultrasonic image data generated in the apparatus main body 10. Or display.

  The external device 6 is a device connected to the device main body 10 via an interface unit 19 described later. For example, the external device 6 is a database of a PACS (Picture Archiving and Communication System) that is a system that manages data of various medical images, a database of an electronic medical record system that manages an electronic medical record to which medical images are attached, and the like. . Alternatively, the external apparatus 6 is various medical image diagnostic apparatuses other than the ultrasonic diagnostic apparatus shown in FIG. 1, such as an X-ray CT (Computed Tomography) apparatus and an MRI (Magnetic Resonance Imaging) apparatus.

  For example, the apparatus main body 10 shown in FIG. 1 can acquire various medical image data unified in an image format conforming to DICOM (Digital Imaging and Communications in Medicine) from the external apparatus 6 via the interface unit 19. it can. For example, the apparatus main body 10 receives various types of medical image data (for example, two-dimensional or three-dimensional X-ray CT image data, to be combined with ultrasonic image data generated by the own apparatus via an interface unit 19 described later. 2D or 3D MRI image data) is acquired from the external device 6.

  The apparatus main body 10 is an apparatus that generates ultrasonic image data based on a reflected wave signal received by the ultrasonic probe 1. The apparatus main body 10 shown in FIG. 1 can generate two-dimensional ultrasound image data based on a two-dimensional reflected wave signal, and can generate three-dimensional ultrasound image data based on a three-dimensional reflected wave signal. Device. However, this embodiment is applicable even when the apparatus main body 10 is an apparatus dedicated to two-dimensional data.

  As shown in FIG. 1, the apparatus body 10 includes a transmission / reception unit 11, a B-mode processing unit 12, a Doppler processing unit 13, an image generation unit 14, an image memory 15, an internal storage unit 16, and an image processing unit. 17, a control unit 18, and an interface unit 19.

  The transmission / reception unit 11 controls ultrasonic transmission / reception performed by the ultrasonic probe 1 based on an instruction from the control unit 18 described later. The transmission / reception unit 11 includes a pulse generator, a transmission delay unit, a pulser, and the like, and supplies a drive signal to the ultrasonic probe 1. The pulse generator repeatedly generates rate pulses for forming transmission ultrasonic waves at a predetermined rate frequency. The transmission delay unit generates a delay time for each piezoelectric vibrator necessary for focusing the ultrasonic wave generated from the ultrasonic probe 1 into a beam and determining transmission directivity. Give for each rate pulse. The pulser applies a drive signal (drive pulse) to the ultrasonic probe 1 at a timing based on the rate pulse. The transmission delay unit arbitrarily adjusts the transmission direction of the ultrasonic wave transmitted from the piezoelectric vibrator surface by changing the delay time given to each rate pulse.

  The transmission / reception unit 11 has a function capable of instantaneously changing a transmission frequency, a transmission drive voltage, and the like in order to execute a predetermined scan sequence based on an instruction from the control unit 18 described later. In particular, the change of the transmission drive voltage is realized by a linear amplifier type transmission circuit capable of instantaneously switching the value or a mechanism for electrically switching a plurality of power supply units.

  The transmission / reception unit 11 includes a preamplifier, an A / D (Analog / Digital) converter, a reception delay unit, an adder, and the like. The transmission / reception unit 11 performs various processing on the reflected wave signal received by the ultrasonic probe 1 and reflects it. Generate wave data. The preamplifier amplifies the reflected wave signal for each channel. The A / D converter A / D converts the amplified reflected wave signal. The reception delay unit gives a delay time necessary for determining the reception directivity. The adder performs an addition process on the reflected wave signal processed by the reception delay unit to generate reflected wave data. By the addition processing of the adder, the reflection component from the direction corresponding to the reception directivity of the reflected wave signal is emphasized, and a comprehensive beam for ultrasonic transmission / reception is formed by the reception directivity and the transmission directivity.

  The transmitter / receiver 11 transmits a two-dimensional ultrasonic beam from the ultrasonic probe 1 when the subject P is two-dimensionally scanned. Then, the transmission / reception unit 11 generates two-dimensional reflected wave data from the two-dimensional reflected wave signal received by the ultrasonic probe 1. Further, when the subject P is three-dimensionally scanned, the transmission / reception unit 11 transmits a three-dimensional ultrasonic beam from the ultrasonic probe 1. Then, the transmission / reception unit 11 generates three-dimensional reflected wave data from the three-dimensional reflected wave signal received by the ultrasonic probe 1.

  The form of the output signal from the transmission / reception unit 11 can be selected from various forms such as a signal including phase information called an RF (Radio Frequency) signal or amplitude information after envelope detection processing. Is possible.

  The B-mode processing unit 12 and the Doppler processing unit 13 are signal processing units that perform various types of signal processing on the reflected wave data generated from the reflected wave signal by the transmission / reception unit 11. The B-mode processing unit 12 receives the reflected wave data from the transmission / reception unit 11, performs logarithmic amplification, envelope detection processing, and the like, and generates data (B-mode data) in which the signal intensity is expressed by brightness. . Further, the Doppler processing unit 13 performs frequency analysis on velocity information from the reflected wave data received from the transmission / reception unit 11 and extracts data (Doppler data) obtained by extracting moving body information such as velocity, dispersion, and power due to the Doppler effect at multiple points. Generate. Here, the moving body is, for example, a blood flow, a tissue such as a heart wall, or a contrast agent. When blood flow is targeted as a moving body, the moving body information becomes blood flow information indicating information such as speed, dispersion, and power at each sample point in a region where blood flow exists.

  Note that the B-mode processing unit 12 and the Doppler processing unit 13 illustrated in FIG. 1 can process both two-dimensional reflected wave data and three-dimensional reflected wave data. That is, the B-mode processing unit 12 generates two-dimensional B-mode data from the two-dimensional reflected wave data, and generates three-dimensional B-mode data from the three-dimensional reflected wave data. The Doppler processing unit 13 generates two-dimensional Doppler data from the two-dimensional reflected wave data, and generates three-dimensional Doppler data from the three-dimensional reflected wave data.

  The image generation unit 14 generates ultrasonic image data from the data generated by the B mode processing unit 12 and the Doppler processing unit 13. That is, the image generation unit 14 generates two-dimensional B-mode image data in which the intensity of the reflected wave is expressed by luminance from the two-dimensional B-mode data generated by the B-mode processing unit 12. Further, the image generation unit 14 generates two-dimensional Doppler image data representing moving body information from the two-dimensional Doppler data generated by the Doppler processing unit 13. The two-dimensional Doppler image data is velocity image data, distributed image data, power image data, or image data obtained by combining these.

  Here, the image generation unit 14 generally converts (scan converts) a scanning line signal sequence of ultrasonic scanning into a scanning line signal sequence of a video format represented by a television or the like, and displays ultrasonic waves for display. Generate image data. Specifically, the image generation unit 14 generates ultrasonic image data for display by performing coordinate conversion in accordance with the ultrasonic scanning mode of the ultrasonic probe 1. In addition to the scan conversion, the image generation unit 14 performs various image processing, such as image processing (smoothing processing) for regenerating an average luminance image using a plurality of image frames after scan conversion, Image processing (edge enhancement processing) using a differential filter is performed in the image. Further, the image generation unit 14 synthesizes incidental information (character information of various parameters, scales, body marks, etc.) with the ultrasonic image data.

  That is, the B-mode data and Doppler data are ultrasonic image data before the scan conversion process, and the data generated by the image generation unit 14 is display ultrasonic image data after the scan conversion process. The B-mode data and the Doppler data are also called raw data (Raw Data). The image generation unit 14 generates “two-dimensional B-mode” that is two-dimensional ultrasonic image data for display from “two-dimensional B-mode data or two-dimensional Doppler data” that is two-dimensional ultrasonic image data before scan conversion processing. Image data and 2D Doppler image data "are generated.

  Further, the image generation unit 14 generates three-dimensional B-mode image data by performing coordinate conversion on the three-dimensional B-mode data generated by the B-mode processing unit 12. Further, the image generation unit 14 generates three-dimensional Doppler image data by performing coordinate conversion on the three-dimensional Doppler data generated by the Doppler processing unit 13. The image generation unit 14 generates “3D B-mode image data or 3D Doppler image data” as “3D ultrasound image data (ultrasound volume data)”.

  Further, the image generation unit 14 performs a rendering process on the volume data in order to generate various two-dimensional image data for displaying the volume data on the monitor 2. The rendering process performed by the image generation unit 14 includes, for example, a process of generating MPR image data from volume data by performing a multi-planar reconstruction (MPR). The rendering processing performed by the image generation unit 14 includes, for example, volume rendering (VR) processing that generates two-dimensional image data reflecting three-dimensional information.

  Further, the image generation unit 14 can perform the above-described rendering processing on volume data captured by another medical image diagnostic apparatus. Such volume data includes three-dimensional X-ray CT image data (hereinafter referred to as X-ray CT volume data) captured by an X-ray CT apparatus, and three-dimensional MRI image data (hereinafter referred to as MRI volume data) captured by an MRI apparatus. ).

  Note that the ultrasound diagnostic apparatus according to the present embodiment can generate and display ultrasound image data that can provide various information in various modes other than the B mode and the Doppler mode described above. For example, the ultrasonic diagnostic apparatus according to the present embodiment performs elastography imaging. In the elast mode, the image generating unit 14 images image data (elastic image) obtained by imaging an index (elastic modulus) indicating tissue hardness from reflected wave data (reflected wave signal) subjected to signal processing by the Doppler processing unit 13. Data). For example, in the elast mode, the operator presses and releases the living tissue with the transducer surface of the ultrasonic probe 1 performing ultrasonic transmission / reception. Thereby, the tissue is deformed and movement occurs in the tissue. Information regarding the movement of the tissue appears as a phase shift of the reflected wave signal (reflected wave data).

  In the elast mode, for example, the Doppler processing unit 13 calculates speed information from the phase shift of the reflected wave data, and measures a displacement that is a time integration of the speed information. Then, the Doppler processing unit 13 calculates strain (strain) by spatially differentiating the displacement. As a method for measuring the displacement, there are “autocorrelation method”, “cross-correlation method”, “composite autocorrelation method”, “zero phase method” and the like. Since a hard biological tissue is less likely to be deformed, the strain value of the hard biological tissue becomes smaller and the strain value of the soft biological tissue becomes larger. That is, the strain value is an index value indicating the hardness (elastic modulus) of the tissue. The image generation unit 14 generates elastic image data whose color tone is changed according to the magnitude of the multi-point strain value calculated by the Doppler processing unit 13. Note that the image generation unit 14 may generate, as elastic image data, image data in which pixels whose color tone is changed are superimposed on B-mode image data according to the magnitude of the strain value.

  In the elast mode, in addition to the method of deforming the tissue by compression and release by the ultrasonic probe 1, the tissue is deformed by “Push Pulse” of high sound pressure transmitted from the ultrasonic probe 1, and a transverse wave propagating through the tissue is used. There is also a method for forming a certain shear wave and evaluating tissue elasticity based on the propagation speed of the shear wave. In the elast mode, in addition to a method of forming a shear wave and a method of manually compressing and releasing the tissue, for example, the elastic modulus is calculated by detecting tissue deformation caused by pulsation or movement of the diaphragm. There is also a method. The elastic image data may be generated from reflected wave data (reflected wave signal) that has been subjected to signal processing by the B-mode processing unit 12. For example, the elasticity image data is a multi-point displacement from the result of tracking a plurality of tracking points by speckle tracking processing using a plurality of B-mode image data (or B-mode data) generated along a time series. It can also be generated by calculating the distortion.

  In addition, for example, the ultrasonic diagnostic apparatus according to the present embodiment performs harmonic imaging such as contrast harmonic imaging (CHI) and tissue harmonic imaging (THI). For example, the B-mode processing unit 12 shown in FIG. 1 can change the frequency band to be visualized by changing the detection frequency by filtering. The B-mode processing unit 12 generates reflected wave data (harmonic data) of harmonic components using a contrast agent (microbubbles, bubbles) as a reflection source from the reflected wave data of the subject P into which the contrast agent has been injected, and the subject. The reflected wave data (fundamental wave data) of the fundamental wave component using the tissue in P as a reflection source can be separated. In the CHI mode, the B-mode processing unit 12 can generate B-mode data for generating contrast ultrasound image data from the reflected wave data of the harmonic component. In addition, by using the filter processing function of the B-mode processing unit 12, harmonic data that is reflected wave data of harmonic components can be separated from reflected wave data of the subject P in the THI mode. The B-mode processing unit 12 can generate B-mode data for generating ultrasonic image data in which noise components such as side lobe components are removed from the reflected wave data of harmonic components.

  Further, when performing the CHI mode or the THI mode, the B-mode processing unit 12 can extract the harmonic component by a method different from the method using the filter processing described above. In harmonic imaging, an imaging method called an AMPM method combining an amplitude modulation (AM) method, a phase modulation (PM) method, an AM method, and a PM method is performed. In the AM method, PM method, and AMPM method, ultrasonic transmission with different amplitudes and phases is performed a plurality of times for the same scanning line. Thereby, the transmission / reception unit 11 generates and outputs a plurality of reflected wave data in each scanning line. Then, the B-mode processing unit 12 extracts a harmonic component by performing addition / subtraction processing on the plurality of reflected wave data of each scanning line according to the modulation method. Then, the B-mode processing unit 12 performs envelope detection processing or the like on the reflected wave data of the harmonic component, and generates B-mode data.

  For example, when the PM method is performed, the transmission / reception unit 11 scans each scan with ultrasonic waves having the same amplitude with the phase polarity reversed, for example (−1, 1), according to the scan sequence set by the control unit 18. Send twice on the line. Then, the transmission / reception unit 11 generates a reception signal by transmission of “−1” and a reception signal by transmission of “1”, and the B-mode processing unit 12 adds these two reception signals. Thereby, the fundamental wave component is removed, and a signal in which the second harmonic component mainly remains is generated. The B-mode processing unit 12 performs envelope detection processing or the like on the signal to generate THI B-mode data or CHI B-mode data. The B-mode data generated in the various CHI modes described above is an ultrasonic image obtained by imaging the region where the contrast agent (that is, blood flow) flowing in the imaging region of the subject P exists by the processing of the image generation unit 14. It becomes contrast image data.

  The various modes described above are merely examples, and the ultrasonic diagnostic apparatus according to the present embodiment can execute other two-dimensional imaging or three-dimensional imaging processes in various known modes.

  The image memory 15 is a memory for storing image data for display generated by the image generation unit 14. The image memory 15 can also store data generated by the B-mode processing unit 12 and the Doppler processing unit 13. The B-mode data and Doppler data stored in the image memory 15 can be called by an operator after diagnosis, for example, and become ultrasonic image data for display via the image generation unit 14. The image memory 15 also stores image data generated by an image processing unit 17 described later.

  The internal storage unit 16 stores a control program for performing ultrasonic transmission / reception, image processing and display processing, diagnostic information (for example, patient ID, doctor's findings, etc.), various data such as a diagnostic protocol and various body marks. To do. The internal storage unit 16 is also used for storing image data stored in the image memory 15 as necessary. In addition, data stored in the internal storage unit 16 can be transferred to the external device 6 via an interface unit 19 described later.

  The image processing unit 17 is installed for image diagnosis support. As illustrated in FIG. 1, the image processing unit 17 according to the present embodiment includes an extraction unit 171, an identification unit 172, and a synthesis unit 173, and includes ultrasonic image data generated by the image generation unit 14 and an external device 6. Image processing using the acquired other kinds of medical image data is performed. Specifically, the image processing unit 17 according to the present embodiment uses ultrasonic image data and other types of medical image data that are aligned based on alignment information that associates spatial positions between image data. Perform synthesis processing. The processing executed by the image processing unit 17 will be described in detail later.

  The control unit 18 controls the entire processing of the ultrasonic diagnostic apparatus. Specifically, the control unit 18 is based on various setting information input from the operator via the input device 3, various control programs and various data read from the internal storage unit 16, and the transmission / reception unit 11, B mode Controls the processing of the processing unit 12, Doppler processing unit 13, image generation unit 14, and image processing unit 17. Further, the control unit 18 controls the monitor 2 to display the display image data stored in the image memory 15 or the internal storage unit 16. Further, the control unit 18 receives medical image data received from the operator via the input device 3 from the external device 6 via the network 100 and the interface unit 19, the internal storage unit 16, the image memory 15, or the image processing unit 17. To be transferred to.

  The interface unit 19 is an interface for the input device 3, the network 100, and the external device 6. Various setting information and various instructions from the operator received by the input device 3 are transferred to the control unit 18 by the interface unit 19. For example, a transfer request for image data received by the input device 3 from the operator is notified to the external device 6 via the network 100 by the interface unit 19. The image data transferred by the external device 6 is stored in the internal storage unit 16, the image memory 15, or the image processing unit 17 by the interface unit 19.

  The overall configuration of the ultrasonic diagnostic apparatus according to this embodiment has been described above. In such a configuration, the ultrasonic diagnostic apparatus according to the present embodiment performs processing for easily obtaining an image from which various pieces of information on the imaging region can be obtained by the image processing unit 17 described below.

  Here, as described above, the image processing unit 17 according to the present embodiment receives the ultrasonic image data and other types of medical image data that are aligned based on the alignment information that associates the spatial positions between the image data. The used image composition processing is performed. Hereinafter, an example of the alignment process for associating the spatial positions between the image data will be described with reference to FIG. FIG. 2 is a diagram illustrating an example of the alignment process according to the present embodiment. In the present embodiment, as an example, alignment by which a spatial position between image data is associated by the processing of the control unit 18 using the position detection system including the position sensor 4 and the transmitter 5 illustrated in FIGS. 1 and 2. Is done.

  For example, the operator obtains X-ray CT volume data 101 (see FIG. 2) obtained by imaging an imaging region (examination region) of the subject P before performing an ultrasound examination of the subject P using the ultrasound probe 1. A transfer request is made. The X-ray CT volume data 101 shown in FIG. 2 is, for example, data obtained by imaging a three-dimensional region including the imaging region of the subject P, and is composed of, for example, 500 axial X-ray CT image data. Under the control of the control unit 18, the monitor 2 displays one of the X-ray CT image data of the plurality of axial surfaces. Then, the operator presses the set button of the input device 3 by bringing the ultrasonic probe 1 into contact with the body surface of the subject P in the vertical direction so as to scan the axial surface of the imaging region of the subject P. The control unit 18 acquires, as initial position information, the three-dimensional position information of the ultrasonic probe 1 acquired from the position sensor 4 when the button is pressed. As shown in FIG. 2, the monitor 2 displays the ultrasonic image data 200 generated by the image generation unit 14 when the set button is pressed.

  Then, the operator operates, for example, the mouse of the input device 3 so that X-ray CT image data having substantially the same cross section as the ultrasound image data 200 is displayed on the monitor 2. Then, when the ultrasound image data 200 and the X-ray CT image data 102 having substantially the same cross section (same axial plane) are displayed, the operator presses the set button again. Thereby, the control unit 18 acquires the position of the substantially same cross section as the scanning cross section of the ultrasonic probe 1 indicated by the initial position information in the X-ray CT volume data 101. At this stage, even when the position of the ultrasonic probe 1 is moved by the operator, the control unit 18 determines the scanning cross section of the ultrasonic probe 1 from the three-dimensional position information of the ultrasonic probe 1 acquired from the position sensor 4. The same cross section can be specified in the X-ray CT volume data 101.

  Further, the operator designates at least one set of feature points corresponding to each of the X-ray CT image data 102 and the ultrasonic image data 200. The control unit 18 uses the position of the feature point specified by the X-ray CT image data 102 and the position of the feature point specified by the ultrasound image data 200 to determine the coordinates of each point of the ultrasound image data 200. Then, a conversion matrix for converting into the coordinates of each corresponding point in the X-ray CT image data 102 (X-ray CT volume data 101) is calculated. Thereby, even when the position of the ultrasonic probe 1 is moved by the operator, the control unit 18 uses the same three-dimensional position information of the ultrasonic probe 1 acquired from the position sensor 4 as the scanning section of the ultrasonic probe 1. A cross section is specified by the X-ray CT volume data 101, and further, coordinates corresponding to each coordinate of ultrasonic image data generated by scanning the ultrasonic probe 1 within the specified cross section are calculated using a transformation matrix. be able to.

  The control unit 18 notifies the image generation unit 14 of the alignment information acquired using the transformation matrix. The image generation unit 14 generates, from the X-ray CT volume data 101, X-ray CT image data (MPR image data) aligned with the ultrasonic image data scanned and generated at this time using the alignment information. To do. Note that the above alignment process can be applied even when three-dimensional scanning is performed by the ultrasonic probe 1. In such a case, the control unit 18 substantially matches the ultrasonic volume data scanned and generated at the present time. A three-dimensional region is specified from the X-ray CT volume data 101, and alignment information indicating which position in the specified three-dimensional region corresponds to which position in the ultrasonic volume data is acquired. Further, the above alignment processing can be applied even if the other type of medical image data is MRI image data (MRI volume data).

  However, in the present embodiment, the alignment processing based on the alignment information that associates the spatial positions between the medical image data generated by different modalities includes cross-correlation, autocorrelation, mutual information amount, standardized mutual information amount, correlation It may be performed by a known technique using a ratio or the like.

  The ultrasonic image data and other types of medical image data that are aligned based on the alignment information that associates the spatial position with the above processing are transferred to the image processing unit 17. Perform the following process. Hereinafter, the outline of the image processing performed by the image processing unit 17 according to the present embodiment will be described with reference to FIG. FIG. 3 is a diagram showing an outline of image processing performed by the image processing unit according to the present embodiment.

  As shown in the upper left diagram of FIG. 3, the extraction unit 171 is one of image data of ultrasonic image data and other types of medical image data that are aligned based on alignment information that associates spatial positions between image data. The specific area of the first image data is extracted by image processing. Specifically, the extraction unit 171 extracts, from the first image data, an area where information that cannot be obtained from the second image data, which is the other image data, is obtained as a specific area. More specifically, the extraction unit 171 extracts a specific region from the first image data by image processing based on information set by the operator. For example, the extraction unit 171 extracts a specific region from the first image data by a segmentation process based on information on the subject region set by the operator.

  And the identification part 172 identifies the corresponding area | region corresponding to a specific area | region in 2nd image data using position alignment information (conversion matrix), as shown in the lower left figure of FIG. That is, the identification unit 172 obtains the position of the corresponding area corresponding to the specific area in the second image data using the alignment information (conversion matrix). The specific area of the first image data is an area where information that cannot be obtained from the corresponding area of the second image data is obtained, or an area where information supplementing information obtained from the corresponding area of the second image data is obtained.

  Then, the composition unit 173 generates composite image data in which the corresponding area of the second image data is changed based on the first image data in the specific area. For example, the composition unit 173 generates composite image data in which the corresponding region of the second image data is replaced with a specific region as illustrated in the right diagram of FIG. As shown in the right diagram of FIG. 3, the composite image data is image data in which the corresponding area is replaced with a specific area based on the second image data. The synthesizing unit 173 performs the synthesizing process based on the alignment information (conversion matrix) to enlarge or reduce the specific area as a whole to make the size of the corresponding area. Alternatively, the synthesizing unit 173 performs the synthesizing process based on the alignment information (conversion matrix) by partially enlarging or reducing the specific area to the size of the corresponding area.

  Then, the control unit 18 causes the monitor 2 to display the composite image data. An operator such as a doctor obtains information that cannot be obtained in the corresponding area from the specified specific area together with information on the area other than the corresponding area of the second image data by referring to the single composite image data. be able to.

  Various specific examples of the image processing shown in the above outline will be described with reference to FIGS. 4 to 7 are diagrams illustrating specific examples of image processing according to the present embodiment. In the following, a case will be described in which the other-type medical image data that is aligned in association with the spatial position of the ultrasound image data is X-ray CT image data. However, the specific examples described below are applicable even when the other types of medical image data are, for example, MRI image data aligned with the ultrasound image data. In a specific example described below, segmentation processing based on information on a subject part set by an operator is performed as image processing in which the extraction unit 171 extracts a specific region.

  In the example shown in FIG. 4 (hereinafter, referred to as a first pattern), the extraction unit 171 uses other regions of the medical image as a specific region that corresponds to a region that is not visualized by ultrasound image data due to a substance that does not propagate ultrasound. Extract from the data. Here, the substance that does not propagate ultrasonic waves is bone in the living body or air such as intestinal gas. Since the ultrasonic wave transmitted under normal imaging conditions is substantially totally reflected by the bone and air, it does not propagate to a position deeper than the bone or air. As a result, in the ultrasonic image data (B-mode image data), as shown in the upper right diagram of FIG. 4, for example, the region at a position deeper than the bone becomes dark with a shape corresponding to the transmission direction of the ultrasonic beam. On the other hand, for example, by irradiating X-rays around the entire subject P, X-ray CT image data reconstructed by collecting projection data of a plurality of views clearly shows bones and tissues around the bones. It is depicted (see the upper left figure in Fig. 4).

  The operator inputs setting information such as “ultrasonic image data: bone” with reference to the aligned ultrasonic image data (B-mode image data) and X-ray CT image data. Then, when the operator instructs the start of the synthesis process, the control unit 18 determines that the first image data is X-ray CT image data as shown in FIG. Is ultrasonic image data. Then, the extraction unit 171 performs extraction of a bone region corresponding to the CT value of the bone, for example, by segmentation processing by a region expansion method in accordance with an instruction from the control unit 18. Then, the extraction unit 171 identifies a line (curve) located at the top of the image in the bone region. Then, the specific area surrounded by the white line is extracted from the X-ray CT image data, which is the first image data, as shown in the upper left diagram of FIG. 4 from the specified curve and the ultrasonic transmission direction. To do. That is, the extraction unit 171 extracts a region located deep from the bone region as a specific region from the X-ray CT image data as the other types of medical image data.

  Then, the identification unit 172 obtains the position of the corresponding region in the ultrasonic image data that is the second image data by using the alignment information (conversion matrix) from the position of the specific region extracted by the extraction unit 171. As shown in the upper right diagram of FIG. Then, as illustrated in the lower diagram of FIG. 4, the synthesis unit 173 generates synthesized image data in which the corresponding area is replaced with a specific area of the X-ray CT image data based on the ultrasound image data. By referring to such composite image data, a doctor can visually recognize a region that is not depicted by the ultrasound image data by the specific region. When “ultrasonic image data: air” is set, the extraction unit 171 extracts a region at a deep position from the air region as a specific region from X-ray CT image data as other types of medical image data.

  Next, in the example shown in FIG. 5 (hereinafter, the second pattern), the ultrasound image data is Doppler image data, that is, ultrasound image data in which blood flow information based on the Doppler effect is depicted. Further, as shown in FIG. 5, in the second pattern, the X-ray CT image data as the other-type medical image data is non-contrast X-ray CT image data obtained by non-contrast imaging. In the second pattern, the extraction unit 171 extracts, from the ultrasound image data (Doppler image data), a region corresponding to the luminal region of other types of medical image data in the region where blood flow information exists as a specific region. . That is, in the non-contrast X-ray CT image data, the tissue structure is clearly depicted, but the inside of the blood vessel (lumen region) where the blood flow exists is depicted darkly as shown in the right diagram of FIG. Yes. It is difficult for the operator to easily determine whether such a dark region is a region where blood flow exists. On the other hand, as shown in the left diagram of FIG. 5, the blood flow information described above is given to the Doppler image data in a range where there is a blood flow in which a Doppler frequency shift occurs. Such blood flow information not only indicates that blood flow exists, but also includes information such as blood flow velocity, dispersion, and power.

  The operator inputs setting information such as “non-contrast X-ray CT image data: lumen” with reference to the aligned ultrasonic image data (Doppler image data) and non-contrast X-ray CT image data. When the operator instructs the start of the synthesis process, the control unit 18 determines that the first image data is Doppler image data as shown in FIG. The contrast X-ray CT image data is determined. Then, the extraction unit 171 specifies all pixels for which blood flow information is obtained from the Doppler image data according to an instruction from the control unit 18. Then, for example, the extraction unit 171 obtains, by binarization processing, a region that becomes a tubular region among the regions occupied by all the specified pixels, and extracts the obtained region as the specific region (see the left diagram in FIG. 5). reference).

  Then, the identification unit 172 uses the alignment information (conversion matrix) from the position of the specific region extracted by the extraction unit 171 to generate the second image data from the non-contrast X-ray CT image data in the right diagram of FIG. As shown, the corresponding region is identified. Thereby, the composition unit 173 generates composite image data in which the corresponding region is replaced with the specific region of the Doppler image data based on the non-contrast X-ray CT image data.

  In the second pattern, the extraction unit 171 identifies a region where blood flow information exists (for example, a region obtained by smoothing the region occupied by all pixels from which blood flow information is obtained by isolated point removal or the like). The area may be used. By referring to such composite image data, a doctor can visually recognize blood flow information that is not depicted by non-contrast X-ray CT image data in a specific region. In the above example, the doctor can obtain blood flow information only by performing Doppler mode ultrasound examination without performing contrast examination in X-ray CT examination. In the above example, the extraction unit 171 extracts the lumen region (corresponding region) of the non-contrast X-ray CT image data, and the identification unit 172 identifies the specific region of the Doppler image data using the alignment information. Also good.

  Next, in the example shown in FIG. 6, three types of patterns are shown. That is, in the example shown in FIG. 6, the third pattern “first image data: ultrasound image data (contrast), second image data: X-ray CT image data (non-contrast)”, and “first image data : X-ray CT image data (contrast), second image data: ultrasonic image data (non-contrast) "," first image data: ultrasonic image data (elastography), second image data : X-ray CT image data ”.

  The third pattern shown in FIG. 6 is a pattern that can reduce the number of contrast examinations. That is, in the third pattern, the ultrasound image data is contrast-enhanced ultrasound image data obtained by contrast imaging, and medical image data (X-ray CT image data) obtained by non-contrast imaging is obtained. Non-contrast X-ray CT image data). In such a case, for example, the operator inputs setting information such as “non-contrast X-ray CT image data: lumen” or “non-contrast X-ray CT image data: blood flow”. When the operator instructs the start of the synthesis process, the control unit 18 determines that the first image data is contrast ultrasound image data, and the second image data serving as a synthesis base is a non-contrast X-ray CT image. Determine that the data. Then, the extraction unit 171 specifies a region corresponding to the lumen region of other types of medical image data (non-contrast X-ray CT image data) in the region where the contrast agent exists or in the region where the contrast agent exists. As extracted from contrast-enhanced ultrasound image data.

  When “non-contrast X-ray CT image data: lumen” is the setting information, for example, the extraction unit 171 specifies all pixels having a luminance equal to or higher than a predetermined threshold value in the contrast ultrasound image data. Then, the extraction unit 171 obtains, for example, a region that becomes a tubular region from among the regions occupied by all the specified pixels, and extracts the obtained region as the specific region. Alternatively, in the case where “non-contrast X-ray CT image data: blood flow” is the setting information, for example, the extraction unit 171 performs smoothing processing on the region occupied by all pixels having luminance equal to or higher than a predetermined threshold in the contrast ultrasound image data. The selected area is extracted as a specific area.

  The fourth pattern shown in FIG. 6 is also a pattern that can reduce the number of contrast examinations. However, in the fourth pattern, ultrasound image data is ultrasound image data (normal B-mode image data) obtained by non-contrast imaging, and other types of medical image data (X-ray CT image data) are obtained by contrast imaging. It is the obtained medical image data (contrast X-ray CT image data). In this case, for example, the operator inputs setting information such as “non-contrast B-mode image data: lumen” or “non-contrast B-mode image data: blood flow”. When the operator instructs the start of the synthesis process, the control unit 18 determines that the first image data is contrast X-ray CT image data, and the second image data serving as a base for the synthesis is a non-contrast B-mode image. Determine that the data. And the extraction part 171 sets the area | region applicable to the luminal area | region of ultrasonic image data (non-contrast B mode image data) in the area | region where a contrast agent exists, or the area | region where a contrast agent exists as a specific area | region. Extracted from other types of medical image data (contrast X-ray CT image data).

  When “non-contrast B-mode image data: lumen” is the setting information, for example, the extraction unit 171 specifies all pixels having luminance equal to or higher than a predetermined threshold in the contrast X-ray CT image data. Then, the extraction unit 171 obtains, for example, a region that becomes a tubular region from among the regions occupied by all the specified pixels, and extracts the obtained region as the specific region. Alternatively, in the case where “non-contrast B-mode image data: blood flow” is the setting information, for example, the extraction unit 171 performs smoothing processing on the region occupied by all pixels having luminance equal to or higher than a predetermined threshold in the contrast X-ray CT image data. The selected area is extracted as a specific area. In addition, in the above second pattern to fourth pattern, when it is desired to obtain information on not only blood flowing in the blood vessel but also blood perfused in the tissue, the operator executes a smoothing process such as isolated point removal. You can specify not to.

  In the fifth pattern shown in FIG. 6, the ultrasonic image data is ultrasonic image data in which an index (elastic modulus) indicating the hardness of the imaging region is mapped by elastography, that is, elastic image data. In the fifth pattern, the X-ray CT image data as the other type of medical image data may be image data obtained by contrast imaging or image data obtained by non-contrast imaging. Hardness information based on objective indices obtained by elastography cannot be obtained from X-ray CT image data. The fifth pattern is selected when X-ray CT image data with a high spatial resolution of the tissue is intended for composite image data in which a region focused on by a doctor is replaced with information obtained by elastography.

  In an example of the fifth pattern, the extraction unit 171 extracts, from the ultrasound image data (elastic image data), a region having an index (elastic modulus) within a predetermined range as a specific region. For example, after imaging in the elast mode or after designating ultrasonic image data in the elast mode, the operator inputs “X-ray CT image data: elastic modulus: equivalent to tumor” as setting information. In such a case, for example, the extraction unit 171 extracts, as the specific region, a region obtained by smoothing all regions occupied by pixels having an elastic modulus equal to or higher than the hardness recognized in general tumor tissue in the elastic image data. To do. Alternatively, when the setting information “X-ray CT image data: elastic modulus: normal equivalent” is input, the extraction unit 171 is elastic image data and indicates the elastic modulus indicating the softness recognized in general normal tissue. A region obtained by smoothing the entire region occupied by pixels having a pixel is extracted as a specific region. Alternatively, when the setting information “X-ray CT image data: elastic modulus: equivalent to tumor and normal” is input, the extraction unit 171 smoothes the entire area occupied by the above-described pixels having both elastic moduli. A region is extracted as a specific region.

  In another example of the fifth pattern, the extraction unit 171 uses the medical image data (X-ray CT image data) as a specific region, and includes ultrasonic image data (elastic image data). Extract from Here, the predetermined site is, for example, a tumor site recognized in X-ray CT image data, and the region including the predetermined site is, for example, a region in which a tumor site and a peripheral site of a tumor tissue are combined. It is. For example, when an operator who refers to X-ray CT image data designates a region-of-interest (ROI) as a tumor site using the input device 3, the extraction unit 171 extracts an area of elastic image data corresponding to the ROI. , Using the transformation matrix, expand the specified area using preset margin information (for example, “omnidirectional: 30 mm”), and extract the expanded area as the specific area. In this example, the operator inputs setting information such as “X-ray CT image data: tumor + margin” after specifying the ROI.

  In the fifth pattern, when the operator gives an instruction to start the synthesis process after the setting information has been input, the control unit 18 determines that the first image data is elastic image data, and is the first base for synthesis. It is determined that the two-image data is X-ray CT image data. Then, the extraction unit 171 extracts a specific region from the elasticity image data by the method described above, and the identification unit 172 identifies a corresponding part of the X-ray CT image data. Then, the synthesizing unit 173 generates an X-ray CT image database, and generates synthetic image data in which a corresponding part of the X-ray CT image database is replaced with a specific region in which elastic modulus information is drawn in a part focused on by the doctor.

  Next, the sixth pattern will be described with reference to FIG. The sixth pattern shows an example in which the image processing according to the present embodiment is applied to radio frequency ablation (RFA). RFA treatment is a treatment method in which an RFA needle (electrode needle) is inserted from the body surface toward a lesioned part (tumor part), and the lesioned part is coagulated and killed by a high temperature generated by radio waves. RFA is usually performed under an ultrasonic guide. And the area | region (treatment plan area | region, ablation area | region) which performs RFA treatment is set omnidirectionally with respect to a tumor boundary, for example, with a margin of about 5 mm, for prevention of recurrence.

  The left diagram in FIG. 7A shows B-mode image data (or ultrasound contrast image data) before cauterization treatment with a puncture needle. 7A, the tumor region depicted in the B-mode image data (or ultrasound contrast image data) is indicated by “T” and is omnidirectional with respect to the boundary of the tumor region T. An ablation area determined by a margin of about 5 mm set to “M” is indicated by “M”. Further, in the left diagram of FIG. 7A, for example, a puncture needle for RFA treatment inserted from a puncture guide attached to the ultrasonic probe 1 is indicated by a line segment.

  Here, during the cauterization treatment, gas is generated by radio waves generated from the puncture needle having the tip inserted to the center of the tumor region T. In such a case, as shown in the right diagram of FIG. 7A, the visibility of the ablation region M is lowered by the gas even if the ultrasonic image data during the ablation is referred to. Therefore, in the sixth pattern, the ultrasound image data is the ultrasound image data of the subject P being cauterized by radio waves, and the other type of medical image data is the medical image data of the subject P before the ablation. For example, in the sixth pattern, as shown in FIG. 7B, the other type of medical image data becomes the X-ray CT image data of the subject P before cauterization.

  In the sixth pattern, the extraction unit 171 extracts the ablation area including the treatment target part (tumor part) from the other types of medical image data (X-ray CT image data before the ablation) as the specific area. For example, after specifying the ROI of the tumor region T with reference to the ultrasound image data before cauterization treatment, the operator inputs “ultrasound image data (after start of cauterization): tumor + margin” as setting information. In such a case, the extraction unit 171 specifies a region corresponding to the tumor region T designated as the ROI in the X-ray CT image data before cauterization using the transformation matrix (region T shown in FIG. 7B). '). Then, the extraction unit 171 expands the region T ′ using preset margin information (for example, “omnidirectional: 5 mm”), and extracts the expanded region M ′ as a specific region.

  Then, the identifying unit 172 identifies the corresponding region (that is, the region M) corresponding to the region M ′ for each B-mode image data generated along the time series during cauterization. Thereby, the composition unit 173 generates composite image data in time series. As a result, during the cauterization treatment, the monitor 2 displays a moving image of the composite image data in which the region M in which the visibility is reduced by the gas is replaced with the region M ′ without the gas. The doctor can easily perform the RFA treatment by referring to the moving image and confirming that the gas is uniformly generated outside the region M ′ while visually recognizing the tumor site.

  In the sixth pattern, as a modification, the first image data may be ultrasonic image data before cauterization shown in the left diagram of FIG. The alignment process in such a case can be executed using a position detection system including the position sensor 4 and the transmitter 5, ultrasonic transmission / reception conditions, and image generation conditions. In the modified example of the sixth pattern, the processing performed by the image processing unit 17 is as follows.

  That is, the extraction unit 171 performs the ultrasonic image data before the cauterization of the subject P that is aligned based on the alignment information that associates the ultrasonic image data of the subject P being cauterized by radio waves with the spatial position. Then, the ablation area including the treatment target part is extracted as a specific area by image processing. And the identification part 172 calculates | requires the position of the corresponding | compatible area | region corresponding to the specific area | region in the ultrasonic image data in cauterization using alignment information. And the synthetic | combination part 173 produces | generates the synthetic image data which changed the corresponding area | region of the ultrasonic image data in cauterization based on the ultrasonic image data before cauterization in a specific area | region. Specifically, the composition unit 173 generates composite image data in which the corresponding region of the ultrasonic image data being cauterized is replaced with a specific region. The RFA treatment can also be supported by displaying a moving image of the composite image data.

  Note that the above-described image processing of the first to sixth patterns can be applied even when the ultrasound image data and the other types of medical image data are volume data.

  Next, the display form of the composite image data executed by the control unit 18 will be described with reference to FIGS. 8 and 9 are diagrams for explaining an example of a display form by the control unit according to the present embodiment.

  The control unit 18 causes the monitor 2 to perform the first display form in which at least one of the first image data and the second image data is displayed together with the composite image data. For example, the control unit 18 displays the composite image data and the base second image data in parallel. Alternatively, the control unit 18 displays the composite image data, the first image data, and the second image data in parallel.

  Alternatively, as shown in FIG. 8, the control unit 18 causes the monitor 2 to perform the second display form in which the composite image data and the second image data are switched and displayed. For example, the second display form shown in FIG. 8 is executed when the operator presses a switching button of the input device 3.

  Or the control part 18 makes the monitor 2 perform the 3rd display form which makes the area | region changed by synthetic | combination image data clear. When the above-described composite image data is used as display image data, the control unit 18 causes the monitor 2 to perform the third display mode in which the region replaced with the composite image data is clearly indicated. For example, the compositing unit 173 generates composite image data in which a specific area is colored or a dotted line is drawn on the boundary of the specific area according to an instruction from the control unit 18 and is output to the monitor 2. Alternatively, the control unit 18 causes the monitor 2 to perform the fourth display form in which the difference value between the composite image data and the second image data is displayed on the composite image data. For example, the synthesizing unit 173 calculates a difference value between the synthesized image data and the second image data for all the pixels, and superimposes a color tone according to the calculated value on the synthesized image data. Display the output image data. Moreover, the control part 18 performs one, two, or all the display forms of said 1st display form, 2nd display form, 3rd display form, and 4th display form by an operator's designation | designated. Is possible.

  Further, the control unit 18 can generate and display composite image data different from the above-described “composite image data in which the corresponding region is replaced with the specific region” in the fifth display mode described below. For example, in the fifth display mode, the composition unit 173 displays the composite image data obtained by superimposing the specific region and the corresponding region by changing the mixing ratio together with the composite image data in which the corresponding region is replaced with the specific region. Generate as data. That is, the combining unit 173 sets the corresponding area of the second image data as the combined image data changed based on the first image data in the specific area, the combined image data in which the corresponding area is replaced with the specific area, and the mixing ratio. It is possible to generate at least one of the combined image data obtained by superimposing and combining the specific area and the corresponding area as display image data.

  For example, the operator operates the slide bar shown in FIG. 9 to adjust the mixing rate. When it is desired to refer to the second image data itself that is the base together with the composite image data, the operator moves the slide bar to the position of “specific area: 0%”. Alternatively, when it is desired to refer to the second image data with the mixing ratio of “specific area: 80%, corresponding area: 20%”, the operator moves the slide bar to the position of “specific area: 80%”. As a result, the combining unit 173 generates image data superimposed and combined at a mixing ratio of “specific area: 80%, corresponding area: 20%”, and the monitor 2 displays the image data. Note that the fifth display form may be a case where only one of the replacement-type composite image data and the mixed-type composite image data is generated and displayed, or a case where both are generated and displayed. Even in the case of performing the fifth display form of the mixed type composite image data, at least one of the first display form to the fourth display form can be used in combination.

  By performing the first display mode to the fifth display mode described above, an operator such as a doctor can refer to the image data that is the combination source of the combined image data, and the specific region and the corresponding region are superimposed and combined at various mixing ratios. The obtained image data can be referred to together with the composite image data. Thereby, for example, the doctor can determine whether the diagnosis result performed on the composite image data can be obtained from the original image data in the same manner, and can guarantee the accuracy of the image diagnosis.

  Next, an example of processing performed by the ultrasonic diagnostic apparatus according to the present embodiment will be described with reference to FIG. FIG. 10 is a flowchart for explaining image processing performed by the ultrasonic diagnostic apparatus according to the present embodiment. In the example illustrated in FIG. 10, processing after the control unit 18 acquires the alignment information will be described. Further, in the example illustrated in FIG. 10, processing when replacement-type composite image data is set as composite image data will be described.

  As illustrated in FIG. 10, the control unit 18 of the ultrasonic diagnostic apparatus according to the present embodiment determines whether setting information related to the synthesis process has been received from the operator (step S <b> 101). Here, when setting information is not received (No at Step S101), the control unit 18 waits until setting information is received.

  On the other hand, when the setting information is received (Yes at Step S101), the control unit 18 notifies the setting unit 171 of the setting information, and further determines whether a synthesis process start request is received from the operator (Step S102). ). Here, when a synthesis process start request is not accepted (No at Step S102), the control unit 18 waits until a synthesis process start request is accepted.

  On the other hand, when the composition processing start request is received (Yes at Step S102), the processing of the image processing unit 17 is started by an instruction from the control unit 18. First, the extraction unit 171 determines first image data and second image data based on the setting information (step S103). That is, based on the setting information, the extraction unit 171 determines whether the first image data having a region (corresponding region) for which information is insufficient is ultrasonic image data or other types of medical image data. .

  Then, the extraction unit 171 extracts a specific region from the first image data (step S104), and the identification unit 172 identifies a corresponding region from the second image data (step S105). That is, the identification unit 172 obtains the position of the corresponding region using the alignment information in the second image data. Then, the synthesizing unit 173 replaces the corresponding area of the second image data with the specific area, and generates synthesized image data (step S106).

  Then, the monitor 2 displays the composite image data under the control of the control unit 18 (step S107), and ends the process. In step S107 or after step S107, at least one of the first display mode to the fifth display mode described above may be performed according to an instruction from the operator.

  As described above, in the present embodiment, one piece of composite image data that can obtain information from a specific area that cannot be obtained in an area where the information is insufficient in the second image data is generated and displayed. In the present embodiment, as described in the first pattern to the sixth pattern, the operator can generate and display the composite image data just by setting simple information. Therefore, in this embodiment, it is possible to easily obtain an image from which various information on the imaging region can be obtained. In this embodiment, the efficiency of image diagnosis (inspection efficiency) can be improved by providing the above-described composite image data as an image for image diagnosis. In the present embodiment, the accuracy of image diagnosis can be improved by performing the first to fifth display modes described above.

  In the present embodiment, the modifications described below may be performed. The corresponding region identified by the alignment information may not match the region corresponding to the specific region depending on, for example, the detection accuracy of the position sensor 4 and noise in the image. Therefore, the image processing unit 17 according to this modification performs the adjustment process illustrated in FIG. FIG. 11 is a diagram for explaining a modification according to the present embodiment.

  In the present modification, together with the corresponding region identification processing by the identification unit 172, the extraction unit 171 extracts a second corresponding region corresponding to the specific region from the second image data by image processing. In FIG. 11, the case where this modification is performed by the 1st pattern mentioned above is illustrated. As shown in the upper left diagram of FIG. 11, the identification unit 172 identifies the corresponding region of the ultrasonic image data that is the second image data from the alignment information. Further, the extraction unit 171 extracts a bone boundary drawn in the ultrasonic image data by image processing such as edge detection processing, for example, and uses the extracted bone boundary and the transmission direction of the ultrasonic wave to 11, the second corresponding area is extracted.

  Then, the identification unit 172 adjusts the position of the corresponding area using the second corresponding area. In the example shown in the left diagram of FIG. 11, since the positions of the corresponding region and the second corresponding region do not match, the identification unit 172 uses the second corresponding region as the corresponding region as shown in the right diagram of FIG. 11. The “adjusted corresponding region” is adjusted by adjusting the position of. And in this modification, the composition unit 173 replaces the adjusted corresponding area of the second image data with the specific area, or mixes the specific area and the adjusted corresponding area at an arbitrary mixing ratio. Composite image data is generated. In this modification, even when the alignment accuracy is low, by using the adjusted corresponding region, it is possible to generate and display composite image data that can reliably support image diagnosis.

  Note that the image processing methods described in the above-described embodiments and modifications may be performed by a medical image diagnostic apparatus other than the ultrasonic diagnostic apparatus. Further, the image processing method described in the above-described embodiments and modifications may be performed by an image processing apparatus that is installed independently of the medical image diagnostic apparatus and in which the function of the image processing unit 17 described above is mounted.

  In addition, the first image data and the second image data on which the image processing apparatus performs the above-described image processing is a case where the ultrasonic image data is not included even when the ultrasonic image data is included. Also good. That is, the image processing apparatus acquires two different types of medical image data, which are registered in association with the spatial position between the image data, from various medical image diagnostic apparatuses or a medical image database. Then, the image processing apparatus extracts the specific area of the first image data, which is one of the two pieces of medical image data, by image processing, and corresponds to the specific area in the second image data, which is the other image data. The position of the corresponding area is determined using the alignment information. Then, the image processing device generates composite image data in which the corresponding area of the second image data is changed based on the first image data in the specific area. Thereby, the image processing apparatus can display the composite image data.

  The image processing method described in this embodiment and the modification can be realized by executing an image processing program prepared in advance on a computer such as a personal computer or a workstation. This image processing program can be distributed via a network such as the Internet. The image processing program is recorded on a computer-readable non-transitory recording medium such as a hard disk, a flexible disk (FD), a CD-ROM, an MO, a DVD, a flash memory such as a USB memory and an SD card memory. It can also be executed by being read from a non-transitory recording medium by a computer.

  As described above, according to the present embodiment and the modification, it is possible to easily obtain an image from which various information on the imaging region can be obtained.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and the equivalents thereof.

17 Image processing unit 171 Extraction unit 172 Identification unit 173 Composition unit 18 Control unit

Claims (15)

The specific area of the first image data which is one image data of the ultrasonic image data and other types medical image data that is aligned on the basis of the alignment information for associating the spatial position between the image data and output extraction, other An extraction unit that extracts the position of the second corresponding region corresponding to the specific region from the second image data that is the image data of the first image processing ;
An identification unit for obtaining a position of a corresponding area corresponding to the specific area in the second image data by a second image process different from the first image process using the alignment information;
If the the position of the corresponding area and the position of the second corresponding region coincide, the corresponding region of the second image data to generate synthesized image data has been changed based on the first image data in the specific region When the position of the corresponding area does not match the position of the second corresponding area, the composite image data obtained by changing the second corresponding area of the second image data based on the first image data in the specific area A synthesis unit for generating
A control unit for displaying the composite image data on a display unit;
Ultrasonic diagnostic apparatus characterized in that e Bei a.   The ultrasonic diagnostic apparatus according to claim 1, wherein the extraction unit extracts, from the first image data, an area where information that cannot be obtained from the second image data is obtained as the specific area.   The ultrasonic diagnostic apparatus according to claim 1, wherein the extraction unit extracts the specific region from the first image data by image processing based on information set by an operator.   The ultrasonic diagnostic apparatus according to claim 3, wherein the extraction unit extracts the specific region from the first image data by a segmentation process based on information on a subject region set by the operator.   The control unit displays a display form for displaying at least one of the first image data and the second image data together with the composite image data, a display form for switching and displaying the composite image data and the second image data, and the composite image The display unit includes at least one display mode for displaying a region changed with data and a display mode for displaying a difference value between the composite image data and the second image data on the composite image data. The ultrasonic diagnostic apparatus according to claim 1, wherein the ultrasonic diagnostic apparatus is performed.   The composition unit is at least one of composite image data in which the corresponding region is replaced with the specific region, and composite image data in which the specific region and the corresponding region are superimposed and synthesized by changing a mixing ratio as the composite image data. Is generated as display image data. The ultrasonic diagnostic apparatus according to any one of claims 1 to 5.   The extraction unit extracts an area corresponding to an area that is not visualized in the ultrasound image data by a substance that does not propagate ultrasound as the specific area from the other medical image data. The ultrasonic diagnostic apparatus according to any one of 1 to 6.   The ultrasonic diagnostic apparatus according to claim 7, wherein the extraction unit extracts a region located deep from a bone region as the specific region from the other-type medical image data. The ultrasound image data is ultrasound image data depicting blood flow information based on the Doppler effect,
The extraction unit extracts, from the ultrasound image data, an area where the blood flow information exists or an area corresponding to a lumen area of the other-type medical image data in the area as the specific area. The ultrasonic diagnostic apparatus according to any one of claims 1 to 6. The ultrasound image data is ultrasound image data obtained by contrast imaging, and the other medical image data is medical image data obtained by non-contrast imaging,
The extraction unit extracts, from the ultrasound image data, a region where a contrast agent exists or a region corresponding to a lumen region of the other medical image data in the region as the specific region. The ultrasonic diagnostic apparatus according to any one of claims 1 to 6. The ultrasound image data is ultrasound image data obtained by non-contrast imaging, and the other medical image data is medical image data obtained by contrast imaging,
The extraction unit extracts, from the other types of medical image data, a region in which a contrast medium exists or a region corresponding to a lumen region of the ultrasound image data in the region as the specific region. The ultrasonic diagnostic apparatus according to any one of claims 1 to 6. The ultrasound image data is ultrasound image data in which an index indicating the hardness of the imaging region is mapped,
The extraction unit extracts, from the ultrasound image data, an area where the index is in a predetermined range or an area including a predetermined part in the imaging part in the other type of medical image data as the specific area. The ultrasonic diagnostic apparatus according to claim 1, wherein the ultrasonic diagnostic apparatus is characterized in that The ultrasound image data is ultrasound image data of a subject undergoing radiofrequency ablation, and the other medical image data is medical image data of the subject before ablation,
The ultrasonic diagnostic apparatus according to claim 1, wherein the extraction unit extracts the ablation area including the treatment target part as a specific area from the other-type medical image data. The specific area of the first image data which is one image data of the two medical image data of different types that are aligned based on the alignment information for associating the spatial position between the image data and output extraction, other image An extraction unit that extracts the position of the second corresponding region corresponding to the specific region from the second image data that is data by the first image processing ;
An identification unit that obtains the position of the corresponding region corresponding to the specific region in the second image data that is the other image data by using a second image process different from the first image process using the alignment information;
If the the position of the corresponding area and the position of the second corresponding region coincide, the corresponding region of the second image data to generate synthesized image data has been changed based on the first image data in the specific region When the position of the corresponding area does not match the position of the second corresponding area, the composite image data obtained by changing the second corresponding area of the second image data based on the first image data in the specific area A synthesis unit for generating
A control unit for displaying the composite image data on a display unit;
The image processing apparatus being characterized in that example Bei a. The specific area of the first image data which is one image data of the two medical image data of different types that are aligned based on the alignment information for associating the spatial position between the image data and output extraction, other image An extraction procedure for extracting the position of the second corresponding area corresponding to the specific area from the second image data as data by the first image processing ;
An identification procedure for obtaining a position of a corresponding area corresponding to the specific area in the second image data, which is the other image data, by a second image process different from the first image process using the alignment information;
If the the position of the corresponding area and the position of the second corresponding region coincide, the corresponding region of the second image data to generate synthesized image data has been changed based on the first image data in the specific region When the position of the corresponding area does not match the position of the second corresponding area, the composite image data obtained by changing the second corresponding area of the second image data based on the first image data in the specific area A synthesis procedure to generate
A control procedure for displaying the composite image data on a display unit;
An image processing program for causing a computer to execute.