JP2009247641A - Ultrasonic image processor and surgery support system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a surgery support system capable of providing a real-time fly-through image using an ultrasonic image. <P>SOLUTION: This ultrasonic processor includes: position detection means for detecting the position of an ultrasonic probe inserted in the interior of a subject; image processing means for processing ultrasonic tomographic image data created using an ultrasonic signal received by the ultrasonic probe; and a display means for displaying an image processed by the image processing means, wherein the image processing means has image rearrangement means for rearranging the ultrasonic tomographic image data captured when detecting the position, in a three-dimensional image space using position information of the ultrasonic probe detected by the position detection means, and fly-through image formation means for forming a fly-through image using a plurality of ultrasonic tomographic image data rearranged in the three-dimensional image space. The surgery supporting function can be reinforced by providing the image processor in the image processing section of the surgery support system. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a technique for supporting a surgical operation using an intracorporeal surgical instrument such as a catheter using a medical image diagnostic apparatus, and more particularly to a technique for creating a fly-through image of a subject lumen using an ultrasonic image.

  Catheter treatment is generally used for blood vessel and heart treatment, and various techniques for supporting catheter treatment using image diagnostic apparatuses such as MRI, CT, and ultrasonic diagnostic apparatuses have been proposed. For example, an MRI image of a region including a catheter is acquired while the catheter is inserted, and a technique for identifying the catheter position from the MRI image (Patent Document 1 or the like) or an ultrasonic element attached to the distal end of the catheter is used. There is a technique for imaging a vertical cross section (Patent Document 2).

In addition, there is a fly-through function as a method for simulating a lumen such as a blood vessel into which a catheter is inserted. A fly-through image is an image depicting the interior of a lumen with a point in the lumen as a viewpoint in the same way as an endoscopic image, and can observe the state of the blood vessel wall and blood vessel running. Useful as. Patent Document 3 discloses a technique for creating a fly-through image using volume data acquired by MRI, CT, or the like.
Japanese Patent No. 3911602 Japanese Patent Publication No. 7-38852 JP 2005-110773 A

  Fly-through images are excellent for observing lumens, but methods that use volume data acquired by MRI, CT, etc. have a problem of lack of real-time performance, and sufficiently perform functions that support the progress of surgery. I can't. On the other hand, the ultrasonic diagnostic apparatus can provide a real-time image, but the image is a cross-sectional image perpendicular to the catheter traveling direction and can obtain information in the depth direction with respect to the wall surface. Not suitable for observation.

  Therefore, an object of the present invention is to provide a surgical operation support apparatus that can provide a fly-through image having real-time characteristics using an ultrasonic image.

  In order to solve the above-described problems, an ultrasonic image processing apparatus according to the present invention includes position detection means for detecting the position of an ultrasonic probe inserted into a subject, and ultrasonic waves received by the ultrasonic probe. Image processing means for processing the ultrasonic tomographic image data created using the signal, and display means for displaying the image processed by the image processing means, the image processing means being detected by the position detection means Image rearrangement means for rearranging the ultrasonic tomographic image data imaged at the time of position detection in a three-dimensional image space using the position information of the ultrasonic probe, and a plurality of rearrangements in the three-dimensional image space And fly-through image generation means for creating a fly-through image using the ultrasonic tomogram data.

  The surgical operation support apparatus according to the present invention includes an ultrasonic imaging unit including an ultrasonic probe, an imaging unit other than the ultrasonic imaging unit, and a plurality of subjects obtained by the ultrasonic imaging unit and the imaging unit. Position detection for processing the image and generating a support image necessary for surgery, a display means for displaying the support image, and a position of the ultrasonic probe inserted in the subject The support image creating means comprises means for detecting ultrasonic tomographic image data generated by the ultrasonic imaging unit at the time of position detection using position information of the ultrasonic probe detected by the position detecting means. Image rearrangement means for rearranging in a three-dimensional image space; and fly-through image generation means for creating a fly-through image using a plurality of ultrasonic tomographic image data rearranged in the three-dimensional image space.

  ADVANTAGE OF THE INVENTION According to this invention, the fly through image useful for surgery assistance in catheter treatment can be displayed in real time. In particular, by displaying a fly-through image obtained from an ultrasonic tomographic image together with an image obtained by an imaging means such as an MRI apparatus, an image of a subject lumen such as a blood vessel wall related to the catheter position can be displayed. A high intraoperative support function can be realized.

DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments of a surgery support apparatus of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing an overall outline of the surgery support apparatus of the present invention. This surgery support apparatus includes a bed 25 on which the subject 10 is laid, an ultrasonic imaging unit 30 including an imaging unit 20 that images the subject, and a probe 35 that can be attached to a catheter that is inserted into the subject. An image processing unit 40 that processes an image captured by the imaging unit 20 and the ultrasonic imaging unit 30, and a display unit 50 that displays an image processed by the image processing unit 40.

  The imaging unit 20 is not particularly limited as long as it can capture an image of a relatively wide range of the subject, and an MRI apparatus, a CT apparatus, or the like can be adopted. However, in the illustrated embodiment, an imaging space is used. On the other hand, an open-type MRI apparatus having a structure in which magnets 21 and 22 are arranged on the upper and lower sides and these magnets are supported by support columns 23 is employed. The open MRI apparatus has a higher degree of freedom in accessing the imaging space than other imaging apparatuses in which the imaging space is limited to the inside of the bore, and is suitable for intraoperative imaging. Although not shown in the figure, a gradient magnetic field coil, a high frequency magnetic field coil, and the like are provided in the vicinity of the magnets 21 and 22, and a high frequency magnetic field coil 24 for detecting a nuclear magnetic resonance signal generated from the subject 10 is provided. It is attached to the subject 10 as necessary. In addition, a drive system and a control system necessary for driving the imaging unit 20, a display unit 50A, and the like are provided.

  The function of the imaging unit 20 is the same as the function of a known MRI apparatus, and description of the imaging method is omitted here, but the functions of the MRI apparatus used for surgical support include, for example, a surgical navigation function and an interactive scan. (ISC). The surgical navigation function is a function for displaying, for example, a three-section image of a position designated by the operator using volume data acquired in advance. The interactive scan (ISC) is a function for imaging a cross section including an arbitrary position designated by the operator in real time. In order to perform surgical navigation or ISC, the position designated by the surgeon may be a specific pointing tool detected by the three-dimensional position detection device when the surgery support device includes a three-dimensional position detection device. However, it is also possible to use catheter position information (tip position, direction, etc.) obtained by the position detection unit 42 of the image processing unit 40 described later.

  As shown in FIG. 2, the ultrasound imaging unit 30 includes an ultrasound probe 35 including a transducer element that irradiates the subject 10 with ultrasound and receives ultrasound reflected from the subject, An ultrasonic transmission / reception unit 31 that transmits / receives a signal, an ultrasonic image configuration unit 32 that forms a tomographic image such as a two-dimensional ultrasonic image (B-mode image) based on the received signal, and a tissue of a subject using the received signal An elastic image constructing unit 33 that calculates elasticity data such as elastic modulus and strain and generates an elastic image, and an ultrasonic image composed of the ultrasonic image constructing unit 32 and an elastic image calibrated by the elastic image constructing unit 33 A display unit 34 for displaying, a control unit 36 for sending commands necessary for the operation to the ultrasonic transmission / reception unit 31, the ultrasonic image configuration unit 32 and the elastic image configuration unit 33, and a control panel 37 for giving instructions to the control unit 36 Have

  Although the ultrasonic probe 35 is shown in a large form in FIG. 2, in this embodiment, the ultrasonic probe 35 is fixed to the distal end of the catheter 60 (FIG. 1) inserted into the subject 10, and the catheter is For example, when inserted into a blood vessel, the contact surface of the probe 35 is configured to contact the blood vessel wall. When imaging is performed over 360 degrees around the blood vessel wall, it is possible to use a plurality of probes 35 fixed to a ring-shaped member so that their contact surfaces face outward.

  The function of the ultrasonic imaging unit 30 is the same as that of a known ultrasonic imaging unit, but will be described briefly. The ultrasonic image constructing unit 32 reads the received signal as frame data, and after A / D conversion, adds a gradation to obtain image frame data. The elastic image is obtained by measurement in a state where the pressure with which the ultrasonic probe 35 is pressed against the subject surface is changed, and the elastic image constructing unit 33 newly acquires the frame data already acquired. From the set of frame data, the elastic modulus and strain amount of the tissue are calculated from the displacement and pressure change of the tissue that occurred at the time when both were acquired, and elastic frame data is created. The pressure change can be generated by utilizing respiratory motion or pulsation of the blood vessel itself. It is also possible to add to the probe 35 a mechanism that causes pressure fluctuations, such as a piezo element.

  A predetermined color or gradation is given to the elastic frame data obtained in this way to obtain elastic image data. On the display unit 34, a black and white image (for example, a B-mode image) configured by the ultrasonic image configuration unit 32 and an elastic image configured by the elastic image configuration unit 33 are displayed separately or superimposed. In the following description, various images generated by the ultrasonic imaging unit 30 are collectively referred to as a US image.

  As shown in FIG. 3, the image processing unit 40 processes the MR image acquired by the imaging unit 20 and generates various images necessary for surgical support, and detects the position information of the catheter from the MR image. A position detection unit 42 that performs image processing, a US image rearrangement unit 43 that arranges an image acquired by the ultrasound imaging unit 30 in the same three-dimensional image space (for example, an orthogonal coordinate system or DICOM coordinates) as the image space of the MR image, and a US image A US image reconstruction unit 44 for reconstructing a fly-through image, a developed image, an extracted image of a specific region, and the like from the three-dimensional image data rearranged by the rearrangement unit 43, the MR image processing unit 41, and the US image reconstruction A display image generation unit 45 that generates an image to be displayed on the display unit 50 using each image generated by the configuration unit 44, data generated by these units, and data necessary for processing of each unit A memory 46 for storing, and a main control unit 47 for controlling these portions.

  As shown in FIG. 1, each unit of the image processing unit 40 can be installed as a program in a personal computer, for example, and inputs such as a GUI, a keyboard, and a mouse displayed on a display 50 </ b> B provided in the personal computer. Instructions necessary for the operation of each unit can be sent via the device 51.

  The MR image processing unit 41 processes the volume data captured by the imaging unit 20 in advance, and displays images necessary for surgical support, for example, three cross-sectional views (Axial diagram, Sagittal diagram, Colonal diagram) and volume rendering images of surgery navigation. In addition to the creation, the image data captured by the imaging unit 20 during the operation is processed to create a tomographic image. Furthermore, the MR image processing unit 41 may have a function of creating a fly-through image with the catheter position as a viewpoint, using the catheter position information obtained by the position detection unit 42. Note that the above-described functions of the MRI image processing unit 41 may be provided in the imaging unit 20 and an image as a processing result may be transferred to the image processing unit 40.

  The position detection unit 42 receives time-series image data acquired by the imaging unit 20 in real time, and detects the position of the catheter from the image data. When the imaging unit 20 is an MRI apparatus, a marker identifiable by the MRI apparatus is attached to the distal end of the catheter, and the position of the catheter distal end is detected by detecting the marker position on the image. Alternatively, it is also possible to discriminate an ultrasonic probe attached to the catheter tip from the image and detect the position as the catheter tip position. As the catheter tip position detection method, for example, the method disclosed in Patent Document 1 can be adopted.

  FIG. 4 shows how the position is detected by the position detector 42. Here, an MR image is acquired while inserting the catheter 401 with the ultrasound probe 402 at the tip into the blood vessel 400, and the marker position P1 (t1) in the three-dimensional orthogonal coordinate system is acquired from the MR image acquired at each time point. , P2 (t2)... Pn (tn). The position of the catheter tip detected by the position detector 42 and the time when the MR image for position detection is acquired are stored in the memory 46 and used for US image rearrangement described below.

  The US image rearrangement unit 43 receives the US image (tomographic image data and / or elasticity image data) acquired by the ultrasonic imaging unit 30 in real time, and based on the position information of the catheter tip detected by the position detection unit 42. And arranged in orthogonal coordinates or DICOM coordinates associated with the real space. FIG. 5 shows processing in the US image rearrangement unit 43. In FIG. 5, orthogonal coordinates are shown as an example of the rearranged image space. As shown in FIG. 5A, the memory 46 of the image processing unit 40 stores, from the MR images acquired in time series, positional information p (x, y, z) is stored together with acquisition times t1, t2, t3,. The US image rearrangement unit 43 converts the US images 511 to 514 formed from the received signal received at time t into a three-dimensional image such that, for example, the center of the image coincides with the position p at time t stored in the memory 46. It arrange | positions in space (FIG.5 (b)). As a result, as shown in FIG. 5C, the US images 510 (511 to 514) obtained in time series become the rearranged three-dimensional image data 520 rearranged along the traveling path of the catheter. The US image data after rearrangement in the image space associated with the real space is hereinafter referred to as three-dimensional US image data.

  FIG. 5 shows a state in which the US images are arranged in parallel to each other. However, when using image information of a tissue deeper than the inner wall, the US images are arranged so as to be orthogonal to the catheter advancing direction. It is necessary. The traveling direction of the catheter in the three-dimensional image space can be calculated from the position of the catheter tip that is temporally adjacent. However, when only the fly-through image of the inner wall is created, they may be arranged in parallel as shown. Further, when the acquisition of the ultrasonic signal of the US image to be rearranged is different from the acquisition of the MR image, the position may be obtained by interpolation using two pieces of position information at the closest time in the MR image acquisition.

  The US image reconstruction unit 44 includes an image extraction unit 441, a fly-through image generation unit 442, a developed image generation unit 443, an extraction region image generation unit 444, and the like.

  The image extraction unit 441 scans all lines of the US image data constituting the three-dimensional US image data in units of pixels, automatically determines the boundary of each tissue from the change in the signal intensity of each pixel, and image data of a specific tissue To extract. In addition, the distance and thickness of a specific tissue are measured. The extracted image data and measurement values are stored in the memory 47 and used for image formation by the fly-through image generation unit 442. In addition, the numerical data itself can be displayed on the display unit 50 as necessary.

  The organ or tissue to be processed by the image extraction unit 441 may be specified by the operator through a GUI displayed on the display unit 50, or may be automatically performed for all tissues having different elastic moduli, for example. May be. Further, it is possible to manually determine the boundary of the tissue. In this case, a US image is displayed on the display unit 50 and a tool (GUI) for determination is provided, and this tool is operated by an input device 51 such as a mouse. However, the boundaries of each tissue may be indicated manually. The image extraction unit 441 calculates the distance and thickness from the pixel position of the tissue that has been manually determined.

FIG. 5 shows a state of discrimination by the image extraction unit 441. FIG. 6A shows a US image 601 displayed on the display unit 50. In FIG. 6B, automatic scanning 602 is performed on the US image 601, and, for example, consecutive pixels with a signal intensity in a certain range are extracted, and are determined to be the same tissue. In this case, the signal intensity may be the elasticity data (elastic modulus and strain amount) of the elastic image in addition to the signal intensity of the B-mode image.
FIG. 6C shows a state of manual discrimination. On the screen of the display unit 50, a drawing tool 603 is displayed together with the US image 601, and the operator operates the drawing tool 603 by operating the mouse 605 or the like, and draws a tissue boundary determined from the image. Even in the case of automatic determination, an operator may input a signal intensity threshold value for determining the tissue. For example, the operator may designate an arbitrary position on the image with a pointing device, and a region of a predetermined signal intensity range including the signal intensity of the pixel corresponding to the position may be determined.

  The fly-through image generation unit 442 generates a fly-through image using the three-dimensional US image data of the specific tissue extracted by the image extraction unit 441. This is shown in FIG. FIG. 7A shows three-dimensional US image data 700 (or three-dimensional image data of a specific tissue extracted by the image extracting unit 441). For example, each US image data includes a blood vessel wall 711, a lymph node 712, A muscle layer 713, a fat layer 714, a skin layer 715 and the like are included. The fly-through image generation unit 442 extracts, for example, only the image data of the blood vessel wall 611 from these tissues 711 to 715 (FIG. 7B), and superimposes the pixels in order from the image on the deepest side in the observation direction. The data is synthesized (FIG. 7 (c)). As a result, a fly-through image 710 as shown in FIG. 7D is obtained. A fly-through image can be similarly created for tissues other than the blood vessel wall. FIG. 8 is a fly-through image 810 created by extracting image data of the fat layer 714.

  Note that the direction (viewing direction) of the fly-through image may be the catheter traveling direction or the direction opposite to the catheter traveling direction as long as the three-dimensional US image data exists.

  The developed image generation unit 443 creates a developed image in which the fly-through image is developed on one surface of three-dimensional coordinates. Specifically, as shown in FIGS. 9A and 9B, the three-dimensional US image data 710 used to create the fly-through image is used by using the catheter position information detected by the position detection unit 42. Rearrange the catheter so that the catheter travels in a straight line. That is, the image is rearranged before being rearranged by the US image rearrangement unit 43. Then, as shown in FIG. 9C, the US image is developed on a line where each of the US images constituting the three-dimensional image data intersects one plane of the three-dimensional coordinates (for example, xz plane), A developed image 910 as shown in FIG. 9D is created. In this development view, it is possible to observe a state in which a blood vessel wall or the like is deployed with respect to the catheter traveling direction even when the traveling direction of the catheter is not a fixed direction.

  The extraction region image generation unit 444 uses the specific organ and tissue information (distance and thickness from the inner wall) extracted by the image extraction unit 441 to generate a US image of the specific organ to be superimposed on the fly-through image. create.

  The display image generation unit 45 synthesizes various surgery support images created by the image processing unit 40 and displays them on the screen of the display unit 50. The display image generation unit 45 is necessary in accordance with a command from the main control unit 47. The image is displayed on the display unit 50. For example, in the example illustrated in FIG. 9, the real-time image 920 captured by the imaging unit 20 is displayed on the window screen on the screen displaying the developed image of the US fly-through image. As a result, the surgeon can grasp where the portion displayed in the developed image 910 is located in the subject. Such display using a window screen is possible not only for a developed image but also for a screen that displays a fly-through image and other images. Further, instead of the window screen, if necessary, two types of images can be displayed in a superimposed manner, for example, by displaying the organ image created by the extraction region image generation unit 444 on the developed image. . Various other aspects of the image displayed on the display unit 50 will be described later.

  As for the display unit 50, the display device 50B attached to the personal computer in which the image processing unit 40 is constructed may also have the function, and the display unit 50A provided in the imaging unit 20 (MRI apparatus) is the operation support device. It may also serve as a display unit. The display unit 50 also has a function of displaying a GUI for operating the surgery support apparatus.

  Next, the operation of the surgery support apparatus having the above-described configuration will be described. FIG. 10 shows an example of a time chart during surgery. In the figure, the vertical lines indicate the progress of the processing of the image processing unit 40, the imaging unit 20, and the ultrasonic imaging unit 30 performed in parallel with the progress of the operation.

  First, the imaging unit 20 performs imaging of 3D volume data necessary for surgical navigation (S101). The 3D volume data is sent to the image processing unit 40 for processing by the image processing unit 40. Real-time slice-through image creation is activated in the image processing unit 40 (S102). When surgery is started (S103) and a catheter is inserted (S104), the imaging unit 20 starts real-time imaging and detects the catheter position (S105). The position information of the catheter and the detection time are sent to the image processing unit 40. In this embodiment, the imaging unit 20 detects the catheter position from the image captured by the imaging unit 20, but the image processing unit 40 has a function of the position detection unit as in the embodiment shown in FIG. In the case of being provided, the image captured by the imaging unit 20 is transferred to the image processing unit 40, and the position detection unit 42 of the image processing unit calculates the catheter position.

  If the catheter is inserted into a predetermined blood vessel, imaging of the blood vessel lumen is started by the ultrasonic imaging unit 30 (S106). The ultrasonic imaging unit 30 continuously performs imaging in parallel with the progress of the catheter, and obtains a plurality of time-series US images. A plurality of time-series US images are sequentially sent to the image processing unit 40. The US image rearrangement unit 43 of the image processing unit 40 rearranges the time series US image sent using the catheter position information in a three-dimensional image space corresponding to the real space, and uses the rearranged US image data. The fly-through image generation unit 442 creates a fly-through image (S107).

  On the other hand, when the imaging unit 20 has a surgical navigation function or an ISC function, in parallel with the fly-through image creation by the image processing unit 40, the creation of a 3D navigation image centered on the point indicated by the operator sequentially, Updating and ISC imaging including a predetermined position of the catheter are performed (S108, S109). The surgeon performs a catheter operation while confirming a surgical site or the like with a fly-through image created by the image processing unit 40 or an image created by the imaging unit 20 (S110).

  The above-described operation of the surgery support apparatus is realized via a GUI displayed on the display unit 50. An example of the GUI is shown in FIGS. FIG. 11 is a diagram showing a GUI before surgery. In the illustrated example, the display screen 1100 includes an image display unit 1110, a 3D image display unit 1120, a surgery information display unit 1130, and an operation button display unit. The operation buttons include a button 1141 for starting 3D imaging by the imaging unit 20, a button 1142 for starting / stopping ISC, a button 1143 for starting / stopping 3D navigation, a button 1144 for starting imaging by the ultrasonic imaging unit 30, and an image. A button 1145 for operating a fly-through image creation function by the processing unit 40, a button 1146 for creating a two-dimensional developed image, and the like are provided.

  The image display unit 1110 displays the slice images 1111 to 1114 of the 3D volume image acquired in advance by the imaging unit 20, and the US surgeon or operator uses the images of these slices to depict the surgical route. Pre-processing is performed. In the 3D image display unit 1120, each slice of the 3D volume image is stereoscopically displayed. The operation information display unit 1130 displays device information such as the position of the bed, the type of coil, and the coil position, information on various functions provided in the operation support device, information on the patient, and surgical tool information.

  FIG. 12 is a diagram illustrating an example of a GUI during surgery. The operation buttons are the same as before surgery. In the illustrated example, the fly-through image 1202 created by the image processing unit 40 is displayed together with the US image 1201 captured by the ultrasonic imaging unit 30 on the left side of the screen, and various functions of the imaging unit 20 are operated in the other portions. Images 1203 to 1027 created by the above are displayed.

  For example, when the operation button 1144 is operated, the process S106 in FIG. 10 is started, and the real-time image 1201 captured by the ultrasonic imaging unit 30 is displayed on the screen. When the fly-through creation operation button 1145 is further operated, the process 107 in FIG. 10 is started, and a real-time fly-through image 1202 is displayed on the screen. When the MR image processing unit 41 of the image processing unit 40 has a fly-through image creation function, a fly-through image 1203 created from previously acquired 3D volume data (MR data) is displayed using the catheter position information. To do. This fly-through image can be used as a navigation image for displaying a surgical route and a crossing.

  The image 1204 is an image captured by the imaging unit 20 in real time by operating the ISC operation button 1142, and in this embodiment, a cross section including the catheter position detected by the position detection unit 42 is captured and displayed. Images 1205 to 1207 are three cross-sectional views created by the image processing unit 40 by operating the 3D navigation operation buttons 1143. In this embodiment, three axes are determined based on the catheter position and direction detected by the position detector 42. That is, the axial image 1205 is centered on the distal end position of the catheter and a cross section perpendicular to the catheter direction is cut out. The sagittal image 1206 is cut out of the cross section with the catheter direction as the horizontal axis. A section taken as an axis is cut out and displayed. An image 1208 is a volume rendering image using 3D volume data, and the catheter tip position 1221 detected by the position detection unit 42 is displayed together with its locus 1222.

  As mentioned above, although one embodiment of the surgery support apparatus of the present invention has been described, the surgery support apparatus of the present invention adds a known surgery support function to the functions described in the above embodiment, When a mirror or the like is provided, an endoscopic video can be displayed. Moreover, it is not necessary to operate all the functions described in this embodiment, and an operator can appropriately select and create and display an image necessary for surgery.

  Moreover, although the case where MRI was used as an imaging device was demonstrated in the said embodiment, imaging devices may be apparatuses other than MRI apparatuses, such as CT apparatus and a X-ray fluoroscope. In addition, it is possible to implement the surgery support device of the present invention by providing a means for detecting the catheter position, for example, a three-dimensional position detection device, instead of the imaging device.

  Further, in the above-described embodiment, the image processing unit of the surgery support apparatus having the configuration of the image processing unit 40 illustrated in FIG. 3 has been described, but the function of the image processing unit is the ultrasonic imaging apparatus illustrated in FIG. It is also possible to incorporate it into the ultrasonic imaging apparatus, thereby greatly enhancing the function of the ultrasonic imaging apparatus. However, in this case, the function of the MR image processing unit 41 of the image processing unit shown in FIG. 3 can be omitted.

The block diagram which shows the whole outline | summary of the surgery assistance apparatus of this invention Block diagram showing the configuration of the ultrasound imaging unit Block diagram showing the configuration of the image processing unit The figure explaining the position detection by a position detection part The figure explaining the process in an ultrasonic image rearrangement part The figure explaining the discrimination | determination process of the structure | tissue by an image extraction part The figure explaining the process by a fly-through image generation part The figure which shows the example of the fly-through image by a fly-through image generation part The figure explaining the process by the expansion | deployment image generation part Time chart showing operation of surgery support device The figure which shows an example (before operation) of GUI displayed on the display part of a surgery assistance apparatus The figure which shows an example (in operation) of GUI displayed on the display part of a surgery assistance apparatus

Explanation of symbols

DESCRIPTION OF SYMBOLS 20 ... Imaging part, 30 ... Ultrasonic imaging part, 40 ... Image processing part, 50 ... Display part, 51 ... Input device, 60 ... Catheter, 41 ... MR image Processing unit 42 ... Position detection unit 43 ... US image rearrangement unit 44 ... US image reconstruction unit 441 ... Image extraction unit 442 ... Fly-through image generation unit 443 ... Development image generation unit, 444 ... Extraction area image generation unit.

Claims (9)

Position detecting means for detecting the position of the ultrasonic probe inserted into the subject; and
Image processing means for processing ultrasonic tomographic image data created using an ultrasonic signal received by the ultrasonic probe;
Display means for displaying the image processed by the image processing means,
The image processing means includes
Image rearrangement means for rearranging the ultrasonic tomographic image data captured at the time of position detection in a three-dimensional image space using the position information of the ultrasonic probe detected by the position detection means;
An ultrasonic image processing apparatus comprising fly-through image generation means for generating a fly-through image using a plurality of ultrasonic tomographic image data rearranged in the three-dimensional image space. The ultrasonic image processing apparatus according to claim 1,
The fly-through image generation means includes an extraction means for extracting image data of a predetermined tissue from a plurality of ultrasonic tomographic image data rearranged in the three-dimensional image space, and the one or more extracted by the extraction means An ultrasonic image processing apparatus that generates a fly-through image using image data of a tissue. The ultrasonic image processing apparatus according to claim 2,
The ultrasonic tomographic image data includes an elastic image of a subject tissue,
The ultrasonic image processing apparatus, wherein the extraction unit extracts a tissue image having a predetermined elasticity data value from ultrasonic tomographic image data including the elasticity image. The ultrasonic image processing apparatus according to any one of claims 1 to 3,
The ultrasonic image processing apparatus, wherein the image processing means further includes a developed image generating means for generating an image obtained by two-dimensionally developing the fly-through image. The ultrasonic image processing apparatus according to any one of claims 1 to 5,
The ultrasonic probe is fixed to a catheter that is inserted into the lumen of a subject,
The ultrasonic image processing apparatus, wherein the fly-through image generated by the fly-through image generation means is a fly-through image of the lumen wall. An ultrasonic imaging unit including an ultrasonic probe, an imaging unit other than the ultrasonic imaging unit, and a plurality of subject images obtained by the ultrasonic imaging unit and the imaging unit are processed, and is necessary for surgery. Surgery support apparatus comprising support image creation means for creating a support image, display means for displaying the support image, and position detection means for detecting the position of the ultrasonic probe inserted into the subject. Because
The support image creation means includes
Image rearrangement means for rearranging ultrasonic tomographic image data generated by the ultrasonic imaging unit at the time of position detection in a three-dimensional image space using position information of the ultrasonic probe detected by the position detection means;
A surgery support apparatus comprising fly-through image generation means for creating a fly-through image using a plurality of ultrasonic tomographic image data rearranged in the three-dimensional image space. The surgical operation support device according to claim 6,
The position detecting unit includes a determining unit that determines the ultrasonic probe from a subject image including the ultrasonic probe acquired by the imaging unit, and the ultrasonic probe determined by the determining unit. An operation support apparatus for detecting a child coordinate as a position of the ultrasonic probe. The surgery support apparatus according to claim 6 or 7,
The three-dimensional image space in which the ultrasonic tomographic image data is rearranged is a space having the same coordinate system as the image space in which the image data acquired by the imaging unit is arranged,
The fly-through image generation means has a function of creating a fly-through image inside the subject into which the ultrasonic probe is inserted, using the volume data imaged by the imaging means, and the ultrasonic tomographic image data The operation support apparatus, wherein the fly-through image created from the image is superimposed on the fly-through image created from the volume data and displayed on the display means. The surgery support apparatus according to any one of claims 6 to 8,
The surgery support apparatus, wherein the imaging means is an MRI apparatus.

Images