JP2007037790A - Ultrasonic diagnostic equipment - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To anatomically precisely accord a pre-recorded point on ultrasonic diagnostic equipment from heterogeneous image data with a point of a subject supposed to anatomically accord with the point. <P>SOLUTION: This ultrasonic diagnostic equipment is provided with: an ultrasonic tomogram formation means forming an ultrasonic tomogram by ultrasonic signals transmitted/received in the living body; a detecting means detecting the position or orientation of the ultrasonic tomogram; a specification means specifying a feature point on the living body; a reference image data retaining means; and a guide image formation means collating anatomical positions of the specified feature point with the reference point on the reference image data and forming an image guiding the anatomical position or orientation of the ultrasonic tomogram based on the detected position or orientation and the reference result. The specification means specifies the feature point on the ultrasonic tomogram and the guide image formation means calculates the position of the feature point in the living body based on the position of the specified feature point on the ultrasonic tomogram and the detected position or orientation of the ultrasonic tomogram. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

    The present invention relates to an ultrasonic diagnostic apparatus that generates an ultrasonic image using an ultrasonic signal obtained by transmitting and receiving ultrasonic waves to a subject.

  2. Description of the Related Art An ultrasonic diagnostic apparatus that transmits ultrasonic waves into a living body, receives reflected waves from living tissue, and observes the state of the living body as an image is popular because it can observe the state in the living body in real time. When using an ultrasonic diagnostic apparatus, the surgeon should keep in mind the known anatomical positional relationship between organs and tissues in the living body in advance, and consider the anatomy of the currently observed ultrasonic tomographic image. Diagnosis is performed by estimating the position and observing the ultrasonic tomogram. In order to support such a diagnosis, an ultrasonic diagnostic apparatus that displays a guide image for guiding a position observed on an ultrasonic tomographic image has been proposed.

  For example, in the ultrasonic diagnostic apparatus disclosed in Japanese Patent Laid-Open No. 10-151131, an image data input device for inputting a plurality of images including a volume image from the outside is provided, and ultrasonic waves are transmitted from an external probe (ultrasonic probe). To obtain an ultrasonic tomographic image of a specific diagnostic region, and obtain a two-dimensional image (tomographic image) of the position corresponding to this ultrasonic tomographic image from an image from the image data input device to obtain an ultrasonic tomographic image. Image position relationship display means for displaying the images side by side, overlaid, or alternately in a fixed time is provided. The image positional relationship display means is provided with a transmitter for transmitting a magnetic field or a receiver for receiving a magnetic field on the side or inside of the probe. This transmitter or receiver is used to know in which part of the subject the probe is located. By using this ultrasonic diagnostic apparatus, it is possible to inspect while comparing the tomographic image of the X-ray CT apparatus or the MRI apparatus corresponding to the ultrasonic tomographic plane under examination with the ultrasonic tomographic image. .

  Further, for example, in the ultrasonic diagnostic apparatus disclosed in Japanese Patent Application Laid-Open No. 2004-113629, ultrasonic scanning position detecting means for detecting the position of a part for transmitting and receiving ultrasonic waves, and ultrasonic tomography based on the ultrasonic signal An ultrasonic image generation unit that generates an image, and an anatomical schematic diagram of a part of the subject corresponding to the position obtained by the ultrasonic scanning position detection unit are acquired from an image information holding unit that includes schematic diagram data of a human body. And a control means for displaying the schematic diagram as a guide image on the same screen as the ultrasonic tomographic image. In this Japanese Patent Application Laid-Open No. 2004-113629, an elongated and flexible ultrasonic probe to be inserted into a subject is provided as means for obtaining an ultrasonic tomographic image, and this ultrasonic probe is provided around the insertion axis. An electronic radial scanning ultrasonic endoscope with an ultrasonic transducer group arranged in an array, an electronic convex ultrasonic endoscope with an ultrasonic transducer group fan-shaped on one of the insertion axes, and an insertion axis An ultrasonic diagnostic apparatus provided with a mechanical scanning ultrasonic endoscope in which an ultrasonic transducer piece rotates at the center is disclosed. These ultrasonic endoscopes are normally provided with an illumination window for irradiating illumination light into the body cavity and an observation window for observing the inside of the body cavity at the distal end of the flexible part into the body cavity. It is customary.

  By the way, as disclosed in Japanese Patent Laid-Open No. 10-151131, a tomographic image of an X-ray CT apparatus corresponding to an ultrasonic tomographic plane under examination, a tomographic image of an MRI apparatus and an ultrasonic tomographic image are compared, In order to compare a schematic diagram corresponding to an ultrasonic tomographic image under examination and an ultrasonic tomographic image as in Japanese Patent No. 113629, the positional relationship between the ultrasonic tomographic image and different types of image data is anatomically matched. It is necessary to let

  This method is disclosed in the above publication as follows.

Japanese Patent Laid-Open No. 10-151131: “stereoscopic image obtained by X-ray CT apparatus, MRI apparatus, etc. —approximately—points a pixel in the stereoscopic image at a position that is the origin of the probe and corresponds in space If this calibration is performed for four or more points that are not on the same plane, the position of the probe or the pixel in the stereoscopic image corresponding to the pixel on the ultrasonic tomographic image can be found. . "
JP 2004-113629 A: There is no particularly clear disclosure.

  Since there is no disclosure in Japanese Patent Laid-Open No. 2004-113629, it will not be described below, and only the matching method of Japanese Patent Laid-Open No. 10-151131 will be described below. Even in the description of Japanese Patent Laid-Open No. 10-151131, a specific method of pointing a pixel in a stereoscopic image at a position that is the “origin of the probe” is not clear. Therefore, for the time being, this method is interpreted as follows from the description in Japanese Patent Laid-Open No. 10-151131.

[1] The surgeon causes the ultrasonic diagnostic apparatus to record four corresponding points on the body surface of a stereoscopic image obtained in advance by an X-ray apparatus, an MRI apparatus, or the like.

[2] The surgeon sequentially brings the probe into contact with the four points on the actual body surface of the subject who anatomically match the four points.

[3] The image positional relationship display means uses a transmitter or a receiver provided on the side or inside of the probe, obtains the position of the “origin of the probe” at each contact time, and The actual position on the body surface of the subject 4 points that match.

[4] The image positional relationship display means includes four positions on the actual body surface of the subject and four points corresponding to the body surface of a stereoscopic image obtained in advance by an X-ray apparatus, an MRI apparatus, or the like. To match the position of.

[5] The surgeon starts scanning the probe and causes the ultrasonic diagnostic apparatus to display an ultrasonic tomographic image of the subject.

[6] The ultrasonic diagnostic apparatus obtains the position of each pixel of the ultrasonic tomographic image based on the output from the receiver.

[7] The ultrasonic diagnostic apparatus sequentially reads each pixel in the stereoscopic image that matches each pixel position of the ultrasonic tomographic image, reconstructs it into a two-dimensional image (tomographic image), and displays it as an image.
JP-A-10-151131 JP 2004-113629 A

  However, in the matching method disclosed in Japanese Patent Laid-Open No. 10-151131, a point recorded in advance in an ultrasonic diagnostic apparatus from different kinds of image data (stereoscopic image in Japanese Patent Laid-Open No. 10-151131), There was a problem that it was difficult to accurately match the points of the subject considered to be anatomically coincident with each other.

  The reason is as follows.

(1) Disagreement due to the operator's recognition Since the operator has made contact with the probe in order for the operator to make an anatomical match, this point should be taken especially in the deep part of the body other than the subject's body surface. I can't. In addition, the body surface of the abdomen where an ultrasonic diagnostic apparatus is often used has few anatomically characteristic objective points. For this reason, the degree of anatomical coincidence between the points recorded from different types of image data and the points on the body surface of the subject with whom the operator has contacted the probe differs depending on the operator's recognition. Occurs. For example, even if the end of the pelvis is taken as an example of the contact point, there is a difference of about 5 cm in the recognition of the end by the operator, and there is a difference by the operator in the point where the probe is contacted. As a result, the pelvic end recorded in advance in the ultrasonic diagnostic apparatus from different types of image data does not exactly match the actual contact point.

(2) Disagreement due to probe contact method The surgeon cannot recognize the position of the “probe origin” from the appearance of the probe, and the probe is not an infinitely small point. Therefore, the degree of anatomical coincidence between the points recorded from different types of image data and the points on the body surface of the subject to which the operator has contacted the probe is related to the contact of the probe. There is a difference depending on how. For example, if a point on the exterior of the probe that is far from the origin of the probe is in contact with a point on the subject's body surface, the former point and the latter point are Mismatch occurs only by the distance to the contact point.

  Therefore, an object of the present invention is to accurately match the point recorded in advance in the ultrasonic diagnostic apparatus from different types of image data and the point of the subject considered to be anatomically coincident with this point. An object of the present invention is to provide an ultrasonic diagnostic apparatus that can be used.

  An ultrasonic diagnostic apparatus according to the present invention includes an ultrasonic tomographic image creating means for creating an ultrasonic tomographic image based on an ultrasonic signal obtained by transmitting and receiving ultrasonic waves into a living body, and detects the position or orientation of the ultrasonic tomographic image. Detecting means for specifying, a specifying means for specifying a feature point on the living body, a reference image data holding means for holding reference image data, a feature point specified by the specifying means, and a reference set on the reference image data A guide for collating an anatomical position with a point and guiding the anatomical position or orientation of the ultrasonic tomogram based on the position or orientation detected by the detection means and the result of the collation Guide image creating means for creating an image, wherein the designating means designates the feature point on the ultrasonic tomographic image created by the ultrasonic tomographic image creating means, and the guide image creating means comprises the finger The position of the feature point in the living body is calculated based on the position of the feature point designated by the means on the ultrasonic tomographic image and the position or orientation of the ultrasonic tomographic image detected by the detection means. It is characterized by that.

The ultrasonic tomographic image creating means corresponds to the ultrasonic observation apparatuses 400 and 400A in FIG. 1 or FIG. The detection means corresponds to the position / orientation calculation apparatus 500 in FIG. 1 or FIG. The designation unit includes the keyboard 720, the mouse 710, and the position detection probe 530 in FIG. 1 or FIG. The reference image data holding unit corresponds to the reference image storage unit 610 in FIG. 1 or FIG. The guide image creating means corresponds to the three-dimensional guide image creating circuit 630 in FIG. 1 or FIG.
According to the apparatus of the present invention, the point recorded in advance in the ultrasonic diagnostic apparatus from different types of image data and the point of the subject considered to be anatomically coincident with this point are accurately anatomically matched. It becomes possible to make it.

  In the present invention, the ultrasonic tomographic image creating means includes a blood flow detecting means for detecting blood flow, and the ultrasonic tomographic image creating means creates an ultrasonic tomographic image in which the blood flow is superimposed in a different manner. The specifying means specifies the feature point on an ultrasonic tomographic image on which the blood flow is superimposed.

The blood flow detection means corresponds to the color flow mapping circuit (CFM circuit) 410 in FIG.
According to the present invention, when a blood vessel branch such as a celiac artery branch is specified as a feature point in the deep part of the body, it can be specified more easily and accurately.

In the present invention, the specifying means includes position acquisition means for directly contacting the surface of a body cavity of a living body to obtain a position of a feature point different from the feature point.
The position acquisition means corresponds to the position detection probe 530 in FIG. 1 or FIG.

  According to the present invention, feature points can be specified for both the deep part of the body and the body cavity surface, and various appropriate feature points can be specified according to the region of interest. For this reason, the degree of freedom of taking the feature point in the vicinity of the region of interest increases, and it can be assumed that the feature point moves together when the region of interest moves along with the movement of the ultrasonic endoscope. Images can be created. Furthermore, feature points can be obtained near the region of interest, especially in pancreatic examinations and lung examinations. The closer to the feature points, the more the space contained in the convex triangular pyramid created by the feature points is the triangular pyramid. Since the calculation of the position and orientation of the ultrasonic tomographic image marker is expected to be more accurate than the outer space, it is more accurate to obtain feature points at appropriate locations near the region of interest within the body cavity. A dimension guide image can be created.

  According to the present invention, a point recorded in advance in an ultrasonic diagnostic apparatus from different types of image data and a subject's point that is considered to be anatomically coincident with this point are accurately matched anatomically. Is possible.

Embodiments of the invention will be described with reference to the drawings.
[First Embodiment]
[Constitution]
FIG. 1 shows the configuration of an ultrasonic diagnostic apparatus according to the first embodiment of the present invention. Of course, the present invention is not limited by this embodiment.

  The ultrasonic diagnostic apparatus 100 according to the first embodiment of the present invention includes an ultrasonic endoscope 200 as an ultrasonic probe, an optical observation apparatus 300, an ultrasonic observation apparatus 400, a position / orientation calculation apparatus 500, The transmission antenna 510, the posture detection plate 520, the position detection probe 530, the ultrasonic image processing device 600, the mouse 710, the keyboard 720, and the display device 700 are connected by signal lines.

  The ultrasonic endoscope 200 is used by being inserted into a body cavity and used with a hard portion 210 made of a hard material such as stainless steel at the tip, and a material flexible on the rear end side from the hard portion 210. It consists of a long flexible portion 220 configured and an operation portion 230 formed of a hard material on the rear end side of the flexible portion 220.

  The hard part 210 includes an optical observation window 211 made of a cover glass, a lens 212, a CCD (Charge Coupled Device) camera 213, and an illumination light irradiation window (not shown) that irradiates illumination light into the body cavity. The CCD camera 213 is connected to the optical observation apparatus 300 through a signal line. An illumination light irradiation window (not shown) is configured to irradiate illumination light and illuminate the body cavity. The image of the surface of the body cavity is formed on the CCD camera 213 from the optical observation window 211 via the lens 212, and the CCD signal from the CCD camera 213 is output to the optical observation apparatus 300 via the signal line. Yes.

  The rigid part 210 is provided with an ultrasonic transducer 214. The operation unit 230 includes a motor 231 and a rotary encoder 232 that is connected to a rotation shaft of the motor 231 and detects and outputs a rotation angle. The ultrasonic endoscope 200 has a flexible structure in which one end is fixed to the rotating shaft of the motor 231 and the other end is fixed to the ultrasonic transducer 214 from the operation unit 230 to the rigid unit 210 through the flexible unit 220. A shaft 221 is provided. The motor 231 is connected to the ultrasonic observation apparatus 400 through a control line 250. The rotary encoder 232 is connected to the ultrasonic observation apparatus 400 through a signal line 260. The ultrasonic transducer 214 is configured to perform so-called radial scanning in which ultrasonic waves are repeatedly transmitted and received while rotating in the direction of the arrow shown around the flexible shaft in FIG. The ultrasonic transducer 214 obtains an ultrasonic signal necessary for an ultrasonic tomographic image on a plane perpendicular to the insertion axis of the ultrasonic endoscope 200 (hereinafter referred to as a radial scanning plane), and outputs a flexible shaft 221, a motor 231, and a rotary. An ultrasonic signal is output to the ultrasonic observation apparatus 400 via the encoder 232. Here, orthonormal bases (unit vectors in each direction) V, V3, and V12 fixed to the rigid portion 210 are defined as shown in FIG. As will be described later, V is a normal direction vector of the radial scan plane, V3 is a 3 o'clock direction vector of the radial scan plane, and V12 is a 12 o'clock direction vector of the radial scan plane.

  In the position / orientation calculation apparatus 500, a transmission antenna 510, an attitude detection plate 520, and a long position detection probe 530 are connected by signal lines 540, 550, and 560.

  FIG. 2 shows the position detection probe 530. The position detection probe 530 is provided with an outer cylinder 531 made of a flexible material, a reception coil 532 is fixed to the front end of the outer cylinder 531, a connector 533 is provided at the rear end, and the outer cylinder 531 A forceps opening marker 534 and a 12 o'clock direction marker 535 are provided on the surface.

  As shown in FIG. 2, the receiving coil 532 is formed by integrating three coils having three orthogonal unit vectors Va, Vb, and Vc fixed to the position detection probe 530 as winding axes. The three coils are provided with a total of six signal lines, two for each pole. The connector 533 is provided with six electrodes (not shown). Six signal lines are separately connected to the six electrodes. The electrodes are connected to the position / orientation calculation apparatus 500 via a cable 560.

  As shown in FIG. 1, the ultrasonic endoscope 200 includes a forceps port 201 as a first opening from the operation unit 230 through the flexible unit 220 to the rigid unit 210, and the rigid unit 210 includes A tubular forceps channel 203 having a projecting opening 202 as a second opening is provided. The forceps channel 203 is configured such that the position detection probe 530 can be inserted from the forceps opening 201 and protrude from the protruding opening 202. The opening direction of the projection port 202 is oriented so that the position detection probe 530 falls within the optical field of view range R of the optical observation window 211 when the position detection probe 530 projects from the projection port 202.

  When the operator inserts the position detection probe 530 from the forceps opening 201, the forceps opening marker 534 has the same position as the tip of the position detection probe 530 when the positions of the forceps opening marker 534 and the opening surface of the forceps opening 201 coincide with each other. The receiving coil 532 is provided at a position on the surface of the outer cylinder 531 such that the position of the opening surface of the projecting opening 202 coincides and the receiving coil 532 is very close to the rotational center of radial scanning of the ultrasonic transducer 214. At this time, the tip of the position detection probe 530 is on the opening surface of the protruding port 202 and does not protrude.

  The operation unit 230 includes a 12 o'clock direction marker 536 around the forceps port 201 in addition to the 12 o'clock direction marker 535 provided on the outer cylinder of the position detection probe 530. The twelve o'clock direction marker 535 provided on the surface of the outer cylinder of the position detection probe 530 and the twelve o'clock direction marker 536 provided in the vicinity of the forceps opening of the operation unit 230 are the position detection probe 530 from the forceps opening 201 by the operator. When the position detection probe 530 is rotated around the vector Vc in FIG. 2 until the positions of both the 12 o'clock direction markers 535 and 536 coincide with each other, the vector Vc in FIG. The vector Va in FIG. 2 is provided so as to match the vector V3 in FIG. 1, and the vector Vb in FIG. 2 is matched with the vector V12 in FIG.

  The operation unit 230 fixes the position detection probe 530 so that the position detection probe 530 is detachably fixed so that the position detection probe 530 does not move in the insertion axis direction and the position detection probe 530 does not rotate further in the forceps channel 203. A jig is provided near the forceps opening 201.

  In the transmission antenna 510, a plurality of transmission coils (not shown) having different winding axis orientations are integrally housed in a cylindrical casing. Each of the plurality of transmission coils is connected to the position / orientation calculation apparatus 500.

FIG. 3 shows the posture detection plate 520.
The attitude detection plate 520 includes three plate coils 520a, 520b, and 520c each having a single winding coil. Here, the orthogonal coordinate axes O ″ -x ″ y ″ z ″ fixed to the posture detection plate 520 and their orthonormal bases (unit vectors in the respective axis directions) i ″, j ″, k ″ are defined as shown in FIG. Of the three plate coils, two are fixed with their winding axes oriented in the i "direction and one with the winding axes oriented in the j" direction. Here, the reference position L of the attitude detection plate 520 is 3 is defined as the center of gravity of the three plate coils, and the posture detection plate 520 uses the attached belt so that the reference position L overlaps the position of the xiphoid process of the subject's rib, and the body surface contact surface BS of FIG. In contact with the subject and fixed.

  The ultrasonic image processing apparatus 600 includes a reference image storage unit 610, an extraction circuit 620, a three-dimensional guide image creation circuit 630, a volume memory 640, a mixing circuit 650, a display circuit 660, and a control circuit 670. ing.

  The display circuit 660 is provided with a switch 661 for switching the input. The switch 661 includes an input terminal α, an input terminal β, an input terminal γ, an input terminal δ, and one output terminal d. The input terminal α is connected to an output terminal (not shown) of the optical observation apparatus 300. The input terminal β is connected to the reference image storage unit 610. The input terminal γ is connected to an output terminal (not shown) of the ultrasonic observation apparatus 400. The input terminal δ is connected to the mixing circuit 650. The output terminal d is connected to the display device 700.

  The control circuit 670 is connected to each unit and circuit by a signal line (not shown) so that a command can be output to each unit and circuit in the ultrasonic image processing apparatus 600. The control circuit 670 is directly connected to the ultrasonic observation apparatus 400, the mouse 710, and the keyboard 720 through control lines a, b, and c.

  The reference image storage unit 610 includes a hard disk drive that can store a large amount of data. The reference image storage unit 610 stores a plurality of reference image data RD as anatomical image information. As shown in FIG. 4, the reference image data RD is obtained by slicing a 60-cm square photo data obtained by slicing a frozen human body other than the subject in parallel at a pitch of 1 mm, and further classifying each pixel by organ and then color-coding it. This is the image data obtained by changing the attribute. The reason why one side of the photographic data is set to 60 cm is that the entire cross section of the human body perpendicular to the body axis from the head to the foot is large. The reference image data RD in the reference image storage unit 610 in FIG. 4 are numbered from 1 to N for convenience of explanation. Here, the orthogonal coordinate axes O′−x′y′z ′ fixed with respect to the plurality of reference image data RD, their orthonormal bases (unit vectors in each axis direction) i ′, j ′, k ′, and the origin O 'Is defined at the bottom left of the first reference image data as shown in FIG.

  The volume memory 640 is configured to store a large amount of data. A voxel space VXS is allocated to a part of the storage area of the volume memory 640. As shown in FIG. 5, the voxel space VXS includes memory cells (hereinafter referred to as voxels) VX having addresses corresponding to the orthogonal coordinate axes O'-x'y'z '.

  The keyboard 720 includes an organ of interest designation key 721, a reference point designation key 722, a body cavity surface feature point designation key 723, a deep body feature point designation key 724, a feature point designation stop key 725, a scanning control key 726, As the display switching key 727, a display switching key α, a display switching key β, a display switching key γ, and a display switching key δ are provided. When the display switching key α, β, γ, or δ is pressed, the control circuit 670 outputs a command to the display circuit 660 to switch the switch 661 to the input terminal α, β, γ, or δ. The switch 661 is switched to the input terminal α when the display switching key α is pressed, to the input terminal β when the display switching key β is pressed, and to the input terminal γ when the display switching key γ is pressed. When is pressed, the input terminal δ is switched.

[Action]
Each arrow line in FIG. 1 shows the flow of signals and data as follows.
1: The dotted line is the flow of signals and data related to optical images.
Second: The broken line shows the flow of signals and data related to ultrasonic tomograms.
Third: The solid line is the flow of signals and data related to the position,
Fourth: Dash-dot line is the flow of reference image data and data created by processing it,
Fifth: A thick line represents a flow of signals and data related to a final display screen obtained by combining three-dimensional guide image data (described later) and ultrasonic tomographic image data (described later).
6th: Curve is the flow of signals and data related to other controls.

  The operation of the first embodiment will be described along the flow of signals and data related to the first optical image.

  An illumination light irradiation window (not shown) of the rigid portion 210 irradiates the optical field range R with illumination light. The CCD camera 213 images an object in the optical visual field range R and outputs a CCD signal. The optical observation device 300 creates image data in the optical visual field range R based on the CCD signal, and outputs this data as optical image data to the input terminal α of the switch 661 of the display circuit 660 in the ultrasonic image processing device 600. To do.

The operation of the first embodiment will be described along the flow of signals and data related to the second ultrasonic tomographic image.
When the operator presses the scan control key 726, the control circuit 670 outputs a scan control signal for instructing on / off control of radial scanning to the ultrasound observation apparatus 400. In response to the scanning control signal, the ultrasonic observation apparatus 400 outputs a rotation control signal for controlling on / off of rotation to the motor 231. The motor 231 receives the rotation control signal and rotates the ultrasonic transducer 214. The ultrasonic transducer 214 repeats transmission of ultrasonic waves and reception of reflected waves while rotating in the body cavity, and converts each reflected wave into an electrical ultrasonic signal. That is, the ultrasonic transducer 214 performs so-called radial scanning in which ultrasonic waves are transmitted and received radially within a plane perpendicular to the insertion axis of the flexible portion 220 and the rigid portion 210. The rotary encoder 232 outputs the angle of the rotation shaft of the motor 231 to the ultrasonic observation apparatus 400 as a rotation angle signal.

  At this time, the ultrasonic observation apparatus 400 drives the ultrasonic transducer 214 and generates ultrasonic vibration from the ultrasonic signal converted from the reflected wave by the ultrasonic transducer 214 and the rotation angle signal from the rotary encoder 232. One digitized ultrasonic tomographic image data perpendicular to the insertion axis of the flexible portion 220 is generated for one radial rotation of the child 214, and a mixing circuit 650 and a display circuit in the ultrasonic image processing apparatus 600 are generated. 660 to the input terminal γ of the switch 661. The rotation angle signal from the rotary encoder 232 determines in which direction the 12:00 direction of the ultrasonic tomographic image data is generated with respect to the ultrasonic endoscope 200 by the ultrasonic observation apparatus 400. Will be determined. The normal angle direction vector V, 3 o'clock direction vector V3, and 12 o'clock direction vector V12 of the radial scanning plane are defined by this rotation angle signal.

The operation of the first embodiment will be described along the flow of signals and data related to the third position.
The position / orientation calculation apparatus 500 excites a transmission coil (not shown) of the transmission antenna 510 a plurality of times in a time division manner. The transmitting antenna 510 divides the alternating magnetic field in the space in six times in total for three coils having different winding axes constituting the receiving coil 532 and three plate coils of the attitude detection plate 520. The three coils and the three plate coils having different winding axes constituting the reception coil 532 detect an alternating magnetic field, convert the detected magnetic field into a position electric signal, and output the position electric signal to the position orientation calculation apparatus 500.

  The position / orientation calculation apparatus 500 calculates three coil positions at which the winding axes of the receiving coil 532 are orthogonal to each other and directions of the winding axes based on each position electric signal input in time division, and receives the calculated values from these calculated values. The position and orientation of the coil 532 are calculated. Details of the calculated values for the position and orientation of the receiving coil 532 will be described later.

  The position / orientation calculation apparatus 500 calculates the positions of the three plate coils of the posture detection plate 520 and the direction of the winding axis based on each position electric signal input in time division. The position / orientation calculation apparatus 500 calculates the center of gravity of the three plate coils, that is, the reference position L of the posture detection plate 520 from the calculated values of the positions of the three plate coils. Further, the position / orientation calculation apparatus 500 calculates the orientation of the attitude detection plate 520 from the calculated values of the directions of the winding axes of the three plate coils. Details of the calculated values for the reference position and orientation of the attitude detection plate 520 will be described later.

The position / orientation calculation apparatus 500 uses the calculated position and orientation of the reception coil 532 and the reference position L and orientation of the attitude detection plate 520 as position / orientation data to the three-dimensional guide image creation circuit 630 of the ultrasonic image processing apparatus 600. Output.
Here, in the first embodiment, the origin O is defined on the transmission antenna 510, and the orthogonal coordinate axis O-xyz and its orthonormal basis (each The unit vectors i, j, and k in the axial direction are defined as shown in FIG. The contents of the position / orientation data are defined below as a function of time t. The reason why the position / orientation data is defined as a function of the time t is that the receiving coil 532 and the attitude detection plate 520 move in space, so that the contents of the position and orientation also change depending on the time t. The position / orientation calculation apparatus 500 normalizes the lengths of Vc (t) and Vb (t) to a unit length in advance and outputs them.

Each directional component in the orthogonal coordinate axis O-xyz of the position vector OC (t) of the position C (t) of the receiving coil 532 The orthogonal coordinate axis O− of the direction unit vector Vc (t) indicating the first winding axis direction of the receiving coil 532 Each direction component in xyz Each direction component in orthogonal coordinate axis O-xyz of direction unit vector Vb (t) indicating the second winding axis direction of receiving coil 532 Position vector OL () of reference position L (t) of posture detection plate 520 Each direction component on the orthogonal coordinate axis O-xyz of t) 3 × 3 rotation matrix T (t) indicating the orientation of the attitude detection plate 520
Here, T (t) is a rotation matrix indicating the orientation of the attitude detection plate 520 with respect to the orthogonal coordinate axis O-xyz in FIG.

  Strictly speaking, the (m, n) component tmn (t) of the 3 × 3 rotation matrix T (t) is defined by the following equation. i ″, j ″, k ″ are orthonormal bases (unit vectors in the direction of the respective axes) of the orthogonal coordinate axes O ″ −x ″ y ″ z ″ fixed to the attitude detection plate 520 shown in FIG. Can be taken anywhere as long as the positional relationship with the posture detection plate 520 is fixed, but in the first embodiment, it is taken as the reference position L (t) of the posture detection plate 520. However, in FIG. 3, for convenience of explanation, the orthogonal coordinate axes O ″ -x ″ y ″ z ″ and their normal orthogonal bases i ″, j ″, k ″ are drawn away from the posture detection plate 520 for easy understanding. In the following formula, “・” means inner product.

このようにＴ（ｔ）を定義すると、下式が成り立つ。 [Equation 1]

When T (t) is defined in this way, the following equation holds.

また、一般に、Ｔ（ｔ）は所謂オイラー角θ、φ、ψを用い、ｚ軸め周りの角度ψの回転、ｙ軸の周りの角度φの回転、ｘ軸の周りの角度θの回転をこの順序で、直交座標軸Ｏ−ｘｙｚに対して施したとき、姿勢検出プレート５２０上に仮想的に固定された直交座標軸Ｏ”−ｘ”ｙ”ｚ”と一致することを想定した行列であり下式でも表現される。被検者は時間とともに体位を変えるため、θ、φ、ψはいずれも時間ｔの関数である。 [Equation 2]

Also, in general, T (t) uses so-called Euler angles θ, φ, and ψ to rotate the angle ψ around the z axis, rotate the angle φ around the y axis, and rotate the angle θ around the x axis. This matrix assumes that when applied to the Cartesian coordinate axis O-xyz in this order, it coincides with the Cartesian coordinate axis O "-x" y "z" virtually fixed on the posture detection plate 520. It is also expressed as an expression. Since the subject changes his / her posture with time, θ, φ, and ψ are all functions of time t.

第４の参照画像データやそれを加工して作成されたデータの流れについては、超音波画像処理装置６００の詳細の作用とともに後述する。 [Equation 3]

The flow of the fourth reference image data and the data created by processing the fourth reference image data will be described later together with the detailed operation of the ultrasonic image processing apparatus 600.

  The operation of the first embodiment will be described along the flow of signals and data related to the final display both surfaces obtained by synthesizing the fifth ultrasonic tomographic image data and three-dimensional guide image data (described later).

  The mixing circuit 650 generates the mixed data for display by arranging the ultrasonic tomographic image data from the ultrasonic observation apparatus 400 and the three-dimensional guide image data from the three-dimensional guide image generation circuit 630 (described later).

  The display circuit 660 converts this mixed data into an analog video signal.

  The display device 700 displays the ultrasonic tomographic image and the three-dimensional guide image side by side based on the analog video signal. A display example is shown in FIG.

Sixth, the operation of the first embodiment will be described along the flow of signals and data related to control.
The three-dimensional guide image creation circuit 630, the mixing circuit 650, the reference image storage unit 610, and the display circuit 660 in the ultrasonic image processing apparatus 600 are controlled by commands from the control circuit 670. Details of the control will be described later together with the detailed operation of the ultrasonic image processing apparatus 600.

  FIG. 7 is a flowchart for explaining the entire operation of the ultrasonic image processing apparatus 600, the keyboard 720, the mouse 710, and the display apparatus 700 according to the first embodiment.

  As shown in FIG. 7, the flow of the first embodiment includes (S-1) an interest organ extraction process, (S-2) a reference point designation process, (S-3) a body cavity surface feature point designation process, (S-4) Deep body feature point designation processing, (S-5) Three-dimensional guide image creation / display processing. FIG. 8, FIG. 11, FIG. 13, FIG. 15 and FIG. In the first embodiment, these processes are executed in this order. Details of these processes will be described below.

(S-1) Description of Interesting Organ Extraction Processing Details of the interested organ extraction processing will be described with reference to FIG. In the first embodiment, processing for extracting pancreas, aorta, superior mesenteric vein, and duodenum as an organ of interest will be described as an example.

(S-1-1)
The surgeon presses the display switching key β. The switch 661 of the display circuit 660 is switched to the input terminal β.
(S-1-2)
The control circuit 670 causes the display circuit 660 to read reference image data from the reference image storage unit 610. The control circuit 670 reads the first reference image data for the first time.

(S-1-3)
The display circuit 660 converts the reference image data into an analog video signal, and outputs the analog video signal to the display device 700. The display device 700 displays a reference image.

(S-1-4)
The surgeon confirms whether or not the organ of interest is shown in the reference image on the display screen of the display device 700. If the organ of interest is shown, the process jumps to (S-1-6), and if not, the process jumps to (S-1-5).

(S-1-5)
The surgeon presses a predetermined key on the keyboard 720 or clicks a menu on the screen with the mouse 710 and reselects the reference image data to be displayed as other reference image data. In the first embodiment, for example, the operator instructs to select the reference image data of the next number. Thereafter, the control circuit 670 jumps the process to (S-1-2) and repeats the process.

(S-1-6)
The surgeon designates the organ of interest shown on the display screen of the display device 700. This is shown in FIG. In FIG. 9, a reference image corresponding to the nth reference image data is displayed. On the display screen, the reference image is displayed in a size that almost fits the entire cross section of the human body perpendicular to the body axis, and is color-coded by organ for each pixel. In FIG. 9, the pancreas is light blue, the aorta is red, the superior mesenteric vein is purple, and the duodenum is yellow. A cursor that can be moved with the mouse 710 is displayed on the display screen. The operator sequentially moves the cursor over the pancreas, aorta, superior mesenteric vein, and duodenum that are of interest, and presses an organ designating key 721 on each.

(S-1-7)
The extraction circuit 620 extracts pixels corresponding to the designated organ of interest over all reference image data from No. 1 to N. For example, if pancreas, aorta, superior mesenteric vein, and duodenum are designated as organs of interest in (S-1-6), pixels of light blue, red, purple, and yellow are extracted over all reference image data. .
(S-1-8)
The extraction circuit 620 interpolates the extracted data for each reference image data so as to allocate data to all the voxels VX in the voxel space VXS. The interpolated pixel data is collectively referred to as extracted data.

(S-1-9)
The extraction circuit 620 writes the extracted data to the voxel space VXS in the volume memory 640. At this time, the extraction circuit 620 writes the extracted data to the voxel VX in the voxel space VXS having an address corresponding to the coordinate on the orthogonal coordinate axis O′-x′y′z ′ of each pixel. Here, the extraction circuit 620 corresponds to the voxel VX corresponding to the pixel extracted in (S-1-7) with the colored data of the pixel between the pixels extracted in (S-1-7). Data obtained by interpolating the pixel is assigned to the voxel VX, 0 (transparent) is assigned to the other voxels VX, and data is assigned to all the voxels VX in the voxel space VXS to construct dense data.

  The extracted data written to the voxel space VXS is shown in FIG. FIG. 10 is a diagram when the pancreas, aorta, superior mesenteric vein, and duodenum are designated as the organs of interest. In FIG. 10, the duodenum is omitted so that the shape of the organ of interest can be easily understood.

(S-2) Description of Reference Point Designation Process Details of the reference point designation process will be described with reference to FIG. In the first embodiment, as a reference point, an example is a process of designating a xiphoid process, duodenal papilla (exit to the duodenum of the common bile duct), cardia (stomach entrance), pylorus (stomach exit), and blood vessel branching I will explain it. In the following description of the first embodiment, the branching of the blood vessel is a branch point where the celiac artery is divided into the common hepatic artery and the splenic artery (hereinafter referred to as the celiac artery branch). The area corresponding to the celiac artery branch is smaller than the “pelvic right end” or the like regardless of the subject. Further, since the short celiac artery is immediately connected to the aorta, it exists at a fixed position that does not move relatively regardless of the posture of the subject. This branch point exists in the deep part of the body immediately behind the pancreas and is not exposed to the digestive tract such as the stomach and duodenum.

(S-2-1) to (S-2-3)
This is the same as (S-1-1) to (S-1-3) in FIG.
(S-2-4)
The surgeon checks on the display screen of the display device 700 whether or not the reference point is shown in the reference image. If the reference point is shown, the process jumps to (S-2-6), and if not, the process jumps to (S-2-5).
(S-2-5)
This is the same as (S-1-5) in FIG.

(S-2-6)
The surgeon designates a reference point shown on the display screen of the display device 700. This is shown in FIG. In FIG. 12, a reference image corresponding to the m-th reference image data is displayed. On the display screen, the reference image is displayed in a size that almost fits the entire cross section of the human body perpendicular to the body axis, and is color-coded by organ for each pixel. In FIG. 12, sword-shaped projections are displayed. A cursor that can be moved with the mouse 710 is displayed on the display screen. The surgeon moves the cursor over a point of interest (the xiphoid process) as a reference point, and presses a reference point designation key 722 thereon.

(S-2-7)
The extraction circuit 620 writes the direction component of the position vector of the designated reference point on the orthogonal coordinate axis O′-x′y′z ′ to the volume memory 640.
(S-2-8)
The control circuit 670 terminates the reference point designation process when the designation of the five reference points has been completed, and otherwise jumps the process to (S-2-2) and repeats the process.

  The five reference points designated by the operator in this way are designated as reference points L ′, P0 ′, P1 ′, P2 ′, and P3 ′ in the designated order. In the first embodiment, the reference point L 'is described as a xiphoid process, P0' is a duodenal papilla, P1 'is a cardia, P2' is a pylorus, and P3 'is a celiac artery branch.

The extraction circuit 620 includes each directional component (x _L ′, y _L ′, z _L ′) on the orthogonal coordinate axis O′-x′y′z ′ of the position vector O′L ′ in the order in which the reference points are designated, Each direction component (xp0 ', yp0', zp0 ') in the orthogonal coordinate axis O'-x'y'z' of the position vector O'P0 'and the orthogonal coordinate axis O'-x'y' of the position vector O'P1 ' Each direction component (xp1 ′, yp1 ′, zp1 ′) in z ′ and each direction component (xp2 ′, yp2 ′, zp2 ′) in the orthogonal coordinate axis O′-x′y′z ′ of the position vector O′P2 ′ And the directional components (xp3 ′, yp3 ′, zp3 ′) of the position vector O′P3 ′ in the orthogonal coordinate axes O′-x′y′z ′ are written in the volume memory 640. Since each reference image data has a fixed size of 60 cm on a side and is parallel at a fixed 1 mm pitch, the extraction circuit 620 can calculate each direction component.
Since each directional component in the orthogonal coordinate axis O′-x′y′z ′ can be defined as described above, the following equation holds.

（Ｓ−３）体腔表面特徴点指定処理の説明
図１３を参照して、体腔表面特徴点指定処理の詳細を説明する。以下に説明する「特徴点」Ｐ０、Ｐ１、Ｐ２、Ｐ３は、「参照点」Ｐ０’、Ｐ１’、Ｐ２’、Ｐ３’に解剖学的に対応する、被検者の体腔表面上もしくは体内深部の実際の点である。本第１の実施の形態では特徴点として、参照点のうち剣状突起を除く十二指腸乳頭、噴門、幽門、腹腔動脈分岐を指定する処理を例にあげて説明する。 [Equation 4]

(S-3) Description of Body Cavity Surface Feature Point Designation Processing Details of the body cavity surface feature point designation processing will be described with reference to FIG. “Feature points” P0, P1, P2, and P3 described below are anatomically corresponding to “reference points” P0 ′, P1 ′, P2 ′, and P3 ′ on the surface of the body cavity of the subject or deep in the body. Is the actual point. In the first embodiment, processing for designating the duodenal papilla excluding the xiphoid process, the cardia, the pylorus, and the celiac artery branch as reference points will be described as an example.

Since the reference point P0 ′ and the feature point P0 are the duodenal papilla, the reference point P1 ′ and the feature point P1 are cardia, the reference point P2 ′ and the feature point P2 are the pylorus, and the reference point P3 ′ and the feature point P3 are the celiac artery branch. The feature points P0, P1, and P2 are points on the surface of the body cavity of the subject, and the feature point P3 is a point deep in the body of the subject.
Therefore, in the flow of the body cavity surface feature point designation process described below (S-3), the duodenal papilla of the feature point P0 on the surface of the body cavity of the subject, the cardia of the feature point P1, and the pylorus of the feature point P2 are included. Only the specified process will be described.

(S-3-1)
The surgeon presses the display switching key α. The switch 661 of the display circuit 660 is switched to the input terminal α.
(S-3-2)
The display circuit 660 converts the optical image data from the optical observation device 300 into an analog video signal, and outputs the analog video signal to the display device 700. The display device 700 displays an optical image.

(S-3-3)
The operator inserts the rigid part 210 and the flexible part 220 into the body cavity of the subject, looks for the feature point while observing the optical image, and moves the rigid part 210 to the vicinity of the feature point (duodenal papilla).

(S-3-4)
While observing the optical image, the operator inserts the position detection probe 530 from the forceps opening 201 and protrudes from the protruding opening 202. Then, the operator brings the tip of the position detection probe 530 into contact with the feature point (duodenal papilla) under the optical image field.
This is shown in FIG. FIG. 14 shows an optical image display screen. In FIG. 14, an optical image is displayed on the display screen. The optical image shows the duodenal papilla and the position detection probe 530. The operator brings the tip of the position detection probe 530 into contact with the duodenal papilla while viewing the optical image.

(S-3-5)
The operator presses the body cavity surface feature point designation key 723. This time is defined as t0.
(S-3-6)
The three-dimensional guide image creation circuit 630 takes in position / orientation data from the position / orientation calculation apparatus 500.
The three-dimensional guide image creation circuit 630 acquires each direction component on the orthogonal coordinate axis O-xyz of the position vector OC (t0) of the position C (t0) of the receiving coil 532 at the tip of the position detection probe 530 from the position / orientation data.

  The position vector OP0 (t0) of the position P0 of the duodenal papilla and its respective direction components xp0 (t0), yp0 (t0), zp0 (t0) on the orthogonal coordinate axis O-xyz have OP0 (t0) equal to OC (t0). Therefore, it can be expressed by the following formula.

このとき、３次元ガイド画像作成回路６３０は位置配向算出装置５００から同時に再び姿勢検出プレート５２０の基準位置Ｌ（ｔ０）の位置ベクトルＯＬ（ｔ０）の直交座標軸Ｏ−ｘｙｚにおける各方向成分を取得する。
さらに、３次元ガイド画像作成回路６３０は位置配向算出装置５００から同時に姿勢検出プレート５２０の基準点Ｌ（ｔ０）の配向を示す回転行列Ｔ（ｔ０）とを取得する。
位置ベクトルＯＬ（ｔ０）と回転行列Ｔ（ｔ０）とは被検者の体位の変化による、特徴点Ｐ０の位置の変化を補正するときに用いる。 [Equation 5]

At this time, the three-dimensional guide image creation circuit 630 simultaneously obtains each direction component on the orthogonal coordinate axis O-xyz of the position vector OL (t0) of the reference position L (t0) of the posture detection plate 520 simultaneously from the position / orientation calculation apparatus 500. .
Furthermore, the three-dimensional guide image creation circuit 630 simultaneously obtains a rotation matrix T (t0) indicating the orientation of the reference point L (t0) of the posture detection plate 520 from the position / orientation calculation device 500.
The position vector OL (t0) and the rotation matrix T (t0) are used when correcting the change in the position of the feature point P0 due to the change in the body position of the subject.

  The three-dimensional guide image creation circuit 630 can acquire the respective direction components of the OP0 (t0) and OL (t0) on the orthogonal coordinate axis O-xyz and the rotation matrix T (t0) at the time t0.

(S-3-7)
The three-dimensional guide image creation circuit 630 writes OP0 (t0), OL (t0) direction components on the orthogonal coordinate axis O-xyz at this time t0, and the rotation matrix T (t0) to the volume memory 640.
(S-3-8)
If there is an input to the feature point designation stop key 725 from the operator by this step, the control circuit 670 ends the body cavity surface feature point designation process. In other cases, the process jumps to (S-3-3), and the processes from (S-3-3) to (S-3-7) are repeated.

  In the above (S-3-3) to (S-3-7), the feature point on the surface of the body cavity has been described as the duodenal papilla. However, in the first embodiment, (S-3-3) to (S In the process of repeating the process up to -3-7), the same action is performed in the order of cardia and pylorus. This action is referred to as (S-3-3) 'to (S-3-7)' in the cardia action, and (S-3-3) "to (S-3-7)" in the pylorus action. Shown in It is omitted from FIG.

After inputting three body cavity surface feature points of the duodenal papilla, cardia, and pylorus, the operator presses a feature point designation stop key 725. It is omitted from FIG.
The second iteration is as follows.

(S-3-3) '
The surgeon inserts the rigid portion 210 and the flexible portion 220 into the body cavity of the subject, looks for the feature point while observing the optical image, and moves the rigid portion 210 to the vicinity of the feature point (cardia).
(S-3-4) '
While observing the optical image, the operator inserts the position detection probe 530 from the forceps opening 201 and protrudes from the protruding opening 202. Then, the operator brings the tip of the position detection probe 530 into contact with the feature point (cardia) under the optical image field.
(S-3-5) '
The operator presses the body cavity surface feature point designation key 723. This time is defined as t1.

(S-3-6) '
The three-dimensional guide image creation circuit 630 takes in position / orientation data from the position / orientation calculation apparatus 500.
The three-dimensional guide image creation circuit 630 obtains each direction component on the orthogonal coordinate axis O-xyz of the position vector OC (t1) of the position C (t1) of the receiving coil 532 at the tip of the position detection probe 530 from the position / orientation data.
The position vector OP1 (t1) of the position P1 of the duodenal papilla and the respective direction components xp1 (t1), yp1 (t1), and zp1 (t1) on the orthogonal coordinate axis O-xyz have OP1 (t1) equal to OC (t1). Therefore, it can be expressed by the following formula.

このとき、３次元ガイド画像作成回路６３０は位置配向算出装置５００から同時に再び姿勢検出プレート５２０の基準位置Ｌ（ｔ１）の位置ベクトＯＬ（ｔ１）の直交座標軸Ｏ−ｘｙｚにおける各方向成分を取得する。
さらに、３次元ガイド画像作成回路６３０は位置配向算出装置５００から同時に姿勢検出プレート５２０の基準点Ｌ（ｔ１）の配向を示す回転行列Ｔ（ｔ１）とを取得する。
位置ベクトルＯＬ（ｔ１）と回転行列Ｔ（ｔ１）とは被検者の体位の変化による、特徴点Ｐ１の位置の変化を補正するときに用いる。 [Equation 6]

At this time, the three-dimensional guide image creation circuit 630 simultaneously obtains each direction component on the orthogonal coordinate axis O-xyz of the position vector OL (t1) of the reference position L (t1) of the posture detection plate 520 simultaneously from the position / orientation calculation apparatus 500. .
Further, the three-dimensional guide image creation circuit 630 simultaneously obtains a rotation matrix T (t1) indicating the orientation of the reference point L (t1) of the posture detection plate 520 from the position / orientation calculation device 500.
The position vector OL (t1) and the rotation matrix T (t1) are used when correcting the change in the position of the feature point P1 due to the change in the posture of the subject.

The three-dimensional guide image creation circuit 630 has acquired each direction component of the OP1 (t1) and OL (t1) on the orthogonal coordinate axis O-xyz and the rotation matrix T (t1) at the time t1.
(S-3-7) '
The three-dimensional guide image creation circuit 630 writes the OP1 (t1), OL (t1) direction components on the orthogonal coordinate axis O-xyz, and the rotation matrix T (t1) at the time t1 to the volume memory 640.
The third iteration is as follows.

(S-3-3) "
The surgeon inserts the rigid portion 210 and the flexible portion 220 into the body cavity of the subject, looks for the feature point while observing the optical image, and moves the rigid portion 210 to the vicinity of the feature point (pylorus).
(S-3-4) "
While observing the optical image, the operator inserts the position detection probe 530 from the forceps opening 201 and protrudes from the protruding opening 202. Then, the operator brings the tip of the position detection probe 530 into contact with the feature point (pylorus) under the optical image field.
(S-3-5) "
The operator presses the body cavity surface feature point designation key 723. This time is defined as t2.

(S-3-6) "
The three-dimensional guide image creation circuit 630 takes in position / orientation data from the position / orientation calculation apparatus 500.
The three-dimensional guide image creation circuit 630 acquires each directional component of the position vector OC (t2) of the position vector OC (t2) of the position C (t2) of the receiving coil 532 at the tip of the position detection probe 530 from the position / orientation data.
The position vector OP2 (t2) of the position P2 of the duodenal papilla and the respective direction components xp2 (t2), yp2 (t2), and zp2 (t2) on the orthogonal coordinate axis O-xyz have OP2 (t2) the same as OC (t2). Therefore, it can be expressed by the following formula.

このとき、３次元ガイド画像作成回路６３０は位置配向算出装置５００から同時に再び姿勢検出プレート５２０の基準位置Ｌ（ｔ２）の位置ベクトルＯＬ（ｔ２）の直交座標軸Ｏ−ｘｙｚにおける各方向成分を取得する。
さらに、３次元ガイド画像作成回路６３０は位置配向算出装置５００から同時に姿勢検出プレート５２０の基準点Ｌ（ｔ２）の配向を示す回転行列Ｔ（ｔ２）とを取得する。
位置ベクトルＯＬ（ｔ２）と回転行列Ｔ（ｔ２）とは被検者の体位の変化による、特徴点Ｐ２の位置の変化を補正するときに用いる。 [Equation 7]

At this time, the three-dimensional guide image creation circuit 630 simultaneously acquires each direction component on the orthogonal coordinate axis O-xyz of the position vector OL (t2) of the reference position L (t2) of the posture detection plate 520 simultaneously from the position / orientation calculation apparatus 500. .
Further, the three-dimensional guide image creation circuit 630 simultaneously obtains a rotation matrix T (t2) indicating the orientation of the reference point L (t2) of the posture detection plate 520 from the position / orientation calculation device 500.
The position vector OL (t2) and the rotation matrix T (t2) are used when correcting the change in the position of the feature point P2 due to the change in the body position of the subject.

The three-dimensional guide image creation circuit 630 is able to acquire the respective directional components of the OP2 (t2) and OL (t2) on the orthogonal coordinate axis O-xyz and the rotation matrix T (t2) at the time t2.
(S-3-7) "
The three-dimensional guide image creation circuit 630 writes OP2 (t2) and each direction component of the OL (t2) on the orthogonal coordinate axis O-xyz and the rotation matrix T (t2) to the volume memory 640 at the time t2.

(S-4) Description of Deep Body Feature Point Specifying Process Details of the deep body feature point specifying process will be described with reference to FIG. Since the reference point P3 ′ and the feature point P3 are celiac artery branches, the feature point P3 is a point deep in the body of the subject.
(S-4-1)
The operator matches the positions of the forceps port marker 534 of the position detection probe 530 and the opening surface of the forceps port 201. At this time, the position of the tip of the position detection probe 530 and the position of the opening surface of the projection port 202 match, and the reception coil 532 is very close to the rotational center of radial scanning of the ultrasonic transducer 214.
Further, the operator rotates the position detection probe 530 until the positions of the 12 o'clock direction marker 535 of the position detection probe 530 and the 12 o'clock direction marker 536 provided around the forceps port 201 of the operation unit 230 coincide with each other. At this time, the vector Vc in FIG. 2 matches the vector V in FIG. 1, the vector Va in FIG. 2 matches the vector V3 in FIG. 1, and the vector Vb in FIG. 2 matches the vector V12 in FIG.

Then, the operator fixes the position detection probe 530 so as not to move in the forceps channel 203.
By fixing in this way, the contents of the position / orientation data are interpreted as follows.

Since the receiving coil 532 is fixed in the vicinity of the ultrasonic transducer 214, the OC (t) of the receiving coil 532 can be considered as a position vector of the rotational center position of the ultrasonic transducer 214 and can be used in actual use.
Since the direction unit vector Vc indicating the first winding axis direction of the receiving coil 532 coincides with the vector V in FIG. 1, Vc (t) is the normal direction of the radial scanning plane of the ultrasonic transducer 214, that is, the ultrasonic tomography. It can be considered as a vector V indicating the normal direction of the image data for practical use.

Since the direction unit vector Vb (t) indicating the second winding axis direction of the receiving coil 532 coincides with the vector V12 in FIG. 1, Vb (t) indicates the 12 o'clock direction of the radial scanning surface of the ultrasonic transducer 214. Considering the vector V12, there is no problem in actual use.
In the following description, V and V12 are time functions V (t) and V12 (t), Vc (t) is replaced with V (t), and Vb (t) is replaced with V12 (t).

(S-4-2)
The surgeon presses the display switching key γ. The switch 661 of the display circuit 660 is switched to the input terminal γ.
(S-4-3)
The operator presses the scan control key 726. The ultrasonic transducer 214 starts radial scanning.
(S-4-4)
The display circuit 660 converts the ultrasonic tomographic image data from the ultrasonic observation apparatus 400 into an analog video signal, and outputs the ultrasonic tomographic image to the display apparatus 700. The display device 700 displays an ultrasonic tomographic image.

(S-4-5)
The operator moves the flexible part 220 and the rigid part 210 while observing the ultrasonic tomographic image, changes the direction, searches for the characteristic point (abdominal artery branch), and draws the characteristic point on the ultrasonic tomographic image.
This is shown in FIG. FIG. 16 shows an ultrasonic tomographic image display screen. In FIG. 16, the ultrasonic tomographic image is displayed with the cursor superimposed on the display screen. The ultrasonic tomogram shows the aorta, celiac artery, common hepatic artery, and splenic artery. The celiac artery branch is a portion where the celiac artery branches into the common hepatic artery and the splenic artery. In FIG. 16, for the convenience of explanation, the celiac artery branch is indicated by a dotted virtual circle. The celiac artery branch is a region that fits within a circle having a diameter of about 1 to 2 cm in the body of an actual subject.

(S-4-6)
The operator uses the mouse 710 to move the cursor as indicated by the dotted arrow in FIG.
FIG. 17 shows an ultrasonic tomographic image display screen at this time. The address from the center of the screen pointed by the operator with the cursor is indicated by (−Q1, −Q2). The address is positive at 12 o'clock in the vertical direction and at 3 o'clock in the horizontal direction.
The surgeon then presses a deep body feature point designation key 724. This time is defined as t3.
(S-4-7)
The three-dimensional guide image creation circuit 630 takes in position / orientation data from the position / orientation calculation apparatus 500. Specifically, it is the following data.

Each direction component in the orthogonal coordinate axis O-xyz of the position vector OC (t3) of the position C (t3) of the reception coil 532 The orthogonal coordinate axis O-xyz of the direction vector V (t3) indicating the first winding axis direction of the reception coil 532 Each direction component in the direction vector V12 (t3) indicating the second winding axis direction of the receiving coil 532 Each direction component in the orthogonal coordinate axis O-xyz The position vector OL (t3) of the reference position L (t3) of the attitude detection plate 520 Each directional component of the orthogonal coordinate axis O-xyz of 3 × 3 rotation matrix T (t3) indicating the orientation of the posture detection plate 520
OC (t3), V (t3), and V12 (t3) are taken in, as will be described later, the position vector OP3 (t3) of the feature point P3 and the directional components xp3 (t3), yp3 on the orthogonal coordinate axes O-xyz. This is because (t3) and zp3 (t3) are calculated.

The reason why OL (t3) and T (t3) are taken in is that the three-dimensional guide image creation circuit 630 always corrects the current position of the feature point P3 that is always moved by the change in the posture of the subject as will be described later.
(S-4-8)
The three-dimensional guide image creation circuit 630 calculates the position vector OP3 (t3) of the feature point P3 at the time t3 using the address (−Q1, −Q2).
The rotation center position of the ultrasonic transducer 214, that is, the center of the ultrasonic tomographic image is the point C (t3), and its position vector is OC (t3).

Since the direction unit vector indicating the normal direction of the ultrasonic tomographic image is V (t3) and the direction unit vector indicating the 12 o'clock direction of the radial scanning plane is V12 (t3), it indicates the 3 o'clock direction of the ultrasonic tomographic image. The direction unit vector can be written as V12 (t3) × V (t3). The positional relationship between these three vectors V (t3), V12 (t3), V12 (t3) × V (t3) and the ultrasonic tomogram is shown in FIG. Here, V12 (t3) × V (t3) is an outer product of V12 (t3) and V (t3).
The position vector OP3 (t3) of the feature point P3 is obtained by the following equation from FIG.

ＯＣ（ｔ３）、Ｖ（ｔ３）、Ｖ１２（ｔ３）はいずれも直交座標軸Ｏ−ｘｙｚにおける各方向成分で表現できるので、腹腔動脈分岐の位置Ｐ３の位置ベクトルＯＰ３（ｔ３）の直交座標軸Ｏ−ｘｙｚにおける各方向成分ｘｐ３（ｔ３）、ｙｐ３（ｔ３）、ｚｐ３（ｔ３）は上式から求めることができる。ＯＰ３（ｔ３）とその直交座標軸Ｏ−ｘｙｚにおける各方向成分ｘｐ３（ｔ３）、ｙｐ３（ｔ３）、ｚｐ３（ｔ３）とは以下の関係がある。 [Equation 8]

Since OC (t3), V (t3), and V12 (t3) can all be expressed by respective direction components on the orthogonal coordinate axis O-xyz, the orthogonal coordinate axis O-xyz of the position vector OP3 (t3) of the position P3 of the celiac artery branch P3. Each direction component xp3 (t3), yp3 (t3), and zp3 (t3) in can be obtained from the above equation. There is the following relationship between OP3 (t3) and each directional component xp3 (t3), yp3 (t3), zp3 (t3) in the orthogonal coordinate axis O-xyz.

（Ｓ−４−９）
３次元ガイド画像作成回路６３０はこの時刻ｔ３でのＯＰ３（ｔ３）、ＯＬ（ｔ３）の直交座標軸Ｏ−ｘｙｚにおける各方向成分と、回転行列Ｔ（ｔ３）とをボリュームメモリ６４０に書き出す。
（Ｓ−４−１０）
制御回路６７０は、このステップまでに術者からの特徴点指定中止キー７２５ヘの入力がある場合には体内深部特徴点指定処理を終了させる。それ以外の場合には処理を（Ｓ−４−５）ヘジャンプさせ、（Ｓ−４−５）〜（Ｓ−４−９）までの処理を繰り返す。 [Equation 9]

(S-4-9)
The three-dimensional guide image creation circuit 630 writes OP3 (t3), OL (t3) direction components on the orthogonal coordinate axis O-xyz at this time t3, and the rotation matrix T (t3) to the volume memory 640.
(S-4-10)
When there is an input to the feature point designation stop key 725 from the operator by this step, the control circuit 670 ends the deep body feature point designation processing. In other cases, the process jumps to (S-4-5), and the processes from (S-4-5) to (S-4-9) are repeated.

  In the first embodiment, the feature point in the deep part of the body is described as one celiac artery branch, and after the operator inputs the celiac artery branch, the feature point designation stop key 725 is pressed and the example is not repeated. To do. This is because 4 feature points are required as well as the reference points, but 3 feature points of duodenal papilla, cardia and pylorus have already been designated in the body cavity surface feature point designation process, and the remaining 1 point is the deep body feature point. This is because there are four feature points.

(S-5) Description of 3D Guide Image Creation / Display Processing Details of the 3D guide image creation / display processing will be described with reference to FIG.
(S-5-1)
Same as (S-4-1).
(S-5-2),
The operator presses the display switching key δ. The switch 661 of the display circuit 660 is switched to the input terminal δ.

(S-5-3)
The three-dimensional guide image creation circuit 630 obtains each direction component from the volume memory 640 on the orthogonal coordinate axes O′-x′y′z ′ of the position vectors of the five reference points L ′, P0 ′, P1 ′, P2 ′, P3 ′. read out. Then, the three-dimensional guide image creation circuit 630 reads from the volume memory 640 each direction component on the orthogonal coordinate axis O-xyz and the rotation matrix of the position vectors of the four feature points P0, P1, P2, and P3.
(S-5-4)
The operator presses the scan control key 726. The ultrasonic transducer 214 starts radial scanning. In accordance with the radial scanning, ultrasonic tomographic image data is sequentially input from the ultrasonic observation apparatus 400 to the mixing circuit 650.

(S-5-5)
Each time the ultrasonic transducer 214 performs one radial scan and the ultrasonic observation apparatus 400 creates ultrasonic tomographic image data, and the ultrasonic tomographic image data is input from the ultrasonic observation apparatus 400 to the mixing circuit 650, The control circuit 670 issues a command to the three-dimensional guide image creation circuit 630.
The three-dimensional guide image creation circuit 630 takes in position / orientation data from the position / orientation calculation apparatus 500 in response to this command. This time is defined as ts.

  The three-dimensional guide image creation circuit 630 acquires the following data from the position / orientation data.

Each direction component in the orthogonal coordinate axis O-xyz of the position vector OC (ts) of the position C (ts) of the receiving coil 532 The orthogonal coordinate axis O-xyz of the direction vector V (ts) indicating the first winding axis direction of the receiving coil 532 Each direction component in the direction vector V12 (ts) indicating the second winding axis direction of the receiving coil 532 Each direction component in the orthogonal coordinate axis O-xyz The position vector OL (ts) of the reference position L (ts) of the attitude detection plate 520 Each directional component of the orthogonal coordinate axis O-xyz of 3 × 3 rotation matrix T (ts) indicating the orientation of the attitude detection plate 520
The reason why OC (ts), V (ts), and V12 (ts) are acquired is that the three-dimensional guide image creation circuit 630 always corrects the position and direction of the radial scanning plane correctly, as will be described later.

OL (ts) and T (ts) are acquired based on the current positions of the feature points P0, P1, P2, and P3 at which the three-dimensional guide image creation circuit 630 always moves due to the change in the posture of the subject, as will be described later. This is to correct correctly.
(S-5-6)
The three-dimensional guide image creation circuit 630 corrects and calculates the positions of the feature points P0, P1, P2, and P3 at time ts in consideration of changes in the body position of the subject. At this time, it is assumed that the body of the subject does not stretch or distort, and the positional relationship between L (sword-shaped projections), P0, P1, P2, and P3 is not changed in time.
Hereinafter, the calculation method of the position of the feature point P0 at the time ts will be specifically described as an example.

  Each direction component of the position vector OL (t) of the reference position L on the orthogonal coordinate axis O-xyz and the rotation matrix T (t) indicating the orientation of the posture detection plate 520 are sequentially changed according to the change in the posture of the subject. Change.

In (S-3-7), each direction component on the orthogonal coordinate axis O-xyz of OL (t0) at time t0 and the rotation matrix T (t0) are simultaneously written and stored in the volume memory 640. .
In (S-5-5), each directional component of the orthogonal coordinate axis O-xyz of OL (ts) at time ts and the rotation matrix T (ts) are simultaneously acquired by the three-dimensional image creation circuit.
Therefore, the change in the posture of the subject from time t0 to ts is a change between each direction component on the orthogonal coordinate axis O-xyz of OL (t0) and each direction component on the orthogonal coordinate axis O-xyz of OL (ts), and The rotation matrix T (t0) and the change between the rotation matrix T (ts) can be calculated.

  In (S-3-7), each directional component xp0 (t0), yp0 (t0), zp0 (t0) on the orthogonal coordinate axis O-xyz of the position vector OP0 (t0) of the feature point P0 at time t0 is a volume memory. 640 is written and stored.

  Therefore, each directional component xp0 (ts), yp0 (ts), zp0 (ts) on the orthogonal coordinate axis O-xyz of the position vector OP0 (ts) of the feature point P0 at time ts is the orthogonal coordinate axis O of OL (t0). Each direction component in -xyz, rotation matrix T (t0), each direction component in orthogonal coordinate axis O-xyz of OL (ts), and rotation matrix T (ts) are used, and xp0 (t0), yp0 (t0) ), Zp0 (t0).

That is, the position of the feature point P0 at time ts is the value already stored in the volume memory 640 in (S-3-7) and the position captured by the 3D guide image creation circuit 630 in (S-5-5). -Corrected and calculated from the orientation data in consideration of changes in the posture of the subject.
The calculation method of the positions of the feature points P1, P2, and P3 at time ts is the same. After all, the following values are corrected and calculated in consideration of the change in the posture of the subject.

Each direction component xp0 (ts), yp0 (ts), zp0 (ts) on the orthogonal coordinate axis O-xyz of OP0 (ts)
Each direction component xp1 (ts), yp1 (ts), zp1 (ts) in the orthogonal coordinate axis O-xyz of OP1 (ts)
Each direction component xp2 (ts), yp2 (ts), zp2 (ts) in the orthogonal coordinate axis O-xyz of OP2 (ts)
Each direction component xp3 (ts), yp3 (ts), zp3 (ts) in the orthogonal coordinate axis O-xyz of OP3 (ts)
(S-5-7)
Hereinafter, a description will be given with reference to FIGS. 19 and 20. FIG. 19 shows feature points and a radial scanning plane 631. FIG. 20 shows reference points and ultrasonic tomographic image markers 632.

  The three-dimensional guide image creation circuit 630 calculates the position and orientation of an index (hereinafter referred to as an ultrasonic tomographic image marker) 632 to indicate an ultrasonic tomographic image in the voxel space VXS together with the extracted data. FIG. 21 shows an ultrasonic tomographic image marker 632. The ultrasonic tomographic image marker 632 includes the ultrasonic tomographic image data output from the ultrasonic observation apparatus 400 with respect to one rotation of the ultrasonic transducer 214 in the voxel space VXS and the anatomical position and orientation. FIG.

  As shown in FIG. 20, the center position of the ultrasonic tomographic image marker 632 is the point C ′ (ts), the normal direction vector of the ultrasonic tomographic image marker 632 is V ′ (ts), and the 12 o'clock direction vector is V12 ′ (ts). ). Then, since the 3 o'clock direction vector is V12 ′ (ts) × V ′ (ts), the ultrasonic tomographic image marker 632 has a position vector at a point R ′ (ts) where the position vector satisfies the following expression as shown in FIG. It becomes a set. Here, V12 ′ (ts) × V ′ (ts) is an outer product of V12 ′ (ts) and V ′ (ts). X ′ is the distance in the 3 o'clock direction between the point R ′ (ts) and the point C ′ (ts), and Y ′ is the 12 o'clock direction between the point R ′ (ts) and the point C ′ (ts). Distance.

以下、（１）超音波断層像データの平面上の任意点と超音波断層像マーカ６３２上の点との対応関係、（２）超音波断層像マーカ６３２の位置と配向として、その中心位置、法線方向、１２時方向の算出方法の原理について説明する。
３次元ガイド画像作成回路６３０はこの原理の説明で最終的に得られる（３３）式と（３４）式とより中心位置Ｃ’（ｔｓ）の位置ベクトルＯ’Ｃ’（ｔｓ）とその直交座標軸Ｏ’−ｘ’ｙ’ｚ’における各方向成分を、（４０）式もしくは、（４４）式と（４５）式とで超音波断層像マーカ６３２の１２時方向ベクトルＶ１２’（ｔｓ）とその直交座標軸Ｏ’−ｘ’ｙ’ｚ’における各方向成分を、（５７）式と（５８）式とで超音波断層像マーカ６３２の法線方向ベクトルＶ’（ｔｓ）とその直交座標軸Ｏ’−ｘ’ｙ’ｚ’における各方向成分を算出する。 [Equation 10]

Hereinafter, (1) the correspondence between an arbitrary point on the plane of the ultrasonic tomographic image data and a point on the ultrasonic tomographic image marker 632, (2) the center position as the position and orientation of the ultrasonic tomographic image marker 632, The principle of the calculation method for the normal direction and the 12 o'clock direction will be described.
The three-dimensional guide image creation circuit 630 obtains the position vector O′C ′ (ts) of the center position C ′ (ts) and its orthogonal coordinate axes based on the equations (33) and (34) finally obtained by explanation of this principle. Each direction component in O′−x′y′z ′ is expressed by the following expression (40) or the 12 o'clock direction vector V12 ′ (ts) of the ultrasonic tomographic image marker 632 by the expressions (44) and (45). Each directional component on the orthogonal coordinate axis O′-x′y′z ′ is expressed by the normal direction vector V ′ (ts) of the ultrasonic tomographic image marker 632 and its orthogonal coordinate axis O ′ by the equations (57) and (58). Each directional component at −x′y′z ′ is calculated.

(1) Correspondence relationship between arbitrary point on plane of ultrasonic tomographic image data and point on ultrasonic tomographic image marker 632 Point R (ts) is an arbitrary point on radial scanning plane 631 on orthogonal coordinate axis O-xyz. And A position vector P0R (ts) between the point P0 and the point R (ts) can be expressed by the following equation by taking appropriate real numbers a, b, and c. In the following equation, all vectors are handled as functions of time.

一方、参照画像データ上の参照点Ｐ０’、Ｐ１’、Ｐ２’、Ｐ３’は、各々特徴点Ｐ０、Ｐ１、Ｐ２、Ｐ３に解剖学的に同じ位置として対応づけられている。さらに、人体の解剖学的な構造が人によらず凡そ同じであると考えられる。そのため、点Ｒ（ｔｓ）が特徴点を頂点とする三角錐Ｐ０Ｐ１Ｐ２Ｐ３に対して特定の位置にあるとすると、参照画像データ上の参照点を頂点とする三角錐Ｐ０’Ｐ１’Ｐ２’Ｐ３’に対して同等の位置にある点Ｒ’（ｔｓ）は点Ｒ（ｔｓ）と解剖学的に同じ器官、もしくは同じ組織上の点に相当すると仮定することができる。この仮定の下では、（１７）式のａ、ｂ、ｃを使って下式のように同様に表現できる直交座標軸Ｏ’−ｘ’ｙ’ｚ’上にある点Ｒ’（ｔｓ）こそが点Ｒ（ｔｓ）の解剖学的な対応点と言える。 [Equation 11]

On the other hand, the reference points P0 ′, P1 ′, P2 ′, and P3 ′ on the reference image data are associated with the feature points P0, P1, P2, and P3 as anatomically the same positions. Furthermore, it is thought that the anatomical structure of the human body is almost the same regardless of the person. Therefore, if the point R (ts) is at a specific position with respect to the triangular pyramid P0P1P2P3 having the feature point as a vertex, the triangular pyramid P0′P1′P2′P3 ′ having the reference point on the reference image data as a vertex is obtained. It can be assumed that the point R ′ (ts) in the equivalent position corresponds to a point on the same organ or tissue anatomically as the point R (ts). Under this assumption, the point R ′ (ts) on the orthogonal coordinate axis O′−x′y′z ′ that can be similarly expressed as in the following equation using a, b, and c in the equation (17). It can be said that it is an anatomical corresponding point of the point R (ts).

ここで、点Ｒ（ｔｓ）の位置ベクトルＯＲ（ｔｓ）の直交座標軸Ｏ−ｘｙｚにおける各方向成分をｘＲ（ｔｓ）、ｙＲ（ｔｓ）、ｚＲ（ｔｓ）と定義し、点Ｒ’（ｔｓ）の位置ベクトルＯ’Ｒ’（ｔｓ）の直交座標軸Ｏ’−ｘ’ｙ’ｚ’における各方向成分をｘＲ’（ｔｓ）、ｙＲ’（ｔｓ）、ｚＲ’（ｔｓ）と定義すると、下式が成り立つ。 [Equation 12]

Here, each direction component in the orthogonal coordinate axis O-xyz of the position vector OR (ts) of the point R (ts) is defined as xR (ts), yR (ts), zR (ts), and the point R ′ (ts) If each directional component of the position vector O′R ′ (ts) in the orthogonal coordinate axes O′−x′y′z ′ is defined as xR ′ (ts), yR ′ (ts), zR ′ (ts), Holds.

以下では、これまでに出てきた式を基に、直交座標軸Ｏ−ｘｙｚ上にあるラジアル走査面上の任意点Ｒ（ｔｓ）の解剖学的な対応点Ｒ’（ｔｓ）の位置ベクトルＯ’Ｒ’とその直交座標軸Ｏ’−ｘ’ｙ’ｚ’における各方向成分ｘＲ’（ｔｓ）、ｙＲ’（ｔｓ）、ｚＲ’（ｔｓ）とを求める。
まず、（１７）式より、下式が成り立つ。 [Equation 13]

In the following, based on the equations given so far, the position vector O ′ of the anatomical corresponding point R ′ (ts) of the arbitrary point R (ts) on the radial scanning plane on the orthogonal coordinate axis O-xyz. R ′ and each directional component xR ′ (ts), yR ′ (ts), zR ′ (ts) in the orthogonal coordinate axes O′-x′y′z ′ are obtained.
First, the following equation is established from the equation (17).

（２１）式に（１９）式と、（Ｓ−５−６）で算出したＯＰ０（ｔｓ）、ＯＰ１（ｔｓ）、ＯＰ２（ｔｓ）、ＯＰ３（ｔｓ）の直交座標軸Ｏ−ｘｙｚにおける各方向成分（ｘｐ０（ｔｓ）、ｙｐ０（ｔｓ）、ｚｐ０（ｔｓ））と、（ｘｐ１（ｔｓ）、ｙｐ１（ｔｓ）、ｚｐ１（ｔｓ））と、（ｘｐ２（ｔｓ）、ｙｐ２（ｔｓ）、ｚｐ２（ｔｓ））と、（ｘｐ３（ｔｓ）、ｙｐ３（ｔｓ）、ｚｐ３（ｔｓ））とを代入して、下式が成り立つ。 [Formula 14]

Each directional component on the orthogonal coordinate axis O-xyz of OP0 (ts), OP1 (ts), OP2 (ts), and OP3 (ts) calculated in (19) and (S-5-6) in (21) (Xp0 (ts), yp0 (ts), zp0 (ts)), (xp1 (ts), yp1 (ts), zp1 (ts)), (xp2 (ts), yp2 (ts), zp2 (ts) )) And (xp3 (ts), yp3 (ts), zp3 (ts)) are substituted, and the following equation is established.

ここで、以降の式を簡単に表現するために３×３行列Ｑ（ｔｓ）を下式で定義する。
［数１６］

（２３）式を（２２）式に代入して
［数１７］

故に下式が成り立つ。ここでＱ（ｔｓ）^−１はＱ（ｔｓ）の逆行列を意味する。 [Equation 15]

Here, a 3 × 3 matrix Q (ts) is defined by the following equation in order to easily express the following equations.
[Equation 16]

Substituting equation (23) into equation (22), [Equation 17]

Therefore, the following equation holds. Here, Q (ts) ⁻¹ means an inverse matrix of Q (ts).

一方、（１８）式からも（２１）式と同様な式が導かれる。
［数１９］

（２１）式に（１９）式と、（Ｓ−５−６）で算出したＯＰ０（ｔｓ）、ＯＰ１（ｔｓ）、ＯＰ２（ｔｓ）、OＰ３（ｔｓ）の直交座標軸Ｏ−ｘｙｚにおける各方向成分とを代入して（２２）式が導かれたのと同様に、（２６）式に（７）〜（１０）式と（２０）式とを代入して下式が成り立つ。 [Equation 18]

On the other hand, an equation similar to the equation (21) is derived from the equation (18).
[Equation 19]

Each directional component on the orthogonal coordinate axis O-xyz of OP0 (ts), OP1 (ts), OP2 (ts), and OP3 (ts) calculated in (19) and (S-5-6) in (21) In the same manner as the expression (22) is derived by substituting, the following expression is established by substituting the expressions (7) to (10) and (20) into the expression (26).

ここで、以降の式を簡単に表現するために３×３行列Ｑ’を下式で定義する。
［数２１］

（２８）式を（２７）式に代入して下式を得る。 [Equation 20]

Here, in order to easily express the following equations, a 3 × 3 matrix Q ′ is defined by the following equation.
[Equation 21]

Substituting equation (28) into equation (27) gives the following equation:

（２５）式で、ａ、ｂ、ｃが求まっているから、これを（２９）式へ代入して、下式を得る。
［数２３］

故に、下式を得る
［数２４］

こうして、（２０）式と（３１）式とで直交座標軸Ｏ−ｘｙｚ上にあるラジアル走査面６３１上の任意点Ｒ（ｔｓ）の解剖学的な対応点Ｒ’（ｔｓ）の位置ベクトルＯ’Ｒ’（ｔｓ）とその直交座標軸Ｏ’−ｘ’ｙ’ｚ’における各方向成分ｘＲ’（ｔｓ）、ｙＲ’（ｔｓ）、ｚＲ’（ｔｓ）とを求めることができた。つまり、ラジアル走査面６３１上の任意点Ｒ（ｔｓ）に対し、３次元ガイド画像作成回路６３０が（Ｓ−５−３）でボリュームメモリ６４０から読み出した５つの参照点と４特徴点の位置ベクトルの方向成分と回転行列とを用い、（２０）式と（３１）式とで計算できる点Ｒ’（ｔｓ）の集合こそが超音波断層像マーカ６３２なのである。 [Equation 22]

Since a, b, and c are obtained in equation (25), these are substituted into equation (29) to obtain the following equation.
[Equation 23]

Therefore, the following formula is obtained [Equation 24]

Thus, the position vector O ′ of the anatomical corresponding point R ′ (ts) of the arbitrary point R (ts) on the radial scanning plane 631 on the orthogonal coordinate axis O-xyz in the expressions (20) and (31). R ′ (ts) and each directional component xR ′ (ts), yR ′ (ts), zR ′ (ts) in the orthogonal coordinate axes O′−x′y′z ′ could be obtained. That is, the position vector of the five reference points and the four feature points read from the volume memory 640 by the three-dimensional guide image creation circuit 630 in (S-5-3) with respect to the arbitrary point R (ts) on the radial scanning plane 631. The set of points R ′ (ts) that can be calculated by the equations (20) and (31) using the direction component and the rotation matrix is the ultrasonic tomographic image marker 632.

(2) Calculation method of the center position, normal direction, and 12 o'clock direction of the ultrasonic tomographic image marker 632 The arbitrary point R (ts) on the radial scanning plane 631 on the orthogonal coordinate axis O-xyz in the equation (31), The correspondence relationship with the anatomical corresponding point R ′ (ts) on the orthogonal coordinate axis O′−x′y′z ′ could be obtained.
Hereinafter, calculation methods of the center position C ′ (ts), the normal direction vector V ′ (ts), and the 12 o'clock direction vector V12 ′ (ts) of the ultrasonic tomographic image marker 632 will be described.

(2) -1: Calculation of Center Position C ′ (ts) of Ultrasonic Tomographic Image Marker An anatomical corresponding point of the position C (ts) of the receiving coil 532 is defined as C ′ (ts). Since the point C (ts) is the rotation center of the ultrasonic transducer 214, it is eventually the center of the radial scanning surface 631. Therefore, the point C ′ (ts) is the center of the ultrasonic tomographic image marker 632.

  If each directional component of OC (ts) on the orthogonal coordinate axis O-xyz is xc (ts), yc (ts), zc (ts), the following equation is established.

Ｏ’Ｃ’（ｔｓ）の直交座標軸Ｏ’−ｘ’ｙ’ｚ’における各方向成分をｘｃ’（ｔｓ）、ｙｃ’（ｔｓ）、ｚｃ’（ｔｓ）とすれば下式が成り立つ。 [Equation 25]

If each directional component of O′C ′ (ts) on the orthogonal coordinate axis O′−x′y′z ′ is xc ′ (ts), yc ′ (ts), zc ′ (ts), the following equation is established.

（３１）式では点Ｒ（ｔｓ）がラジアル走査面６３１上の任意点であったため、点Ｒ（ｔｓ）を点Ｃ（ｔｓ）、点Ｒ’（ｔｓ）を点Ｃ’（ｔｓ）と考えれば、下式が成り立つ。 [Equation 26]

In the equation (31), the point R (ts) is an arbitrary point on the radial scanning plane 631, so the point R (ts) can be considered as the point C (ts) and the point R ′ (ts) as the point C ′ (ts). For example, the following equation holds.

（３３）式、（３４）式より参照画像データ上における超音波断層像マーカ６３２の中心位置Ｃ’（ｔｓ）が求められた。 [Equation 27]

The center position C ′ (ts) of the ultrasonic tomographic image marker 632 on the reference image data was obtained from the equations (33) and (34).

(2) -2: Calculation of the 12 o'clock direction vector V12 ′ (ts) of the ultrasonic tomographic image marker 632 On the radial scanning plane 631, a point at a unit distance in the 12 o'clock direction from the center C (ts) of the plane is R12. (Ts). Point R12 (ts) is not shown. An anatomical corresponding point of the point R12 (ts) is defined as a point R12 ′ (ts).
If each directional component of the OR12 (ts) on the orthogonal coordinate axis O-xyz is xR12 (ts), yR12 (ts), zR12 (ts), the following equation is established.

Ｏ’Ｒ１２’（ｔｓ）の直交座標軸Ｏ’−ｘ’ｙ’ｚ’における各方向成分をｘＲ１２’（ｔｓ）、ｙＲ１２’（ｔｓ）、ｚＲ１２’（ｔｓ）とすれば下式が成り立つ。
［数２９］

（３１）式では点Ｒ（ｔｓ）がラジアル走査面６３１上の任意点であったため、点Ｒ（ｔｓ）を点Ｒ１２（ｔｓ）、点Ｒ’（ｔｓ）を点Ｒ１２’（ｔｓ）と考えれば（３１）式より以下の式を得る。 [Equation 28]

If each directional component of O′R12 ′ (ts) on the orthogonal coordinate axes O′−x′y′z ′ is xR12 ′ (ts), yR12 ′ (ts), zR12 ′ (ts), the following equation is established.
[Equation 29]

In the equation (31), the point R (ts) is an arbitrary point on the radial scanning plane 631, so the point R (ts) can be considered as the point R12 (ts) and the point R ′ (ts) can be considered as the point R12 ′ (ts). From the formula (31), the following formula is obtained.

（３６）式、（３７）式よりラジアル走査面６３１の中心Ｃ（ｔｓ）から１２時方向へ単位距離にある点Ｒ１２（ｔｓ）の解剖学上の対応点Ｒ１２’（ｔｓ）が求められた。 [Equation 30]

From the equations (36) and (37), the anatomical corresponding point R12 ′ (ts) of the point R12 (ts) at a unit distance in the 12 o'clock direction from the center C (ts) of the radial scanning surface 631 was obtained. .

（３９）式より、Ｏ’Ｒ１２’（ｔｓ）−Ｏ’Ｃ’（ｔｓ）の方向が超音波断層像マーカ６３２の１２時方向ベクトルＶ１２’（ｔｓ）の方向であり、１２時方向ベクトルＶ１２’（ｔｓ）を求めるにはこれを単位長に規格化すれば良い。すなわち、下式が成り立つ。 At this time, with respect to the position vector of the point R12 (ts), the following expression is established by using the 12 o'clock direction vector V12 (ts) of the ultrasonic tomographic image.
[Equation 31]

From the equation (39), the direction of O′R12 ′ (ts) −O′C ′ (ts) is the direction of the 12 o'clock direction vector V12 ′ (ts) of the ultrasonic tomographic image marker 632, and the 12 o'clock direction vector V12. In order to obtain '(ts), this may be normalized to a unit length. That is, the following equation holds.

（３３）式と（３４）式とでＯ’Ｃ’（ｔｓ）が、（３６）式と（３７）式とでＯ’Ｒ１２’（ｔｓ）が既に求められているため、（４０）式でＶ１２’（ｔｓ）が求められたことになる。
ここで、より明快にＶ１２’（ｔｓ）を求める方法を述べておく。
因みに、（３４）式と（３７）式とを両辺差し引くと、下式が成り立つ。この左辺はＯ’Ｒ１２’（ｔｓ）−Ｏ’Ｃ’（ｔｓ）の直交座標軸Ｏ’−ｘ’ｙ’ｚ’における各方向成分である。右辺の｛ ｝の中はＯＲ１２（ｔｓ）−ＯＣ（ｔｓ）の直交座標軸Ｏ−ｘｙｚにおける各方向成分である。 [Formula 32]

Since O'C '(ts) is already obtained from the equations (33) and (34), and O'R12' (ts) is already obtained from the equations (36) and (37), the equation (40) Thus, V12 ′ (ts) is obtained.
Here, a method for obtaining V12 ′ (ts) more clearly will be described.
Incidentally, the following equation is established by subtracting both sides of Equation (34) and Equation (37). This left side is each directional component on the orthogonal coordinate axis O′-x′y′z ′ of O′R12 ′ (ts) −O′C ′ (ts). Items in {} on the right side are each direction component on the orthogonal coordinate axis O-xyz of OR12 (ts) -OC (ts).

さらに右辺の｛ ｝の中は、（３９）式よりＶ１２（ｔｓ）の直交座標軸Ｏ−ｘｙｚにおける各方向成分であり、３次元ガイド画像作成回路６３０は位置配向算出装置５００からこの各方向成分を（Ｓ−５−５）で取得している。Ｖ１２（ｔｓ）の直交座標軸Ｏ−ｘｙｚにおける各方向成分をｘｖ１２（ｔｓ）、ｙｖ１２（ｔｓ）、ｚｖ１２（ｔｓ）とすれば、（４１）式より下式を得る。
［数３４］

結局、これを（４０）式のように規格化して、超音波断層像マーカ６３２の１２時方向ベクトルＶ１２’（ｔｓ）が求められる。なお、こでＶ１２’（ｔｓ）の直交座標軸Ｏ’−ｘ’ｙ’ｚ’における各方向成分をｘｖ１２’（ｔｓ）、ｙｖ１２’（ｔｓ）、ｚｖ１２’（ｔｓ）とする。 [Equation 33]

Further, in {} on the right side is each direction component on the orthogonal coordinate axis O-xyz of V12 (ts) according to the equation (39). (S-5-5). If each direction component of the orthogonal coordinate axis O-xyz of V12 (ts) is xv12 (ts), yv12 (ts), zv12 (ts), the following expression is obtained from the expression (41).
[Formula 34]

Eventually, this is normalized as shown in Equation (40), and the 12 o'clock direction vector V12 ′ (ts) of the ultrasonic tomographic image marker 632 is obtained. Here, each direction component of the orthogonal coordinate axis O′−x′y′z ′ of V12 ′ (ts) is assumed to be xv12 ′ (ts), yv12 ′ (ts), and zv12 ′ (ts).

（２）−３：超音波断層像マーカ６３２の法線方向ベクトルＶ’（ｔｓ）の算出
超音波断層像マーカ６３２の法線方向ベクトルをＶ’（ｔｓ）とすると、Ｖ’（ｔｓ）は結局、超音波断層像マーカ６３２上の如何なるベクトルとも直交すれば良い。 [Equation 35]

(2) -3: Calculation of normal direction vector V ′ (ts) of ultrasonic tomographic image marker 632 If the normal direction vector of ultrasonic tomographic image marker 632 is V ′ (ts), V ′ (ts) is After all, it is sufficient to be orthogonal to any vector on the ultrasonic tomographic image marker 632.

  Incidentally, the points R 1 ′ (ts) and R 2 ′ (ts) are set as arbitrary points on the ultrasonic tomographic image marker 632.

  Since the points R1 ′ (ts) and R2 ′ (ts) should have anatomical corresponding points on the radial scanning plane 631, they are designated as points R1 (ts) and R2 (ts).

  Each directional component of the point R1 (ts) on the orthogonal coordinate axis O-xyz is expressed as xR1 (ts), yR1 (ts), zR1 (ts), and each directional component of the point R2 (ts) on the orthogonal coordinate axis O-xyz is expressed as xR2 (ts). ), YR2 (ts), zR2 (ts). At this time, the following equation holds.

点Ｒ１’（ｔｓ）の直交座標軸Ｏ’−ｘ’ｙ’ｚ’における各方向成分をｘＲ１’（ｔｓ）、ｙＲ１’（ｔｓ）、ｚＲ１’（ｔｓ）、点Ｒ２’（ｔｓ）の直交座標軸Ｏ’−ｘ’ｙ’ｚ’における各方向成分をｘＲ２’（ｔｓ）、ｙＲ２’（ｔｓ）、ｚＲ２’（ｔｓ）とする。このとき下式が成り立つ。
［数３７］

（３１）式より点Ｒ１（ｔｓ）と点Ｒ１’（ｔｓ）、点Ｒ２（ｔｓ）と点Ｒ２’（ｔｓ）には下式の関係がある。 [Equation 36]

Each directional component of the point R1 ′ (ts) on the orthogonal coordinate axis O′−x′y′z ′ is represented by xR1 ′ (ts), yR1 ′ (ts), zR1 ′ (ts), and the orthogonal coordinate axis of the point R2 ′ (ts). The directional components in O′−x′y′z ′ are xR2 ′ (ts), yR2 ′ (ts), and zR2 ′ (ts). At this time, the following equation holds.
[Equation 37]

From the equation (31), there is a relationship of the following equation between the point R1 (ts) and the point R1 ′ (ts), and the point R2 (ts) and the point R2 ′ (ts).

（５０）式と（５１）式とを両辺差し引くと、下式が成り立つ。 [Equation 38]

Subtracting both sides of equation (50) and equation (51) gives the following equation.

両辺に左からＱ（ｔｓ）Ｑ’^−１（ｔｓ）をかけると下式が成り立つ。
［数４０］

ここでラジアル走査面６３１の法線方向ベクトルＶ（ｔｓ）の直交座標軸Ｏ−ｘｙｚにおける各方向成分をｘｖ（ｔｓ）、ｙｖ（ｔｓ）、ｚｖ（ｔｓ）とすれば下式を得る。
［数４１］

ここで、Ｖ（ｔｓ）はラジアル走査面６３１の法線方向ベクトルだから、点Ｒ２（ｔｓ）を始点としＲ１（ｔｓ）を終点とするベクトルＲ２Ｒ１（ｔｓ）とは直交する。従って、下式が成り立つ。 [Equation 39]

When Q (ts) Q ′ ⁻¹ (ts) is applied to both sides from the left, the following equation is established.
[Equation 40]

Here, if the respective direction components on the orthogonal coordinate axis O-xyz of the normal direction vector V (ts) of the radial scanning plane 631 are xv (ts), yv (ts), and zv (ts), the following expression is obtained.
[Equation 41]

Here, since V (ts) is a normal vector of the radial scanning plane 631, it is orthogonal to the vector R2R1 (ts) starting from the point R2 (ts) and ending at R1 (ts). Therefore, the following equation holds.

右辺の｛ ｝の中に（５３）式を代入して、下式を得る。
［数４３］

ここで、Ｖ’（ｔｓ）の直交座標軸Ｏ’−ｘ’ｙ’ｚ’における各方向成分をｘｖ’（ｔｓ）、ｙｖ’（ｔｓ）、ｚｖ’（ｔｓ）とすれば下式を得る。
［数４４］

さらに各方向成分を以下のように定義する。 [Formula 42]

Substituting Expression (53) into {} on the right side, the following expression is obtained.
[Equation 43]

Here, if each directional component of the orthogonal coordinate axis O′-x′y′z ′ of V ′ (ts) is xv ′ (ts), yv ′ (ts), zv ′ (ts), the following equation is obtained.
[Equation 44]

Furthermore, each direction component is defined as follows.

このように定義して、（５６）式に代入すると下式が得られる。
［数４６］

すなわち、下式が得られる。
［数４７］

（６０）式は結局、Ｖ’（ｔｓ）が超音波断層像マーカ６３２上の任意の２点を結ぶベクトルに常に直交するということを意味しているから、（５７）式、（５８）式で与えられるＶ’（ｔｓ）は超音波断層像マーカ６３２の法線方向ベクトルである。このようにして（５７）式と（５８）式とで超音波断層像マーカ６３２の法線方向ベクトルＶ’（ｔｓ）が求められた。 [Equation 45]

By defining in this way and substituting into the equation (56), the following equation is obtained.
[Equation 46]

That is, the following formula is obtained.
[Equation 47]

After all, equation (60) means that V ′ (ts) is always orthogonal to a vector connecting any two points on the ultrasonic tomographic image marker 632, so equations (57) and (58) V ′ (ts) given by is a normal direction vector of the ultrasonic tomographic image marker 632. In this way, the normal direction vector V ′ (ts) of the ultrasonic tomographic image marker 632 was obtained by the equations (57) and (58).

(S-5-8)
Based on the position and orientation (center position, normal direction, 12 o'clock direction) of the ultrasonic tomographic image marker 632 obtained in (S-5-7), the three-dimensional guide image creation circuit 630 is shown in FIG. A parallelogram ultrasonic tomographic marker 632 with a time direction marker 633 is created. Then, the three-dimensional guide image creation circuit 630 writes the ultrasonic tomographic image marker 632 to the corresponding voxel VX in the voxel space VXS in the volume memory 640 based on the position and orientation thereof. Since the extracted data extracted and interpolated by the extraction circuit 620 has already been written in the voxel space VXS, the ultrasonic tomographic image marker 632 and the extracted data are combined data (hereinafter referred to as combined data). FIG. 22 shows the composite data.

Although the duodenum is omitted in FIG. 10, an index indicating the duodenal wall is superimposed on the ultrasonic tomographic image marker 632 in FIG. 22.
(S-5-9)
The three-dimensional guide image creation circuit 630 reads the composite data from the voxel space VXS in the volume memory 640. The three-dimensional guide image creation circuit 630 erases the ultrasonic tomographic image marker 632 from the voxel space VXS immediately after the reading.
(S-5-10)
The three-dimensional guide image creation circuit 630 creates known three-dimensional guide image data by applying known three-dimensional image processing such as shadow removal, shading addition, and coordinate transformation accompanying line-of-sight transformation based on the composite data. Then, the three-dimensional guide image creation circuit 630 outputs the three-dimensional guide image data to the mixing circuit 650.

(S-5-11)
The mixing circuit 650 arranges the ultrasonic tomographic image data input from the ultrasonic observation apparatus 400 and the three-dimensional guide image data input from the three-dimensional guide image creation circuit 630, and outputs them to the display circuit 660 as mixed data. The display circuit 660 converts the mixed data into an analog video signal and outputs it to the display device 700. FIG. 23 shows a display example in which a three-dimensional guide image 701 and an ultrasonic tomographic image 702 are displayed side by side. As shown in FIG. 23, the display device 700 displays an ultrasonic tomographic image 702 and a three-dimensional guide image 701 side by side. Each organ represented on the three-dimensional guide image 701 is displayed in a color that is originally color-coded by organ in the reference image data. On the three-dimensional guide image 701 in FIG. 23, the pancreas is displayed in light blue, the aorta is displayed in red, the superior mesenteric vein is displayed in purple, and the duodenum is displayed in yellow. An arrow A indicates that the ultrasonic tomographic image marker 632 moves in conjunction with the movement of the radial scanning plane 631.

(S-5-12)
The control circuit 670 confirms whether or not the operator presses the scan control key again during (S-5-5) to (S-5-11).
When the surgeon presses the scan control key 726 again, the above processing is ended here, and a scan control signal for commanding the radial scan control OFF is output to the ultrasound observation apparatus 400. In response to the scanning control signal, the ultrasonic observation apparatus 400 outputs a rotation control signal for controlling the rotation off to the motor 231. The motor 231 receives the rotation control signal and stops the rotation of the ultrasonic transducer 214.
If the surgeon has not pressed the scan control key 726 again, the process jumps to (S-5-5).

  In this way, by repeating the processing described in (S-5-5) to (S-5-11), the ultrasonic transducer 214 performs one radial scan, and the ultrasonic observation apparatus 400 performs the ultrasonic scanning. Each time ultrasonic tomographic image data is generated and ultrasonic tomographic image data is input from the ultrasonic observation apparatus 400 to the mixing circuit 650, a new three-dimensional guide image 701 is generated and displayed together with the new ultrasonic tomographic image 702. 700 is displayed on both sides of the display while being updated in real time. That is, the ultrasonic tomographic image marker 632 of the three-dimensional guide image 701 moves with respect to the extracted data along with the movement of the radial scanning surface 631 accompanying the manual operation of the flexible portion 220 and the rigid portion 210 of the surgeon. Go.

<Effect>
According to the first embodiment, an operator inputs the position of the celiac artery branch using the keyboard 720 and the mouse 710 on the ultrasonic tomographic image 702 created by the ultrasonic observation apparatus 400, and is three-dimensional. Since the guide image creation circuit 630 is configured to calculate the position of the celiac artery branch based on the position / orientation data, a smaller celiac artery branch such as “pelvic end” can be selected as a feature point. The point recorded in advance in the ultrasonic diagnostic apparatus 100 and the subject's point that is considered to be anatomically coincident with this point can be accurately matched anatomically without depending on the subject. Furthermore, since the short celiac artery is immediately connected to the aorta in the branch of the celiac artery and is present at a certain position where it does not move relatively regardless of the body position of the subject, the above-mentioned matching can be made accurate. Furthermore, the celiac artery branch is relatively easy to draw the ultrasonic tomographic image 702 than the pancreas, etc. and does not depend on the skill level of the operator, so these deep points in the body are adopted as feature points for creating a guide image. There is reasonableness to do.

  In the ultrasonic diagnostic apparatus 100, an ultrasonic endoscope 200 made of a flexible material that is inserted into the body cavity of the subject, in particular, allows the operator to determine the position of the scanning plane in the body cavity. Since it cannot be directly viewed, a lesioned part is estimated and displayed on an ultrasonic image, and a considerable level of skill is required to read this, which hinders the spread of the ultrasonic endoscope 200. According to the first embodiment, using the ultrasonic endoscope 200 provided with the flexible portion 220 made of a flexible material to be inserted into the body cavity of the subject, the reference image data Assuming that the arrangement of the organ of interest and the arrangement of the actual subject's organ are the same, the observation position of the ultrasonic tomographic image 702 corrected from the feature points is a three-dimensional guide constructed from reference image data by a calculation formula Since the ultrasonic tomographic image marker 632 is configured and operated so as to be superimposed on the image 701, the observation position by the ultrasonic tomographic image 702 can be displayed as a more easily understood three-dimensional guide image 701. For example, the surgeon observes the three-dimensional guide image 701 color-coded for each organ, and more accurately while recognizing which position of the subject is observed anatomically from the ultrasonic tomographic image 702. Diagnosis can be made. This is far more medically useful than the ultrasonic diagnostic apparatus 100 of the type irradiated from outside the subject's body, and particularly contributes to shortening the examination time and the learning time for beginners.

  Further, according to the first embodiment, it is assumed that the triangular pyramid generated by the four feature points and the four reference points corresponds anatomically if they have the same positional relationship. Since the image 701 is configured and operated, a change in the posture of the subject and a difference in physique can be automatically corrected, and the three-dimensional guide image 701 can be created more accurately.

  In addition, according to the first embodiment, the operator is configured and operated so that the ultrasonic image and the three-dimensional guide image 701 can be observed automatically and in real time during radial scanning. It can be seen where the ultrasonic tomographic image 702 corresponding to the living body anatomically corresponds to, and the operator can change the scanning plane of the ultrasonic endoscope 200 to various angles while viewing the three-dimensional guide image 701. The region of interest can be accurately observed using the three-dimensional guide image 701.

  In addition, according to the first embodiment, since the configuration and operation of generating the three-dimensional guide image 701 by detecting not only the position of the scanning plane but also the orientation of the three-dimensional guide image 701, the scanning plane for radial scanning is used. The orientation of the ultrasonic tomographic image marker 632 also changes, and the three-dimensional guide image 701 is created more accurately. Therefore, the surgeon accurately observes the region of interest using the three-dimensional guide image 701 even when the scanning plane of the ultrasonic endoscope 200 is changed to various angles near the region of interest while viewing the three-dimensional guide image 701. be able to.

  Further, according to the first embodiment, the reference image data obtained by changing the attributes of organs such as the pancreas, pancreatic duct, common bile duct, and portal vein in advance by color-coding or the like is used as the reference image data. As a result, the organ serving as an index on the three-dimensional guide image 701 can be observed in an easy-to-understand manner, and can be observed in the body cavity while viewing the three-dimensional guide image 701. The scanning plane of the ultrasonic endoscope 200 can be changed. This leads to a faster approach to a region of interest such as a lesion, which also contributes to shortening the examination time.

  Further, according to the first embodiment, the position detection probe 530 is brought into contact with a feature point different from the feature point in the deep part of the body, and the position of this other feature point is acquired. Can be specified for both the deep part of the body and the surface of the body cavity, and various appropriate feature points can be specified according to the region of interest. Therefore, the degree of freedom of taking the feature point in the vicinity of the region of interest is increased, and it can be assumed that the feature point moves together even when the region of interest moves with the movement of the ultrasonic endoscope 200. A guide image 701 can be created. Furthermore, feature points can be obtained near the region of interest, especially in pancreatic examinations and lung examinations. The closer to the feature points, the more the space contained in the convex triangular pyramid created by the feature points is the triangular pyramid. Since the calculation of the position and orientation of the ultrasonic tomographic image marker 632 is expected to be more accurate than the outer space, it is more accurate in the vicinity of the region of interest by acquiring feature points at appropriate locations in the body cavity. A three-dimensional guide image 701 can be created.

  In the first embodiment, reference points and feature points are set in the duodenal papilla, cardia, pylorus, and celiac artery branch. However, according to the first embodiment, the reference point and the feature point are configured and operated so that both the reference point and the feature point can be set through the instruction from the mouse 710 and the keyboard 720 and the output of the position / orientation calculation apparatus 500. If the bifurcation point between the bile duct and the main pancreatic duct is added, or if the region of interest is known before surgery, it is easy to set the reference point or feature point close to the region of interest. It is expected that the calculation of the position and orientation of the ultrasonic tomographic image marker 632 will be more accurate in the space included in the convex triangular pyramid created by the characteristic point, closer to the feature point than in the space outside the triangular pyramid. Therefore, a more accurate three-dimensional guide image 701 can be created near the region of interest. In addition, it is easy to change a point that is easy to draw in the ultrasonic tomographic image 702 to a feature point and adopt it.

  In addition, according to the first embodiment, the reference position of the posture detection plate 520 is fixed to the subject so as to overlap the xiphoid process, and the change in the position and orientation of the posture detection plate 520 is always acquired. Since the point is corrected and the 3D guide image 701 is created and operated, a more accurate 3D guide image 701 is created even if the subject's body position changes during feature point acquisition or radial scanning. can do.

  In the ultrasonic diagnostic apparatus known so far, such as the ultrasonic diagnostic apparatus disclosed in Japanese Patent Application Laid-Open No. 2004-113629 described as the prior art, the position and orientation of an external image and an ultrasonic image are determined. The method of specifying feature points for matching is unclear. In particular, in an ultrasonic diagnostic apparatus in which a flexible ultrasonic probe such as an ultrasonic endoscope is inserted into a body cavity, the organ of interest may be moved by the operation of the ultrasonic probe itself. There is a problem that the guide image becomes inaccurate when only the point on the body surface is referred to as the point and the position of the reference image is collated. However, according to the first embodiment, the ultrasonic endoscope 200 is provided with a bellows channel, and the position detection probe 530 is inserted from the forceps port 201 of the forceps channel 203 and protrudes from the protrusion port 202, and an optical image is obtained. Since the feature point of the body cavity surface is designated by bringing the tip of the position detection probe 530 into contact with the feature point under the field of view, the feature point of the body cavity surface is designated accurately under the optical image field of view, and an accurate guide image Can be created.

<Modification>
In the first embodiment, the reference points and feature points on the surface of the body cavity are taken as three points of the duodenal papilla, cardia and pylorus, and the reference point in the deep part of the body and one point of the celiac artery branch as the feature point. Examples of reference points and feature points are not limited thereto. For example, a reference point on the surface of the body cavity, two points of the duodenal papilla and pylorus as feature points, a reference point in the deep part of the body, and a celiac artery branch and other vascular branches as feature points (for example, the junction of the bile duct and main pancreatic duct) A total of four points may be designated, including two points. At this time, the body cavity surface feature point designation process (S-3-3) to (S-3-7) is performed twice, and the deep body feature point designation process (S-4-5) to (S-4). The process of -9) is repeated twice.

In the first embodiment, the ultrasonic endoscope 200 including the forceps channel 203 and the position detection probe 530 inserted through the forceps channel 203 are provided. However, the configuration is not limited thereto. Instead of providing the forceps channel 203, the rigid portion 210 may be provided with a dedicated ultrasonic endoscope having the receiving coil 532 incorporated therein.
In the first embodiment, the operator is configured and operated so that the operator can set the reference point through an instruction from the mouse 710 and the keyboard 720. However, if the region of interest and protocol to be examined are determined in advance, the factory At the time of shipment or the like, a set of several types of reference points is stored in the reference image storage unit by default, and the reference image storage unit 610 is obtained before acquiring feature points by an instruction from the operator via the control circuit from the mouse 710 and the keyboard 720. It may be configured and operated so as to read out an appropriate set from.

In the first embodiment, when setting the reference point, the surgeon sequentially presses a predetermined key on the keyboard 720 or clicks a menu on the screen with the mouse 710, so that the reference image is displayed. The data is arranged and operated so that the data is displayed on the screen of the display device 700 in the order from the smallest number to the second, third, fourth,... It may be configured and operated so that it is read and displayed on the display device 700 side by side.
In the first embodiment, the ultrasonic tomographic image marker 632 is configured and operated so that the center position, the normal direction, and the 12 o'clock direction are obtained, and the ultrasonic tomographic image marker 632 is obtained based on the obtained position. It was. However, the configuration and operation are such that four corresponding points are obtained by anatomically corresponding the four corner points of the ultrasonic tomographic image data, and the ultrasonic tomographic image marker 632 is obtained based on the four corresponding points. You may let them. In addition, the size of the three-dimensional guide image 701 is not determined from the four corner points of the ultrasonic tomographic image data, but the size of the three-dimensional guide image 701 is taken into account in consideration of the display size and the display magnification. The surgeon may specify the size by inputting a numerical value from the keyboard 720 or by selecting a size menu on the screen with the mouse 710.

In the first embodiment, the transmitting antenna 510 and the receiving coil 532 are used as the position detecting means and the position and orientation are detected and detected by the magnetic field. However, transmission and reception may be reversed. Alternatively, the position and orientation may be detected and detected by acceleration or other means instead of a magnetic field.
In the first embodiment, the origin O is set to a specific position of the receiving coil 532. However, the origin O may be set to another place where the positional relationship with the receiving coil 532 does not change. .

In the first embodiment, a radial scanning type ultrasonic endoscope is provided as the ultrasonic endoscope 200. However, instead of the radial scanning type ultrasonic endoscope, as a conventional technique, As disclosed in Japanese Patent Application Laid-Open No. 2004-113629, an electronic convex ultrasonic endoscope in which an ultrasonic transducer group is provided in a fan shape on one of the insertion shafts may be provided. The method is not limited.
Further, in the first embodiment, the reference image data is obtained by classifying the reference image data for each organ and then changing the attribute by color coding. However, the attribute is not limited to the color, and the luminance value and other modes are used. But it ’s okay.
Further, in the first embodiment, each organ on the three-dimensional guide image 701 is configured to be displayed by being color-coded for each organ. However, the present invention is not limited to the color-coded mode, and brightness, brightness, saturation, etc. Other embodiments may be used.

[Second Embodiment]
<Configuration>
FIG. 24 is a diagram showing a configuration of an ultrasonic diagnostic apparatus according to the second embodiment of the present invention.
The ultrasonic diagnostic apparatus 100A of the second embodiment differs from the ultrasonic diagnostic apparatus 100 of the first embodiment shown in FIG. 1 in the following points.
In the first embodiment, the flexible shaft 220 of the ultrasonic endoscope 200 is provided with the flexible shaft 221, and the motor 231 and the rotary encoder 232 are provided in the operation unit 230. However, in the second embodiment, they are provided. Instead, an electronic radial scanning type ultrasonic endoscope is used as the ultrasonic endoscope 200A. As shown in FIG. 24, in the rigid portion 210A of the ultrasonic endoscope 200A of the second embodiment, ultrasonic transducers are cut into strips and arranged as an annular array around the insertion axis. (Hereinafter, ultrasonic transducer array 215). Each ultrasonic transducer constituting the ultrasonic transducer array 215 is connected to the ultrasonic observation apparatus 400A via the operation unit 230A via a signal line.

The ultrasonic observation apparatus 400A of the second embodiment is equipped with a color flow mapping circuit (hereinafter referred to as CFM circuit) 410 as blood flow detection means.
Other configurations are the same as those of the first embodiment.

<Action>
The second embodiment differs from the first embodiment in the action of acquiring the ultrasonic tomographic image 702, particularly the action of radial scanning and blood flow detection. Only different points will be described below.

  Among the ultrasonic transducers constituting the ultrasonic transducer array 215, some and a plurality of ultrasonic transducers receive an excitation signal in the form of a pulse voltage from the ultrasonic observation apparatus 400A and are superconducting waves of the medium. Convert to sound waves. At this time, the ultrasonic observation apparatus 400A delays each excitation signal so that the time at which each excitation signal arrives at each ultrasonic transducer is different. This delay is applied so that a single ultrasonic beam is formed when the ultrasonic waves excited by the ultrasonic transducers are superimposed in the subject. The ultrasonic beam is irradiated to the outside of the ultrasonic endoscope 200A, and the reflected wave from the inside of the subject returns to each ultrasonic transducer along a path opposite to the ultrasonic beam. Each ultrasonic transducer converts the reflected wave into an electrical echo signal and transmits it to the ultrasonic observation apparatus 400A through a path opposite to the excitation signal.

The ultrasonic observation apparatus 400A performs ultrasonic scanning so that the ultrasonic beam rotates in a plane similar to that of the first embodiment in a plane perpendicular to the flexible part 220A (hereinafter, radial scanning surface 631). A plurality of ultrasonic transducers involved in beam formation are selected again, and an excitation signal is transmitted again. In this way, the transmission angle of the ultrasonic beam changes. By repeating this repeatedly, so-called electronic radial scanning is realized.
In the first embodiment, the rotary encoder 400A determines in which direction the 12 o'clock direction of the ultrasonic tomographic image data is generated with respect to the ultrasonic endoscope is the rotary encoder. In the second embodiment, the ultrasonic observation apparatus 400A reselects a plurality of ultrasonic transducers involved in the formation of the ultrasonic beam, and generates the excitation signal again. In order to transmit, it is determined by which ultrasonic transducer 400A selects as the 12 o'clock direction.

Furthermore, the CFM circuit 410 of the ultrasonic observation apparatus 400A performs a known process that applies the Doppler effect to the echo signal, detects the blood flow by detecting the movement of the moving body, particularly the red blood cells, and detects the blood flow velocity. For example, coloring data of a red hue is created.
The ultrasonic observation apparatus 400A creates one piece of digital ultrasonic tomographic image data perpendicular to the insertion axis of the flexible portion 220A for one electronic radial scan of the ultrasonic transducer array 215. At this time, the ultrasonic observation apparatus 400A superimposes coloring data on the blood flow portion of the ultrasonic tomographic image data. The ultrasonic observation apparatus 400A outputs the ultrasonic tomographic image data to the mixing circuit 650 in the ultrasonic image processing apparatus 600 and the input terminal γ of the switch 661 of the display circuit 660.

In the second embodiment, during the process of designating the celiac artery branch described in (S-4-6) of the first embodiment as a feature point in the deep part of the body, the aorta, the celiac artery, the common hepatic artery, The carotid artery is colored. This is shown in FIG. FIG. 25 shows an ultrasonic tomographic image display screen. The hatched portion in FIG. 25 is a blood vessel colored as a blood flow by the ultrasonic observation apparatus 400A.
Other operations are the same as those in the first embodiment.

<Effect>
In the second embodiment, the CFM circuit 410 detects the blood flow, the ultrasonic observation apparatus 400A superimposes the colored data on the blood flow portion of the ultrasonic tomographic image data, and the operator uses the keyboard 720 and the mouse 710. Since the position of the celiac artery branch is input on the ultrasonic tomographic image 702 in which the blood vessel is colored and the three-dimensional guide image creation circuit 630 calculates the position of the celiac artery branch based on the position / orientation data. When a blood vessel branch such as an arterial branch is specified as a feature point in the deep part of the body, it can be specified more easily and accurately.

  Further, in the first embodiment, mechanical radial scanning that rotates the ultrasonic transducer is employed, so that the flexible shaft 221 may be twisted. Then, due to the torsion of the flexible shaft 221, there is a possibility that an angle shift occurs between the angle output of the rotary encoder 232 and the actual ultrasonic transducer, and this is based on the ultrasonic tomographic images 702 and 3. There is a possibility that the 12 o'clock directions of the dimension guide image 701 are shifted from each other. However, according to the second embodiment, an ultrasonic endoscope that performs electronic radial scanning is adopted as the ultrasonic endoscope 200A, and the ultrasonic observation apparatus 400A has 12 in the 12 o'clock direction of the ultrasonic tomographic image 702. Since it is determined according to which ultrasonic transducer is selected as the time direction, it is possible to prevent this 12 o'clock direction deviation from occurring, and the three-dimensional guide image 701 displayed on the display screen of the display device 700 can be prevented. There is little shift in the 12 o'clock direction between the ultrasonic tomographic image marker 632 and its 12 o'clock direction marker and the ultrasonic tomographic image, and an accurate three-dimensional guide image 701 can be constructed.

Other effects are the same as those of the first embodiment.

<Modification>
In the second embodiment, the ultrasonic transducer is cut into strips at the tip of the flexible portion 220A of the ultrasonic endoscope 200A and arranged in an annular array around the flexible portion 220A. The acoustic transducer array 215 may be provided on the entire circumference of 360 ° or may be missing. For example, you may provide in a part of circumference | surroundings of the flexible part 220A like 270 degrees and 180 degrees.
Other modifications are the same as those in the first embodiment.

  According to the ultrasonic diagnostic apparatus of the present invention described above, the points recorded in advance in the ultrasonic diagnostic apparatus from different types of image data, and the points of the subject considered to be anatomically coincident with this point, Can be accurately matched anatomically.

  The present invention can be widely applied to ultrasonic diagnostic apparatuses.

1 is a diagram showing a configuration of an ultrasonic diagnostic apparatus according to a first embodiment of the present invention. The perspective view which shows the structure of the position detection probe of FIG. The perspective view which shows the structure of the attitude | position detection plate of FIG. The perspective view explaining the structure of the reference image memory | storage part of FIG. The figure which shows voxel space and a voxel. When the origin O is defined on the receiving coil and the orthogonal coordinate axis O-xyz and its orthonormal basis (unit vector in each axis direction) i, j, k are defined on the actual space where the operator examines the subject. Illustration. 3 is a flowchart for explaining operations of the ultrasonic image processing apparatus, the keyboard, the mouse, and the display device according to the first embodiment of the present invention. 8 is a flowchart for explaining details of an interest organ extraction process in FIG. 7. The figure which shows the state by which the reference image corresponding to the nth reference image data in a reference image memory | storage part was displayed. The figure which shows the extraction data written in the voxel space. The flowchart explaining the detail of the reference point estimation process in FIG. The figure which shows the state by which the reference image corresponding to the mth reference image data in a reference image memory | storage part was displayed. The flowchart explaining the detail of the body cavity surface feature point designation | designated process in FIG. The figure which shows the display screen of an optical image. FIG. 8 is a flowchart for explaining details of deep body feature point designation processing in FIG. 7. FIG. The figure which shows the display screen of an ultrasonic tomogram. The figure which shows the display screen of an ultrasonic tomogram. 8 is a flowchart for explaining details of the three-dimensional guide image creation / display process in FIG. 7. The figure which shows a feature point and a radial scanning surface. The figure which shows a reference point and an ultrasonic tomographic image marker. The figure which shows an ultrasonic tomogram marker. The figure which shows the data by which the ultrasonic tomogram marker and extraction data were synthesize | combined. The figure which shows the example of a display which displayed the ultrasonic tomogram and the three-dimensional guide image side by side. The figure which shows the structure of the ultrasonic diagnosing device of the 2nd Embodiment of this invention. The figure which shows the display screen of the ultrasonic tomogram from which the blood flow was detected.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100,100A ... Ultrasonic diagnostic apparatus 200,200A ... Ultrasound endoscope 214 ... Ultrasonic transducer 215 ... Ultrasonic transducer array 400,400A ... Ultrasonic observation apparatus (ultrasonic tomographic image creating means)
410 ... CFM circuit (blood flow detection means)
500 ... Position / orientation calculation device (detection means)
520 ... Posture detection plate 530 ... Position detection probe (position acquisition means)
610 ... Reference image storage unit (reference image data holding means)
630... 3D guide image creation circuit (guide image creation means)
660 ... display circuit 700 ... display device 710 ... mouse (designating means)
720 ... Keyboard (designating means)
723 ... Body cavity surface feature point designation key 724 ... Deep body feature point designation key

Claims (3)

An ultrasonic tomographic image creating means for creating an ultrasonic tomographic image from an ultrasonic signal obtained by transmitting and receiving ultrasonic waves into a living body;
Detecting means for detecting the position or orientation of the ultrasonic tomographic image;
Designating means for designating feature points on the living body;
Reference image data holding means for holding reference image data;
Collating the anatomical position of the feature point designated by the designation means and the reference point set on the reference image data, the position or orientation detected by the detection means, and the result of the collation And a guide image creating means for creating a guide image for guiding the anatomical position or orientation of the ultrasonic tomographic image,
The designating means designates the feature point on the ultrasonic tomographic image created by the ultrasonic tomographic image creating means;
The guide image creating unit is configured to determine the feature point based on the position of the feature point specified by the specifying unit on the ultrasonic tomographic image and the position or orientation of the ultrasonic tomographic image detected by the detecting unit. An ultrasonic diagnostic apparatus characterized in that a position in the living body is calculated. The ultrasonic tomographic image creation means includes blood flow detection means for detecting blood flow,
The ultrasonic tomographic image creating means creates an ultrasonic tomographic image in which blood flow is superimposed in a different manner,
The designation means designates the feature point on an ultrasonic tomographic image on which the blood flow is superimposed;
The ultrasonic diagnostic apparatus according to claim 1. The designation means comprises a position acquisition means for directly contacting the surface of the body cavity of a living body to obtain a position of a feature point different from the feature point;
The ultrasonic diagnostic apparatus according to claim 1.

Images