JP2002336188A - Endoscope system for measurement - Google Patents

Abstract

PROBLEM TO BE SOLVED: To grasp a distance from a subject and present data by displaying a three-dimensional image. SOLUTION: An endoscope system is equipped with a stereoscopic video image endoscope (hereinbelow referred to as an endoscope) 301, video processors 310L and 310R for processing the respective image signals of left and right images picked up by the endoscope 301 and a host computer 311 for storing the RGB signals outputted from the video processors 310L and 310R to apply gradation drawing processing to them.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a measurement endoscope apparatus capable of stereoscopically observing an observation object.

[0002]

2. Description of the Related Art In recent years, a stereoscopic endoscope apparatus in which two systems of image pickup means are provided at a position having a parallax of an endoscope so as to obtain a visual field having a three-dimensional effect is disclosed in, for example, Japanese Patent Application Laid-Open No. 2-119936. Various proposals have been made.

[0003] For observation of stereoscopic images, or
In order to use it as a display of a virtual reality device, a binocular spectacle type display (also called a head mounted display) used by wearing it on the head,
For example, Japanese Patent Application Laid-Open No. 11-69383 proposes a stereoscopic display device in which the stereoscopic display device including the head mounted display is applied to the medical field.

[0004]

Problems to be Solved by the Invention
The problems of the third publication are as follows: Since the device information is simply drawn on the left and right images, the device information may have a parallax exceeding the fusion limit and may not be stereoscopically viewed. In that case, the device information may hinder the stereoscopic viewing of the endoscope image. Also, at this time, when performing stereoscopic viewing of the endoscope image and when performing stereoscopic viewing of the device information, it is necessary for the user to adjust the device for stereoscopic viewing, and the user is forced to perform additional work. Become.

Japanese Patent Application Laid-Open No. 2-119836 has the following problems.

It is sometimes difficult to grasp the positional relationship between the cursor and the object only by displaying a three-dimensionally visible cursor, but there is no means for assisting this.・ The size of the target cannot be grasped unless the cursor is moved and measurement is not performed.

An object of the present invention is to display a figure which looks three-dimensional, grasp the distance to an object, present information, and confirm the parallax of the displayed figure.

A second object of the present invention is to make it possible to grasp the size of an object by displaying a scale corresponding to the distance to the object.

A third object of the present invention is to make it easy to grasp the positional relationship between the object and the graphic by displaying a plurality of three-dimensional graphics.

A fourth object of the present invention is to provide means for stopping the display of a graphic according to the distance to the object.

A fifth object of the present invention is to provide a means for executing a predetermined process according to an input in a stereoscopic state.

It is an object of the present invention to provide a means for executing a process corresponding to a designated place by designating an inside of a field of view in a stereoscopic state.

It is an object of the present invention to provide a means for changing a parameter of a figure to be displayed in a visual field in a stereoscopic state and displaying the figure according to the parameter.

An object of the present invention is to display information input to the input processing device in a visual field in a stereoscopic state,
It is to provide a means to confirm.

An object of a ninth aspect is to provide means for measuring a designated area in a stereoscopic state.

[0016]

According to the measurement endoscope apparatus of the present invention, a first image obtained by capturing an image of a predetermined observation target with a first image capturing means and the first image capturing means are different from each other. In a measurement endoscope apparatus capable of stereoscopic viewing, each of which displays a second image obtained by imaging the predetermined observation target from a different direction with a second imaging unit, the first imaging unit or the first imaging unit A distance setting means for setting a distance between one of the second imaging means and a virtual or actual object;
Corresponding point determining means for finding a corresponding point at which the same point of the object is imaged in the image obtained by the image capturing means and the image obtained by the second image capturing means; The apparatus comprises a figure drawing means for drawing a corresponding figure on at least one of the corresponding points, and an inspection means for deriving a parallax of the corresponding point and inspecting the parallax.

[0017]

Embodiments of the present invention will be described below with reference to the drawings.

FIGS. 1 to 34 relate to a first embodiment of the present invention. FIG. 1 is a configuration diagram showing a configuration of an endoscope apparatus.
2 is a block diagram showing a configuration of the host computer of FIG. 1, FIG. 3 is a configuration diagram showing a configuration of a distal end portion of the endoscope of FIG. 1,
FIG. 4 is a view for explaining a coordinate system in the imaging system of the endoscope in FIG. 1, and FIG. 5 is a first view for explaining distortion correction of the endoscope in FIG.
6, FIG. 6 is a second diagram for explaining the distortion correction of the endoscope in FIG. 1, FIG. 7 is a first diagram for explaining the operation of the host computer in FIG. 2, and FIG. 8 is a host computer in FIG. FIG. 9 is a third diagram illustrating the operation of the host computer of FIG. 2, FIG. 10 is a fourth diagram illustrating the operation of the host computer of FIG. 2, and FIG. FIG. 12 is a first flowchart showing a flow of main processing performed by the host computer of FIG. 2, and FIG. 12 is a second flowchart showing a flow of main processing processed by the host computer of FIG.
FIG. 13 is a flowchart showing the flow of processing of algorithm A used in the main processing of FIGS. 11 and 12, and FIG. 14 shows the operation of processing of algorithm B used in the main processing of FIGS. 11 and 12. FIG. 15 is a first diagram for explaining the operation, and FIG. 15 is a second diagram for explaining the operation of the algorithm B used in the main process in FIGS. 11 and 12, and FIG. 16 is a diagram used in the main process in FIGS. 17 is a flowchart showing the flow of processing of algorithm A ′, FIG. 17 is a flowchart showing the flow of processing of algorithm B used in the main processing of FIGS. 11 and 12,
FIG. 18 is a flowchart showing the flow of processing of algorithm C used in the main processing of FIGS. 11 and 12, and FIG.
9 is a first diagram illustrating the operation of the process of the algorithm C in FIG. 18, FIG. 20 is a second diagram illustrating the operation of the process of the algorithm C in FIG. 18, and FIG. 21 is the main process of FIGS. FIG. 22 is a flowchart showing the operation of algorithm D in FIG. 21, and FIG. 23 is a flowchart showing the operation of algorithm D in FIG. 21. 2 and FIG. 24 are flowcharts showing the processing flow of the algorithm E used in the main processing of FIGS. 11 and 12, and FIG.
FIG. 26 is a second diagram illustrating the operation of the algorithm E of FIG. 24, and FIG. 27 is used in the main process of FIG. 11 and FIG. 27 is a flowchart showing the flow of the processing of the algorithm F, FIG. 28 is a first diagram illustrating the operation of the process of the algorithm F in FIG. 27, FIG. 29 is a second diagram illustrating the operation of the process of the algorithm F in FIG. FIG. 30 is a flowchart showing the flow of processing of the algorithm G used in the main processing of FIGS. 11 and 12, FIG. 31 is a first diagram for explaining the operation of the processing of the algorithm G of FIG. 30, and FIG.
FIG. 2 is a second diagram for explaining the operation of the processing of the algorithm G of 0;
FIG. 33 is a first flowchart showing the flow of processing of algorithm H used in the main processing of FIGS. 11 and 12, and FIG. 34 shows the flow of processing of algorithm H used in the main processing of FIGS. 11 and 12. It is a 2nd flowchart shown.

As shown in FIG. 1, the endoscope apparatus according to the present embodiment has a stereo video image endoscope (hereinafter, referred to as an endoscope) 301 and a left image captured by the endoscope 301. And video processors 310L and 310R for processing each image signal of the right and right images,
A host computer 311 for storing RGB signals output from the video processors 310L and 310R and performing scale drawing processing and the like;
L, 310R video signals based on RGB signals are input, and an observation monitor 312L, 312R that displays a left image and a right image
A mouse 331 connected to the host computer 311 for inputting parameters and the like; a keyboard 332 connected to the host computer 311 for inputting parameters and the like; and a left image and a right image processed by the host computer 311 are displayed. The video processor 312L,
The 312R performs signal processing synchronized with each other.

The host computer 311 is configured as shown in FIG. That is, the host computer 311 communicates with the CP connected to the host bus 329.
U321, main memory 322, local bus 3
Image input / output board 324 for the left image connected to 30
L, an image input / output board for right image 324R, peripheral device interfaces 326 and 327, and a video board 328.

The host bus 329 and the local bus 330
Are connected by a host-local bridge 323,
Data transfer is performed via the host-local bridge 323.

Image input / output boards 324L and 324
R has an image capture function and a video output function, and at the time of image capture and video output,
A memory 334 (not shown) existing on each board
L, 334R to temporarily store the image.

The left image input / output board 324L is connected to the left image video processor 310L, and transfers the left image taken into the memory 334L on the board (not shown) to the main memory 322. The right image image input / output board 324R is connected to the right image video processor 310R, and transfers the right image captured in the memory 334R on the board (not shown) to the main memory 322.

The CPU 321 performs processing such as scale drawing on the left image and the right image transferred to the main memory 322, and converts the left image into a left image image input / output board 324L and the right image into a right image image input / output board. 324R, and can be displayed on the head mounted display 313.

The peripheral device interface 326
Is connected to the mouse 331, and the peripheral device interface 327 is connected to the keyboard 332. Video board 328 is connected to display 333.

The endoscope 301 has an elongated insertion portion 302 as shown in FIG. 3, and a plurality of, for example, two observation windows and an illumination window are provided at the tip of the insertion portion 302. Inside the observation windows, a left-eye objective lens system 303L and a right-eye objective lens system 304R are provided at positions having parallax with each other. The solid-state imaging devices 304L and 304R are located at the image forming positions of the objective lens systems 303L and 303R, respectively.
Are arranged.

A light distribution lens 305 is provided inside the illumination window, and a light guide 306 made of a fiber bundle is connected to the rear end of the light distribution lens 305. The light guide 306 is inserted into the insertion portion 302, and the incident end is connected to a light source device (not shown).

The illumination light output from the light source device is applied to the subject via the light guide 306 and the light distribution lens 305. The light from this subject is converted into a left image and a right image by the objective lenses 303R and 303L, respectively, as a solid-state image sensor 3
04L and 304R are formed.

First, a schematic flow of the scale drawing of the measurement endoscope apparatus configured as described above will be described.

In the image obtained by the endoscope 301, distortion due to distortion cannot be largely ignored. Images before correction are referred to as original images, particularly left and right images, respectively, as left original images and right original images, and corrected images are simply referred to as left images and right images.

In the present embodiment, processing such as scale drawing is performed on the original image on which distortion correction is not performed, and stereoscopic viewing is performed. However, scale drawing may be performed on the corrected image to perform stereoscopic viewing. is there.

FIG. 4 shows a coordinate system in the imaging system. The origin of the world coordinate system X ^L Y ^L Z ^L and left optical center O _left. The right optical center is called O _right .

As shown in FIG. 4, the local coordinate system of the left original image is i ^L j ^L , and the local coordinate system of the right original image is i ^R j ^R.
In both coordinate systems of the left and right original images, the origin is at the upper left of the image. FIG.
The left and right images in are the images after the distortion correction.

Further, the coordinate system of the left image coordinate system i ^{L 'j L',} right image is i ^{R 'j R'.} In both coordinate systems of the left and right images, the origin is at the upper left of the image. A ^L (i ^L , j ^L ) represents the coordinates of the pixel A on the left original image in the local coordinates of the left original image, and the same coordinate system of the pixel corresponding to the pixel A in the left image after distortion correction. The coordinates obtained are represented by A ^{L ′} (i ^{L ′} , j ^{L ′} ).

Similarly, the pixel A on the right original image is represented by A ^R (i ^R , j ^R ) in the local coordinates of the right original image, and the pixel A corresponding to the pixel A in the right image after the distortion correction is obtained. ^{Let the} coordinates be represented by A ^{R ′} (i ^{R ′} , j ^{R ′} ).

A^L(I^L, J^L) With the left optical center as the origin
A is the coordinate expressed in world coordinates^L _W(X^L, Y^L,
z^L), A^R(I^R, J^R) With the origin at the right optical center
Field coordinates A ^R _W(X^R, Y^R, Z^R). A^{L '}(I^{L '},
j^{L '}) In world coordinates with the origin at the left optical center
Coordinate A^{L '} _W(X^{L '}, Y^{L '}, Z^{L '}), A^{R '}(I^{R '},
j^{R '}) Is the world coordinate A with the right optical center as the origin.
^{R '} _W(X^{R '}, Y^{R '}, Z^{R '}).

For the correction of distortion, for example, the following method is used. Hereinafter, the left image will be described as an example, but the same applies to the right image. When the square grid of FIG. 5 is imaged by the endoscope 301, the obtained image is as shown in FIG. The coordinate transformation of each surface element is obtained in advance so that the distortion image becomes a square, and a distortion aberration corrected image is created from the original image. The coordinate transformation is represented by the following equation as a function of the pixel coordinates (i ^L , j ^L ).

[0038]

[Number 1] ^{i L '= p (i L} , j L) j L' = q (i L, j L) (1) p, q is i ^L, is expressed by a polynomial of j ^L. A more specific approach is disclosed in US Pat. No. 4,895,431.

Actually, (i ^{L ′} , j
^{L ′} ) is input and (i ^L , j ^L ) is output as a lookup table, and the pixel values of the corrected image coordinates (i ^{L ′} , j ^{L ′} ) are converted to the original image coordinates (i ^L , j ^L , By determining the pixel value from j ^L ), the distortion correction processing is realized. By using the look-up table, it is possible to determine from which pixel of the original image a certain pixel (i ^{L ′} , j ^{L ′} ) on the distortion-corrected image is created. In the following, it will be referred to as inverse correction corresponding to ^{(i L ', j L'} ) corresponding to the (i ^L, j ^L).
Also, it will be referred to as forward correction corresponding to (i ^L, j ^L) corresponding to the ^{(i L ', j L'} ).

Similarly, (i ^R , j ^R ) corresponding to (i ^R , j ^R ) corresponds to the inverse correction, and (i ^R , j ^R ) corresponding to (i ^R , j ^R ) corresponds to (i ^R , j ^R ). This is referred to as forward correction correspondence.

In the following, as shown in FIG.
An example of drawing a solid scale existing at is1 [mm] will be described.

[0042] Assume a perpendicular vertical plane Z ^L axis in the endoscope tip Dis1 [mm] away. Give the scale drawing position on the left image in advance. As shown in FIG. 8, five horizontal scales and five vertical scales are considered, and the coordinates of each scale are represented by M _[k] ^{L '} (i _[k] ^L' , j _{[k ]} ^{L '} ) (k = 0,1, ‥
９, 9).

Next, for each scale, the pixel M _[k] on the left image
The point γ on the vertical plane shown in ^{L ′} (i _[k] ^{L ′} , j _[k] ^{L ′} ) is obtained (see FIG. 4), and the point M in the right image showing the point γ as shown in FIG. _[k] ^{R '} (i _[k] ^R' , j _[k] ^{R '} ) (k = 0,1, ‥
‥, 9). In other words, a certain point γ on the vertical plane
The position of the image in the left and right images is determined.

Further, the positions of the left and right original images are obtained from the positions of the images of the left and right images by using the inverse correction correspondence, and a scale is drawn at the positions, so that Dis1 [mm] from the tip of the endoscope.
3D scale that can be seen floating in the image can be observed in the field of view of a head mounted display (hereinafter, HMD) 313.

In the first embodiment, a solid figure, parameters, graphical user
An interface (hereinafter, GUI) and a mouse cursor are three-dimensionally drawn in the visual field. Drawing is performed on the overlay image.
The coordinate system of the left overlay image is the same as the local coordinate system of the left original image, and the coordinate system of the right overlay image is the same as the local coordinate system of the right original image.

The drawing state of the left and right overlay images is as shown in FIG. The overlay image is synthesized with the left and right original images by a known keying technique.

The keying is described in detail in Document 1 (Practical image processing learned in C language, written by Seiki Inoue et al., Ohmsha).

The horizontal size of the left and right original images is W [pixel], the vertical size is H [pixel], the horizontal size of the left and right images is Wd [pixel], and the vertical size is Hd [pixel]. The size of the left and right overlay images is also W [pixels], and the vertical size is H [pixels]
It is.

A specific flow of the main processing according to the first embodiment will be described with reference to FIGS.

The parallax that can be stereoscopically viewed by the HMD 313 is checked in advance. For example, it can be clarified by drawing figures with various parallaxes and observing them with an HMD.

Here, the parallax means the difference between the figure drawing i ^R coordinate in the right overlay image and the figure drawing i ^L coordinate in the left overlay image. The range is D1 [pixel] <
(Parallax) <D2 [pixel].

In step S1, an interval [pixel] of one scale with respect to Sc [mm] is determined by an algorithm B described later. The left image horizontal scale interval Spx ^{L '} ,
Left image vertical scale interval Spy ^{L '} , right image horizontal scale interval Sp
xR ^' , the right image vertical scale interval Spy ^R' .

In step S2, the left image horizontal scale
Spax Spx^{L '}, Left image vertical scale interval Spy ^{L '}Using the left image
Scale drawing coordinate M at_[k] ^{L '}(K = 0,1, ‥‥, 1
0) is determined. Scale drawing of the left image given in advance
Image center coordinates (Cx^{L '}, Cy^{L '}) And is determined by equation (2)
Is done.

[0054]

## EQU2 ## Horizontal scale M _[k] ^{L '} = (CxL ^' + SpxL ^'. (K-2), CyL ^' ) = (i _[k] ^{L '} , j _[k] ^L' ) (0≤ k ≦ 4) Vertical scale M _[k] ^{L ′} = (CxL ^′ , CyL ^′ + SpyL ^′ · (k−6)) = (i _[k] ^{L ′} , j _[k] ^{L ′} ) (5 ≦ (2) In step S3, the distance from the endoscope tip to the scale is set to Dis1 [mm], and M _[k] ^{L '} (i _[k] ^L' , j _{[k ] is determined} by algorithm A described later. _] ^{L ′} ) (k = 0, 1, ‥‥,
9) The inverse correction corresponding pixel M _[k] ^L (i _[k] ^{L '} , j _[k] ^L' ) (k
= 0, 1, ‥‥, 9) and the corresponding point M _[k] ^R (i
_[k] ^{L ′} , j _[k] ^{L ′} ) (k = 0, 1, ‥‥, 10). Using M _[k] ^L and M _[k] ^R (k = 0, 1,..., 9), the algorithm C described later uses a solid scale 600L for the left overlay image and a 600R for the right overlay image as shown in FIG. To draw.

In step S4, the drawing coordinates of the auxiliary figure in the left overlay image are determined. Here, (C
X2 ^L , Cy2 ^L ) are given in advance.

[0056] In step S5, the distance from the endoscope tip to solid figure as D is2 [mm] as shown in FIG. 7, obtaining the corresponding points from the algorithm A 'to be described later (CX2 ^L, Cy2 ^L). FIG.
, A three-dimensional figure 601L is drawn on the left overlay image and a 601R is drawn on the right overlay image. It is easy to see whether the object is in front of or behind the scale that appears in the air, but it is difficult to grasp that it is just beside. Therefore, by drawing auxiliary figures having different distances from the scale and the endoscope end, it is easy to grasp the positional relationship between the target and the scale. Also, a plurality of auxiliary figures may be drawn, and the distance from the endoscope end to the three-dimensional figure is given to each figure,
By performing the processing in steps S4 and S5, drawing can be performed.

In step S6, buttons 604, 6
The drawing position in the left overlay image 05 is given by Expression (3). Button size is given width Bx, height By
And then, each of the center coordinates of the button 604 (CX3 ^L, Cy
3 ^L ), the coordinates of the four vertices are determined. The same applies to the button 605.

[0058]

[Number 3] vertex ^{1: (CX3 L -Bx / 2} , Cy3 L -By / 2) vertex ^{2: (CX3 L + Bx /} 2, Cy3 L -By / 2) vertex ^{3: (CX3 L -Bx / 2} , Cy3 ^L + by / 2) vertices 4: (in ^{CX3 L + Bx / 2, Cy3} L + by / 2) (3) step S7, the distance from the endoscope tip to GUI (buttons in this case) as shown in FIG. 7 dis3 As [mm], the GUI drawing coordinates of the right overlay image are obtained by the algorithm A ′ described later. A button 604 is displayed as a GUI as shown in FIG.
L, 605L to the left overlay image, 604R, 60
5R is drawn on the right overlay image.

In step S8, the coordinates of a reference line for drawing parameters in the left overlay image are determined. For each parameter, two coordinates necessary for determining a reference line, that is, a straight line, are given.

In step S9, the distance from the tip of the endoscope to the parameter is set to Dis4 [mm] as shown in FIG. 7, and the parameter drawing coordinates of the right image are obtained by an algorithm A 'described later. As shown in FIG. 10, a parameter 602 is added to the left overlay image by an algorithm F described later.
L, 602R is drawn on the right overlay image. FIG.
In this example, “20.0 mm” is drawn.

In step S10, coordinates in the mouse cursor movable area in the left overlay image are given. The mouse cursor is represented by one point at the head of the arrow as shown in FIGS. Here, the cursor movable area is the entire left original image.

In step S11, the distance from the distal end of the endoscope to the cursor is set to Dis5 [mm] as shown in FIG. 7, and the right mouse cursor coordinates with respect to the left mouse cursor coordinates are derived by algorithm A 'described later. Also,
A mouse cursor 606L is drawn on the left overlay image and 606R is drawn on the right overlay image as shown in FIG.

In step S12, the parallax of each of the scales, auxiliary figures, buttons, parameters, and mouse cursor figures is checked to determine whether they are within a stereoscopically viewable range. For example, in the case of a scale, the following expression is used.

If D1 <M _[k] ^R- M _[k] ^L <D2, the process proceeds to step S13.
A warning is displayed to reset the distance from the endoscope tip to each figure, and the process proceeds to step S14. If the distance is within the range, the process directly proceeds to step S14. In step S13, the distance may be input using an input device (such as the keyboard 332).

In step S14, a parameter changing means using a GUI and a keyboard is executed by an algorithm H described later.

[Algorithm A] The flow of algorithm A, which is a process for determining corresponding points, will be described with reference to FIG.

In step S21, an image is obtained. The images output from the video processors 310L and 310R are transferred to the main memory 322 by the image input / output boards 324L and 324R.

In step S22, a distance τ [m from the end of the endoscope to a drawing figure set in advance.
m], figure drawing coordinates α ^{L ′} (i ^{L ′} , j ^{L ′} ) in the left image
To get.

In step S23, a pixel α ^L corresponding to the inverse correction of α ^{L ′} is obtained. α ^L is the drawing position of the left overlay image. The method of obtaining the inverse correction countermeasure element can be obtained by performing a process reverse to the expression (1), or by searching for an input value of a lookup table that outputs (i ^{L ′} , j ^{L ′} ). Let the obtained coordinates be α ^L (i ^L , j ^L ).

In step S24, α ^{L ′} is converted into a world coordinate system by the following coordinate conversion. Α ^{L '}
_W (xL ^' , yL ^' , zL ^' ). In the following equation, f ^L is a focal length, C ^L of the left optical system is left CCD1 horizontal size per pixel or left CCD1 vertical size per pixel.

[0071]

Equation 4] ^{x L '= (i L'} -Wd / 2) in ^{× Cx L y L '= (} Hd / 2-j L') × Cy L z L '= f L (4) step S25, the vertical Find the virtual target point coordinates on the plane. The intersection of a straight line L passing through α ^{L ′} _W and the left optical center Oleft (0, 0, 0) and a vertical plane passing through (0, 0, τ) is obtained. This intersection is called a virtual target point, and the coordinates γ ^L _{W
^{(X L γ, y L γ}} , z L γ) and. Actually, it is possible to obtain the coordinates of the virtual target point by using the similarity of the triangles seen in the imaging systems of FIGS. 14 and 15.

[0072]

In Equation 5] _{^{x L γ = (x L '}} × τ) / z L y L γ = (y L' × τ) / z L z L γ = τ (5) Step S26, the corresponding point in the right image The coordinates are derived. Determining the coordinates of the virtual object point gamma ^R _W in the coordinate system X ^R Y ^R Z ^R with the origin of the right optical system centered by the coordinate transformation. The coordinates _{^{γ R W (x R γ,}} y R γ, z R γ) to. R is the rotation matrix, (Mx, My, Mz) are the coordinates of the right camera in the X ^L Y ^L Z ^L.

[0073]

(Equation 6) R is a rotation matrix, which can be expressed in various ways. For example, the rotation amount θ1 is centered on the Z ^L axis in the polar coordinate system, .theta.2
Represents the amount of rotation about the ^XL axis as follows.

[0074]

(Equation 7)γ^R _WAnd right optical center O_rightAnd a straight line L 'passing through
By finding the intersection with the image, the virtual target point γ^R _WAgainst
Find the coordinates of the image in the right image. The coordinates are β
^R _W(X^R, Y^R, Z^R). FIG. 14 and FIG.
5 using the similarity of triangles found in the imaging system
Can be sought. f^RIs the focal length of the right optical system
It is separation.

[0075]

In Equation 8] _{^{x R = (x R γ /}} z R γ) × f R y R = (y R γ / z R γ) × f R z R = f R (8) Step S27, the coordinate transformation β converting the ^R _W in the local coordinate system of the right image. ^Let the coordinates be β ^{R '} (i ^R' ,
j ^{R ′} ). Cx ^R horizontal size per right CCD1 pixel, the Cy ^R a longitudinal size per right CCD1 pixel.

[0076]

Equation 9] i ^R for ^{'= Wd / 2 + x R} / Cx R j R' = Hd / 2-y R / Cy R (9) β R ' is representing the coordinates of the image in the distortion correction image, right The image β ^R (i ^R , j ^R ) in the original image is obtained by a look-up table.

As described above, α ^L (i ^L , j ^L ) is the drawing position of the left overlay image, and β ^R (i ^R , j ^R ) is the drawing position of the right overlay image.

[Algorithm A '] The flow of algorithm A', which is the processing for determining corresponding points, will be described with reference to FIG. The difference from the algorithm A is that the coordinates to be processed are not the coordinates of the left image but the coordinates of the left original image (that is, the left overlay image).

In step S31, an image is obtained. The images output from the video processors 310L and 310R are transferred to the main memory 322 by the image input / output boards 324L and 324R.

In step S32, a predetermined distance τ [mm] from the endoscope tip and a figure drawing coordinate α ^L (i ^L , j ^L ) in the left overlay image are acquired.

[0081] In step S33, obtaining the alpha forward correction corresponding pixels of ^L α ^{L '.} Using (i ^L , j ^L ) as an input to the lookup table, its output is α ^{L ′} (i ^{L ′} , j ^{L ′} ). The relationship between α ^L and α ^{L ′} is that the coordinate α ^L becomes the coordinate α ^{L ′} due to distortion correction.

In step S34, α ^{L ′} is converted into a world coordinate system by the following coordinate conversion. Α ^{L '}
_W (xL ^' , yL ^' , zL ^' ). In the following equation, f ^L is the focal length of the left optical system, Cx ^L is the horizontal size per left CCD pixel, and Cy ^L is the vertical size per left CCD pixel.

[0083]

Equation 10] ^{x L '= (i L'} -Wd / 2) in ^{× Cx L y L '= (} Hd / 2-j L') × Cy L z L '= f L (10) Step S35, the vertical Find the virtual target point coordinates on the plane. The intersection of a straight line L passing through α ^{L ′} _W and the left optical center Oleft (0, 0, 0) and a vertical plane passing through (0, 0, τ) is obtained. This intersection is called a virtual target point, and the coordinates γ ^L _{W
^{(X L γ, y L γ}} , z L γ) and. Actually, it is possible to obtain the coordinates of the virtual target point by using the similarity of the triangles seen in the imaging systems of FIGS. 14 and 15.

[0084]

In Equation 11] _{^{x L γ = (x L '}} × τ) / z L y L γ = (y L' × τ) / z L z L γ = τ (11) step S36, the corresponding point in the right image The coordinates are derived. Determining the coordinates of the virtual object point gamma ^R _W in the coordinate system X ^R Y ^R Z ^R with the origin of the right optical system centered by the coordinate transformation. The coordinates _{^{γ R W (x R γ,}} y R γ, z R γ) to. R is the rotation matrix, (MX, My, Mz) are the coordinates of the right camera in the X ^L Y ^L Z ^L.

[0085]

(Equation 12) R is a rotation matrix, which is expressed by the following equation as in the algorithm A.

[0086]

(Equation 13)γ^R _WAnd the left optical center O_rightAnd a straight line L 'passing through
By finding the intersection with the image, the virtual target point γ^R _WAgainst
Find the coordinates of the image in the right image. The coordinates are β
^R _W(X^R, Y^R, Z^R). FIG. 14 and FIG.
5 using the similarity of triangles found in the imaging system
Can be sought. f^RIs the focal length of the right optical system
It is separation.

[0087]

In Equation 14] _{^{x R = (x R γ /}} z R γ) × f R y R = (y R γ / z R γ) × f R z R = f R (14) step S37, the coordinate transformation β converting the ^R _W in the local coordinate system of the right image. ^Let the coordinates be β ^{R '} (i ^R' ,
j ^{R ′} ). Cx ^R horizontal size per right CCD1 pixel, the Cy ^R a longitudinal size per right CCD1 pixel.

[0088]

(15) i ^{R ′} = Wd / 2 + x ^R / Cx ^R j ^{R ′} = Hd / 2−y ^R / Cy ^R (15) Since β ^{R ′} represents the coordinates of the image in the distortion-corrected image, the right The image β ^R (i ^R , j ^R ) in the original image is obtained by a look-up table.

As described above, α ^L (i ^L , j ^L ) and β ^R (i ^R ,
j ^R ) is the drawing position in the overlay image.

[Algorithm B] The flow of algorithm B for determining the scale interval will be described with reference to FIG. In step S41, a scale interval S given in advance
Get c [mm].

In step S42, the proportional relation of the imaging system is determined.
Determines the horizontal scale interval [pixels] of the left image
You. When creating a scale of Sc [mm] in the horizontal direction,
The X coordinate on the image plane where the sculpture is drawn
From FIG. 14 as viewed from the side, the following is obtained. In FIG.
X and Y axes are X in the global coordinate system.^L, Y^LOr
X ^R, Y^RRepresents

[0092]

Equation 16] for converting _{Sc / z γ = Sx / f} L Sx = a _{^{(Scf L) z γ (16}} ) units from the [mm] in the pixel.

[0093]

Equation 17] ^{SPx L '= Sx / Cx L} (17) Thus the number of pixels corresponding to Sc [mm] is SPx ^L' becomes.

In step S43, the drawing interval of the right image in the horizontal direction is given by the following equation.

[0095]

Equation 18] for converting Sc / z _gamma a ^{'= Sx / f R Sx =} (Scf R) z γ' (18) units from [mm] to [pixel].

[0096]

Equation 19] ^{SPx R '= Sx / Cx R} (19) Thus the number of pixels corresponding to Sc [mm] is SPx ^R' becomes.

In step S44, the vertical scale interval of the left image is determined based on the proportional relationship of the imaging system. When a scale of Sc [mm] is created in the vertical direction, the Y coordinate on the image plane on which the scale is drawn is as follows based on a proportional relationship from FIG. 15 when the imaging system is viewed from the X-axis side. FIG.
Y axis in, Z-axis represents the Y ^L, Z ^L or Y ^R, Z ^R of the global coordinate system.

[0098]

Equation 20] for converting _{Sc / z γ = Sy / f} L Sy = a _{^{(Scf L) z γ (20}} ) units from [mm] to [pixel].

[0099]

SPy ^{L ′} = Sy / Cy ^L (21) Therefore, the number of pixels corresponding to Sc [mm] is SPy ^{L ′} .

In step S45, similarly, the drawing interval of the right image in the vertical direction is given by equation (22).

[0101]

Equation 22] for converting _{Sc / z γ '= Sy /} f R Sy = (Scf R) z γ' (22) units from [mm] to [pixel].

[0102]

SPy ^{R ′} = Sy / Cy ^R (23) Therefore, the number of pixels corresponding to Sc [mm] is SPy ^{R ′} .

[Algorithm C] A vertical line with a length SL1 given in advance as a horizontal scale is drawn around the given coordinates. Draw a horizontal line of length SL2 as a vertical scale. A straight line is drawn between the center coordinates on any scale. Here, a case where the number of scales is 5 horizontally and 5 vertically will be described. The flow of algorithm C, which is a scale drawing process, will be described with reference to FIG.

In step S51, k = 0.
In step S52, the scales for left and right overlay drawing are represented by M _[k] ^L and M _[k] ^R as shown in FIGS. Get the scale drawing coordinates. The acquired coordinates are M
_[k] ^L (i _[k] ^L , j _[k] ^L ) and M _[k] ^R (i _[k] ^R , j _[k] ^R ). Here, as can be seen from FIGS. 19 and 20, and further FIGS. 8 and 9, the coordinates of k ≦ 4 are the coordinates of the horizontal scale, and the coordinates of k> 4 are the coordinates of the vertical scale.

In step S53, if k ≦ 4, step S54 is performed. Otherwise, step S5
Perform Step 5.

In step S54, a horizontal scale is drawn on the left overlay image. That is, (i _[k] ^L ,
j _[k] ^L ) and a vertical line is drawn as shown in FIG. Draw a horizontal scale on the right overlay image. That is, a vertical line is drawn centering on (i _[k] ^R , j _[k] ^R ) as shown in FIG. Step S56 is performed.

In step S55, a vertical scale is drawn on the left overlay image. A horizontal line is drawn centering on (i _{[k ]} ^L , j _[k] ^L ) as shown in FIG. Draw a vertical scale on the right overlay image. (I _[k] ^R ,
A horizontal line is drawn centered on j _[k] ^R ) as shown in FIG.

In step S56, k> 0 and k ≠
If it is 5, in the left overlay image (i _[k] ^L ,
j _[k] ^L ) and a straight line between (i _[k-1] ^L , j _[k-1] ^L ). In the right overlay image (i _[k] ^R , j _[k] ^R ),
A straight line is drawn between (i _[k-1] ^R , j _[k-1] ^R ). The condition of k ≠ 5 is to prevent a straight line from being drawn between the horizontal scale and the vertical scale.

In step S57, k is incremented by one. If k <10 in step S58, the process returns to step S53.

The resulting scales are the scales 600L and 600R shown in FIG.

[Algorithm D] A figure is drawn around the coordinates given in advance. Here, a cross-shaped auxiliary graphic whose vertical line and horizontal line lengths are SL3 given in advance is drawn with the acquired coordinates as the center. The shape is not limited to the cross shape. For example, by performing the same processing as in steps S1 and S2, it is possible to draw a solid scale as an auxiliary figure. The flow of auxiliary figure drawing will be described with reference to FIG.

In step S61, coordinates (i _[0] ^L , j _[0] ^L ) are obtained as drawing coordinates of a cross-shaped figure in the left image. The coordinates (i _[0] ^R , j _[0] ^R ) are acquired as the drawing coordinates of the cross-shaped figure in the right image.

In step S62, the left overlay
Draw a cross shape on the image. (I _[0] ^L, J_[0] ^L) Inside
As shown in FIG. 22, the length of the vertical and horizontal lines is not more than SL3.
Draw a glyph. The drawing state is the cross-shaped figure 6 in FIG.
01L.

In step S63, the right overlay
Draw a cross shape on the image. (I _[0] ^R, J_[0] ^R) Inside
As shown in FIG. 23, the length of the vertical and horizontal lines is 10
Draw a glyph. The drawing state is the cross-shaped figure 6 in FIG.
01R.

[Algorithm E] A button is drawn using the acquired coordinates. The flow of algorithm E, which is a button drawing process, will be described with reference to FIG.

In step S71, the left GUI coordinates are obtained. The acquired coordinates are represented by (i _[0] ^L , j _[0] ^L ) to (i
_[3] ^L , j _[3] ^L ). Get the right GUI coordinates. ^{Let the} acquired coordinates be (i _[0] ^R , j _[0] ^R ) to (i _[3] ^R , j _[3] ^R ).

In step S72, if the GUI visibility attribute is TRUE, step S73 is executed. If not, step S74 is executed.

In step S73, a button is drawn on the left overlay image. (I _[0] ^L , j _[0] ^L ) to (i
_[3] ^L , j _[3] ^L ) is a rectangular button with four vertices.
Is drawn on the overlay image as follows. Draw a button on the right image. (I _[0] ^R , j _[0] ^R ) to (i _[3] ^R , j _[3] ^R )
Is drawn on the overlay image as shown in FIG. 26. The drawing state is 604L in FIG.
It looks like a 605L, 604R, 605R button. The process ends.

In step S74, if the GUI visibility attribute is FALSE, the buttons are deleted from the left and right overlay images. The process ends.

[Algorithm F] The parameters are drawn using the obtained coordinates. The flow of algorithm F, which is a parameter drawing process, will be described with reference to FIG.

In step S81, the coordinates of a reference line for drawing parameters in the left overlay image are obtained. The acquired coordinates are (i _[0] ^L , j _[0] ^L ), (i _[1] ^L ,
j _{[1 ]} ^L ). The coordinates of a reference line for drawing a parameter in the right overlay image are acquired. The acquired coordinates are (i _[0] ^R , j _[0] ^R ) and (i _[1] ^R , j _[1] ^R ).

In step S82, if the parameter visibility attribute is TRUE, step S83 is executed. Otherwise, execute step S84.

In step S83, the left overlay
Draw parameters on the image. (I _[0] ^L, J_[0] ^L),
(I_[1] ^L, J_[1] ^L) As a reference line, as shown in FIG.
Draw the meter on the overlay image. Reference line is horizontal
When tilted with respect to the direction, the equation (24) for the horizontal direction
Angle θ^LUse a font that is tilted according to

Equation 24] _{^{θ L = tan -1 (jj [}} 1] L -jj [0] L) / (i [1] L -i [0] L) jj [k] L = H-j [k] L (K = 0, 1) (24) Draw parameters on the right overlay image.
Using (i _[0] ^R , j _[0] ^R ) and (i _[1] ^R , j _[1] ^R ) as reference lines, parameters are drawn on the overlay image as shown in FIG. Angle θ ^R of equation (25) with respect to the reference line and the horizontal direction
Draw parameters using a font tilted according to. The process ends. The drawing state is the parameters 602L and 602R in FIG.

[0124]

Θ ^R = tan ⁻¹ (JJ _[1] ^R− JJ _[0] ^R ) / (i _[1] ^R− i _[0] ^R ) jjk _] ^R = H−j _[k] ^R ( k = 0,1) (25) In step S84, the parameter visibility attribute is set to FAL.
If it is SE, the parameters are deleted from the left and right overlay images. The process ends.

[Algorithm G] A mouse cursor is drawn using the acquired coordinates. The flow of algorithm G, which is cursor drawing, will be described with reference to FIG.

In step S91, the coordinates (i ^L , j ^L ) of the mouse cursor at the coordinates of the current left overlay image are obtained. ^Let the coordinates of the corresponding point of the right overlay image corresponding to that be (i ^R , j ^R ). As described above, the coordinate system of the left overlay image is the same as the local coordinates of the left original image, and the coordinate system of the right overlay image is the same as the local coordinates of the right original image.

At step S92, if the cursor visibility attribute is TRUE, step S93 is executed. Otherwise, step S94 is performed.

In step S93, a mouse cursor is drawn on the left overlay image. The mouse cursor is drawn on the overlay image as shown in FIG. 31 around (i ^L , j ^L ). Draw the mouse cursor on the right overlay image. The mouse cursor is drawn on the overlay image as shown in FIG. 32 around (i ^R , j ^R ). The drawing state is the cursors 606L and 606R in FIG. The process ends.

At step S94, if the cursor visibility attribute is FALSE, the mouse cursor is deleted from the left and right overlay images. The process ends.

[Algorithm H] A parameter changing function is provided by an input device such as a keyboard or a mouse. The flow of algorithm H, which is a parameter change process, will be described with reference to FIGS.

The changeable parameter is, for example, a scale interval, but other parameters can be used. The mouse cursor, parameters, and GUI are deleted from the screen when there is no input by the input means for a predetermined time or more. When the input is restarted, the parameters and GUI are automatically redrawn.

In step S101, it is determined whether or not there is no input from the keyboard, mouse, etc., that is, whether or not the idle state has exceeded a predetermined time. If it exceeds, step S1O2 is executed, and if not, step S103 is executed.

In step S102, the GUI visible attribute and the parameter visible attribute are changed to FALSE, and the algorithm and the algorithm F are used to erase the GUI and the parameter. The cursor visibility attribute is changed to FALSE, and the mouse cursor is erased by the algorithm G.

In step S103, keyboard input is determined. If there is no input, step S106 is executed.

In step S104, the parameter visibility attribute is changed to TRUE, and redrawing is performed using algorithm F.

In step S105, the entered key is determined. For example, the numeric keypad “1” decreases by −1 parameter, and the numeric keypad “3” increases +1 parameter.
When "1" is pressed, the scale interval is set to -1 [mm],
When "3" is pressed, the scale interval is increased by +1 [mm]. After that, the call scale is redrawn in the order of the algorithms B and A. In step S106, the parameters after the change are redrawn by the algorithm F.

In step S107, a mouse input is determined. If there is no mouse input, step S112
Is carried out.

If the mouse is moved in step S108, the GUI visibility attribute and the parameter visibility attribute are changed to TRUE, and the GUI
I, draw parameters. TRU cursor visibility attribute
Change to E and draw the cursor using algorithm G.

In step S109, when the left mouse button is pressed, it is determined whether or not the cursor exists in the GUI area. If it is in the GUI area, step S110 is performed; otherwise, step S112 is performed.

In step S110, a GUI area is determined. If the "-" button, the scale interval is -1 [m
m], and the call scale is redrawn in the order of algorithm B and algorithm A. In the case of the "+" button, the scale interval is set to +1 [mm], and the call scale is redrawn in the order of algorithm B and algorithm A.

In step S111, the changed parameters are redrawn by the algorithm F.

At step S112, the termination of the application is determined. If completed, the process ends. Otherwise, the process returns to step S101.

FIGS. 35 to 48 relate to the second embodiment of the present invention. FIG. 35 is a first flowchart showing the flow of processing by the host computer, and FIG.
A second flowchart following the first flowchart of
FIG. 37 is a first diagram illustrating the operation of the processing in FIGS. 35 and 36.
38, FIG. 38 is a second diagram illustrating the operation of the process of FIGS. 35 and 36, FIG. 39 is a third diagram illustrating the operation of the process of FIGS. 35 and 36, and FIG. FIG. 41 is a fourth diagram illustrating the operation of the process of FIG. 36, FIG. 41 is a flowchart illustrating the flow of a process of a modification of the second flowchart of FIG. 36, and FIG. 42 is a first diagram illustrating the operation of the flowchart of FIG. 43 is a second diagram illustrating the operation of the flowchart of FIG. 41, FIG. 44 is a third diagram illustrating the operation of the flowchart of FIG. 41, and FIG. 45 is a fourth diagram illustrating the operation of the flowchart of FIG. FIG. 46 is a fifth diagram illustrating the operation of the flowchart of FIG. 41, FIG. 47 is a sixth diagram illustrating the operation of the flowchart of FIG. 41, and FIG. 48 is a seventh diagram illustrating the operation of the flowchart of FIG. FIG.

Since the second embodiment is almost the same as the first embodiment, only different points will be described, and the same components will be denoted by the same reference numerals and description thereof will be omitted.

In the second embodiment, the device configuration is the same as in the first embodiment. The present embodiment will be described in the industrial field, but can be similarly performed in the medical field and other fields. Here, a description will be given of an example of measurement of a turbine blade of an engine.

More specifically, the size of a crack (crack) shown in FIG. 37 of the blade is measured. The blade replacement is determined based on the measured crack size.

In the following, a description will be given of a flow in which both ends of a crack are designated by an input device, the size of the crack is measured, and the measurement result is displayed in a situation where the object is stereoscopically viewed, with reference to FIGS. 35 and 36. Although a mouse is described as an example of the input device, other devices are also possible.

In step S200, the turbine blade is stereoscopically viewed according to the first embodiment.

In step S201, the mouse cursors 621L and 621R are drawn on the overlay image as shown in FIG. 37 by the algorithm G as in steps S10 and S11 of the first embodiment. The cursor movable area is the entire left original image.

In step S202, it is determined whether a mouse button has been input. If there is a left button input, step S2
Execute 05.

In step S203, the mouse cursor is drawn on the overlay image by algorithm G according to the movement of the mouse cursor.

In step S204, the termination of the application is determined. If completed, the process ends. Otherwise, the process returns to step S202.

In step S205, the figures 620L, 622L, 6
20R and 622R are drawn. Drawing is performed on the left and right overlapping images in the same manner as in the algorithm D of the first embodiment.

The distance from the tip of the endoscope to the figure is set to the same Dis5 [mm] as the mouse cursor. A total of two points are designated one by one with the mouse, and both ends of the crack are designated. These are called designated points, and are designated as P _[0] ^L and P _[1] ^L , respectively. If the first point is specified in step S206, step S2
Return to 02. If the second point is designated, step S2
Proceed to 07. Note that when the first point is specified, if the previous two specified points have already been drawn, they are deleted before drawing the figure at the position of the mouse cursor.

In step S207, the left and right original images are corrected for distortion. The distortion corrected images are referred to as a left image and a right image.

In step S208, preprocessing is performed on the left and right images. The edges of the left and right images are detected by a known edge detection method. The edge image is binarized by a predetermined threshold Th1. Next, after the edges are adjusted to the width of one pixel by thinning, small edge groups are erased. First, the edge groups are numbered by known labeling. The number of pixels of the edge group assigned the same number is checked, and edges having the number of pixels smaller than the threshold Th2 are deleted. Each edge in the left image is E
_[i] Let ^L (i = 0, 1, 2,...). Reference 1
(Practical image processing in C language, written by Seiki Inoue et al., Ohmsha).

In step S209, the three-dimensional positions of the designated point and the edge are obtained. By the same method as in the first embodiment, P _[0] ^L , P _[1] ^L , and E _[i] ^L (i = 0, 1, 2,.
画素) A pixel corresponding to the forward correction is obtained. P _[0] ^{L '} , P _[1] ^L' , E _[i] ^{L '} (i = 0, 1, 2,...)
And P _[0] ^{L '} , P _[1] ^L' , E _[i]
The three-dimensional position of the image corresponding to ^{L '} is obtained.

The corresponding points are detected by template matching using known epipolar constraints on the left and right images.

As shown in FIG. 38, the template is P _[0] ^{L '}
Alternatively, a small area centered on the coordinates of P _[1] ^{L '} or E _[i] ^L' is cut out and created.

FIG. 38 shows a template centered on the coordinates of E _[i] ^{L '} . The corresponding points are P _[0] ^{R '} and P _[1]
^{R ′} , E _[i] ^{R ′} (i = 0, 1, 2, ‥‥) (these pixels corresponding to the inverse correction are P _[0] ^R , P _[1] ^R , E _[i] ^R (i = 0,
1, 2, ‥‥)).

According to the principle of triangulation, P_[0] ^{L '}, P_[1] ^{L '},
E_[i] ^{L '}Three-dimensional position P of the image corresponding to_[0] ^{L '} _W(X_[0] ^{L '},
y_[0] ^{L '}, Z_[0] ^{L '}), P_[1] ^{L '} _W(X_[1] ^{L '}, Y_[1] ^{L '}, Z
_[1] ^{L '}), E_[i] ^{L '} _W(X_{E [i]} ^{L '}, Y_{E [i]} ^{L '}, Z_{E [i]} ^{L '})
To determine. The principle of triangulation is described in JP-A-6-33945.
No.4 is detailed.

In step S209, a straight line Lm passing through the three-dimensional positions P _[0] ^{L '} _W and P _[1] ^L' _W of the designated point is determined.

[0163]

(Equation 26) In step S210, all distances between E _[i] ^L' _W and Lm are obtained as shown in FIG. The maximum distance Mmax is determined from the values.

In step S211, E _[i] ^{L ′} indicating the maximum distance on the overlap image as shown in FIG.
Is drawn at the inverse correction corresponding position coordinate E _[i] ^L and its corresponding point E _[i] ^R , and the distance Mmax is displayed near the figure (8.5 mm in FIG. 23). The method of marking is performed in the same manner as in the algorithm D in the first embodiment, and the method of displaying the distance is performed in the same manner as the algorithm F in the first embodiment to display three-dimensionally. It returns to step S202.

As described above, it is possible to measure the size of the target while viewing the target in a stereoscopic manner, and furthermore, it is possible to see the measurement result in a state where the target is viewed in a stereoscopic manner. When inputting the designated point, the screen may be frozen.

The following method is also conceivable for designating the measurement object. In the following, a description will be given of a flow of displaying a measurement result after specifying a region including a crack with the input device and measuring the size of the crack in a situation where the target is viewed stereoscopically, with reference to FIG. 41. Although a mouse is described as an example of the input device, other devices are also possible. Although the figure surrounding the crack portion will be described as a rectangle, a circle, a rectangle, a free curve, and the like can be considered, and the figure is not limited to a rectangle.

Steps S201 to S204 are the same. However, the step executed when the left button is input in step S202 is referred to as step S220. Hereinafter, different processes will be described.

In step S220, as shown in FIG. 42, a rectangular area 630L is designated by designating two points, a starting point and a diagonal point forming a diagonal point, with the mouse cursor 621L. This rectangular area 630L, as shown in FIG.
By changing the mouse cursor 621L by specifying a diagonal point, the change can be performed.
Is drawn on the left and right overlap images in the same manner as the algorithm D of the embodiment. The distance from the tip of the endoscope to the rectangle is set to the same Dis5 [mm] as the mouse cursor 621L.

In step S221, the left and right original images are corrected for distortion. The distortion-corrected images are called a left image and a right image.

In step S222, in the left image, the outline in the rectangular area is extracted as a binary image by a known edge detection method. 1 edge by thinning
Trim to pixel width.

In step S223, in the rectangular area of the left image, numbers are assigned to the outlines by known labeling, and the outlines are distinguished. The number of pixels of the edge group assigned the same number is checked, edges having a pixel number smaller than the threshold Th2 are deleted, and small edge groups are deleted. Thereafter, the number Lv included in the maximum number in the rectangular area is obtained.

In step S224, as shown in FIG. 44, a contour line having a number Lv in the rectangular area of the left image is searched by scanning the area from left to right and from top to bottom. Let the coordinates of the contour image having the first Lv be St ^{L ′} (i ^{L ′} st, j ^{L ′} st).

Using St ^{L ′} (i ^{L ′} st, j ^{L ′} st) as the tracking start position, all the contour lines E _[n] ^{L ′} (n = 0, 1, 2,.
{, NE) is traced. Here, NE is the number of contour lines. If it reaches outside the area during tracking, tracking ends at that point. Pixels included in each contour are denoted by e
_{[n, m]} ^{L ′} (m = 0, 1, 2, ‥‥, NPn). Here, NPn is the number of edges included in the contour line E _[n] ^{L '} . FIG. 45 shows an example in which E _[0] ^{L ′} , E _[1] ^{L ′} , and E _[2] ^{L ′} are extracted. E _[0] ^{L '} is extracted from the start point of St ^L' (i ^{L '} st, j ^L' st) to the tracking end point of E _[0] ^{L '} as shown in FIG. E _[1] ^{L '} is St ^L' (i ^{L '} st, j ^L' s
Starting from t), as shown in FIG. 45, up to the tracking end point of E _[1] ^{L '} is extracted. The same applies to E _[2] ^{L '} .

In step S225, a main contour is determined from the extracted contours. If only one contour is extracted, the following processing is not performed. As a method for determining the main contour from E _[n] ^{L ',} two methods are selected here that satisfy the following conditions. The younger the number, the higher the priority.

Condition 1: Starting from the start point of E _[n] ^{L ', a} point that reaches the rectangle indicating the measurement area is selected. Condition 2: The longest contour line is selected from E _[n] ^{L '} .

In FIG. 45, E satisfying both the conditions 1 and 2 is satisfied.
_[0] ^{L '} and E _[1] ^L' are selected. E _{[0 ]} ^{L '} and E _[1] ^L' are both S
A contour line starting from t ^{L ′} (i ^{L ′} st, j ^{L ′} st) is created from the two contour lines. This can be done by inverting the pixel order of one contour line and combining it with the other. However, both contour lines are St ^{L '
(I L 'st, j L} ' st) because it contains, it is necessary to remove the St ^L of one contour line ^{'(i L' st, j} L 'st).

E _[0] ^{L '} ｛e _[0,0] ^L' , e shown in FIG.
_[0,1] ^{L '} , ‥‥, e _{[0, NP0]} ^L' ｝, E _[1] ^{L '} ｛e _[1,0] ^L' ,
In e _[1,1] ^{L '} , _{ , e _{[1, NP1]} ^L' }, E _[0] ^{L '} is inverted to obtain a pixel sequence E _[0] E _[1] ^L' ｛e _{[0 , NP0]} ^{L '} , e
_{[0, NP-1]} ^{L '} , ‥‥, e _[0,1] ^L' , e _{[1,0 ]} ^{L '} ,
e _[1,1] ^{L '} , _{ , e _{[1, NP1]} ^L' }. In this example, the start point e _[0,0] ^{L '} of the contour line of E0 is deleted. Said E
_[0] E _[1] ^{L '} is changed to E _[0] E _[1] ^L' ｛e _[0] ^{L '} , as shown in FIG.
e _[1] ^{L '} , ‥‥, e _{[NP0 + NP1-1]} ^L' ｝.

In step S226, step S2
08 and pattern matching using epipolar constraints
And a pixel row E based on the principle of triangulation._[0]E_[1] ^{L '}3rd
Original position E_[0]E_[1] ^{L '} _W｛E_[0] ^{L '}, E_[1] ^{L '}, ‥‥, e
_{[NP0 + NP1-1]} ^{L '}Determine｝. In particular, e_[0] ^{L '}To
P_[0] ^{L '}, E_{[NP0 + NP1-1]} ^{L '}To P_[1] ^{L '}And the third order
Original position is P _[0] ^{L '} _W(X_[0] ^{L '}, Y_[0] ^{L '}, Z_[0] ^{L '}), P
_[1] ^{L '} _W(X_[1] ^{L '}, Y_[1] ^{L '}, Z_[1] ^{L '}).

In step S227, P _[0] ^{L '} _W ,
A straight line Lm is determined from P _[1] ^L' _W .

[0180]

[Equation 27] E _[0] E _[1] The distance between each edge of ^L' _W and the straight line Lm is obtained, and the maximum distance and the edge at that time are obtained from the distance.

In step S228, E _[0] E _[1] indicating the maximum distance on the overlap image as shown in FIG.
^A figure is drawn at the position coordinates E _[0] E _[1] ^L corresponding to the inverse correction of ^{L '} (633L, 633R), and a distance Mmax is provided near the figure.
(632L, 632R) is displayed. The method of marking is performed in the same manner as in the algorithm D in the first embodiment, and the method of displaying the distance is performed in the same manner as the algorithm F in the first embodiment to display three-dimensionally. It returns to step S202.

As described above, the measurement can be easily performed by surrounding the object to be measured while the object is stereoscopically viewed, and the measurement result can be viewed in a stereoscopic state. When inputting the designated point, the screen may be frozen.

FIGS. 49 to 52 relate to the third embodiment of the present invention. FIG. 49 is a flowchart showing the flow of processing by the host computer, and FIG. 50 is a first diagram for explaining the operation of the flowchart of FIG. FIG. 51 shows FIG.
FIG. 52 is a flow chart showing the flow of processing of algorithm J used in the flow chart of FIG. 49.

Since the third embodiment is almost the same as the first embodiment, only different points will be described, and the same components will be denoted by the same reference numerals and description thereof will be omitted.

As in the first embodiment, it is assumed that the solid scale is displayed in Dis1 [mm] from the endoscope end. If the solid scale is displayed even when the distance from the endoscope end to the object is shorter than Dis1 [mm], there is a sense of discomfort because the solid scale to be hidden by the object is visible.

Therefore, a stereoscopic vision system that does not display a stereoscopic figure when the distance from the endoscope end to the object is shorter than Dis1 [mm] will be described below.

In the following, a three-dimensional scale will be described as an example of a three-dimensional figure whose display is stopped in accordance with the distance. However, the figure is not limited to the three-dimensional scale. For example, the figure is not limited to the three-dimensional scale. Even in the case of a cross figure or the like, display can be stopped according to the distance by performing the same processing on the cross figure. In addition, there may be a plurality of three-dimensional figures for which the display is stopped, and the display can be stopped according to the distance by performing the same stop processing for each figure.

As in the first embodiment, it is assumed that five scales in the horizontal direction and five scales in the vertical direction are drawn on the left overlay image and the right overlay image as shown in FIGS. In the first embodiment, the distance from the endoscope tip to the scale is given in advance, but in the present embodiment,
The actual three-dimensional coordinates of the object are obtained, and a scale is drawn at that position.

As shown in FIG. 8, each scale drawing coordinate of the left image is M
_[k] ^{L '} (i _[k] ^L' , j _[k] ^{L '} ) (k = 0, 1, ‥‥,
9), as shown in FIG.
_[k] ^{R '} (i _[k] ^R' , j _[k] ^{R '} ) (k = 0,1, ‥‥, 9)
It expresses. As shown in FIG. 8, the center of the horizontal scale is M _{[ 2]} ^{L ′} (i _[2]
^{L '} , j _[2] ^L' ). As a representative, M
_[2] ^{L '} (i _[2] ^L' , j _[2] ^{L '} ) (or the center of the vertical scale M _[7] ^L' (i _[7] ^{L '} , j _[7] ^L' )) reflected by the three-dimensional position γ of the object are ^{_{(x L W, y L W}} , z L W) determined. Assuming that the target is on a vertical plane passing through z ^L _W , the scale interval and the scale drawing coordinates in the left and right overlay images are determined as in the first embodiment, and the scale is drawn on the left and right overlay images. The overlay image is synthesized with the left and right original images by a known keying technique.

[0190] Next, subject to the distal end of the endoscope is z ^L _W [m
m] will be described.

In FIG. 50, it is assumed that the positions of the left and right cameras are fixed. Now, let ξ be the distance between the left camera and the target. The point Q on the object corresponding to the point Q1 on the left image is Q1
And the straight line passing through the left optical center O _left and the intersection with the object. The image Q2 on the right image for Q is Q
And the right image passing through the right optical center O _right .

Next, the distance between the left camera and the target is ξ ′.
(Ξ '<ξ). The point Q 'on the object corresponding to the point Q1 on the left image can be obtained from the intersection of Q1 and a straight line passing through the left optical center with the object. The coordinates of the image Q2 'on the right image corresponding to the point Q' on the object can be obtained in the same manner as Q2. That is, the coordinates of the corresponding point on the right image with respect to a certain point on the left image (referred to as a designated measurement point) vary depending on the distance between the camera and the object, but can be uniquely determined.

Further, Q1, Q2 and Q2 'are the same objects.
Since the image is an image, there is a high possibility that its vicinity is similar.
Therefore, the distance between the left camera and the target is as shown in FIG.
To z ^L _WApproaching from [mm] to 0 [mm] by ρ [mm]
While specifying the measurement of the left image in k ways (k is a positive integer)
Point (Cx^{L '}, Cy^{L '}) In the right image_[k] ^{R '}
(I_[k] ^{R '}, J_[k] ^{R '}) (K = 0,1,2, ‥‥)
(Ρ is a real number). Currently, the left and right images (Cx
^{L '}, Cy^{L '}) And M_[k] ^{R '}(I_[k] ^{R '}, J_[k] ^{R '}) (K = 0,
If the similarity of (1, 2, ‥‥) is high, the target is Dis1-
It can be determined that the distance is (ρ × k).

[0194] First, to determine the drawing position of the solid scale in z ^L _W [mm] from the endoscope distal end, to draw to the left and right overlay image (from step S251 step S254).
Next, 0 distance from the endoscope tip from the z ^L _W [mm] [m
m] between stepwise left image ^{(Cx L ', Cy L'} ) determine the corresponding point in the right image for the stores (step S255~
Step S257). Distance from the endoscope tip to a subject by similarity scale drawing to determine whether closer z ^L _W [mm], realized in the flow of erasing (step S261~267). The processing flow will be described with reference to FIG.

In step S251, the center coordinates M _[2] ^{L '} (i
_[2] ^{L ′} , j _[2] ^{L ′} ) The three-dimensional coordinates of the object shown in the figure are obtained. That is, that position becomes the three-dimensional position of the scale. In the present embodiment, the coordinates of M _[2] ^{L '} (i _{[ 2]} ^L' , j _[2] ^{L '} ) on the left image are measured designated points (CxL ^' , Cy) in the present embodiment.
^{L '} ). A corresponding point in the right image of M _[2] ^{L '} (i _[2] ^L' , j _[2] ^{L '} ) in the left image given in advance is determined by template matching using a known epipolar constraint. The template is M _[2] ^{L '} (i _[2]
A small area of a predetermined size (Tx × Ty) centering on ^{L ′} , j _[2] ^{L ′} ) is created. Using the coordinates of the corresponding point, the designated measurement point (Cx ^{L '} , C
y ^{L ′} ) is obtained. Let the coordinates be γ (x ^L _W , y ^L
_W , z ^L _W ).

[0196] In step S252, the distance to the scale of the endoscope tip as z ^L _W, the algorithm B to determine the Sc 1 scale interval for [mm] [pixels]. These are a left image horizontal scale interval Spx ^{L '} , a left image vertical scale interval Spy ^L' , a right image horizontal scale interval Spx ^{R '} , and a right image vertical scale interval Spy ^R' .

In step S253, the drawing position is determined by the left image horizontal scale interval Spx ^{L '} and the left image vertical scale interval Spy ^L' . M _[n] ^{L '} (n = 0, 1, ‥‥, 9)
Is the designated measurement point (C
x ^{L ′} , Cy ^{L ′} ) is determined by the following equation.

[0198]

Equation 28] horizontal scale ^{_{M [n] L '= (}} Cx L' + Spx L '· (n-2), Cy L') vertical scale ^{_{M [n] L '= (}} Cx L', Cy L '+ Spy in ^{L '· (n-6)} ) (28) step S254, the distance from the endoscope tip to the scale as z ^L _W [mm], M by algorithm a
_[n] ^{L ′} (n = 0, 1, ‥‥, 9) inverse correction corresponding pixel M _[n]
^L (n = 0, 1, ‥‥, 9) and its corresponding point M _[n] ^R
(N = 0, 1,..., 9) is obtained. M _[n] ^L and M
By using algorithm _[n] ^R (n = 0, 1, ‥‥, 10), a solid scale 600L is drawn on the left overlay image and 600R is drawn on the right overlay image as shown in FIG.

In step S255, k = 0. In step S256, the distance [mm] to scale the endoscope tip z ^L _W - and (ρ × k). Where ρ
Is a distance change step. While changing the distance, the corresponding point M _[k] ^{R '} of M _[2] ^L' is obtained by the algorithm A.
M _[k] ^{R 'coordinates} of ^{_{M [k] R' (i}} [k] R ', j [k] R') to be.

[0200] In step S257, the k 1 increased, z ^L _W in step S258 - returns to the (ρ × k)> 0 if step S256.

In step S259, scale drawing is performed using algorithm J described later. Step S260
In, an application end determination is made. If completed, the process ends. Otherwise, the process returns to step S259.

[Algorithm J] The flow of processing for drawing and erasing a scale according to the distance to an object will be described with reference to FIG.

At step S261, k = 0. In step S262, the designated measurement point (Cx ^{L '} ,
Cy ^{L ′} ) and M _[k] ^{R ′} (k = 0, 1, 2, ‥‥) are correlated with small areas centered on M _[k] ^{R ′} (i _[k] ^{R ′} ₎ by template matching. , J _[k] ^{R ′} ) in a predetermined search area.

In step S263, M _[k] ^{R '} (i
If the maximum correlation value in the search area of _[k] ^{R '} , j _[k] ^R' ) is larger than a predetermined threshold Th3, it is determined that the target exists less than Dis1 from the tip. In that case, step S266 is executed.

In step S264, when the correlation between the designated measurement point and all M _[k] ^{R '} (i _[k] ^R' , j _[k] ^{R '} ) has been checked, step S267 is executed.

In step S265, k is incremented by 1, and the flow returns to step S262. In step S266, M _[n] ^L (i _[n] ^L , j _[n] ^L ) of the left overlay image,
M _[n] ^R (i _[n] ^L , j _[n] ^L ) of the right image (n = 0, 1, 2,
(4) Delete the scale of (9). The process ends.

In step S267, M _[n] ^L (i _[n] ^L , j _[n] ^L ) of the left overlay image and M _[n] ^{L of} the right image
_[k] ^R (i _[n] ^L , j _[n] ^L ) (n = 0,1,2, ‥‥, 9)
Draw a scale on. The process ends.

[Appendix] The first, second, and third described above
According to the embodiment, the contents shown in the following additional items can be said to be the characteristic items.

(Additional Item 1) A first image obtained by capturing an image of a predetermined observation target by the first imaging means and a second image obtained by imaging the predetermined observation target from a direction different from that of the first imaging means are converted to a second image. In a measurement endoscope apparatus capable of displaying a second image obtained by imaging by an imaging unit and capable of displaying a stereoscopic view, a virtual image is generated by either one of the first imaging unit or the second imaging unit. Alternatively, in a distance setting means for setting a distance to an actual object, an image obtained by the first imaging means, and an image obtained by the second imaging means, the same point of the object is imaged. Corresponding point determining means for obtaining a corresponding point, graphic drawing means for drawing a graphic corresponding to the distance to the object at at least one of the corresponding points, deriving parallax of the corresponding point, Inspection means for inspection A measuring endoscope apparatus characterized by the following.

(Additional Item 2) The measurement endoscope apparatus according to Additional Item 1, wherein the graphic drawing means has a scale drawing function of drawing a scale according to a distance to the object.

According to this, the size of the object can be grasped by comparing the scale with the object.

(Additional Item 3) The graphic drawing means is characterized in that the graphic drawing means draws the graphic at corresponding points determined by the corresponding point determining means with respect to a plurality of distances to the object. 2. The measurement endoscope apparatus according to claim 1.

According to this, it is easy to grasp the positional relationship between the object and the figure.

(Additional Item 4) The measurement drawing according to Additional Item 3, wherein the figure drawn at a plurality of distances to the object is a scale corresponding to the distance to the object. Endoscope device.

According to this, it is easy to grasp the positional relationship between the object and the figure and to grasp the size of the object.

(Additional Item 5) The endoscope apparatus for measurement according to additional items 1 to 4, characterized in that the graphic drawing means has a function of stopping graphic drawing according to the distance to the object. .

According to this, it is possible to stop displaying the graphic according to the distance to the object.

(Additional Item 6) An input processing device for performing a predetermined process in accordance with an input.
5. The measurement endoscope device according to any one of items 4 to 4.

According to this, in a stereoscopic state,
Processing can be executed according to the input.

(Additional Item 7) The measurement apparatus according to additional item 6, wherein the input processing device specifies the figure or the inside of the image and performs a predetermined process according to the specified location. Endoscope device.

According to this, when stereoscopically viewed,
By designating the inside of the field of view, a predetermined process can be executed.

(Additional Item 8) The measurement endoscope apparatus according to additional item 6 or 7, wherein the input device of the input processing device is a keyboard.

According to this, when stereoscopically viewed,
A predetermined process can be executed according to a keyboard input.

(Additional Item 9) The measurement endoscope apparatus according to additional item 6 or 7, wherein the input device of the input processing device is a mouse.

According to this, in a stereoscopic state,
A predetermined process corresponding to a mouse input can be executed.

(Additional Item 10) The measurement endoscope apparatus according to additional item 6 or 7, wherein the graphic drawing means changes parameters related to the graphic based on an instruction from the input processing device. .

According to this, when stereoscopically viewed,
It is possible to change the parameters of the graphic displayed in the field of view and display the graphic according to the parameters.

(Additional Item 11) The measurement endoscope apparatus according to additional item 6 or 7, wherein the graphic drawing means draws an instruction content from the input processing device.

According to this, in a stereoscopic state,
It is possible to confirm the instruction content.

(Additional Item 12) The measurement endoscope apparatus according to additional item 6 or 7, wherein the graphic drawing means draws a parameter relating to a graphic.

According to this, in a stereoscopic state,
It is possible to check parameters related to the figure.

(Additional Item 13) The measurement endoscope according to additional item 7, wherein the input processing device has a measuring means for designating a point on the object in the image and measuring the designated point. Mirror device.

According to this, in a stereoscopic state,
Measurement at the specified point becomes possible.

(Additional Item 14) The additional item is characterized in that the measuring means performs measurement by a distance between a straight line composed of two points on the object in the image designated by the input processing device and an arbitrary point. 14. The measurement endoscope apparatus according to claim 13.

According to this, in a stereoscopic state,
It is possible to measure a straight line by two designated points and an arbitrary point.

(Supplementary note 15) The measuring instrument according to Supplementary note 7, wherein the input processing device includes a measuring unit for designating a measurement area of the object in the image and measuring the designated area. Endoscope device.

According to this, by specifying the measurement area, measurement for the area becomes possible.

(Additional Item 16) The measurement endoscope according to Additional Item 15, wherein the measuring means extracts a main contour in the measurement area and measures the contour. apparatus.

According to this, in a stereoscopic state,
The measurement for the contour line becomes possible.

(Additional Item 17) The additional measurement item 1 is characterized in that the measuring means performs measurement based on a distance between a straight line composed of two edges on the main contour and an arbitrary point.
6. The measurement endoscope apparatus according to 5.

According to this, in a stereoscopic state,
It is possible to measure a straight line by a point edge and an arbitrary point.

(Additional Item 18) The measurement endoscope apparatus according to additional items 13 to 17, wherein the graphic drawing means draws a result obtained by the measuring means.

[0243]

As described above, according to the present invention, there is an effect that a figure which looks three-dimensional can be displayed, the distance to the object can be grasped, and information can be presented.

[Brief description of the drawings]

FIG. 1 is a configuration diagram showing a configuration of an endoscope apparatus according to a first embodiment of the present invention.

FIG. 2 is a block diagram showing the configuration of the host computer shown in FIG. 1;

FIG. 3 is a configuration diagram showing a configuration of a distal end portion of the endoscope in FIG. 1;

FIG. 4 is a view for explaining a coordinate system in the imaging system of the endoscope in FIG. 1;

FIG. 5 is a first diagram illustrating distortion correction of the endoscope in FIG. 1;

FIG. 6 is a second diagram illustrating distortion correction of the endoscope in FIG. 1;

FIG. 7 is a first diagram illustrating the operation of the host computer in FIG. 2;

FIG. 8 is a second diagram illustrating the operation of the host computer in FIG. 2;

FIG. 9 is a third diagram illustrating the operation of the host computer in FIG. 2;

FIG. 10 is a fourth diagram illustrating the operation of the host computer in FIG. 2;

FIG. 11 is a first flowchart showing a flow of a main process performed by the host computer of FIG. 2;

FIG. 12 is a second flowchart showing the flow of a main process performed by the host computer of FIG. 2;

FIG. 13 is a flowchart showing a processing flow of an algorithm A used in the main processing of FIGS. 11 and 12;

FIG. 14 is a first diagram illustrating an operation of processing of algorithm B used in the main processing of FIGS. 11 and 12,

FIG. 15 is a second diagram illustrating the operation of the processing of algorithm B used in the main processing of FIGS. 11 and 12,

FIG. 16 is a flowchart showing a processing flow of an algorithm A ′ used in the main processing of FIGS. 11 and 12;

FIG. 17 is a flowchart showing a processing flow of an algorithm B used in the main processing of FIGS. 11 and 12;

FIG. 18 is a flowchart showing the flow of processing of an algorithm C used in the main processing of FIGS. 11 and 12;

FIG. 19 is a first diagram illustrating the operation of the process of algorithm C in FIG. 18;

20 is a second diagram illustrating the operation of the process of algorithm C in FIG. 18;

FIG. 21 is a flowchart showing a processing flow of an algorithm D used in the main processing of FIGS. 11 and 12;

FIG. 22 is a first diagram illustrating the operation of the processing of algorithm D in FIG. 21;

FIG. 23 is a second diagram illustrating the operation of the process of algorithm D in FIG. 21;

FIG. 24 is a flowchart showing the flow of processing of an algorithm E used in the main processing of FIGS. 11 and 12;

FIG. 25 is a first diagram illustrating the operation of the processing of algorithm E in FIG. 24;

FIG. 26 is a second diagram illustrating the operation of the processing of algorithm E in FIG. 24;

FIG. 27 is a flowchart showing the flow of processing of an algorithm F used in the main processing of FIGS. 11 and 12;

FIG. 28 is a first diagram illustrating the operation of the processing of algorithm F in FIG. 27;

FIG. 29 is a second diagram illustrating the operation of the process of the algorithm F in FIG. 27;

FIG. 30 is a flowchart showing the flow of processing of an algorithm G used in the main processing of FIGS. 11 and 12;

FIG. 31 is a first diagram illustrating the operation of the processing of algorithm G in FIG. 30;

FIG. 32 is a second diagram illustrating the operation of the processing of algorithm G in FIG. 30;

FIG. 33 is a first flowchart showing a processing flow of an algorithm H used in the main processing of FIGS. 11 and 12;

FIG. 34 is a second flowchart showing the flow of processing of algorithm H used in the main processing of FIGS. 11 and 12;

FIG. 35 is a first flowchart showing a processing flow by a host computer according to the second embodiment of the present invention;

FIG. 36 is a second flowchart following the first flowchart of FIG. 35;

FIG. 37 is a first diagram illustrating the operation of the processing in FIGS. 35 and 36;

FIG. 38 is a second diagram illustrating the operation of the processing in FIGS. 35 and 36;

FIG. 39 is a third diagram illustrating the operation of the processing in FIGS. 35 and 36;

FIG. 40 is a fourth diagram illustrating the operation of the processing in FIGS. 35 and 36;

FIG. 41 is a flowchart showing the flow of processing of a modification of the second flowchart in FIG. 36;

FIG. 42 is a first diagram illustrating the operation of the flowchart in FIG. 41;

FIG. 43 is a second diagram illustrating the operation of the flowchart in FIG. 41;

FIG. 44 is a third diagram illustrating the operation of the flowchart in FIG. 41;

FIG. 45 is a fourth diagram illustrating the operation of the flowchart in FIG. 41;

FIG. 46 is a fifth diagram for explaining the operation of the flowchart in FIG. 41;

FIG. 47 is a sixth diagram for explaining the operation of the flowchart in FIG. 41;

FIG. 48 is a seventh diagram illustrating the operation of the flowchart in FIG. 41;

FIG. 49 is a flowchart showing the flow of processing by the host computer according to the third embodiment of the present invention.

FIG. 50 is a first diagram illustrating the operation of the flowchart in FIG. 49;

FIG. 51 is a second diagram illustrating the operation of the flowchart in FIG. 49;

FIG. 52 is a flowchart showing the processing flow of algorithm J used in the flowchart of FIG. 49;

[Explanation of symbols]

301 ... endoscope (stereo video image end scope) 310L, 310R ... video processor 311 ... host computer 312L, 312R ... observation monitor 313 ... head mounted display 321 ... CPU 322 ... main memory 323 ... host-local bridge 324L, 324R ... Image input / output board 325 ... External storage interface 326,327 ... Peripheral device interface 328 ... Video board 329 ... Host bus 330 ... Local bus 331 ... Mouse 332 ... Keyboard 333 ... Display

────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] June 8, 2001 (2001.6.8)

[Procedure amendment 1]

[Document name to be amended] Statement

[Correction target item name] 0004

[Correction method] Change

[Correction contents]

[0004]

Problems to be Solved by the Invention
The problems of the third publication are as follows: Since the device information is simply drawn on the left and right images, the device information may have a parallax exceeding the fusion limit and the device information may not be stereoscopically viewed. In that case, the device information may hinder the stereoscopic viewing of the endoscope image. Also,
At this time, when performing stereoscopic viewing of the endoscope image and when performing stereoscopic viewing of the device information, it is necessary for the user to adjust the device for stereoscopic viewing, thereby forcing the user to perform additional work.

[Procedure amendment 2]

[Document name to be amended] Statement

[Correction target item name] 0053

[Correction method] Change

[Correction contents]

In step S2, the left image horizontal scale
Spax Spx^{L '}, Left image vertical scale interval Spy ^{L '}Using the left image
Scale drawing coordinate M at_[k] ^{L '}(K = 0,1, ‥‥,
9). Scale drawing of the left image given in advance
Image center coordinates (Cx^{L '}, Cy^{L '}) And is determined by equation (2)
Is done.

[Procedure amendment 3]

[Document name to be amended] Statement

[Correction target item name] 0054

[Correction method] Change

[Correction contents]

[0054]

## EQU2 ## Horizontal scale M _[k] ^{L '} = (CxL ^' + SpxL ^'. (K-2), CyL ^' ) = (i _[k] ^{L '} , j _[k] ^L' ) (0≤ k ≦ 4) Vertical scale M _[k] ^{L ′} = (CxL ^′ , CyL ^′ + SpyL ^′ · (k−6)) = (i _[k] ^{L ′} , j _[k] ^{L ′} ) (5 ≦ (2) In step S3, the distance from the endoscope tip to the scale is set to Dis1 [mm], and M _[k] ^{L '} (i _[k] ^L' , j _{[k ] is determined} by algorithm A described later. _] ^{L ′} ) (k = 0, 1, ‥‥,
9) The inverse correction corresponding pixel M _[k] ^L (i _[k] ^{L '} , j _[k] ^L' ) (k
= 0, 1, ‥‥, 9) and the corresponding point M _[k] ^R (i
_[k] ^{L ′} , j _[k] ^{L ′} ) (k = 0, 1, ‥‥, 9 ).
Using M _[k] ^L and M _[k] ^R (k = 0, 1,..., 9), the algorithm C described later uses a solid scale 600L for the left overlay image and a 600R for the right overlay image as shown in FIG. To draw.

[Procedure amendment 4]

[Document name to be amended] Statement

[Correction target item name]

[Correction method] Change

[Correction contents]

The parameter that can be changed is, for example, the scale interval, but other parameters are also possible. The mouse cursor, parameters, and GUI are deleted from the screen when there is no input by the input means for a predetermined time or more. When the input is restarted, the parameters and GUI are automatically redrawn.

[Procedure amendment 5]

[Document name to be amended] Statement

[Correction target item name] 0173

[Correction method] Change

[Correction contents]

Using St ^{L ′} (i ^{L ′} st, j ^{L ′} st) as the tracking start position, all the contour lines E _[n] ^{L ′} (n = 0, 1, 2,.
{, NE) is traced. Here, NE is the number of contour lines. If it reaches outside the area during tracking, tracking ends at that point. Pixels included in each contour are denoted by e
_{[n, m]} ^{L ′} (m = 0, 1, 2, ‥‥, NPn). Here, NPn is the number of edges included in the contour line E _[n] ^{L '} . FIG. 45 shows an example in which E _[0] ^{L ′} , E _[1] ^{L ′} , and E _[2] ^{L ′} are extracted. E _[0] ^{L '} is extracted from the start point of St ^L' (i ^{L '} st, j ^L' st) to the tracking end point of E _[0] ^{L '} as shown in FIG. E _[1] ^{L '} is St ^L' (i ^{L '} st, j ^L' s
Starting from t), the tracking end point of E _[1] ^{L '} is extracted as shown in FIG. The same applies to E _[2] ^{L '} .

[Procedure amendment 6]

[Document name to be amended] Statement

[Correction target item name]

[Correction method] Change

[Correction contents]

[0198]

Equation 28] horizontal scale ^{_{M [n] L '= (}} Cx L' + Spx L '· (n-2), Cy L') vertical scale ^{_{M [n] L '= (}} Cx L', Cy L '+ Spy in ^{L '· (n-6)} ) (28) step S254, the distance from the endoscope tip to the scale as z ^L _W [mm], M by algorithm a
_[n] ^{L ′} (n = 0, 1, ‥‥, 9) inverse correction corresponding pixel M _[n]
^L (n = 0, 1, ‥‥, 9) and its corresponding point M _[n] ^R
(N = 0, 1,..., 9) is obtained. M _[n] ^L and M
_[n] ^R (n = 0,1, ‥‥, 9 )
As shown in FIG. 10, the solid scale 6 is displayed on the left overlay image.
00L and 600R are drawn on the right overlay image.

[Procedure amendment 7]

[Document name to be amended] Drawing

[Correction target item name] FIG.

[Correction method] Change

[Correction contents]

FIG. 51

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G06T 1/00 315 G06T 1/00 315 5C061 17/40 17/40 F H04N 7/18 H04N 7/18 M 13/00 13/00 F-term (reference) 2H040 BA15 BA22 GA02 GA11 4C061 CC06 HH51 JJ17 LL08 NN05 WW12 5B050 AA02 BA09 EA19 EA28 FA02 FA06 FA14 5B057 AA07 CA12 CB13 CC01 CD14 DA07 DA16 DB03 DC03 5C051 CA04 CC03 FA02 AA01 AA21 AB04 AB18 AB20

Claims (9)

[Claims] 1. A first image obtained by capturing an image of a predetermined observation target by a first imaging unit, and the predetermined observation target by a second imaging unit from a direction different from the first imaging unit. In a measurement endoscope apparatus capable of displaying a second image obtained by imaging in a stereoscopic view, either one of the first imaging means or the second imaging means and a virtual or actual A distance setting unit for setting a distance to an object; an image obtained by the first imaging unit;
In an image obtained by the imaging means, a corresponding point determining means for finding a corresponding point at which the same point of the object is imaged, and a figure corresponding to the distance to the object is converted into at least one corresponding point. An endoscope apparatus for measurement, comprising: a figure drawing means for drawing; and an inspection means for deriving a parallax of the corresponding point and inspecting the parallax. 2. The measurement endoscope apparatus according to claim 1, wherein the graphic drawing means has a scale drawing function for drawing a scale according to a distance to the object. 3. The apparatus according to claim 1, wherein said figure drawing means draws said figure at a corresponding point determined by said corresponding point determining means for a plurality of distances to said object. The endoscope device for measurement according to 1. 4. The measurement endoscope apparatus according to claim 1, wherein the graphic drawing means has a function of stopping graphic drawing according to a distance to the object. 5. The measurement endoscope apparatus according to claim 1, further comprising an input processing device for performing a predetermined process according to an input. 6. The input processing device according to claim 1, wherein
The measurement endoscope apparatus according to claim 5, wherein an inside of the image is designated, and a predetermined process is performed according to the designated place. 7. The measurement endoscope apparatus according to claim 5, wherein the graphic drawing unit changes parameters related to the graphic based on an instruction from the input processing device. 8. The measurement endoscope apparatus according to claim 5, wherein the graphic drawing means draws an instruction content from the input processing device. 9. The measurement endoscope apparatus according to claim 6, further comprising a measurement unit that specifies a point on the object in the image by the input processing device and measures the specified point.