JP2001167272A - Device and method for measuring and displaying object - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily grasp the positional relation of a three-dimensional(3D) form and an object. SOLUTION: When a measure line L is designated as a part to be measured in the left image of an object displayed on a monitor 130L for left image pickup, the correspondent points of respective points on the measure line L are found on the right image (not shown in the figure) of the object and the 3D form (3D data) of the measure line L is found while using the principle of triangulation. Then, the result is displayed as a profile T. The correlation of the measure line L and the profile T is found and a marker clearly showing the correlation is displayed on the measure line L and the profile T so that the correlation of the 3D form and the object can be easily grasped.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a subject measuring and displaying apparatus and a subject measuring and displaying method capable of easily measuring the distance of an arbitrary position of a captured subject image.

[0002]

2. Description of the Related Art In recent years, in the medical field and the industrial field,
Endoscopes have become widely used. In a normal endoscope observation image, the object is generally planar,
It is difficult to recognize irregularities. For this reason, recently, two systems of left and right imaging means are arranged at the tip of the endoscope, and from the left and right images taken by the two systems of imaging means, a three-dimensional shape of the target is estimated by a stereo method. Are displayed by using a profile or a wire frame.

For example, Japanese Patent Application Laid-Open No. Hei 6-339454 discloses a method using a stereo endoscope based on the principle of triangulation, and Japanese Patent Application Laid-Open No. 63-246716 discloses a method using a monocular endoscope based on the principle of triangulation. A technique for calculating a distance and a distance between a plurality of target points is disclosed.

[0004]

However, in the technique disclosed in the above-mentioned prior art, means for displaying a profile of a subject to be measured (a three-dimensional shape converted into a two-dimensional graph) is displayed. However, even if there is a means for displaying a three-dimensional shape using a wire frame or the like, there is a problem that it is difficult to grasp the correspondence between the displayed three-dimensional shape and the subject image.

The present invention has been made in view of the above circumstances, and has as its object to provide a subject measurement and display device and a subject measurement and display method capable of easily grasping the correspondence between a three-dimensional shape and a subject image.

[0006]

In order to achieve the above object, a first object measuring and displaying apparatus according to the present invention comprises: an image pickup means for picking up an object image; and a plurality of image signals obtained from the image pickup means. Three-dimensional shape estimating means for estimating the three-dimensional shape of the subject image; profile information creating means for creating profile information of a predetermined portion in the subject image based on the three-dimensional shape information estimated by the three-dimensional shape estimating means; Profile information display means for displaying profile information created by the profile information creation means, and correspondence relationship for acquiring correspondence information between the profile information displayed by the profile information display means and the imaging signal acquired from the imaging means Information acquisition means, and correspondence information display for displaying the correspondence information acquired by the correspondence information acquisition information means Characterized by comprising a stage.

The first subject measurement and display method includes an imaging step of capturing a subject image, a three-dimensional shape estimation step of estimating a three-dimensional shape of the subject image from a plurality of image signals acquired in the imaging step, A profile information creation step of creating profile information of a predetermined portion in the subject image based on the three-dimensional shape information estimated in the three-dimensional shape estimation step;
A profile information display step of displaying profile information created in the profile information creation step, and correspondence information between the profile information displayed in the profile information display step and the imaging signal acquired in the imaging step are acquired. It is characterized by comprising a correspondence information acquisition step and a correspondence information display step of displaying the correspondence information acquired in the correspondence information acquisition step.

The second object measurement and display device includes an image pickup means for picking up an object image, a three-dimensional shape estimating means for estimating a three-dimensional shape of the object image from a plurality of image signals obtained by the image pickup means, Profile information creating means for creating profile information of a predetermined portion in the subject image based on the three-dimensional shape information estimated by the three-dimensional shape estimation means,
Profile information display means for displaying profile information created by the profile information creation means, and correspondence for acquiring correspondence information between the profile information displayed by the profile information display means and the imaging signal acquired from the imaging means. Relationship information acquisition means, designation means capable of designating an arbitrary position on the profile information displayed by the profile information creation means, designation information designated by the designation means and acquired by the correspondence information acquisition means Position information acquisition means for acquiring the first position information designated by the designation means as second position information in the image signal, based on the correspondence information, and the second position information acquired by the position information acquisition means. Based on the position information of 2, the designated position information by the designation means is placed on the subject image displayed based on the imaging signal. Characterized by comprising a Shimesuru designated-position information display means.

The second subject measurement and display method includes an imaging step of capturing a subject image, a three-dimensional shape estimation step of estimating a three-dimensional shape of the subject image from a plurality of image signals acquired in the imaging step, and A profile information creation step of creating profile information of a predetermined portion in the subject image based on the three-dimensional shape information estimated in the three-dimensional shape estimation step;
A profile information display step of displaying profile information created by the profile information creation means; and a correspondence acquisition of correspondence information between the profile information displayed in the profile information creation step and the imaging signal acquired in the imaging step. A relation information acquisition step, a designation step capable of designating an arbitrary position on the profile information displayed in the profile information display step, and designation information designated by the designation means and acquired by the correspondence information acquisition means. A position information acquisition step of acquiring the first position information designated in the designated step as second position information in the imaging signal, based on the corresponding relationship information, and a second position information acquired in the position information acquisition step. Based on the position information of 2, the designated position information by the designated process is displayed on the subject image displayed based on the imaging signal. Characterized by comprising the designated-position information display step for.

[0010]

Embodiments of the present invention will be described below with reference to the drawings. In the present embodiment, as a measurement endoscope to be applied, a stereo endoscope provided with two objective lenses at the tip is used.

(First Embodiment) FIGS. 1 to 11 relate to a first embodiment of the present invention, FIG. 1 is a block diagram showing a configuration of an endoscope apparatus for measurement, and FIG. FIG. 3 is a block diagram showing a functional configuration of an endoscope device for measurement, FIG. 4 is a block diagram showing a configuration of a host computer, FIG. 5 is a block diagram showing a configuration of an external storage device, 6 is an explanatory diagram of a three-dimensional coordinate system, FIG. 7 is an explanatory diagram for the principle of three-dimensional coordinate calculation, FIG. 8 is a flowchart showing a flow of drawing a measurement result, FIG. 9 is a window display example, and FIG. FIG. 11 is an explanatory view showing a means for converting a three-dimensional shape (three-dimensional data) into a figure (two-dimensional data).

(Configuration) As shown in FIG. 3, a measuring endoscope apparatus employed in the present embodiment is a stereo video endoscope (hereinafter, simply referred to as "endoscope") 101, and A video processor for right image 110R and a video processor for left image 110L that perform signal processing on each of image signals of a right image and a left image captured by the endoscope 101, and the video processors 110R and 11 respectively.
0L, for example, a right image frame memory 112R and a left image frame memory 112L for storing video signals based on RGB signals, and video signals based on, for example, RGB signals output from the frame memories 112R and 112L. A right image monitor 130R and a left image monitor 130L that input and display a right image and a left image
And a host computer 120 that performs a stereoscopic measurement operation using the images stored in the frame memories 112R and 112L, an external storage 140 connected to the host computer 120, and a monitor 130R connected to the host computer 120. The mouse 145 is used to operate a cursor displayed on 130L, specify a measurement target point, set a measurement target area, and the like.

Both video processors 110R, 110L
Are set to perform signal processing synchronized with each other. Also, in the present embodiment, each frame memory 112
The R and 112L have a plurality of sets of memories for R, G and B. One set stores an image, the other set stores a cursor, and adds the signals written to each set. Thereby, an image and a cursor can be displayed on the screen of each of the monitors 130R and 130L.

As shown in FIG. 4, the host computer 120 includes a CPU 121, a right frame memory interface 122R, a left frame memory interface 122L, a main memory 123, an external storage interface 124, a mouse interface 125, a keyboard 126 and a CRT 127. , These are connected to each other by a bus.

Also, the right frame memory interface 1
22R, left frame memory interface 122L
Are connected to a right image frame memory 112R and a left image frame memory 112L, respectively, for transmitting and receiving image data to and from the right image frame memory 112R and the left image frame memory 112L via the interfaces 122R and 122L.
It is set to perform cursor control for R and 112L. The external storage interface 124 is connected to the external storage 140 and is set to transmit and receive image data and measurement target point position information. The mouse interface 125 is connected to the mouse 145.

As shown in FIG. 5, the external storage 140 stores the position information of the measurement target point,
0, a measurement data information file 181 that sends and receives various measurement data, a stereo image manager 182 that sends and receives image data to and from the host computer 120, and a left image that is connected to the stereo image manager 182 and stores left image data. A right image file 183R, which is connected to the stereo image manager 182 and stores the right image data, is provided.

In the present embodiment, stereo images obtained by the endoscope 101 are handled as a pair of left and right images, and a stereo image sent from the host computer 120 in pairs is a stereo image manager. 182, the image files are sorted into left and right image files 183L and 183R and recorded. Also, stereo image manager 1
According to 82, the images recorded in each of the image files 183L and 183R are called in pairs.

As shown in FIG. 2, the endoscope 101 has an elongated insertion section 102, and a plurality of, for example, two observation windows and an illumination window are provided at the distal end of the insertion section 102. ing. Inside each observation window, a right-eye objective lens system 103R and a left-eye objective lens system 103L are provided at positions having parallax with each other. Each objective lens system 103
Left and right image pickup units 104L and 104R using solid-state image pickup devices are provided at image forming positions of R and 103L, respectively.

A light distribution lens 105 is provided inside the illumination window.
At the rear end of the light distribution lens 105, a light guide 106 made of a fiber bundle is provided continuously. The light guide 106 is inserted into the insertion portion 102, and a light source device (not shown) is connected to the incident end.

Illumination light output from the light source device is applied to a subject via a light guide 106 and a light distribution lens 105, and reflected light from the subject is transmitted to the right and left images by objective lens systems 103R and 103L, respectively. An image is formed on the imaging units 104R and 104L as a left image.

Next, a functional block configuration of the measuring endoscope apparatus will be described with reference to FIG.

Each image signal of the subject image picked up by the image pickup means 104R, 104L is input to video processors 110R, 110L, respectively, and subjected to video signal processing. Each image signal output from each of the video processors 110R and 110L is supplied to an A / D converter 11
After being converted into digital signals by the 1R and 111L, the image memory, that is, each frame memory 112R and 112R
L is stored in the image memory.

The image signals read from the right and left image frame memories 112R and 112L are subjected to predetermined image processing by the image processing means 200, respectively.
Via the display control means 201, the D / A converter 158R, 1
After being converted to an analog signal, it is input to the right and left image monitors 130R and 130L. The two monitors 130R, 13R
A right image and a left image are displayed at 0L, respectively.

On the other hand, the display of the cursor is processed by the cursor display means 151, and the display of the cursor is managed by the display control means 201. The cursor is superimposed on the screen,
It is displayed on either one of the monitors 130R and 130L.

The image processing means 200 performs processing such as deriving a three-dimensional position and displaying the correspondence between the three-dimensional shape and the measurement points on the image with respect to the measurement points designated by the cursor. Is displayed on the monitor.

The display control means 201 performs not only management of cursor display but also processing necessary for multi-window display. The processing and control functions of the image processing means 200 and the display control means 201 are achieved by operating the host computer 120. In this case, the function of the display control means 201 is usually provided by the operating system.

Although the right image and left image frame memories 112R and 112L shown in FIGS. 3 and 4 are provided outside the host computer 120 in the present embodiment, the host computer 120 has a PCI board type frame. It may be built in using a memory. In FIG. 3, the display of the stereo image is a right image, a left image monitor 130R,
Going to 130L, but CRT12 of personal computer
7 may be performed.

Next, the process of estimating the three-dimensional shape and displaying the correspondence between the estimated three-dimensional shape and the measurement points on the image by the endoscope apparatus for measurement configured as described above will be described with reference to FIG. This will be described with reference to a flowchart showing the flow of the operation.

In the following, a measurement line L is designated in the left image,
The description will be made assuming that the corresponding point is obtained for the right image. However, there is no problem if the measurement line to be measured is specified in the right image and the corresponding point is obtained in the left image.

As shown in FIG. 9, the operator designates a measurement line L as a portion to be measured in the left image of the stereo image displayed on the left image monitor 130L.
Then, in the right image, a corresponding point of each point on the measurement line L is obtained. Then, by using the principle of triangulation, a three-dimensional shape (three-dimensional data) of the measurement line L is obtained and displayed as a profile.

In the following description, it is assumed that the image A is A (p, q), and p and q indicate the positions of the pixels. Unless otherwise specified, the size of the image is SX and SY in the horizontal and vertical directions, respectively.

First, in S1-1, a measurement line L is designated by an input device on the left image of the stereo image. A point on the measurement line L is represented by Li (i = 0,..., N-
1). The measurement line is designated by a device such as a mouse, for example, by designating a start point L0 and an end point LN-1.

In S1-2, a corresponding point on the right image corresponding to the point Li specified on the left image is obtained. The corresponding points can be detected by, for example, template matching using epipolar lines.

In S1-3, a three-dimensional position is obtained using the principle of triangulation. The three-dimensional position of each point Li on the measurement line L is represented by Di (i = 0,..., N−1), and the coordinates thereof are represented by Di (xi, yi, zi).

In S1-4, the three-dimensional position Di is displayed on a two-dimensional graph (profile). The two-dimensional technique will be described later.

Then, as shown in FIG.
If the point on the profile corresponding to i is Ti, markers (points 1, 2, 3, and 4 in the figure) are drawn at predetermined intervals It for this point Ti. That is, i
When% It = 0, a marker is drawn at the point Ti (% represents the remainder of the integer division). At other points Ti, no marker is drawn.

Note that the marker here is a mark for clearly indicating the drawing position, and specifically, a figure such as a circle or a square is used. Also, a serial number is assigned to the marker, and the number is drawn near the marker.

In S1-5, a measurement line L is drawn on the image as shown in FIG. In this case, markers (points 1, 2, 3, and 4 in the figure) are drawn on the measurement line L at predetermined intervals It, as in the case of the profile. When i% It = 0, a marker is drawn at the point Li on the measurement line L (% represents the remainder of the integer division).

At other points Li, no marker is drawn. In this case, similar to the profile, the markers are assigned consecutive numbers, and the numbers are drawn near the markers. However, the serial numbers are given so that the corresponding points Li and Ti have the same number.

By the above processing, the correspondence between the profile and the image becomes clear. In this case, the method of distinguishing the markers is not limited to the method of assigning consecutive numbers and drawing near the markers. For example, 1) changing the shape of the marker for each corresponding marker. 2) Change the color for each corresponding marker. And so on.

Next, the principle of calculating the distance to the target in S1-3 will be described more specifically.

As shown in FIG. 6, the three-dimensional coordinates of the target object are obtained by setting the direction of a straight line passing through the center of the two objective lenses 103R and 103L at the tip of the endoscope insertion section 102 as the X axis, and the left objective lens Is represented by an XYZ coordinate system with the origin OL. FIGS. 7A and 7B show an X-ray image pickup system of the endoscope 101 in the obtained three-dimensional coordinate system.
It is represented based on a Z, YZ coordinate system.

The measurement target point P is designated on the frame memories 112R and 112L by an operation using the mouse 145. The measurement target point P corresponds to the objective lens systems 103R and 103R.
The image is formed on the imaging means 104R and 104L through 03L. Here, the left and right optical axes are parallel,
It is assumed that they are orthogonal to the imaging units 104L and 104R, respectively. Also, the objective lens system 103R and the imaging means 104
R, the objective lens system 103L and the imaging means 104L are arranged in parallel with each other.

The three-dimensional coordinates (Xp, Yp,
Zp) is calculated using trigonometry. In FIG. 7, the image forming positions of the measurement target point P on the imaging units 104R and 104L are (Lpx, Lpy) and (Rpx, Rpy) with respect to the origin OL (the center of the left objective lens system).
It is expressed as However, since the origin OL and the center OR of the right objective lens system have the same Y coordinate and Lpy = Rpy,
FIG. 7B shows only the left imaging system. Also, imaging means 1
Intersections of the optical axes 04R and 104L with the optical axis are R1 and L1.

Since an image picked up by an endoscope generally uses a wide-angle lens, there is distortion due to distortion. In the present embodiment, the image forming positions after the distortion correction processing is applied are (Lpx, Lpy) and (Rpx, Rpy). The distortion correction processing is performed, for example, as follows.

When, for example, a square grid shown in FIG. 10A is picked up by using an endoscope, the obtained image is shown in FIG.
The result is as shown in FIG. Correction values for each pixel are determined in advance so that this distorted image becomes a square grid image, and distortion correction processing can be realized by performing correction on an actual captured image. Become. A more specific processing method is disclosed in, for example, US Pat. No. 4,895,431. Hereinafter, in the present embodiment, it is assumed that the image after the distortion aberration correction processing is applied is handled.

In FIG. 7, D is the distance between the left and right optical axes, F
Indicates the distance between the objective lens system 103R and the imaging means 104R, and this distance F is the distance between the objective lens system 103L and the imaging means 1R.
The same applies to 04L.

Assuming that a foot of a perpendicular line lowered from the point P to be measured on the distal end surface of the endoscope is Q, the following (A), (B), and (C)
Holds.

From FIG. 7A, (A) ΔPQOL∽ΔOLL1Lpx (B) ΔPOR Q∽ΔOR RPX R1 From FIG. 7B, (C) ΔPQOL∽ΔOLL1Lpy

The values of L1Lpx, R1Rpx and L1Lpy can be obtained as distances on the image sensor surface.
That is, it may be obtained from the image formation position on the frame memories 112R and 112L and the pixel size per pixel.

FIG. 7A shows that (D) OLQ + QOR = D, and FIGS. 7A and 7B show that (E) OLL1 = ORR1 = F.

Under these conditions ((A) to (E)), the coordinates (Xp, Yp, Zp) of the measurement target point P can be obtained from the following equations (1), (2), and (3).

Xp = DLpx / (Lpx−Rpx) (1) Yp = DLpy / (Lpx−Rpx) (2) Zp = DF / (Rpx−Lpx) (3)

From the three-dimensional coordinates of the measurement target point obtained as described above, the distance from the endoscope end to the measurement target point and the distance between a plurality of measurement target points can be obtained.

The method of displaying the three-dimensional coordinates on the profile in S1-4 is performed as follows.

The coordinates obtained by the equations (1) to (3) are coordinate values in the coordinate system of FIG. 6 or, in the case of FIG. 11A, the XYZ coordinate system. The horizontal axis represents the measurement line L and the vertical axis represents the coordinate value.
The data is converted to a two-dimensional coordinate system with the Z axis of the YZ coordinate system and plotted to make it two-dimensional.

That is, the value in the XYZ coordinate system is converted to the X'Y'Z coordinate system in FIG. 11 (X 'is a coordinate parallel to the measurement line, and Y' is an axis perpendicular to X 'and Z). A two-dimensional graph is created using X ′ and Z among the coordinate values after the conversion. The start point and end point of the measurement line L specified in S1-1 are defined as L0 (XL0, YL0, F), LN- in the XYZ coordinate system.
1 (XLN-1, YLN-1, F).

Assuming that the left and right optical axes are parallel and the image plane is perpendicular to the optical axis, the relationship between the X axis and the X 'axis is as follows regardless of what measurement line L is specified on the image. Only rotation about the Z axis occurs. Therefore, it is only necessary to consider the angle between the X axis and the X 'axis considering only the two dimensions of the XY axes.

At this time, the start point and the end point of the measurement line L are as follows: L0 (XL0, YL) LN-1 (XLN-1, YLN-1) (4)

The angle θ between the X axis and the X ′ axis is cos θ = (XL0XLN-1 + YL0YLN-1) / ｛(XL0 ² + XLN-1 ² ) ^1/2 + (YL0 ² + YLN-1 ² ) ^{1 /} 2} is obtained by (5). The value of the XY coordinate system is converted into the X ′ axis and the Y ′ axis using the obtained θ.

[0061]

(Equation 1)

Then, the obtained X ′ and Z according to the equation (3) are plotted on the X′Z coordinate system as shown in FIG. 11B, and are displayed as a profile T (see FIG. 9).

(Second Embodiment) FIGS. 12 and 13 show a second embodiment of the present invention. Note that the configuration of the measurement endoscope apparatus according to the present embodiment is the same as that of the first embodiment, and thus description thereof will be omitted.

When three-dimensional measurement is performed, it may be desired that the correspondence between an arbitrary point on the profile and a point on the image can be confirmed. In the following description, a function of designating an arbitrary point of the profile and clarifying the correspondence by specifying the point on the image corresponding to the arbitrary point will be described with reference to the flowchart of FIG.

In step S1-1 to S1-5 in the flowchart shown in FIG. 8 of the first embodiment, a profile is displayed and a measurement line L is drawn on an image. In this case, the points Li, Di, and Ti (0 ≦ i ≦ N−1) indicate the same contents as in the first embodiment.

Then, first, an arbitrary point Ti on the measurement line T displayed in the profile is designated in S2-1. An input device such as a mouse is used as the designation means, and the cursor is moved to a desired point. If the input device is a mouse, the designation is made by pressing the left button. This designated point (hereinafter referred to as “designated point”) is defined as Tα.

In S2-2, a point on the measurement line L with respect to the designated point Tα (hereinafter referred to as a "measurement point") is obtained. Assuming that the coordinates of the point Li on the image are Li (pi, qi), T
When α (0 ≦ α ≦ N-1) is specified, the three-dimensional position is Dα
It is. The measurement point corresponding to this point Dα is Lα, and its coordinates are Lα (pα, qα).

In S2-3, as shown in FIG. 13, a temporary marker is drawn at the point Lα (pα, qα) on the measurement line L, and the measurement point for Lα is specified. In this case, the graphic may be flashed at regular intervals as a temporary marker.

The temporary marker referred to here is a marker displayed only while the left button of the mouse, a specific button of the keyboard or the like is kept pressed.

In S2-4, it is checked whether the left button of the mouse is kept pressed. S2-3 if pressed
Return to

If the left button of the mouse is released, the display of the temporary marker is deleted in S2-5, and the routine ends.

By drawing the temporary marker on the measurement line L by the above processing, the correspondence between an arbitrary designated point on the measurement line T displayed in the profile and a point on the measurement line L on the image can be determined. You can easily find out.

Note that the method of specifying the measurement point is not limited to drawing a graphic on the measurement point, and for example, there is the following method.

1) Draw a figure such as an arrow pointing to the measurement point near the measurement point. 2) Move the mouse cursor over the corresponding measurement point. However, the movement of the cursor is prohibited for a predetermined period so that the careless movement of the cursor does not make the correspondence unclear. Conversely, a measurement point on the image may be specified by the specifying means, and a temporary marker may be drawn at a corresponding point on the profile.

(Third Embodiment) FIGS. 14 to 16 show a third embodiment of the present invention. Note that the configuration of the measurement endoscope apparatus according to the present embodiment is the same as that of the first embodiment, and thus description thereof will be omitted.

When the measurement is performed, the reliability of each measurement result may be evaluated, and it may be desired to view the evaluation simultaneously with the measurement result. In the following, a function that can grasp the reliability simultaneously with the measurement result will be described with reference to the flowchart shown in FIG.

First, in S3-1, a measurement line L is specified on the left image of the stereo image. A point on the measurement line L is represented as Li (i = 0,..., N−1). The line is specified by an input device such as a mouse.

At S3-2, a corresponding point of the right image is obtained.

In S3-3, a three-dimensional position is obtained from the equations (1) to (3). The three-dimensional position of the point Li is Di (i = 0,..., N−1), and its coordinates are Di (x
i, yi, zi).

At the same time, the reliability of the measurement is determined. The reliability for the point Di is Tri (i = 0,..., N−1).
The method for deriving the reliability will be described later.

In S3-4, a three-dimensional position is drawn as a profile T as shown in FIG.
The background of the profile Di is colored by ri. The coloring method will be described together with the method of deriving reliability described later.

In this case, the method of presenting the reliability to the user is not limited to the background coloring of the profile. For example, the following method is available.

1) The reliability is drawn as a separate graph (eg, a histogram) in an area overlapping with the profile or in the vicinity of the profile.

2) By clicking on an arbitrary position in the profile, the reliability of measurement at that point is displayed in a small window. If necessary, display the measurement results simultaneously.

In this case, the reliability is obtained as follows.

The reliability of the measurement is obtained, for example, by evaluating the sharpness of the cross-correlation value of the block matching. Correlation value Cr
Takes a value of -1.0 to 1.0, and becomes larger as the matched images are more similar.

In general, the sharper the graph of the correlation value Cr, the higher the texture of the matching becomes. Therefore, it is considered that the reliability increases as the correlation value Cr approaches 1.0, in other words, the sharpness increases, and the reliability Tr is calculated by the following equation.

Tr = Crmax (p, q) · g (w) Here, Crmax (p, q) is determined within a predetermined area using a template centered on the point (p, q) on the image. The maximum value of the correlation value Cr (p, q) when matching is performed. g (w) is a coefficient, and is a monotonically decreasing function with respect to w as shown in FIG. This w is a width in which the correlation value Cr (p, q) has a value larger than a predetermined value Th, as shown in FIG. Note that information such as the presence or absence of halation may be used for reliability evaluation.

When the reliability is expressed as coloring the background, for example, a method of 1) “blue” if reliability> 0.8 and 2) “red” if reliability <0.2 is considered.

In the present embodiment, the reliability is displayed as the background of the profile, but may be displayed by changing the color of the measurement line L drawn on the left image based on the reliability.

(Fourth Embodiment) FIGS. 17 to 20 show a fourth embodiment of the present invention. Note that the configuration of the measurement endoscope apparatus according to the present embodiment is the same as that of the first embodiment, and thus description thereof will be omitted.

Assume that three-dimensional measurement is performed, a vertical direction on the left image is designated as a measurement line L, and a profile corresponding to the measurement line L is displayed by a two-dimensional graph.

As described in the first embodiment, when a three-dimensional position is drawn on a two-dimensional graph, a vertical measurement line is selected on the horizontal axis.

However, as shown in FIG. 18, when the horizontal axis of the two-dimensional graph cannot be changed because the horizontal axis is drawn in the horizontal direction due to the limitation of the display screen size and the configuration of the window, etc. Although the measurement line L on the image is drawn in the vertical direction, the three-dimensional position of a point along the vertical direction is converted to a two-dimensional position and drawn along the horizontal direction.

As described above, when the direction of the measurement line L is different from the plotting direction of the profile, it is difficult to make the correspondence between the measurement line and the profile on the image. is there. Alternatively, there is a demand to simply change the direction of an image or the direction of a profile during observation.

Hereinafter, the function of adjusting the measurement line L and the profile direction will be described with reference to the flowchart of FIG. 17 showing the flow of image rotation processing.

S1-1 to S1-1 in FIG. 8 of the first embodiment.
In S1-5, the measurement line T is displayed on the profile, and the measurement line L is drawn on the image.

FIG. 19A shows a drawing example at this time. However, the marker is omitted in FIG.
In this case, the image size is SX / SY, and the image display window size is WSX / WSY.

In the vicinity of the object to be rotated on the window,
An arrow as shown in FIG. 19A is drawn. in this case,
An image is a rotation target, and an image will be described below as an example.

First, in S4-1, the mouse cursor is moved on the arrow, and the left button of the mouse is pressed to start dragging.

Dragging is a general method of using a mouse in which a mouse cursor is moved over an object to be moved, a button is pressed, and the object is moved by moving the mouse cursor while keeping the state as it is.

In S4-2, the arrow is rotated according to the moving direction and the moving amount of the mouse cursor. As shown in FIG. 20B, an angle between a vertical line and a straight line connecting the current position of the mouse cursor and the center of the arrow is defined as θ, and the rotation of the arrow and the update of the angle θ are performed by moving the mouse cursor.

In S4-3, the image is rotated and displayed according to the rotation of the arrow. Considering the XY coordinates with the origin at the center of the image display window, and letting the image be a point A (p, q),

(Equation 2) As a result, the image after rotation becomes a point A ′ (p ′, q ′), and this point A ′ (p ′, q ′) is displayed on the window.

At this time, assuming that the smallest rectangle surrounding the rotated image is a new image, the image size is (SXcos θ + SY sin θ) · (SX sin θ + S
Ycos θ).

Therefore, depending on the value of θ, there may be a case where a portion exceeding the display size of the window cannot be displayed. At this time, all parts of the image can be displayed by scrolling. In this case, the image display window size may be changed according to the screen size and the window size so that the area that cannot be displayed is reduced as much as possible.

Also, there may be a portion where no display data is present in the display area depending on the rotation angle, as shown in the area (1) shown in FIG. 20 (a). For example, padding with 0).

In S4-4, whether the drag has been completed
That is, it is determined whether the left button has been released. If the button is kept pressed, the process returns to S4-2. When released, the routine ends.

FIG. 19B shows a state in which the image is rotated. Since the measurement line L is oriented horizontally by the rotation of the image, the measurement line L of the image coincides with the direction of the horizontal axis of the profile.

When the image is larger than a predetermined size, in order to prevent the image display from flickering, the image is not rotated and displayed while the arrow is being dragged, and the rotated image is preferably drawn at the end of the drag. Specifically, S4-3 and S4
The processing performed in step 4-4 is exchanged, and the processing for drawing after the drag is completed is performed.

The specification of the rotation amount of the image is not limited to the rotation of the arrow, but simply prepares a selection of the rotation amount in advance in a menu, a button, or the like, and selects the rotation amount, or a rotation angle using a keyboard. Is also conceivable. Some of these may be used in combination.

Although the image is rotated in the present embodiment, the same effect can be obtained by rotating the profile.

(Fifth Embodiment) FIGS. 21 and 22 show a fifth embodiment of the present invention. Note that the configuration of the measurement endoscope apparatus according to the present embodiment is the same as that of the first embodiment, and thus description thereof will be omitted.

In the left and right images of the stereo image, substantially the same region is imaged. There is a demand that the other image window be automatically scrolled in synchronization with the scroll operation of one image display window in order to facilitate observation. Hereinafter, a measurement endoscope apparatus having a function of realizing synchronous scroll will be described.

FIG. 22A shows the left image display window W.
1 shows a display example of a right image display window W2. Here, scrolling of both image display windows W1 and W2 is performed.
An example will be described in which the scroll bar is moved and the display of the image is changed according to the amount of movement.

It is assumed that the scroll of each of the image display windows W1 and W2 is controlled by the values of the scroll position and the scroll amount. The scroll position indicates the position of the scroll bar. The scroll amount is the amount of movement of the scroll bar moved by the user.

The scroll state is changed by adding or subtracting the scroll amount to or from the scroll position. That is,
The position of the scroll bar and the display area of the image are managed and determined by the scroll position.

When the check box shown in FIG. 22 is set, the scrolling is performed synchronously. When the check box is not checked, the image display windows W1 and W2 scroll independently.

In this case, depending on the display space of the screen and the convenience of the design, the image may be full-scale (1 ×).
May not be displayed. Here, it is assumed that the vertical size and the horizontal size of the left image display window W1 are displayed by K1 times, and the vertical size and the horizontal size of the right image display window W2 are displayed by K2 times (K1 and K2 are real numbers).

In the following, the function of realizing the synchronous scroll will be described with reference to the flowchart of FIG. 21 showing the flow of the synchronous scroll.

First, in S5-1, the movement of the scroll bar of the image display window Wi (i = 1, 2) is detected. The window in which scrolling is detected is referred to as a main window Wα, and the main window on the non-detection side is referred to as a subwindow Wβ. The scroll amount of the main window Wα is set to Mα.

At S5-2, it is checked whether the check box is set. If it is set, go to 5-3. If it is not set, go to S5-
Proceed to 4.

In S5-3, scroll processing of the main window Wα is performed. The scroll processing refers to movement of a scroll bar according to the scroll amount and drawing of an image on the main window Wα.

Then, the scroll position Stα after the scroll of the main window Wα is obtained. Further, the scroll position of the sub-window Wβ is set to Stα ·
(Kβ / Kα).

Next, after the scroll processing of the sub-window Wβ is performed, the synchronous scroll processing is terminated.

As a result, as shown by a broken line in FIG. 22B, one sub-window of both windows W1 and W2 scrolls synchronously with the other sub-window.

If it is determined in S5-2 that the check box is not set and the process proceeds to S5-4, the scroll processing is performed only on the main window Wα, and the routine ends.

As described above, according to the present embodiment, by scrolling one of the stereo images, the same place in the other image can be automatically displayed.

When the check box shown in FIG. 22 is set, one of the image windows W1 and W2 is set as the main window Wα before the scroll bar is operated, and S5-3 is set. After execution, the scroll positions of both windows W1 and W2 are made to coincide with each other, and then a transition is made to synchronous scrolling.

Incidentally, the synchronous scrolling may be performed on the profile T. In this case, for the image, the section of the measurement line L displayed for each scroll position and the section of the profile T corresponding to the three-dimensional position displayed for each scroll position for the profile are stored, and in response to a scroll request. , So that the corresponding part is displayed.

(Sixth Embodiment) FIGS. 23 to 26 show a sixth embodiment of the present invention. Note that the configuration of the measurement endoscope apparatus according to the present embodiment is the same as that of the first embodiment, and thus description thereof will be omitted.

There is a demand for easily measuring the distance between any two points on an image and finely adjusting the measurement position. Therefore, the function of easily performing measurement at an arbitrary point and adjusting the measurement position will be described with reference to the flowcharts of FIGS.

First, in S6-1, two points on the left image are designated (hereinafter referred to as designated points). The designation of the two points is performed using an input device such as a mouse. At this time, a straight line passing through the two points is defined as a measurement line L, a point on the measurement line L is defined as Li (i = 0,..., N−1), and the selected two points are defined as L.
α, Lβ, and their coordinates are Lα (pα, qα), Lβ
(Pβ, qβ).

In S6-2, a corresponding point on the right image is determined for a point on the measurement line L. Then, (1) to (3)
The three-dimensional position up to a point on the measurement line L is obtained by the equation. Then, the three-dimensional position with respect to the point Li is represented by Di (i =
0,..., N−1) and the coordinates are Di (xi, yi, z).
i).

At S6-3, the three-dimensional position is displayed as a two-dimensional graph (profile) as at S1-4.

FIG. 26 shows an example in which an image and a profile are displayed on the left image monitor 130L. The point Di (xi,
The point on the two-dimensional graph corresponding to (yi, zi) is defined as Ti (i
= 0, ..., N-1).

In S6-4, as shown in FIG. 26A, a measurement line L and a vertical line of a predetermined length passing through the designated points Lα and Lβ are drawn. The distance d (for example, 3 mm) is displayed near the designated point. This distance d is obtained by the following equation.

D = {(xβ−xα) ² + (yβ−yα) ² + (zβ−zα) ² } ^1/2 (6) In S6-5, as shown in FIG. Draws a vertical line of a predetermined length passing through the portions Tα and Tβ corresponding to the designated point, and draws a distance d between the vertical lines near the vertical line.

In step S6-6, it is determined whether the mouse input is a drag or a click. If the mouse input is a drag, the process proceeds to S6-7, and in S6-7 to S6-9, the drag state is determined.

That is, first, in S6-7, it is determined whether the drag is on the image or on the profile. If the drag is on the image, the process proceeds to S6-8.
Go to S9, otherwise go to S6-12.

In S6-8, if the drag start position is on or near the vertical line of the left image, the flow advances to measurement position change mode 1 (S6-10); otherwise, S6-12.
Proceed to.

In S6-9, if the drag start position is on or near the vertical line of the profile, the process proceeds to measurement position change mode 2 (S6-11).
Proceed to 6-12.

In S6-10, as shown in FIG. 26B, the perpendicular in the image is moved according to the drag of the mouse, and when the drag is completed, the movement of the perpendicular is terminated.

Then, two points on the measurement line L intersecting with the perpendicular at the position where the dragging is stopped are defined as Lα and Lβ, respectively, and the distance d (for example, 6 mm) between the perpendiculars is recalculated and displayed by the equation (6). .

This makes it possible to finely adjust the measurement position for the same measurement line L. At this time, the vertical line on the profile side also moves according to the change in the vertical position, and at the same time, the distance d is displayed.

In S6-11, the vertical line on the profile side is moved in accordance with the drag of the mouse, and when the drag is completed, the movement of the vertical line ends.

Then, two points on the profile T which intersect with the vertical line are re-calculated and displayed according to the equation (6) using Lα and Lβ corresponding to Tα and Tβ, respectively. This allows fine adjustment of the measurement position for the same measurement line. At this time, the vertical line of the left image also moves in accordance with the position of the vertical line, and displays the distance d after the movement.

If it is determined in step S6-6 that the mouse input is a click, the flow advances to step S6-14.
In step 6-15, the mouse click position is checked. If the position is on the left image and near the perpendicular, the process proceeds to step S6-12. Otherwise, the process returns to step S6-1 to reset the measurement line L.

In S6-12, the input of the button displayed on the screen is checked, and if the button is pressed, the processing corresponding thereto is performed, and after the processing is completed, the routine ends. If the button has not been pressed, the routine ends.

As shown in FIG. 26, buttons for moving the vertical line left, moving the vertical line right, moving the vertical line left, and moving the vertical line right are displayed on the screen. It can be moved in minute units such as units.

The following processing is performed according to the pressed button. However, when the angle between the perpendicular and the horizontal direction is between 45 and 135 degrees, the leftward movement of the perpendicular corresponds to the movement on the perpendicular. A rightward vertical movement also corresponds to a rightward vertical movement.

1) Vertical left (up) movement The vertical is moved left (up). Recalculate the distance between the perpendiculars and redraw near the perpendicular. Move the vertical line of the profile according to the perpendicular movement, and redraw the distance. 2) Move the perpendicular right (down) Move the perpendicular right (down). Recalculate the distance between the perpendiculars and redraw near the perpendicular. Move the vertical line of the profile according to the perpendicular movement, and redraw the distance. 3) Move vertical line left Move the vertical line to the left. Recalculate the distance between vertical lines and redraw near the vertical line. The vertical line of the image is moved according to the vertical line movement, and the distance is redrawn. 4) Move vertical line right Move the vertical line to the right. Recalculate the distance between vertical lines and redraw near the vertical line. The vertical line of the image is moved according to the vertical line movement, and the distance is redrawn.

Although the distance between two points is displayed in the present embodiment, other measured values such as the height between two points may be displayed.

[Appendix] As described above, according to the present invention, the following configuration can be obtained. (Additional Item 1) Imaging means for imaging the same object from different positions, means for estimating the three-dimensional shape of the measurement object from a plurality of images obtained by the imaging means, and means for converting the estimated three-dimensional shape into profile information A means for obtaining a correspondence between the converted profile information and the image, a display means for displaying the correspondence between the profile information and the image,
An endoscope device for measurement, comprising: According to such a configuration, the correspondence between the three-dimensional shape of the measurement target and the measurement points on the image can be displayed on the display means.

(Additional Item 2) A means for rotating the profile information converted by the conversion means and / or the image captured by the imaging means, and a means for displaying the rotated profile information and / or the image are provided. The measurement endoscope apparatus according to claim 1, characterized in that: According to such a configuration, a three-dimensional shape and an image to be measured can be displayed in an arbitrary direction.

(Additional Item 3) Input means for designating a designated point in the converted profile information, means for obtaining a measurement point on a measurement line corresponding to the designated point in an image, and means for specifying the measurement point The measurement endoscope apparatus according to claim 1, further comprising: According to such a configuration, it is possible to display the correspondence between an arbitrary point of the measurement target three-dimensional shape and the measurement point on the image.

(Additional Item 4) Input means for specifying a measurement point on a measurement line in the image, means for obtaining a corresponding point corresponding to the specified measurement point in the converted profile information, The measuring endoscope apparatus according to claim 1, further comprising means for specifying. According to such a configuration, the correspondence between an arbitrary measurement point on the image and the three-dimensional shape to be measured can be displayed.

(Additional Item 5) The measurement device according to additional item 1, characterized by comprising means for obtaining the reliability of the three-dimensional shape estimation result and means for displaying a numerical value and / or graphic of the reliability. Endoscope device. According to such a configuration, the reliability of measurement can be displayed together with the three-dimensional shape.

(Additional Item 6) An additional item 1 characterized by comprising means for displaying two or more images picked up by the image pickup means, and means for scrolling the two or more images synchronously. The endoscope device for measurement according to the above. According to such a configuration, images can be scrolled in synchronization with each other.

(Supplementary note 7) It is provided that there are provided means for displaying one or more images picked up by the image pickup means, and means for synchronously scrolling one or more images and the converted profile information. The measurement endoscope apparatus according to claim 1, characterized in that: According to such a configuration, the image and the profile information can be scrolled synchronously.

(Additional Item 8) Input means for designating two points on the converted profile information or image, means for obtaining the distance between the designated two points, and means having means for displaying the distance The measurement endoscope apparatus according to claim 1, further comprising: According to such a configuration, measurement of a distance between any two points on an image or a profile and fine adjustment of the measurement point can be performed.

[0161]

As described above, according to the present invention,
At the same time, the subject image picked up by the image pickup means and profile information corresponding to the three-dimensional shape of a predetermined part of the subject image are displayed, and correspondence information indicating the correspondence between the profile information and an image pickup signal of the subject image is displayed. Since the display is performed, the positional relationship between the three-dimensional shape and the subject image can be easily grasped.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a measurement endoscope apparatus according to a first embodiment.

FIG. 2 is an explanatory view of a distal end portion of an insertion portion of the endoscope.

FIG. 3 is a block diagram showing a functional configuration of the measurement endoscope apparatus.

FIG. 4 is a block diagram showing a configuration of the host computer.

FIG. 5 is a block diagram showing a configuration of the external storage device.

FIG. 6 is an explanatory diagram of a three-dimensional coordinate system.

FIG. 7 is an explanatory view showing the principle of calculating three-dimensional coordinates.

FIG. 8 is a flowchart showing a flow of drawing a measurement result.

FIG. 9 is an explanatory diagram showing an example of window display

FIG. 10 is an explanatory diagram relating to distortion correction.

FIG. 11 is an explanatory view showing means for converting a three-dimensional shape (three-dimensional data) into a figure (two-dimensional data);

FIG. 12 is a flowchart showing a flow of drawing a temporary marker according to the second embodiment;

FIG. 13 is an explanatory view showing an example of window display.

FIG. 14 is a flowchart showing a flow for obtaining reliability of a measurement result according to the third embodiment;

FIG. 15 is an explanatory diagram showing a window display example.

FIG. 16 is an explanatory diagram showing a principle for obtaining reliability.

FIG. 17 is a flowchart showing a flow of an image rotation process according to the fourth embodiment;

FIG. 18 is an explanatory diagram showing an example of a window display

FIG. 19 is an explanatory diagram showing an example of a window display at the time of image rotation processing.

FIG. 20 is an explanatory diagram when the image is rotated.

FIG. 21 is a flowchart showing the flow of synchronous scrolling according to the fifth embodiment.

FIG. 22 is an explanatory diagram showing a window display example.

FIG. 23 is a flowchart illustrating a flow of distance measurement according to the sixth embodiment (part 1);

FIG. 24 is a flowchart showing the flow of distance measurement (part 2).

FIG. 25 is a flowchart showing the flow of distance measurement (part 3).

FIG. 26 is an explanatory view showing a window display example of the embodiment.

[Explanation of symbols]

 101 Endoscope 104L Left imaging means 104R Right imaging means 120 Host computer 130L Left image monitor 130R Right image monitor L Measurement line T Profile Lα, Lβ Designated point P Measurement target point

────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] December 24, 1999 (1999.12.
24)

[Procedure amendment 1]

[Document name to be amended] Statement

[Correction target item name] 0004

[Correction method] Change

[Correction contents]

[0004]

However, in the technique disclosed in the above-mentioned prior art, means for displaying a profile of a subject to be measured (a three-dimensional shape converted into a two-dimensional graph) is displayed. However, even if there is a means for displaying a three-dimensional shape using a wire frame or the like, there is a problem that it is difficult to grasp the correspondence between the displayed three-dimensional shape and the subject image.

[Procedure amendment 2]

[Document name to be amended] Statement

[Correction target item name] 0044

[Correction method] Change

[Correction contents]

The three-dimensional coordinates (Xp, Yp,
Zp) is calculated using trigonometry. Each imaging unit 104R of the measurement target point P in FIG. 7, the imaging position on the 104L, based on the origin _{O L} (left center of the objective lens system), respectively (Lpx, Lpy), (Rpx , Rpy)
It is expressed as However, since the Y coordinate of the origin _{O L} and the right objective lens system center _{O R} of equal and are Lpy = RPy,
FIG. 7B shows only the left imaging system. Also, imaging means 1
Intersections of the optical axes 04R and 104L with the optical axis are R1 and L1.

[Procedure amendment 3]

[Document name to be amended] Statement

[Correction target item name] 0049

[Correction method] Change

[Correction contents]

[0049] FIG. 7 than (a) (A) ΔPQO _L ∽ΔO _{L L1Lpx (B) ΔPO R Q∽ΔO R} R px R1 FIG 7 (b) from (C) ΔPQO _L ∽ΔO _L L1Lpy

[Procedure amendment 4]

[Document name to be amended] Statement

[Correction target item name] 0051

[Correction method] Change

[Correction contents]

FIG. 7A shows that (D) _OL Q + _QO R = D holds is, FIGS. 7 (a) and (b) from (E) _{O L} L1 ₌ O R R1 ₌ F holds.

[Procedure amendment 5]

[Document name to be amended] Statement

[Correction target item name] 0057

[Correction method] Change

[Correction contents]

That is, the value in the XYZ coordinate system is converted to the X'Y'Z coordinate system in FIG. 11 (X 'is a coordinate parallel to the measurement line, and Y' is an axis perpendicular to X 'and Z). A two-dimensional graph is created using X ′ and Z among the coordinate values after the conversion. In the XYZ coordinate system a starting point and end point of the measurement line L specified by _{S1-1, L 0 (X L0,} Y L0,
F), L _N-1 (X _LN-1 , Y _LN-1 , F).

[Procedure amendment 6]

[Document name to be amended] Statement

[Correction target item name] 0059

[Correction method] Change

[Correction contents]

[0059] The start and end points of the measurement line L in this _{case, L 0 (X L0, Y} LO) L N-1 (X LN-1, Y LN-1) ... a (4).

[Procedure amendment 7]

[Document name to be amended] Statement

[Correction target item name] 0060

[Correction method] Change

[Correction contents]

[0060] Further, the angle θ between the X-axis and the X _{'axis, cosθ = (X L0 X LN} -1 + Y L0 Y LN-1) / {(X L0 2 + X LN -1 2) 1/2 + ^{_{(Y L0 2 + Y LN-}} 1 2) is obtained by 1/2} ... (5). The value of the XY coordinate system is converted into the X ′ axis and the Y ′ axis using the obtained θ.

[Procedure amendment 8]

[Document name to be amended] Statement

[Correction target item name] 0066

[Correction method] Change

[Correction contents]

Then, first, an arbitrary point Ti on the measurement line T displayed in the profile is designated in S2-1. An input device such as a mouse is used as the designation means, and the cursor is moved to a desired point. If the input device is a mouse, the designation is made by pressing the left button. The specified point (hereinafter referred to as "designated point") and T _α.

[Procedure amendment 9]

[Document name to be amended] Statement

[Correction target item name] 0067

[Correction method] Change

[Correction contents]

[0067] S2-2 by determining the point on the measurement line L with respect to the specified point T α _(hereinafter referred to as "measurement point"). If the coordinates of the point Li on the image are Li (p _i , q _i ), T
_{When α} (0 ≦ α ≦ N-1) is specified, the three-dimensional position is D _α
It is. Measurement point corresponding to the point D _alpha is L _alpha, its coordinates becomes _{L α (p α, q α} ).

[Procedure amendment 10]

[Document name to be amended] Statement

[Correction target item name]

[Correction method] Change

[Correction contents]

In S2-3, as shown in FIG. 13, a temporary marker is drawn at a point L _α (p _α , q _α ) on the measurement line L, and the measurement point for L _α is specified. In this case, the graphic may be flashed at regular intervals as a temporary marker.

[Procedure amendment 11]

[Document name to be amended] Statement

[Correction target item name] 0120

[Correction method] Change

[Correction contents]

First, in S5-1, the movement of the scroll bar of the image display window Wi (i = 1, 2) is detected. The detected window scrolling, the main window W _α, the main window of the non-detection side is set to be a sub-window W _β. In addition, the scroll amount of the main window _W α and M _α.

[Procedure amendment 12]

[Document name to be amended] Statement

[Correction target item name] 0122

[Correction method] Change

[Correction contents]

Perform the scroll processing of the main window _W α in [0122] S5-3. A scroll processing refers to be carried out movement and the scroll bar in accordance with the scroll amount, the drawing of the image to the main window W _α.

[Procedure amendment 13]

[Document name to be amended] Statement

[Correction target item name] 0123

[Correction method] Change

[Correction contents]

[0123] Then, to get the scroll position _St α after scrolling of the main window W _α. Further, the scroll position of the sub-window W _β is set to St _α ·
( _Kβ / _Kα ).

[Procedure amendment 14]

[Document name to be amended] Statement

[Correction target item name] 0124

[Correction method] Change

[Correction contents]

[0124] Then, after the scroll processing of the sub-window W _β, and ends the process of synchronization scroll.

[Procedure amendment 15]

[Document name to be amended] Statement

[Correction target item name] 0126

[Correction method] Change

[Correction contents]

[0126] In addition, a check box is judged not to be set in S5-2, and advance f S5-4, performs a scroll processing only the main window W _α, to end the routine.

[Procedure amendment 16]

[Document name to be amended] Statement

[Correction target item name] 0128

[Correction method] Change

[Correction contents]

[0128] Incidentally, when the check box shown in Figure 22 is set, before operating the scroll bar, and set one of the windows of the image window W1, W2 as the main window W _alpha, S5-3 Is executed, and the scroll positions of both windows W1 and W2 are made to coincide with each other, and then the process proceeds to synchronous scrolling.

[Procedure amendment 17]

[Document name to be amended] Statement

[Correction target item name]

[Correction method] Change

[Correction contents]

First, in S6-1, two points on the left image are designated (hereinafter referred to as designated points). The designation of the two points is performed using an input device such as a mouse. At this time, a straight line passing through the two points is defined as a measurement line L, points on the measurement line L are defined as Li (i = 0,..., N−1), and the selected two points are defined as L _α and L _β. Let the coordinates be L _α (p _α , q _α ), L _β
(P _β , q _β ).

[Procedure amendment 18]

[Document name to be amended] Statement

[Correction target item name] 0136

[Correction method] Change

[Correction contents]

In S6-4, as shown in FIG. 26A, a measurement line L and a vertical line having a predetermined length passing through the designated points L _α and L _β are drawn. The distance d (for example, 3 mm) is displayed near the designated point. This distance d is obtained by the following equation.

[Procedure amendment 19]

[Document name to be amended] Statement

[Correction target item name]

[Correction method] Change

[Correction contents]

[0137] _{d = {(x β -x α} ) 2 + (y β -y α) 2 + (z β -z α) 2} in ^1/2 ... (6) S6-5, FIG 26 (a) As shown in (1), a vertical line having a predetermined length passing through the portions T _α and T _β corresponding to the designated points is drawn on the profile, and a distance d between the vertical lines is drawn near the vertical line.

[Procedure amendment 20]

[Document name to be amended] Statement

[Correction target item name] 0143

[Correction method] Change

[Correction contents]

Then, two points on the measurement line L intersecting with the perpendicular at the position where the dragging is stopped are defined as L _α and L _β , respectively, and the distance d (for example, 6 mm) between the perpendiculars is recalculated by the equation (6). indicate.

[Procedure amendment 21]

[Document name to be amended] Statement

[Correction target item name] 0146

[Correction method] Change

[Correction contents]

Then, two points on the profile T which intersect with the vertical line are recalculated and displayed by the formula (6) using the distances L _α and L _β corresponding to T _α and T _β , respectively. This allows fine adjustment of the measurement position for the same measurement line. At this time, the vertical line of the left image also moves in accordance with the position of the vertical line, and displays the distance d after the movement.

[Procedure amendment 22]

[Document name to be amended] Statement

[Correction target item name] Explanation of sign

[Correction method] Change

[Correction contents]

For [Reference Numerals] 101 endoscope 104L left imaging unit 104R right imaging unit 120 the host computer 130L left image monitor 130R right image monitor L measurement line T profile L _alpha, L _beta designated point P measurement target point

[Procedure amendment 23]

[Document name to be amended] Drawing

[Correction target item name] FIG.

[Correction method] Change

[Correction contents]

FIG. 16

[Procedure amendment 24]

[Document name to be amended] Drawing

[Correction target item name] FIG.

[Correction method] Change

[Correction contents]

FIG. 21

[Procedure amendment 25]

[Document name to be amended] Drawing

[Correction target item name] FIG. 26

[Correction method] Change

[Correction contents]

FIG. 26

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI theme coat ゛ (reference) G06T 1/00 G06F 15/62 390Z F term (reference) 2F065 AA04 AA06 AA53 BB05 CC16 DD00 DD06 FF01 FF05 FF09 JJ03 JJ05 JJ26 LL02 NN00 PP22 QQ03 QQ24 QQ36 QQ38 QQ40 SS13 UU05 2F112 AC06 BA02 BA05 CA02 CA08 DA30 FA03 FA07 FA21 FA23 FA36 FA38 FA45 2H040 BA15 BA22 GA02 GA10 GA11 4C061 AA00 AA29 BB06 CC06 DD00 HH52 WW01 NN WW WW NN WW NN WW NN WW NN WW NN WW18 WW20 YY02 5B057 AA07 BA02 BA17 BA29 CA08 CA13 CB08 CB13 CC03 CE09 CH11 DA04 DA08 DA16 DB03 DB09 DC03 DC09

Claims (4)

[Claims] An imaging unit configured to image a subject image; a plurality of image signals obtained from the imaging unit; a three-dimensional shape estimation unit configured to estimate a three-dimensional shape of the subject image; Profile information creating means for creating profile information of a predetermined portion in the subject image based on the obtained three-dimensional shape information; profile information display means for displaying profile information created by the profile information creating means; Correspondence information acquiring means for acquiring correspondence information between the profile information displayed by the means and the imaging signal acquired from the imaging means; correspondence for displaying the correspondence information acquired by the correspondence information acquisition information means A subject measurement display device, comprising: relation information display means. 2. An imaging process for capturing a subject image, a three-dimensional shape estimation process for estimating a three-dimensional shape of the subject image from a plurality of image signals acquired in the imaging process, and a three-dimensional shape estimation process. A profile information creating step of creating profile information of a predetermined portion in the subject image based on the obtained three-dimensional shape information; a profile information displaying step of displaying profile information created in the profile information creating step; and the profile information display. A correspondence information acquisition step of acquiring correspondence information between the profile information displayed in the step and the imaging signal acquired in the imaging step; and a correspondence displaying the correspondence information acquired in the correspondence information acquisition step. And a related information display step. 3. An image pickup means for picking up an image of a subject, a three-dimensional shape estimating means for estimating a three-dimensional shape of the subject image from a plurality of image signals obtained by the image pick-up means, and an estimating means for estimating the three-dimensional shape. Profile information creating means for creating profile information of a predetermined portion in the subject image based on the obtained three-dimensional shape information; profile information display means for displaying profile information created by the profile information creating means; Correspondence information acquisition means for acquiring correspondence information between the profile information displayed by the means and the imaging signal acquired from the imaging means; and an arbitrary position on the profile information displayed by the profile information creation means. A designating means that can be designated; and a designation information designated by the designation means and a correspondence information acquisition means. A position information acquisition unit that acquires the first position information designated by the designation unit as second position information in the imaging signal based on the correspondence information acquired in the step; A designated position information display unit for displaying designated position information by the designation unit on a subject image displayed based on the image pickup signal based on the obtained second position information; Display device. 4. An imaging process for imaging a subject image; a three-dimensional shape estimation process for estimating a three-dimensional shape of the subject image from a plurality of image signals acquired in the imaging process; A profile information creating step of creating profile information of a predetermined portion in the subject image based on the obtained three-dimensional shape information; a profile information displaying step of displaying profile information created by the profile information creating means; A correspondence information acquisition step of acquiring correspondence information between the profile information displayed in the process and the imaging signal acquired in the imaging step; and an arbitrary position on the profile information displayed in the profile information display step. A designation process that can be designated, designation information designated by the designation unit, and the correspondence information acquisition unit A position information acquisition step of acquiring the first position information designated in the designated step as second position information in the imaging signal, based on the correspondence information acquired in A designated position information display step of displaying designated position information by the designated step on a subject image displayed based on the image pickup signal based on the obtained second position information. Method.