JP2002159021A - Apparatus and method for displaying scale - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable to display a scale display position of an imaged object at a suitable position and to enable switching of a method for forming a scale in response to a state of an image. SOLUTION: An apparatus for displaying the scale comprises a video signal acquiring means for acquiring an image signal of the subject by a plurality of imaging means, a feature value calculating means for calculating the feature value regarding an intensity of the image signal on divided regions in the image based on the image signal of at least one imaging means, and a display means for displaying scale information on the prescribed region based on a calculated result of the calculating means. The apparatus further comprises a means for switching the method for forming the scale based on the calculated result of the feature value calculating means.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a scale display device and a scale display method for displaying a scale on a photographed subject.

[0002]

2. Description of the Related Art In recent years, in the medical field, the industrial field, and the like,
Endoscopes have become widely used. In a normal endoscope observation image, the target object is generally planar,
Unevenness is difficult to recognize. For this reason, recently, two systems of left and right imaging means are arranged at the tip of the endoscope, and from the left and right images taken by the two systems of imaging means, a target stereoscopic image is estimated by a stereo method, and the three-dimensional shape is estimated. Is displayed by using a profile or a wire frame.

For example, Japanese Patent Application Laid-Open No. Hei 6-339454 discloses a technique for calculating a distance to a target and a distance between a plurality of target points using a stereo endoscope based on the principle of triangulation.

In particular, as shown in FIG. 12 of Japanese Patent Application Laid-Open No. Hei 6-339454, the length and size of an object are objectively determined based on the calculated three-dimensional coordinate values. Recognizable scale markers are displayed according to the shape of the object.

According to the method disclosed therein, after obtaining the three-dimensional coordinates of a specified target point, a point having the same X coordinate as that point is obtained at regular intervals in the horizontal direction. Points having the same coordinates are obtained at regular intervals in the vertical direction. The obtained point group becomes a scale at a fixed interval, and is displayed as a curved line with the scale superimposed on the image.

[0006]

However, according to this method, the scale display position is fixed, and the display position is fixed at a predetermined place and cannot be changed. Therefore, for example, depending on the state of the background image, the displayed scale may not be easy to see.

[0007]

SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to provide a method for displaying a scale on an imaged subject.
It is an object of the present invention to provide a scale display device and a method capable of displaying a scale display position at an appropriate position.

Still another object of the present invention is to provide a scale display apparatus and a scale display method capable of changing a scale creation method according to a captured image of a subject.

[0009] A scale display device according to the present invention comprises a video signal obtaining means for obtaining an image signal of a subject by a plurality of imaging means, and an image signal in a divided area in an image based on the image signal of at least one of the imaging means. The image processing apparatus includes a feature value calculation unit that calculates a feature value related to intensity, and a display unit that displays scale information in a predetermined area based on a calculation result of the feature value calculation unit.

Further, the display means has means for switching the scale creation method based on the calculation result of the feature quantity calculation means.

Further, according to the scale display method of the present invention, an image signal of a subject is obtained by a plurality of image pickup means, and the intensity of the image signal in a divided area in the image based on the image signal of at least one image pickup means. And calculates scale information in a predetermined area based on the calculation result of the feature amount.

[0012]

Embodiments of the present invention will be described below with reference to the drawings. Hereinafter, a description will be given of a case where a stereo endoscope provided with two objective lenses at the tip is used as a measurement endoscope to be applied.

In a series of embodiments of the present invention, for example, a pixel R is represented by R (x, y), and x and y represent the position of the pixel on the X axis and the Y axis on the image. . Further, for example, the three-dimensional position coordinates of the point P are P ′ (x
', Y', z '), and x', y ', z' represent coordinate positions on the X 'axis, Y' axis, and Z 'axis in a three-dimensional space. Unless otherwise specified, the number of gradations of an image will be described as 256.

Further, when distortion is present in an image, each process is basically performed on the image after the distortion is corrected. The correction of distortion is described in, for example,
This is performed as described in Japanese Patent Application Laid-Open No. 9454/94. That is, the correction value for each pixel is determined in advance, and the correction is performed on the actual captured image, whereby the distortion correction processing can be realized. Specifically, US Pat. No. 4,895,431 discloses a specific processing method. This is not the case when there is no distortion aberration in the image. Further, when distortion is present in the image, each processing is basically performed on the image after the distortion is corrected. However, when the correction is not performed, the influence of the distortion is small. It is desirable to select the pixel of.

(First Embodiment) A first embodiment will be described with reference to FIGS.

The description of the first embodiment will be made in the following order.

[1]. Overall configuration (FIGS. 1 to 6) [2]. Scale drawing method (1) Vertical plane generation method (FIGS. 7 to 10) (1-1) P point automatic determination method (FIGS. 11 and 12) (1-2) Scale creation method (FIG. 13) (2) 3 Point virtual plane generation method (FIG. 14) [3]. Method of Displaying Scale at Appropriate Position Based on Feature (FIGS. 15 to 17) The order will be described below.

[1]. Overall Configuration FIG. 1 is a block diagram of a measurement endoscope apparatus according to the first embodiment. FIG. 2 is a diagram illustrating the structure of the distal end portion of the insertion section of the endoscope. FIG. 3 is a block diagram illustrating a functional configuration of the measurement endoscope apparatus. FIG. 4 is a diagram illustrating the configuration of the host computer. FIG. 5 is a block diagram illustrating a configuration of the external storage device. FIG. 6 is a diagram for explaining a three-dimensional coordinate system at the end of the endoscope.

First, as shown in FIG. 3, a measuring endoscope apparatus used in the present embodiment is a stereo video image endoscope (hereinafter, simply referred to as “endoscope”) 101, and Right image video processor 110R and left image video processor 1 for performing signal processing on each image signal of a right image and a left image captured by endoscope 101
Has 10L. Furthermore, each video processor 11
Output from 0R and 110L, for example, RGB
The right image frame memory 112R and the left image frame memory 112L that store each video signal based on the signal, and the video signal based on, for example, an RGB signal output from each of the frame memories 112R and 112L is input, and Monitor 13 for right image displaying left image
0R and a left image monitor 130L. In addition, the host computer 1 that performs stereoscopic measurement calculation using the images stored in the frame memories 112R and 112L.
20, an external storage 140 connected to the host computer 120, and a mouse connected to the host computer 120 for operating a cursor displayed on the monitors 130R and 130L, specifying a measurement target point, setting a measurement target area, and the like. 145.

Both video processors 110R, 110L
Are set to perform signal processing synchronized with each other. In this embodiment, each frame memory 112R
And 112L each include a plurality of sets of memories for R, G, and B. One set stores an image, and the other set stores a cursor. By adding the signals written to each set, an image and a cursor are displayed on the respective screens of the monitors 130R and 130L.

Next, as shown in FIG. 4, the host computer 120 includes a CPU 121, a right frame memory interface 122R, a left frame memory interface 122L, a main memory 123, an external storage interface 124, a mouse interface 125, a keyboard 126, and a CRT 127. And they are connected to each other by a bus.

The right frame memory interface 1
22R and left frame memory interface 122
L is connected to the right image frame memory 112R and the left image frame memory 112L, respectively, and transmits and receives image data to and from them. Further, cursor control is performed on the corresponding frame memories 112R and 112L via the interfaces 122R and 122L. The external storage interface 124 is connected to the external storage 140 and is set to transmit and receive image data and measurement target point position information. The mouse interface 125 is connected to the mouse 145.

As shown in FIG. 5, the external storage 140 stores the position information of the measurement target point and stores it in the host computer 12.
0 and a measurement data information file 181 for transmitting and receiving various measurement data, and a stereo image manager 182 for transmitting and receiving image data to and from the host computer 120. Further, the left image file 18 is connected to the stereo image manager 182 and stores left image data.
3L and a right image file 183 similarly connected to the stereo image manager 182 and storing the right image data.
R.

In this embodiment, a stereo image obtained by the endoscope 101 is handled as a pair of left and right images. That is,
The two synchronized image frame data are stored in the storage device as one set of data. The stereo images sent in pairs from the host computer 120 are distributed to the left and right image files 183L and 183R by the stereo image manager 182 and recorded. The images recorded in each of the image files 183L and 183R are called by the stereo image manager 182 in pairs.

As shown in FIG. 2, the endoscope 101
An elongated insertion portion 102 is provided, and a plurality of, for example, two observation windows and an illumination window are provided at a distal end portion of the insertion portion 102. Inside each observation window, a right-eye objective lens system 103R and a left-eye objective lens system 103L are provided at positions having parallax with each other. The imaging positions of the objective lens systems 103R and 103L are respectively provided at left and right imaging units 104L and 104R using a solid-state imaging device such as a CCD.
Are arranged.

A light distribution lens 105 is provided inside the illumination window.
At the rear end of the light distribution lens 105, a light guide 106 made of a fiber bundle is provided continuously. The light guide 106 is inserted into the insertion portion 102, and a light source device (not shown) is connected to the incident end.

Illumination light output from the light source device is applied to a subject via a light guide 106 and a light distribution lens 105, and reflected light from the subject is reflected by the objective lens systems 103R and 103L on the right image and the right image, respectively. The left image is formed on the imaging units 104R and 104L.

Next, a functional block configuration of the measurement endoscope apparatus will be described with reference to FIG. Charge coupled device (CCD: C
The respective image signals of the subject image captured by the image capturing means 104R and 104L using the charge coupled device (charge coupled device) are respectively supplied to the video processor 110.
Input to R and 110L, and video signal processing is performed. Each image signal output from the video processors 110R and 110L is converted into a digital signal by A / D converters 111R and 111L, respectively, and then stored in an image memory, that is, an image memory among the frame memories 112R and 112L. It is memorized.

The right image frame memory 112R
And the image signal read from the left image 112L are subjected to predetermined image processing by the image processing means 200 and then passed through the display control means 201 to the D / A converter 158R.
And 158L. D / A converters 158R and 15
After being converted to an analog signal by 8L, the right image monitor 1
30R and 130L for the left image. Then, a right image and a left image are displayed on the monitors 130R and 130L, respectively.

On the other hand, the display of the cursor is first processed by the cursor display means 151, and the display is controlled by the display control means 201. The cursor is on both monitors 130R,
130L is superimposed and displayed on one of the screens.

When the measurement point is designated by the cursor, the image processing means 200 determines the measurement point based on the designated point data.
Processing such as derivation of a three-dimensional position, creation and display of a scale, and the like are performed, and the processing result is displayed on a monitor through the display control unit 201.

The display control means 201 performs not only management and control of cursor display but also processing necessary for multi-window display. Image processing means 200 and display control means 2
01 are achieved by operating the host computer 120. in this case,
Normally, the function of the display control means 201 is included in the operating system.

The frame memories 112R and 112L for the right image and the left image shown in FIGS. 3 and 4 are provided outside the host computer 120 in the present embodiment. May be built in using the frame memory of the above. Further, in FIG. 3, the stereo image is displayed on the right image monitor 130R and the left image monitor 130L, but both may be displayed on the CRT 127 of the personal computer.

In the first embodiment, a description will be given of the fact that the scale drawing can be selected or switched between a manual mode and an automatic mode. Before that, however, two methods of the scale drawing will be described.

[2]. Scale drawing method A scale drawing method will be described below.

FIG. 7 is a flowchart showing the flow of determining a vertical plane based on the three-dimensional position information of one pixel and drawing a scale. FIG. 8 is a diagram for explaining the scale creation principle. FIG. 9 is a diagram for explaining scale drawing in the left image. FIG. 10 is a diagram for explaining evaluation of flatness. FIG. 11 is a flowchart illustrating a flow of selecting a pixel. FIG. 12 is an explanatory diagram of the area division. FIG. 13 is a flowchart showing the flow of adjusting the scale interval.

(1) Vertical Plane Generation Method In the measurement endoscope apparatus having the above-described configuration, a scale drawing process by the vertical plane method for grasping the size of an object will be described with reference to the flowchart of FIG.

It should be noted that the method described below is
This is called the vertical plane generation method.

In the following description, a description will be given assuming that a corresponding point of the right image is obtained for a predetermined pixel of the left image.
Conversely, a corresponding point in the left image may be obtained for a predetermined pixel in the right image.

As shown in FIG. 6, the target coordinate system of the target object is defined such that a straight line passing through the center of the two objective lenses 103R and 103L at the end of the endoscope insertion section 102 is the X 'axis. The center of the left objective lens is the origin OL, the optical axis of the left objective lens is the Z ′ axis, and the Y ′ axis is perpendicular to the X ′ axis and the Z ′ axis. The target coordinate system is represented by such an X′Y′Z ′ coordinate system. FIG. 8 shows an image pickup system of the endoscope in the three-dimensional coordinate system represented by X ′.
This is based on the -Z 'coordinate system and the Y'-Z' coordinate system.

First, a point P in an image displayed on the left image monitor is designated by the operator. Point P is 2
It corresponds to a certain pixel on the dimensional plane. Hereinafter, the point P is
A pixel P is set, and the coordinates of P in the left image are set to P (x, y).

In step 11 (hereinafter abbreviated as S11) of FIG. 7, a corresponding point on the right image of the designated pixel P is detected. Further, a pixel P in the left image,
From the corresponding point of the pixel P in the right image, the three-dimensional position coordinates of the pixel P are obtained by calculation. Let the three-dimensional coordinate position of P be P '(x', y ', z'). The corresponding points may be detected by well-known template matching using epipolar lines. Further, the three-dimensional coordinate position is obtained based on the principle of triangulation. A specific method of obtaining the information is disclosed in, for example, Japanese Patent Application Laid-Open No. Hei 6-339454.

The coordinates of the pixel P in the left image may be specified in advance as described above, but the P coordinates may be automatically determined from the image. The method of this automatic determination will be described later.

Next, in S12, a virtual vertical plane α passing through the point P 'is determined. The vertical plane α is X ′
-A plane perpendicular to the Z 'plane, that is, a plane parallel to the X'-Y' plane. The equation of the plane is as follows.

(Z'-z ') = 0 At S13, the drawing coordinates of the horizontal scale are determined. The scale is drawn on an image plane γ for drawing the scale. A [mm] horizontally from the center of the image
When the scale is created, the X coordinate on the image plane γ is as follows based on the proportional relationship from FIG. 8A when the imaging system is viewed from the Y ′ axis side. Here, the unit of the length or the distance is millimeter [mm].

A is a unit length (for example, 5 [mm]) to be used as a reference as a scale on a vertical plane. z 'is
This is the distance from OL to the vertical plane α. f is the focal length of the lens, which is the distance from OL in the figure to the image plane γ. Mx ₁ corresponds to the distance A on the vertical plane alpha, is a distance on the image plane gamma.

The converted _{A / z'= Mx 1 / f} Mx 1 = a (A × f) / z'units from [mm] to [number of pixels], the Mx ₁ = Mx ₁ / Cx.

Here, Cx is the horizontal size [mm] per pixel of the CCD.

Therefore, the scale drawing position is (Mx ₁ , Y
c) (Yc: Y coordinate given in advance).

Similarly, 2 × A [mm] (for example, when A is 5
If [mm], to create a graduation at) of 10 [mm], the drawing position (Mx _2, Yc) Tosureba obtained as follows.

2A / z ′ = Mx ₂ / f Mx ₂ = (2A × f) / z ′ The unit is converted from [mm] to [number of pixels], and Mx ₂ = Mx ₂ / Cx.

Therefore, the X coordinate to be drawn in n × A [mm] (n is a positive integer) is ((n × A × f) / (z ′ × Cx))
Becomes

Next, in S14, the drawing coordinates of the vertical scale are determined. This scale also corresponds to the image plane γ described above.
Draw on top. When a scale of B [mm] is created in the vertical direction from the center of the image, the Y coordinate on the image plane γ is as follows based on a proportional relationship from FIG. 8B when the imaging system is viewed from the X ′ axis side. Here, the unit of the distance is millimeter [mm].

B / z ′ = My ₁ / f My ₁ = (B × f) / z ′ The unit is converted from [mm] to [the number of pixels], so that My ₁ = My ₁ / Cy.

Here, Cy is the vertical size [mm] per pixel of the CCD.

Therefore, the drawing position of the scale is (Xc, My)
₁ ) (Xc: X coordinate given in advance).

Similarly, 2 × B [mm] (for example, if B is 5 [m
m], a scale is created at 10 [mm], and the drawing coordinates are (Xc, My ₂ ), as follows.

2B / z ′ = My ₂ / f My ₂ = (2B × f) / z ′ The unit is converted from [mm] to [number of pixels], and My ₂ = My ₂ / Cy, so that n × B The Y coordinate drawn in [mm] (n is a positive integer) is ((n × B × f) / (z ′ × Cy)).

In step S15, using the coordinates obtained in steps S13 and S14, a scale is drawn so that the scale interval indicating the size of one scale (for example, 5 [mm]) can be known as shown in FIG. The scale interval may be drawn in the image or in a window as shown in FIG. here,
Both horizontal and vertical are displayed as 5 [mm].

Then, in S16, a figure Sp as shown in FIG. 9 is drawn at the position of the coordinates P (x, y) of the pixel used for the assumption of the vertical plane. This figure Sp serves as a reference for creating a scale for three-dimensional measurement. Therefore, the point of the graphic SP is not a reference point of the scale. In addition,
A figure may be drawn at the position of the corresponding point of P on the right image.
Thereby, since the pixels used for the plane assumption can be checked, the validity of the scale can be easily checked. That is, the scale width can be determined to be correct when the vicinity of the point P is viewed from the front, when the point P and the surroundings are a bright part, and when the correspondence of the point P is correctly detected. .

Next, in S17, it is determined whether or not the pixel P has been moved by the input device. If there is a movement, S18 is executed. Otherwise, the process ends. The movement of the pixel P is
This is to deal with a case where a plane cannot be determined by a point designated in advance or an automatically determined point described later, or a case where the determined plane is not appropriate. The movement of the pixel P is performed by selecting the figure Sp with the mouse on the left image and dragging it.

When the point P has moved, the vertical plane α is recalculated using the moved point P in S18.
Further, the evaluation of the vertical plane is performed, the result is output to the screen, and the process returns to S13. When returning to S13, the scale drawing coordinates in the horizontal and vertical directions are calculated again (S13, S1).
4) A scale is drawn (S15), and a figure Sp is drawn at the position of the pixel P after the movement, that is, the pixel used for the assumption of the vertical plane (S16).

The evaluation of the vertical plane is performed, for example, as follows. N pixels Mi (i = 0, 1,..., N) are selected in the order of the edge intensity in the vicinity of the pixel P, that is, within a predetermined range of the pixel P. By the pattern matching method such as the template matching described above,
The corresponding point of Mi in the right image is determined, and the three-dimensional position M′i is determined by the method using the principle of triangulation as described above.
(I = 0, 1,..., N).

Then, the distance between the vertical plane α and M′i is calculated, and the maximum distance, the minimum distance, and the average distance are displayed as the evaluation of the vertical plane as shown in FIG. 10 based on the distance data. The smaller these distances are, the better the target is approximated, and it can be evaluated that the planes are suitable for scale creation.

When the pixel P moves after displaying the scale, as shown in FIG. 10, the current distance after movement (maximum distance, minimum distance, average distance) and the distance before movement are determined. The difference between the current distance and the current distance is shown in parentheses. Accordingly, the operator can evaluate which plane is better by looking at these difference values, and can select a more appropriate plane for scale creation.

In the above example, when the pixel P moves, various distances after the movement are always recalculated. However, whether or not to perform the recalculation may be specified by a so-called check box on the screen or the like, and the recalculation may be performed in response to an operator request.

According to the above method, since the scale is created on the assumption that the object exists on the vertical plane assumed at the three-dimensional position of the pixel, the scale can be quickly displayed.

(1-1) Method of Automatically Determining P Point Next, a method of automatically determining the P point assuming the vertical plane α will be described with reference to FIG. The following shows a more general example of selecting N pixels from an image. Hereinafter, this method is referred to as a P-point automatic determination method.

First, when it is specified that the point P for assuming the vertical plane α is automatically determined, the processing of FIG. 11 is executed, and thereafter, the process proceeds to S11 of FIG. If the automatic determination is not specified, the above-described processing in FIG. 7 is executed.

If it is specified that the point P is to be automatically determined, first, at step 21 (S21), FIG.
The left image (or the right image) is divided into small regions of arbitrary equal size as shown in FIG. Here, split the left image,
It is assumed that the area is divided into 3 × 3 small areas of Ssi (i = 0, 1,..., 8).

The average pixel value of each small area is obtained (S2
2). Since the image has 256 gradations, the average value Avi (0, 1,..., 8) for each small area based on each pixel value data
Ask for.

In S23, N small areas are selected from the small area group, and the set Sk (k = 0, 1, 2, 2) is selected.
・ ・) Calculate the evaluation value. Here, a combination of the selected N small areas is defined as one set. Also,
The selection of the area may be to select adjacent small areas or to select non-adjacent small areas. Usually, the number N of small areas to be selected may be three or more.

There are various evaluation formulas for obtaining the evaluation value. Here, the average pixel value is large within a range where the average pixel value is smaller than a predetermined threshold value C ₀ , and The following equation is used so that a set with a small difference between the average pixel values gets a good evaluation. In the following equation, it is assumed that the larger the Esk is, the higher the evaluation is. C ₀ and C ₁ are constants. For example, C ₀ is 200 and C ₁ is 255.
When N = 1, there is no second term of Esk, α = 1.0, β
= 0.0. That is, the evaluation is based on only the average pixel value.

[0074]

(Equation 1) Dij is the absolute value of the difference between the average pixel values of the selected small area (| Avi-Avj | = Dij).

C ₀ and C ₁ are predetermined constants.

Α and β are real numbers, but α + β = 1.0
Is a number that satisfies.

Next, Esk is checked for all combinations of small areas, and a small area set Sk that maximizes Esk is selected (S24).

Then, in step S25, a pixel is selected from each small area of Sk according to the characteristic state of the pixel and its surroundings. Specifically, the edge intensity is the largest from each of the small regions of Sk and the vicinity of the pixel (for example, 5 × 5)
Pixels are selected based on the criterion that the average value obtained in (1) is smaller than the threshold (because no halation portion is selected), and each pixel is set to Pi (i = 0, 1,. From this Pi, for example, the one with the largest edge strength is selected,
Let P be the point for determining the vertical plane α.

In this way, the point P for determining the vertical plane α is automatically determined. Subsequently, the above-described steps after S11 in the flowchart of FIG. 7 are executed to determine the vertical plane, and the scale is set. Is displayed.

(1-2) Scale Creation Method Next, a scale creation method will be described.

When drawing at the same scale interval [mm] regardless of the distance to the target as in S13 and S14 in FIG.
There is a problem that the scale interval [pixels] becomes smaller in proportion to the distance from the endoscope tip to the target, making it difficult to grasp the size of the target. Therefore, a method of creating a scale by setting the scale interval to a certain pixel size or more irrespective of the distance to the target will be described with reference to FIG. In the following, a case in which the scale drawing interval is set to be at least η pixels or more will be considered. This method is hereinafter referred to as an appropriate scale interval creation method.

First, the actual distance (length) indicated by the horizontal scale corresponding to η pixels (the minimum number of pixels for the scale drawing interval) on the left image (or the right image) is calculated (S31). . When a scale is created at the position of η pixel in the horizontal direction from the center of the image, the scale at X ′ coordinate on the target object
[Mm] is as follows by a proportional relationship from FIG. 8A when the imaging system is viewed from the Y ′ axis.

A / z ′ = (η × Cx) / f A = (η × Cx × z ′) / f where f is the focal length of the lens. Cx is CC
D is the width length per pixel, that is, the horizontal size [mm].

The actual distance indicated by the horizontal scale obtained in step 31 is converted into an integer value (S32). Since the length represented by one scale is easier to understand with an integer, an integer larger than A [mm] and the closest integer is set to A '[mm], and the subsequent processing is performed. For example, if A is 4.6 [mm], A '
Is 5 [mm].

Next, the scale drawing position of the horizontal scale is obtained by calculation (S33). The horizontal drawing position Mx ₃ [pixel] on the scale drawing plane is expressed by the following equation as in S13, and the drawing position is (Mx ₃ , Yc) (Yc is the Y coordinate given in advance). Mx ₃ is the distance on the image plane γ corresponding to the distance A'on vertical plane alpha.

A ′ / z ′ = Mx ₃ / f Mx ₃ = (A ′ × f) / z ′ The unit is converted from [mm] to [number of pixels].

Mx ₃ = Mx ₃ / Cx Similarly, the drawing position of the scale of n × A ′ (mm) (n is an integer) is (Mx ₃ × n, Yc).

When the width of a scale drawing area is W [pixels], the number Nh of scales drawn in W is given by the following equation.

Nh = (W / Mx ₃ ) +1 Similarly, the actual distance (length) indicated by the vertical scale of the η pixel on the image is calculated (S 34). When a scale is created at an η pixel position in the vertical direction from the center of the image, the X ′ coordinate B [mm] on the target object is expressed by the following proportional relationship from FIG. 8B when the imaging system is viewed from the Y ′ axis side. Become like

B / z ′ = (η × Cy) / f B = (η × Cy × z ′) / f where f is the focal length of the lens. Cy is CC
D is the vertical length per pixel, that is, the vertical size [mm].

Then, the actual distance indicated by the vertical scale obtained in S34 is converted into an integer value (S35). Since the length represented by one scale is more easily understood by an integer, an integer larger than B [mm] and the closest integer is set to B '[mm], and the subsequent processing is performed.

In S36, the drawing position of the vertical scale is obtained by calculation. Horizontal drawing position M on the image
y ₃ [pixel] is given by the following equation as in S14. Drawing position (Xc, My ₃₎ become (Xc X-coordinates given in advance). My ₃ is a distance on the image plane γ corresponding to the distance B ′ on the vertical plane α.

B ′ / z ′ = My ₃ / f My ₃ = (B ′ × f) / z ′ The unit is converted from [mm] to [number of pixels].

My ₃ = My ₃ / Cy Similarly, the drawing position of the scale of n × B ′ [mm] (n is an integer) is (Xc, My ₃ × n).

Assuming that the height of the area where the scale is drawn is H [pixels], the number Nv of the scales drawn in H is given by the following equation.

Nv = (H / My ₃ ) +1 As described above, the scale can be created at high speed by assuming that the target object exists on the vertical plane. Further, it is possible to automatically determine the pixels used for the scale creation from the characteristics of the image, and to change the scale interval depending on the distance to the target.

(2) Three-Point Virtual Plane Generation Method Next, from three predetermined pixels, corresponding points on the other side of the stereo image are obtained, and three-dimensional coordinate positions of the three points are calculated therefrom. A method for assuming a virtual plane passing through the dimensional coordinates will be described. The configuration of an apparatus to which the three-point virtual plane generation method is applied is the same as that of the vertical plane generation method.

FIG. 14 is a flowchart showing a process of drawing a scale by approximating an object on an image with an appropriate plane in the three-point virtual plane generation method. Hereinafter, this method is referred to as a virtual plane generation method.

When performing scale drawing, there is a possibility that a more accurate scale can be created by drawing a scale by approximating an object on an image with an appropriate plane. In the following, a measurement endoscope apparatus that approximates a target with a plane corresponding to the target and draws a scale will be described with reference to FIG. In the following description, a description will be given assuming that a corresponding point of the right image is obtained for a predetermined pixel of the left image. However, a corresponding point of the left image may be obtained for a predetermined pixel of the right image.

First, corresponding points of the right image are determined for three pixels Pi (xi, yi) (i = 0, 1, 2) specified by the operator on the left image (S41). If the distortion has not been corrected, it is desirable to select these three points from the center of the image. Further, the three-dimensional position of Pi is calculated from the points on the left image and the corresponding points on the right image. As described in the first embodiment, the corresponding points may be detected by well-known template matching using epipolar lines. The three-dimensional coordinate position is obtained based on the principle of triangulation. The three-dimensional coordinate position of each Pi is represented by P'i (i = 0, 1, 2), and the coordinates thereof are represented by P'i (x'i, y
'I, z'i).

Then, a virtual plane β passing through P′i is determined (S42).

[0102]

(Equation 2) Then, the outer product of V ₀ and V ₁ is

[0103]

(Equation 3) Becomes Therefore, the virtual plane β passing through P ′ ₀ is as follows (u is a real number).

[0104]

(Equation 4) Then, the three-dimensional position Q′i of the pixel used for creating the scale is determined (S43). A pixel that is Dh [pixel] apart from one pixel of Pi (for example, hereinafter P ₀ ) in the horizontal direction is Q ₀
A pixel separated by (xq ₀ , yq ₀ ) and Dv [pixel] in the vertical direction is defined as Q ₁ (xq ₁ , yq ₁ ). Dh and Dv are given in advance. The corresponding point Q'i of Qi is obtained and the three-dimensional position Q'i
(X'qi, y'qi, z'qi) are determined.

At S44, the origin of the three-dimensional coordinates and Q'i
Is obtained. Further, an intersection between the straight line Li and the virtual plane β is obtained. Let the intersection be R'i (x'ri, y'ri,
z′ri), the intersection can be obtained as follows.

The equation for Li is as follows.

[0107]

(Equation 5) A point on the straight line is represented by a variable t, and is substituted into the equation of the virtual plane β.

[0108]

(Equation 6) When we ask for t

[0109]

Equation 7 Therefore, R'i is obtained by the following equation.

[0110]

(Equation 8) In S45, it obtains a distance per pixel in the horizontal direction and the vertical direction by _P'0 and R'i.

[0111]

[Equation 9] In S46, as shown in FIG. 9, the scale is drawn so that the scale interval indicating the size of one scale can be recognized. When the scale is drawn every C [mm], the horizontal drawing interval D in the image
rh and the vertical drawing interval Drv are as follows. The scale interval may be determined by a method similar to S31 to S36 of the vertical plane method.

[0112]

(Equation 10) In S47, three figures Sp are drawn at the positions of the coordinates Pi (xi, yi) (i = 0, 1, 2) of the predetermined pixel used for the assumption of the virtual plane as shown in FIG. Similarly, a graphic may be displayed at the corresponding point of Pi in the right image. Thus, since the pixels used for the plane assumption can be confirmed, it is possible to easily confirm the validity of the scale. That is, it is determined that the scale is correct when the point Pi and the periphery are a bright part, and when the corresponding point of the point Pi is correctly detected.

In S48, it is determined whether or not the pixel P has been moved by the input device. If it has been moved, S49 is performed. If not, the process ends.

In S49, after the movement, the virtual plane β is recalculated. Evaluate the virtual plane and output the result to the screen. The evaluation of the virtual plane is performed by, for example, the same method as in the first embodiment. The movement at this time may be recalculated when all three points have moved, or may be calculated again when one or two points have moved.

Further, the evaluation of the virtual plane is based on the designated 3
Some pixels may be selected in descending order of edge strength from the pixels in the area surrounded by the point. Then, the three-dimensional coordinates of the selected pixel are calculated, the distance to the virtual plane is calculated based on the three-dimensional coordinates, and the maximum distance, the minimum distance, and the average distance are obtained and displayed.

Also, Pi in S41 and Qi in S43 are calculated from S21 to S25 in the above-described vertical plane generation method.
It is also possible to automatically select from the image data by the same method as described above. That is, automatic selection can be performed by selecting the top three among the Pis selected based on a predetermined criterion in the small area having the largest evaluation value.

Further, as shown in S18 of FIG. 7 of the vertical plane generation method, after recalculating in S49 and displaying the evaluation, the process may return to S43 and repeat the processing.

Further, when drawing the scale, Q
₀ and Q ₁ a plurality sets respectively, from the horizontal drawing interval Drh and vertical drawing interval Drv which was calculated to correspond to them, may be selected median. According to this, the influence of erroneous detection of the corresponding point can be eliminated.

According to the three-point virtual plane generation method, it is assumed that an object exists on a plane determined by the three-dimensional position of three pixels, and a scale is created based on the distance to the plane.
Therefore, even when the object is not perpendicular to the optical axis of the endoscope or the like, it is possible to create a more accurate scale.

[3]. Method of Displaying Scale at Appropriate Position Based on Feature Amount In the measurement endoscope apparatus configured as described above, a process of drawing a scale for grasping the size of an object in accordance with the feature of an image is shown in FIG. 17 will be described with reference to FIG.

FIG. 15 is a flowchart showing the flow of processing for drawing a scale. Here, the scale display position will be described in the case of having a manual mode and an automatic mode. FIG. 16 is a screen display example for explaining the mode setting. FIG. 17 is a screen display example for explaining scale drawing.

In the following, an example of scale drawing by the above-described vertical plane generation method will be described. However, as described later, regarding the switching between the manual mode and the automatic mode, the above-mentioned three-point virtual plane generation method May be.

When an instruction to draw a scale is given for a captured image of a subject, a scale drawing application program shown in the flowchart of FIG. 15 is executed.

First, the scale data necessary for drawing the scale is calculated by the above-described vertical plane generation method. In this case, the scale data is, for example, data of the number of pixels of a unit length desired to be a reference of the scale on the vertical plane, which is obtained in step 13 of FIG. At this time, when the above-described P point or the like serving as a reference of the vertical plane calculation is not specified, the scale is calculated by the vertical plane method using the center position of the left image as the P point to generate the scale data. Although the scale is created by the vertical plane generation method here, the scale may be created by another method.

Then, it is determined whether the scale drawing mode is the manual mode or the automatic mode (S51). This determination is made, for example, based on the selection state of the so-called radio button 61 on the screen of FIG. The state shown in the screen example of FIG. 16 is when the manual mode in which the user specifies the drawing position is checked. If the manual mode has been selected, the process of S52 is executed. In the case of the automatic mode, the process of S57 is executed.

In the case of the manual mode, the scale drawing position specified by the user is read out, and the scale drawing is performed at the specified position using the scale data calculated and obtained (S52).
At this time, if no scale drawing position is specified in advance, for example, another window is displayed on the screen, and the operator is instructed to specify the scale drawing position. The operator moves the cursor using the mouse to specify the scale drawing position. If a predetermined position coordinate on the screen is previously set on a predetermined memory as an initial value, a scale is drawn at the predetermined position.

Next, it is determined whether or not the scale has been moved by the input device (S53). S54 when moved
Is executed. Otherwise, the process of S59 is performed.

If the scale is difficult to see at the position of the scale drawn and displayed on the screen in S52, the operator moves the cursor using the mouse and clicks on the displayed scale frame to select it. Move the scale while dragging on the screen. FIG. 16 illustrates the situation before and after the movement of the scale in the left image display window. The position of the scale before the movement is indicated by a dotted line, and the position after the movement is indicated by a solid line. In FIG. 16, the arrow indicates the cursor.

When the movement of the scale is finished, that is, when the scale is dragged and dropped, the dropped is detected by the cursor display means. In response to the detection, it is determined that it has been moved, and the process proceeds to S54, where a new scale drawing position is updated and scale drawing is performed.

Next, in S55, it is determined whether or not to perform recalculation. The mode is determined based on, for example, the state of selection of a so-called check box 62 on the screen of FIG. If the recalculation mode is checked, the process of S56 is performed to perform the recalculation, and if not, the process of S59 is performed.

In S56, a scale is created again by the above-described vertical plane generation method using a predetermined pixel, here, the center position of the scale or a position coordinate in the vicinity thereof, and the scale is drawn. After that, S59 is performed.

In this recalculation, a corresponding point is obtained based on, for example, the coordinates of the center position of the scale of the left image.
Then, a three-dimensional coordinate position is calculated based on the corresponding point, a scale is created by the above-described vertical plane generation method, and the scale is drawn at the same position. Note that the coordinates of the center of the scale are not
A vertical plane may be generated using other points and the number of scale pixels may be recalculated.

In step S59, when there is an instruction to terminate the application on the screen, such as termination of the scale drawing instruction or termination of the entire screen display, the application is terminated.

If the drawing mode is automatic in S51, the process of S57 is executed.

In S57, the scale drawing area or position is automatically determined. This automatic determination is performed based on the feature amount of the image obtained from the video signal of the imaging unit.

More specifically, in the automatic selection method, first, an image is divided into small regions of arbitrary equal size. Here, for example, as shown in FIG. 12, it is assumed that the image is divided into 3 × 3 (9) Ssi (i = 0, 1,..., 8). Then, an average pixel value of each small area is obtained. The average pixel value is one of the feature amounts relating to the intensity of the video signal. Each of the average pixel values of each small area is represented by Avi
(I = 0, 1,..., 8). From among combinations of small areas given in advance, a set having a small sum of Avi is selected. In other words, in a region where the pixel value is small, that is, in a dark region,
Scale drawing is performed assuming that there is no object to be observed. For example, Ss ₁ and Ss ₃ in FIG. 12, Ss ₁ and Ss _5, Ss ₃ and Ss _7, S
and a small region located vertically with respect to the center Ss _4, such as s ₅ and Ss _7, given as a set of small regions in the horizontal direction with respect to Ss _4, is selected.

Next, the scale is drawn in the selected small area using the scale data calculated at the beginning of this processing (such as the data of the number of pixels on the drawing plane with respect to the desired unit length) ( S58). For the arrangement of the 3 × 3, the horizontal scale in a small region located vertically with respect to the center Ss _4,
A vertical scale is drawn in a small area located in the left-right direction. S
For s ₁ and Ss _5, drawing a vertical scale to the horizontal scale, Ss ₅ to Ss ₁ as shown in FIG. At the same time, scale intervals indicating the size of one scale are drawn in the image as shown in FIG. The scale interval may be shown in an appropriate window on the screen as shown in FIG. In FIG. 16, 5 [mm] is displayed in both the horizontal and vertical directions.

The position where the scale is drawn in the small area may be a position passing through the center of each small area, or may be another predetermined position in the small area.

In S59, it is determined whether or not the application has been terminated as described above. If not, S5
Return to 1. In S60, in the case of the manual mode, the drawing position is updated using the current scale drawing position, and the processing ends.

As described above, the display position of the scale can be changed manually or automatically.

Further, by recalculating the scale when the scale is manually moved, a scale using a pixel at a desired position can be obtained.

(Second Embodiment) When a scale is created by using a scale creation method corresponding to a subject, an error between the scale and the size of an actual object may be reduced in some cases. Accordingly, a measurement endoscope apparatus that switches a method of creating a scale according to characteristics of an image will be described.

Here, an example of switching between the vertical plane generation method and the three-point virtual plane generation method described in the first embodiment will be described with reference to FIG. FIG. 18 is a flowchart showing the flow of the process for switching the scale creation method.

Since the configuration of the apparatus in this embodiment is the same as that of the first embodiment, FIGS.
The description of item 4 is omitted because it is redundant.

First, when an instruction to draw a scale is made on a captured image of a subject, the processing shown in the flowchart of FIG. 18 is executed.

In step S71, the left image or the right image is divided into arbitrary small areas of equal size. Here, as shown in FIG. 12, Ssi (i = 0, 1,...,
8) It is assumed that the image is divided into 3 × 3 pieces.

In S72, the average pixel value of each small area is obtained. Avi (i = 0, 1,..., 8).

In S73, two adjacent areas are selected from the small areas, and the absolute value Dij of the difference between the average values is obtained. Assuming that the selected two regions are Ssi and Ssj, and their average values are Avi and Avj, respectively, Dij can be obtained from the following equation.

| Avi-Avj | = Dji In S74, the state of the image is determined from the average value and the difference as follows.

(1) When all Avis are smaller than the threshold value Ta, there is only a dark portion (a portion having a small pixel value), so it is assumed that there is no target to be noted.

(2) Threshold Ta at average pixel value of area
When there are both a smaller value and a value larger than the threshold value Ta, there is a dark portion, and therefore, an area other than the dark portion is set as a measurement target.

(3) If the average pixel value of all the regions is larger than Ta, no dark portion exists. Further, a. When all the differences Dji are smaller than the threshold value Tb, it is assumed that the target is viewed from the front.

B. Other than that, it is not a front view.

Then, in S75, the scale display processing is switched according to the state of the image determined in S74.

In the case of (1), no scale display is performed.

In the case of (2), a scale is created using pixels in a small area whose average value is larger than Ta. (3) described later
b. A virtual plane is obtained from three points in the same manner as described above, and a scale is created using the virtual plane method.

(3) In the case of a, the three-dimensional coordinates of one point in a small area whose average value is larger than Ta are obtained, and a scale is created by the vertical plane method.

(3) In the case of b, a virtual plane is obtained from three points in a small area whose average value is larger than Ta, and a scale is created using the virtual plane.

In S76, the scale created in S75 and the scale interval are displayed in the same form as in FIG.

Furthermore, as described in the first embodiment, the scale created by the scale creation method determined in this way is displayed at a position corresponding to the image data so that the scale can be easily viewed. Can be.

According to the above-described processing, according to the second embodiment, it is possible to switch to a scale creation method according to the characteristics of the image of the captured subject.

[Supplementary Notes] According to the first and second embodiments described above, the contents shown in the following supplementary items can be said to be characteristic matters.

[Appendix 1] A video signal obtaining unit for obtaining an image signal of a subject by a plurality of image pickup units, and an intensity of an image signal in a divided region in an image based on an image signal of at least one of the image pickup units. A feature value calculating means for calculating a feature value; a display means for displaying scale information in a predetermined area based on a calculation result of the feature value calculating means;
A scale display device comprising:

[Additional Item 2] A video signal of a subject is obtained by a plurality of image pickup means, and a characteristic quantity relating to the intensity of the image signal in a divided area in the image is calculated based on the image signal of at least one of the image pickup means. A scale display method for displaying scale information in a predetermined area based on the calculation result of the feature amount.

With such a configuration, the displayed scale is easy to see.

[Additional Item 3] The scale display device according to Additional Item 1, wherein the feature amount is an average pixel value for each divided area.

With such a configuration, the feature can be determined based on the pixel value.

[Additional Item 4] The display means displays scale information in a small (dark) area with an average pixel value for each of the divided areas based on the calculation result of the feature amount calculating means. A scale display device according to claim 3.

With this configuration, the scale is displayed at a position where it is easy to see.

[Additional Item 5] The display means can further change the scale display position according to the automatic mode and the manual mode. In the automatic mode, the scale display position is determined based on the result of the characteristic amount calculation. The scale display device according to claim 1, wherein the scale information is displayed in an area designated by the user, and the scale information is displayed in an area designated by the user in the manual mode.

With such a configuration, the scale is created based on appropriate data, and the displayed position is a position that is easy to see.

[Appendix 6] The scale display device according to Appendix 5, wherein the display means moves the display position of the scale information to an area designated by the user in the manual mode.

With this configuration, the scale can be moved to a position where the user can easily see.

[Additional Item 7] The display means recalculates the scale information when the display position of the scale information is moved to an area designated by the user in the manual mode. Item 6. The scale display device according to Item 6.

With such a configuration, when the user moves the scale, the scale information is calculated again, and a more accurate scale of the area to which the user pays attention is displayed.

[Additional Item 8] The scale display device according to Additional Item 1, wherein the display means further comprises means for switching a scale creation method based on the calculation result of the feature amount calculating means.

With such a configuration, a scale is created by an appropriate method according to an image.

[Additional Item 9] The scale display device according to Additional Item 8, wherein the feature amount is an average pixel value for each divided area.

[Supplementary note 10] The display means determines a scale creation method according to the difference between the average pixel values between the divided areas based on the calculation result of the feature quantity calculation means. A scale display device according to claim 9.

[Supplementary Item 11] The display means may further include:
The scale display device according to claim 10, wherein a scale creation method is determined according to an average pixel value for each area.

[Supplementary Item 12] The display means calculates an average pixel value for each region, a difference between the average pixel values for each region,
The scale display device according to claim 10, wherein a scale creation method is determined according to a result of comparison with a predetermined threshold value for each of the predetermined scales.

With such a configuration, the scale becomes more accurate.

[0183]

As described above, according to the present invention, when displaying a scale on an imaged subject,
The scale display position can be displayed at an appropriate position. Further, according to the present invention, the scale creation method can be switched according to the state of the image.

[Brief description of the drawings]

FIG. 1 is a block diagram of a measurement endoscope apparatus according to a first embodiment.

FIG. 2 is a diagram illustrating a structure of a distal end portion of an insertion portion of the endoscope.

FIG. 3 is a block diagram showing a functional configuration of an endoscope device for measurement.

FIG. 4 illustrates a configuration of a host computer.

FIG. 5 is a block diagram illustrating a configuration of an external storage device.

FIG. 6 is a view for explaining a three-dimensional coordinate system at the end of the endoscope.

FIG. 7 is a flowchart showing a flow of determining a vertical plane based on three-dimensional position information of one pixel and drawing a scale;

FIG. 8 is a diagram for explaining the principle of scale creation;

FIG. 9 is a diagram for explaining scale drawing in a left image.

FIG. 10 is a diagram for explaining evaluation of a plane.

FIG. 11 is a flowchart showing a flow of selecting a pixel.

FIG. 12 is an explanatory diagram of area division.

FIG. 13 is a flowchart showing a flow of adjusting a scale interval.

FIG. 14 is a flowchart showing the flow of a scale drawing process according to the three-point virtual plane method;

FIG. 15 is a flowchart showing a flow of processing in which a scale is drawn in a manual mode and an automatic mode.

FIG. 16 is a screen display example for explaining a mode setting;

FIG. 17 is a screen display example for explaining scale drawing.

FIG. 18 is a flowchart showing a flow of processing for switching a method of creating a scale;

[Explanation of symbols]

 61 radio button 62 check box 101 endoscope 102 insertion section 103R, 103L lens 104L left imaging means 104R right imaging means 105 Light distribution lens 106 Light guide 120 Host computer 130L Left image monitor 130R Right image monitor

[Procedure amendment]

[Submission date] December 25, 2000 (200.12.
25)

[Procedure amendment 1]

[Document name to be amended] Statement

[Correction target item name] 0036

[Correction method] Change

[Correction contents]

FIG. 7 is a flowchart showing the flow of determining a vertical plane based on the three-dimensional position information of one pixel and drawing a scale. FIG. 8 is a diagram for explaining the scale creation principle. FIG. 9 is a diagram for explaining scale drawing in the left image. Figure 10 is a diagram for illustrating the evaluation of the flat surface. FIG. 11 is a flowchart illustrating a flow of selecting a pixel. FIG. 12 is an explanatory diagram of the area division. FIG. 13 is a flowchart showing the flow of adjusting the scale interval.

[Procedure amendment 2]

[Document name to be amended] Statement

[Correction target item name] 0046

[Correction method] Change

[Correction contents]

[0046] A is a unit length to be the criteria for eye Sheng on vertical plane (e.g., 5 [mm]). z ′ is the distance from OL to the vertical plane α. f is the focal length of the lens, which is the distance from OL in the figure to the image plane γ. Next formula
In Mx ₁ corresponds to the distance A on the vertical plane alpha, is a distance on the image plane gamma [mm].

[Procedure amendment 3]

[Document name to be amended] Statement

[Correction target item name] 0047

[Correction method] Change

[Correction contents]

A / z ′ = Mx ₁ / f Mx ₁ = (A × f) / z ′ The unit is converted from [mm] to [number of pixels], and M Mx ₁ = Mx ₁ / Cx.

[Procedure amendment 4]

[Document name to be amended] Statement

[Correction target item name] 0049

[Correction method] Change

[Correction contents]

[0049] Thus, the drawing position of the scale (M Mx _1,
Yc) (Yc: Y coordinate given in advance).

[Procedure amendment 5]

[Document name to be amended] Statement

[Correction target item name] 0050

[Correction method] Change

[Correction contents]

Similarly, 2 × A [mm] (for example, when A is 5
If [mm], to create a graduation at) of 10 [mm], the drawing position (M Mx _2, Yc) Tosureba obtained as follows.

[Procedure amendment 6]

[Document name to be amended] Statement

[Correction target item name] 0051

[Correction method] Change

[Correction contents]

2A / z ′ = Mx ₂ / f Mx ₂ = (2A × f) / z ′ The unit is converted from [mm] to [the number of pixels], and M Mx ₂ = Mx ₂ / Cx.

[Procedure amendment 7]

[Document name to be amended] Statement

[Correction target item name] 0054

[Correction method] Change

[Correction contents]

B / z ′ = My ₁ / f My ₁ = (B × f) / z ′ The unit is converted from [mm] to [number of pixels], and M My ₁ = My ₁ / Cy.

[Procedure amendment 8]

[Document name to be amended] Statement

[Correction target item name] 0056

[Correction method] Change

[Correction contents]

Accordingly, the scale drawing position is (Xc, M
My ₁ ) (Xc: X coordinate given in advance).

[Procedure amendment 9]

[Document name to be amended] Statement

[Correction target item name] 0057

[Correction method] Change

[Correction contents]

Similarly, 2 × B [mm] (for example, if B is 5 [m
m], the scale is created at 10 [mm], assuming that the drawing coordinates are (Xc, M My ₂ ).

[Procedure amendment 10]

[Document name to be amended] Statement

[Correction target item name] 0058

[Correction method] Change

[Correction contents]

2B / z ′ = My ₂ / f My ₂ = (2B × f) / z ′ The unit is converted from [mm] to [number of pixels], and M My ₂ = My ₂ / Cy. The Y coordinate drawn on B [mm] (n is a positive integer) is ((n × B × f) / (z ′ × Cy)).

[Procedure amendment 11]

[Document name to be amended] Statement

[Correction target item name] 0085

[Correction method] Change

[Correction contents]

Next, the scale drawing position of the horizontal scale is obtained by calculation (S33). Horizontal drawing position M Mx ₃ [pixels] on the image plane is given by the following formula in the same manner as S13, the drawing position is _{(M Mx 3, Yc) (} Yc is previously gave Y coordinates). In the following equation, Mx ₃ is the distance A on the vertical plane α.
'Is a distance [mm] on the image plane γ corresponding to'.

[Procedure amendment 12]

[Document name to be amended] Statement

[Correction target item name] 0087

[Correction method] Change

[Correction contents]

[0087] drawing position of the scale (n is an integer) of M Mx ₃ = Mx ₃ / Cx less Likewise n × A'(mm) becomes _{(M Mx 3 × n, Yc} ).

[Procedure amendment 13]

[Document name to be amended] Statement

[Correction target item name] 0089

[Correction method] Change

[Correction contents]

[0089] _{Nh = (W / M Mx 3} ) +1 Likewise, actual distance (length) showing the vertical scale η pixels on an image obtained by the calculation (S34). When a scale is created at an η pixel position in the vertical direction from the center of the image, the X ′ coordinate B [mm] on the target object is expressed by the following proportional relationship from FIG. 8B when the imaging system is viewed from the Y ′ axis side. Become like

[Procedure amendment 14]

[Document name to be amended] Statement

[Correction target item name] 0092

[Correction method] Change

[Correction contents]

In S36, the drawing position of the vertical scale is obtained by calculation. Horizontal drawing position M on the image
My ₃ [pixel] is Ri Do the following equation similarly to S14, the drawing position is (Xc, M My ₃₎ become (Xc are given in advance X
Coordinate). In the following equation, My ₃ is a distance B ′ on the vertical plane α.
Is the distance [mm] on the image plane γ corresponding to.

[Procedure amendment 15]

[Document name to be amended] Statement

[Correction target item name]

[Correction method] Change

[Correction contents]

M My ₃ = My ₃ / Cy Hereinafter, similarly, the drawing position of the scale of n × B ′ [mm] (n is an integer) is (Xc, M My ₃ × n).

[Procedure amendment 16]

[Document name to be amended] Statement

[Correction target item name] 0096

[Correction method] Change

[Correction contents]

Nv = (H / M My ₃ ) +1 As described above, the scale can be created at high speed by assuming that the target object exists on the vertical plane. Further, it is possible to automatically determine the pixels used for the scale creation from the characteristics of the image, and to change the scale interval depending on the distance to the target.

[Procedure amendment 17]

[Document name to be amended] Statement

[Correction target item name] 0100

[Correction method] Change

[Correction contents]

First, corresponding points of the right image are determined for three pixels Pi (xi , yi) (i = 0, 1, 2) specified by the operator on the left image (S41). If the distortion has not been corrected, it is desirable to select these three points from the center of the image. Further, the three-dimensional position of Pi is calculated from the points on the left image and the corresponding points on the right image. As described in the first embodiment, the corresponding points may be detected by well-known template matching using epipolar lines. The three-dimensional coordinate position is obtained based on the principle of triangulation. The three-dimensional coordinate position of each Pi is represented by P'i (i = 0, 1, 2), and the coordinates thereof are represented by P'i (x'i, y
'I, z'i).

[Procedure amendment 18]

[Document name to be amended] Statement

[Correction target item name] 0112

[Correction method] Change

[Correction contents]

[0112]

(Equation 10) In S47, three figures Sp are drawn as shown in FIG. 9 at the positions of the coordinates Pi (xi , yi) (i = 0, 1, 2) of the predetermined pixel used for the assumption of the virtual plane. Similarly, a graphic may be displayed at the corresponding point of Pi in the right image. Thus, since the pixels used for the plane assumption can be confirmed, it is possible to easily confirm the validity of the scale. That is, it is determined that the scale is correct when the point Pi and the periphery are a bright part, and when the corresponding point of the point Pi is correctly detected.

[Procedure amendment 19]

[Document name to be amended] Drawing

[Correction target item name] Fig. 9

[Correction method] Change

[Correction contents]

FIG. 9

[Procedure amendment 20]

[Document name to be amended] Drawing

[Correction target item name] FIG.

[Correction method] Change

[Correction contents]

FIG.

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI theme coat ゛ (Reference) H04N 7/18 H04N 7/18 U F term (Reference) 2H040 AA01 BA22 CA23 GA02 GA10 GA12 4C061 CC06 HH52 LL08 SS21 WW04 WW12 5C023 AA18 AA37 AA38 BA01 CA01 CA06 EA03 5C054 CH01 EA01 EA05 FB03 FC15 FD01 FE12 HA12 5C061 AA06 AB04 AB18

Claims (3)

[Claims] A video signal acquiring unit for obtaining an image signal of a subject by a plurality of image capturing units; and a characteristic amount relating to an intensity of the image signal in a divided region in an image based on at least one image signal of the image capturing unit. A scale display device comprising: a feature value calculating means for calculating; and a display means for displaying scale information in a predetermined area based on the calculation result of the feature value calculating means. 2. An image signal of a subject is obtained by a plurality of image pickup means, and a characteristic quantity relating to an intensity of the image signal in a divided area in an image is calculated based on the image signal of at least one of the image pickup means. A scale display method characterized by displaying scale information in a predetermined area based on a calculation result of an amount. 3. The scale display device according to claim 1, wherein said scale display means further comprises means for switching a scale creation method based on a calculation result of said characteristic amount calculation means.