JP2001280956A - Stereoscopic image measuring instrument - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enhance the speed, efficiency, and reliability of measurement as a whole. SOLUTION: A surveying instrument 1 measures several reference points on a site. As a camera 3, a digital camera, a film camera, or the like is used. A reference point search part 10 relates the reference points previously measured by the instrument 1 to an image. From the reference points related by the search part 10, a search area setting part 20 sets an area of search for performing correlative image processing and sets blocks such as a reference data block and a search data block. A computing part 30 performs orienting computation and correlative image processing (stereoscopic matching) with respect to the area of search set by the setting part 20. As a display part 40, a stereoscopic monitor, a monitor for a personal computer, or the like is used that can provide stereoscopic vision. A measuring part 50 demands additional measurement if the correlative processing results in a poor correlation.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a measuring device for stereo images, and more particularly to a measuring device for measuring a stereo image into a three-dimensional image.

[0002]

2. Description of the Related Art Conventionally, when measuring a relatively large object as a three-dimensional image, the number of measurement points is increased by scanning the measurement points using a surveying instrument such as a total station (TS) or moving the GPS. A three-dimensional image is represented by a set of many small surfaces surrounded by a plurality of measurement points. Further, as another method, three-dimensional measurement is performed by performing image correlation processing (stereo matching) or the like from an image captured in stereo to represent a three-dimensional image as a set of many surface shapes.

[0003]

In the case of the former prior art, especially when using a total station (TS),
There is a method in which a measurement point is automatically scanned by using a surveying instrument with an automatic collimation function (non-prism TS) to acquire three-dimensional data. The non-prism TS is a total station that does not require a reflecting mirror such as a prism. However, this method takes an enormous amount of measurement time because measurement is performed while driving the TS with a motor. For example, assuming that one point measurement takes one second, measuring only 200 × 200 points requires about 11 hours. In addition, there are other problems such as the inability to measure where the distance measuring light does not return, and the deterioration of accuracy as the distance measuring beam becomes farther.

When the GPS is moved as in the latter conventional technique, the coordinates of the measurement points are measured while moving the platform on which the GPS is mounted, so that much time and effort are required. In addition, since the GPS must be moved to the measurement target point, measurement in a dangerous place cannot be performed.

[0005] In addition, in the case of GPS, when a photograph is taken from a stereo image for measurement, it is necessary that a reference point (orientation point) be provided on the object to be measured. There is a problem that it is impossible to know whether or not the measurement can be performed only by observing it, and that the stereo matching (image correlation processing) requires a long calculation time and that a portion having no feature cannot be measured. In particular, coarse to dense search (Coarse to
What is called an image correlation method by Fine) is used. This method does not perform correlation processing on high-resolution images from the beginning, but gradually performs correlation processing from low-resolution images to gradually higher images, thereby reducing local errors, This is a method for improving reliability (see Mikio Takagi and Hirohisa Shimoda, Image Analysis Handbook, p. 709). However, according to this method, image correlation processing must be performed for each image having a different resolution, which requires a great deal of calculation time.

[0006] Further, when measurement is performed with a stereo camera having a fixed base line, although there is an advantage that a reference point is not required, the angle of view (measurement range) is limited. A connection point (reference point) is required.

[0007] In view of the above, the present invention, by using the non-prism TS and image correlation processing in combination, is an area where conventional measurement is impossible, such as an inaccessible place or a dangerous place. It is also an object of the present invention to make it possible to perform non-contact measurement, and to further increase the speed, efficiency, and reliability of the entire measurement as compared with the case where measurement is performed by each method alone.

[0008]

According to the present invention, in order to solve the above-mentioned problems, for example, the following features are provided. By these processes, measurement at higher speed and with higher reliability than before can be performed. Becomes 1. 1. Take a stereo image of the area you want to measure 2. Measure 6 or more reference (orientation) points in a region included in both stereo images with the non-prism TS. 3. Measure (orient) a reference (orientation) point measured by TS on the stereo image. 4. Determine an area for image correlation processing from the measurement points Performing image correlation processing on the area An image created from the correlation coefficient of 6.5 or the three-dimensional coordinate values obtained from the result of the correlation processing is displayed on a display.

Further, the present invention has the following features. By adding these processes, higher-speed and more reliable measurement becomes possible. Further, when the reliability of the measurement is considered to be insufficient, the following processes 7 to
You may repeat until 10 is satisfied. 7. 7. If the area with low correlation coefficient or the image created from the measurement data is unsatisfactory, perform additional measurement at that location. 8. Redetermine the area for image correlation processing from the measurement points 9. Perform image correlation processing on the region An image created from the correlation result is displayed on a display.

In the present invention, an area which cannot be measured by the non-prism TS and does not work well by the image correlation processing can be calculated by interpolating the elevation value from the information of the measurement area of the non-prism TS. The advantages of these processes are:
It is as follows.・ Since the three-dimensional measurement value of the non-prism TS can be used as the initial value of stereo matching, coarse / dense search (Coa
(rse to Fine) Image correlation processing can be omitted, and the calculation time is shortened.-The combination measurement of the stereo image and the non-prism TS makes the entire measurement time much shorter than the measurement of the TS alone or the measurement by the stereo image.・ Locations where measurement is difficult with images (ex. Places without features)
Etc. are non-prism TS, and conversely, non-prism TS
It is possible to measure an area where light does not return with an image. As described above, it is possible to interpolate the measurement difficult portions of both. A portion having poor image correlation is interpolated by the measurement of the non-prism TS, and the measured value is used as an initial value, so that the measurement area can be more densely and appropriately, so that reliability is improved.・ Measurement can be performed on-site, and measurement can be performed while confirming error locations, so measurement reliability is increased and failures are eliminated.

According to the solution of the present invention, for a stereo image including at least three measurement points whose position data is known, at least a part of the measurement points is set as a division point and a plurality of measurement points are determined. And a setting unit for setting a search area based on at least three of the segment points, and an image of the corresponding search areas on each stereo image based on the search area set by the setting unit. An arithmetic unit for performing a correlation process; and a measuring unit for measuring coordinates of a point at an arbitrary position from the correlation result by the arithmetic unit. The arithmetic unit determines a new measurement point according to the result of the correlation process. Provided is a stereo image measurement device that creates information on a required measurement area.

[0012]

Embodiments of the present invention will be described below with reference to the drawings.

A. 1. Hardware FIG. 1 shows an overall block diagram of the measuring apparatus of the present invention. The system includes a surveying instrument 1, a measuring device 2, and a camera 3. The measurement device 2 includes a reference point search unit 10, a search area setting unit 20, a calculation unit 30, a display unit 40, a measurement unit 50, and a bus 60.
Is provided. These reference point searching section 10 and search area setting section 2
The arithmetic unit 30, the display unit 40, and the measuring unit 50 are mounted on, for example, a personal computer, and are mutually connected by a bus 60.

The surveying instrument 1 measures several reference points on site. This is not necessary when a stereo image including several reference points is obtained in advance and measured. The camera 3 acquires an image, and for example, a digital camera or a film camera can be used. Even if the camera 3 is not provided, an image in which several reference points are imprinted may be obtained and analyzed in advance.

The reference point searching unit 10 has a surveying instrument 1
The reference point measured by the above is associated with the image. The search area setting unit 20 sets a search area when performing image correlation processing and sets each block of a reference data block and a search data block from the reference points associated by the reference point search unit 10. The operation unit 30 performs orientation calculation,
Image correlation processing (stereo matching) is performed on the search area set by the search area setting unit 20. Display section 40
Is a stereo monitor capable of stereoscopic viewing, a monitor of a personal computer, or the like. The use of a three-dimensional monitor enables more accurate three-dimensional measurement and confirmation of measurement results. The display unit 40 graphically displays a captured stereo image, a correlation coefficient value obtained from a result of the correlation processing, a point created by three-dimensional coordinates obtained from the result of the correlation processing, a contour line, a bird's eye view, an ortho image, and the like. I do. Display section 40
Accordingly, the method of performing the graphic display and confirming or additionally measuring can be used, for example, when it is desired to visually determine the measurement result appropriately, or for an object whose correlation coefficient is not highly reliable. The graphic display can be performed in real time.

The measuring section 50 causes an additional measurement to be performed when the correlation result shows that the correlation result is bad. As the additional measurement by the measuring unit 50, there is a method of displaying a correlation coefficient and performing additional measurement of a portion having a small correlation. In addition, in addition to this, a point based on the three-dimensional coordinates of each point, a contour line, a wire frame model, a surface model with a surface attached, or
There is a method of creating a bird's-eye view or an orthorectified image on which an image is pasted, displaying the image graphically, and performing additional measurement for a defect of the displayed image. In addition, the method of displaying or displaying the correlation coefficient on the screen to check or correct the image includes displaying an area having a low correlation coefficient and manually or semi-automatically measuring and correcting based on the screen, and , There is a method of automatically measuring an area below a predetermined threshold.

An advantage of the present invention is that display, confirmation, and correction can be easily performed in real time at a measurement site without using a stereo monitor or the like, so that the overall measurement time is shortened and reliability is improved. And three-dimensional measurement can be performed reliably.

Next, a description will be given in detail of a case where measurement is performed manually / semi-automatically and a case where measurement is performed automatically based on the correlation result.

B. Basic Measurement Flow FIG. 2 shows a flowchart for online measurement. In online measurement, for example, an image is measured at a measurement site and a three-dimensional image is displayed.

In step S10, first, a stereo image of a region to be measured on site is photographed. FIG. 3 is an explanatory diagram of a stereo image. As shown in this figure,
Two overlapping stereo images (left image L, right image R) are photographed. In step S20, the surveying instrument 1 measures three or more reference (orientation) points in an area included in both the left and right images photographed in stereo. The number of reference points may be three or more. However, if the number of reference points is six or more, the orientation process (see step S40 described later) is performed more stably, and the subsequent analysis becomes highly reliable. Therefore, here, six points will be described as an example. That is, in the area where the left and right images in FIG. 3 overlap, six points are set as reference (orientation) points C1.
Measured by the surveying instrument 1 as C6. Next, the stereo image data and the reference point coordinate values measured by the surveying instrument are
Transfer to a place where a stereo image measuring device such as an office is located. For transfer, use a memory medium that stores images,
It can be transmitted over a telephone line or the like.

Next, at step S40, the display unit 40
The reference (orientation) point measured by the surveying instrument 1 is measured (correlated) on the left and right images displayed in (1). That is, in this example, the points C1 to C6 measured by the surveying instrument 1 are located on the left image L and the right image R, respectively.

In step S80, the measurement points measured in step S40 are connected to each other to create a triangle. FIG.
FIG. 7 shows an explanatory diagram of creating a triangle based on measurement points. In this example, points are connected from measurement points C1 to C6 to form a triangle. In this case, the points may be connected to each other to form a quadrangle instead of a triangle. However, a triangle can classify the area more finely. (A triangular shape can form two planes with four points. Only a plane can be configured), and accuracy and reliability can be improved. As a method of interpolating three-dimensional coordinates from such random points, an irregular triangular network (Triangula) is used.
ted Irragler Network, TI
N). TIN is for generating a mesh having triangles as constituent units. For more information about TIN,
"Masao Iri, Takeshi Koshizuka: Computational Geometry and Geographic Information Processing, p
p127 ", Franz Aurenhammer, Atsuyoshi Sugihara: Voronoi diagram, introduction to one basic geometric data structure, ACM Computing S
urveys, Vol. 23, pp345-405 "
See etc.

In step S91, the reference point searching unit 10
Detects data related to a reference data block (template) from one image, for example, the left image. For example, this data is a triangle formed by three section points and a quadrangle including the triangle. Step S92
Then, the position and size of the template are determined based on the distance from the division point of the template and the like. In step S93,
From the template of the left image measured on the image, a search area for performing image correlation processing is determined by the search area setting unit 20. In step S94, the search area determined for the other image, for example, the right image, is scanned using the template of the left image. The setting and scanning of the template and the search area will be described later.

Then, in step S100, an image correlation process between the template and the search area is performed by the arithmetic unit 30 on each of the divided areas. That is, by obtaining a large correlation, a corresponding point of the right image corresponding to the left image is obtained (or a corresponding point of the left image corresponding to the right image), and its three-dimensional coordinates are calculated. The image correlation processing (stereo matching) is performed by the residual sequential test method (SSDA).
Method) and the cross-correlation coefficient method.

Here, a cross-correlation coefficient method suitable for automatic measurement will be described. (Method Using Cross-Correlation Coefficient) FIG. 5 is an explanatory diagram of an input image and a template image. N as shown in the figure
A template image of ₁ × N ₁ pixel is converted to a larger M
Search range (M ₁ −N ₁ +) in the input image of ₁ × M ₁ pixel
1) Move on ² to find the upper left position of the template image such that the correlation coefficient r in the following equation is maximized, and consider that the template image has been searched.

FIG. 6 is a diagram showing an example of 3 × 3 pixels. In the case of this example, a comparison image I having the same or a large correlation value as the template image T of the left image is searched from the right image on the same line (epipor line). That is, the above equation is calculated for each pixel. This is shifted one pixel at a time or by a predetermined number to obtain a pixel having a high correlation value.

[0027]

(Equation 1)

In the present invention, as an example, the reference data block is used as a template image from the left image, and a search is performed in units of a search data block in a search area of the right image.
A search data block which is a matching part of the right image is searched for a reference data block (template) of the left image. The coincident portion is a point where the cross-correlation coefficient is maximum (close to 1).

Since the corresponding points of the left and right images are obtained by the above correlation processing, three-dimensional coordinate values at each measurement point are calculated based on the principle of the stereo method. Here, FIG. 7 shows an explanatory diagram of the stereo method.

(Stereo method) For simplicity, two identical cameras are used, their optical axes are parallel, the distance a from the principal point of the camera lens to the CCD surface is equal, and the CCD is placed at right angles to the optical axis. It is assumed that Let L be the distance between two optical axes (base length).

Points P ₁ (x ₁ , y ₁ ) and P ₂ (x
₂ , y ₂ ) have the following relationship. However, it is assumed that the origin of the entire coordinate system (x, y, z) is taken as the lens principal point of the camera 1. When the image captured by the camera 1 and the left image, the image by the camera 2 and the right image, the image correlation processing, the position of x ₂ relative to the position of the left image x ₁ is obtained (the highest point of the similarity). Therefore, z is obtained from the equation, and x and y are obtained from the equation using the equation.

Returning to the flowchart for measurement, the description will be continued. In step S110, the obtained correlation result between the left and right images is displayed on the display unit 40. In this case, as a result of searching the search data block, the largest correlation coefficient (point having a high similarity) is set as a similar point in the data block, and the correlation results are displayed on the image.

FIG. 8 is a diagram for explaining the display of the correlation result. For example, as shown in the E1 region of FIG. 8A or the area or the point having a low correlation coefficient as shown in FIG. As a display method, an actual correlation coefficient value may be displayed for each color or the like, or a threshold value may be provided in several stages and displayed. If the displayed result is satisfactory in step S120, the process ends.

Next, an additional measurement flow will be described. In step S130, if the displayed result is not satisfactory, the process proceeds to step S150. Step S150
Then, additional measurement is performed by the surveying instrument 1 for an area required for measurement, for example, an area having a low correlation coefficient. If the measurement is a manual measurement, the process proceeds to step S80.

C. Semi-automatic measurement Semi-automatic measurement is performed using an automatic collimating total station. FIG. 9 shows a flowchart for semi-automatic measurement.

In step S510, for example, as shown in FIG. 8, the E1 area where an area having a low correlation coefficient is displayed is pointed to by a pointing device (cursor for a notebook computer, cursor for a pen computer, Pen). In step S520, the position information of the x and y coordinates of the designated point is transferred from the personal computer to the surveying instrument 1, and the surveying instrument 1 is driven by a motor so as to measure the position and measured. Measurement instructions are sent from a personal computer to the surveying instrument 1.
Automatically sent to Alternatively, the measurement instruction may be issued by a person confirming the position of the surveying instrument. If the point to be measured is an unmeasurable point, another measurable point in the area is searched for and measured. Compared with manual measurement, the semi-automatic measurement work using these images can omit the procedure of performing measurement while collimating a surveying instrument, thereby achieving considerable work efficiency. That is, since the image that can be collimated by the surveying instrument reflects only the local region of the target object, collimating to a point where additional measurement is desired is a very difficult and time-consuming operation. However,
If you just click on the image, it's a simple task that anyone can do.

Next, the process proceeds to a step S80. Step S8
At 0, additional measured points are added to create a triangle.
FIG. 10 is an explanatory diagram of adding a measurement point. That is, the connection of the measurement point is performed again by the additional point C7 shown in FIG. In this case, a more detailed triangle may be created for the additional point as shown in the figure, or all triangles may be created again regardless of the previous connection point.

In step 150, when no additional measurement is performed or when there is no selectable point, the triangle is rearranged. FIG. 11 is a diagram illustrating the rearrangement of triangles. For example, when a triangle is created as shown in FIG. 11A, the triangle can be rearranged as shown in FIG. 11B. In addition, if a point measured linearly in step S150 is additionally selected, a triangular shape will be created finer, and both reliability and density will be higher.

Thereafter, the processing after step S91 is executed in the same manner as described above, and in step S100, image correlation processing is performed for a new area. In this case, when a more detailed triangle is created, image correlation processing is performed only on a newly added triangle region. In step S110, the correlation result is displayed on the display unit. If the result is good in step S120, the process ends. If the displayed result is not satisfactory in step S130, the process proceeds to additional measurement again. In step S150, additional measurement is performed on a region in which additional measurement is to be performed while looking at the monitor. The following procedure is the same as that after step S80, and is repeated until it is satisfied.

If no improvement is obtained even if the additional measurement is performed, or if no additional measurement is performed, step S14 is performed.
Go to 0. In step S140, for example, E10 in FIG.
If no good result is obtained in the region of
The elevation value is interpolated from the plane equation of the plane E10 including the measurement points of 1, C2, and C7. For example, C8 in the E10 region
When you want to find, and the coordinate values of _{C8 (x 8, y 8,} z 8)
If, elevation _{z 8} of C8 is, _z 8 ₌ - is calculated by _{(ax 8 + by 8 + d} ) / c --- (6). (Coefficients a, b, c, d are C1, C
2, calculated from the three-dimensional coordinate value of C7).

D. Automatic Measurement Automatic measurement can be performed by using an automatic collimating total station for the surveying instrument. Hereinafter, the automatic measurement will be described.

In step S150, the measurement can be automated by performing the additional measurement in the following procedure. S1
After the image correlation processing of 00 is performed, a threshold value is provided for the correlation coefficient value, and an additional measurement is performed for an area including a correlation coefficient equal to or less than the threshold value. The threshold value may be fixed to, for example, 0.5 or less from the beginning, or may be determined depending on the object. The measurement area is a triangular area including a correlation coefficient equal to or less than a threshold value. For example, in FIG. 8A, the area is E1. The additional measurement point by the surveying instrument may be the position of the center of gravity of the triangle in the triangular area having a low correlation coefficient, or the point having the lowest correlation coefficient in that area, or the center of the area of the low correlation coefficient area. (Center of gravity) may be used. Hereinafter, a method of obtaining the position of the center of gravity of the triangle and the center of gravity of a region having a low correlation coefficient will be described.

First, the center of gravity of the triangle is, for example, shown in FIG.
When the correlation coefficient in the triangle E1 (C1, C2, C5) of (a) is low, the center of gravity C7 of the triangle (see FIG. 10)
Is determined as the next measurement point in the following manner. C1, C2,
The coordinate values of _{C5 C1 (X 1, Y 1} , Z 1), C2
When (X ₂ , Y ₂ , Z ₂ ) and C5 (X ₅ , Y ₅ , Z ₅ ), X = (X ₁ + X ₂ + X ₅ ) / 3− (7) Y = (Y ₁ + Y ₂ + Y ₅ ) / 3 (8) is the XY coordinate of the next measurement selection point C7.

Alternatively, the center of gravity may be obtained from the distribution of the area having a low correlation coefficient shown in FIG.
Moment method). In the moment method, x _g and y _g obtained by the following equations are set as the X and Y coordinates of the additional measurement selection point.
The number of additional measurement selection points is not limited to one, but may be a plurality of points. _{x g = {Σx * {1} -cor (x, y)}} / Σ {1-cor (x, y)} --- (9) y g = {Σy * {1-cor (x, y) }} / Σ {1-cor (x, y)} --- (10) (x g, y g): coordinates of the center of gravity position, cor (x, y):
(X, y) Correlation value on coordinates

As described above, when the additional measurement point is determined, the surveying instrument is controlled to the coordinate value to drive and measure the surveying instrument.
If these points are unmeasurable or difficult points, the vicinity is searched and measured. Since the other steps are computer processing, a series of steps S80 to S150 is automated.

E. Correction by Graphic Display FIG. 12 shows a flowchart for graphic display. Next, a method of performing graphic display based on the result of the correlation processing to confirm and correct will be described. That is, FIG.
The intermediate step S110 can be realized by performing the following processing.

First, in step S102, a triangle is created by connecting the obtained three-dimensional coordinates of each point. Steps
In S106, a graphic display image is created based on the three-dimensional coordinates. If necessary, create a bird's-eye view in which each point of the three-dimensional coordinates and their connection, a contour line, or a triangle and a surface or an image are pasted on the triangle. The figure in which the image is pasted on the triangle may be an orthographic image. If it is not necessary (if it can be visually judged), these processes may not be performed (for example, a triangle may be simply connected). Anything can be used as long as the three-dimensional coordinates can be visually represented. These are displayed on a stereo monitor or a monitor of a personal computer as a graphic display image that can be appropriately judged. In step S106, the created image is displayed on the display unit 40. As a display method, a method which can be easily determined depending on the situation, such as displaying a triangle in a superimposed manner or displaying a correlation coefficient in a superimposed manner, may be used.

Further, in the bird's-eye view image or the like created and displayed in steps S102 to S106, additional measurement can be performed for an area that is erroneous or unnatural. FIG. 13 is an explanatory diagram of the graphic display. For example, even if only the connected triangles shown in FIG. 13A are displayed, if it is different from the actual situation, it is possible to determine a bad part from the display. For example,
If the obtained three-dimensional coordinate value is originally F1 in FIG. 13A, but is incorrectly moved to F1 ′ in FIG. 13B, an error can be clearly confirmed. Further, if a surface or an image is pasted and displayed in FIG. 13B, the difference from the actual situation is more clearly emphasized. Furthermore, if the display is superimposed on the correlation coefficient value, if the area having a low correlation coefficient matches the bad area, the correction area becomes clearer. In this case, the area around F1 in FIG. 13B may be additionally measured manually or semi-automatically.

Thus, if the measurement is a manual measurement, the process proceeds to step S80. On the other hand, if the automatic collimation type total station is used, semi-automatic measurement becomes possible, and step S1
Move to 10. In the semi-automatic measurement, as described above, for example, as shown in FIG. 13, the F1 area that is wrong or unnatural seems to be a cursor in a notebook computer, and a pen in a pen computer. By pointing, the following processing is executed.

F. Search Area Setting Method Next, setting and scanning of a search area, a reference data block, and a search data block by the search area setting unit 20 will be described. The method of setting the search area is exemplified as follows, and will be described in order. 1. 1. Setting of search area: inclusion rectangle 2. Set each data block based on the size of the search area. 3. Set the position and movement step of each data block according to the distance from the reference point. 4. Set the size of each data block according to the distance from the reference point. Set the size of each data block from the correlation value of the reference point

1. Setting of Search Area: Inclusion Rectangle FIG. 14 is an explanatory diagram (1) for setting of a search area. Basically, in the search area of the left and right images, since the mutual orientation is performed in step S30, the vertical parallax is almost removed, and the points CL1 and CR1 of the left and right images in the figure are placed on the epipolar line P1 by CL2 and CR2. Is CL3 on P2,
CR3 is on P3. As shown in the figure, the triangle created from the points measured by the surveying instrument is CL on the left image.
1, CL2, CL3, CR1, CR2, CR on the right image
When 3 is set, the search area is a quadrangular area including each triangle. Here, for example, a case will be described in which a position corresponding to this image is to be searched for from the right image, using the left image T in the region as a reference data block (template).
In this case, if the vertical image is removed from the right image including the search data block as the search area S, the epipolar line P
3 to perform a search. If the vertical parallax is not completely removed, the vicinity of the corresponding epipolar line is also searched. These processes are performed on each line in the inclusion rectangular area including each reference point. With the setting as shown in the figure, an overlapping area is formed with an adjacent triangle, but a triangle area other than the reference point can be surely searched for.

FIG. 15 is an explanatory diagram of the search area. Although the area inside the triangle in FIG. 14 may be used as the search area, it is uncertain whether the corresponding point other than the reference point is within the triangle area. Therefore, as shown in FIG. 15, it is efficient to set a region including a triangular region as a search region.

FIG. 16 is an explanatory diagram (2) for setting a search area. As shown in FIG. 16, when searching for the position of the search data block corresponding to each of the reference data blocks (templates) T1 and T2, the search areas S1 and S2 for each template overlapping the search area on the line P3 are set. It may be provided. Here, as an example, areas within a predetermined range from the centers of the reference data blocks T1 and T2 are set as search areas S1 and S2. In this way, the area closer to each reference point can be clearly searched efficiently, and the reliability can be improved by overlapping. A plurality of search areas may be set for each template. Further, the corresponding point CR of the right image of the reference point
The search areas in the vicinity of 1, CR2, and CR3 are small, and may be set larger as the distance increases (the minimum value or the maximum value may be set as appropriate). Further, the size of the search area (search area) for the data blocks in the search area may be determined as appropriate according to the position, movement step, and size of each data block described later. This way,
It is possible to efficiently search a range including a triangle.

2. Setting each data block based on the size of the search area FIG. 17 is a diagram illustrating the determination of the size of the search area. As shown in FIG. 17A, since the size of each triangle as a search area differs depending on the measurement point, a data block having a size corresponding to each triangle area is used. For example, each data block is determined based on the inclusion rectangle (see FIG. 14) set as the search area as described above. The size of 1 / k with respect to this circumscribed rectangle is defined as a data block. The coefficient k may be fixed in advance, or may be appropriately determined depending on the accuracy and the size of the region to be obtained. As another example, as shown in FIG. 17B, the radius r of the inscribed circle of the triangle may be obtained, and r, 、, の, or the like of r may be used as the data block size. r is obtained by the following equation. r = {(s−o) (s−p) (s−q) / s} − (11) where s = １ ／ (o + p + q) −− (12) where o, p, and q are The length of each of the three sides of the triangle, which is calculated from the measurement coordinates of the three points. The method for obtaining the data block size is not limited to this, and any method may be used as long as it is in accordance with the size of each triangle.

3. Setting Position and Movement Step of Each Data Block According to Distance from Reference Point FIG. 18 is an explanatory diagram (3) of setting a search area.

3.1 Position of Search Data Block with respect to Reference Data Block As shown in FIG. 18A, for example, using T1 on the left image as a template (reference data block),
When it is desired to search for the same image from the right image, the point S1 where the position of T1 is distributed from the right image to CR and CR3 on the epipolar line P3 according to the distances a and b from the reference points CL1 and CL3. Is set as the center of the search area, and the line is searched within a predetermined range. By doing so, the search in the vicinity of the S1 area estimated as the corresponding point of T1 can be performed, so that the processing can be efficiently performed in a short time.

3.2 Position and Movement Step in Search Area For example, as shown in FIG. 18B, the movement step is made coarser as the distance from the center of the search area set in 3.1 to both sides increases. By doing so, the search is performed more efficiently. In other words, a more efficient search is performed by making the area near the center of the estimated area denser and rougher away from the center. A coefficient may be simply set for the roughening ratio, or the position of the center of gravity of the triangle (see 3.2.1)
Alternatively, it may be changed in proportion to the distance from the ratio closer to the two reference points (see 3.2.2).

3.2.1 Triangle Centroid Method A method for determining the position and moving step of each data block from the position of the centroid of a triangle will be described. FIG. 19 is a diagram illustrating the center of gravity of a triangle and a data block.
As shown in FIG. 19A, the position G of the center of gravity of the triangle is equidistant from each reference point, but the distance between the point T other than the center of gravity and the reference point is smaller than the distance from the center of gravity to the reference point. There is a shorter distance. As a result, the position of the center of gravity is set as the maximum size of the data block, and the steps are made smaller as the position approaches the reference point. C1 (x1, y1), C2 (x2, y
2) When C3 (x3, y3) and T (x, y), the distance of each point from T is as follows. T-C1: L1 = {(x-x1) ² + (y-y1) ² } --- (13) T-C2: L2 = {(x-x2) ² + (y-y2) ² } −− (14) T−C3: L3 = {(x−x3) ² + (y−y3) ² } −− (15) Further, the distance from each barycentric position G is G−C1. = determined by G-C2 = G-C3 = Lg = √ {(xg-x1) 2 + (yg-y1) 2} --- (16) equation (13) to (15), the distance L1, L2 ,
From L3, the smallest one is determined, the ratio of the distance to the center of gravity is determined, and the moving step is determined. For example, the step size of the center of gravity G is set to be the maximum,
Assuming that the minimum step size is SMIN, the step size of the point at the minimum distance L (L2 in FIG. 19A) from the template position is: step size = L / Lg × (SGT-SMIN) + SMIN --- (17) ).

3.2.2 Distance method from the ratio of two reference points of the close method: distance from each reference point to T (closer 2
And the step size is variable according to the distance to be searched. For example, Expressions (13) to (15)
Is determined from the ratio of the two smaller distances. FIG.
As shown in (a), when L2 <L3 <L1, L2 =
The point which becomes L3 is set as the maximum step width SMAX, and the step may be set as L2 / L3 × SMAX (18).

The method of determining the position in the search area and the moving step is not limited to this method. 4. Set the size of each data block and search area according to the distance from the reference point. 4.1 Obtain the distance of the image position to be a data block from each reference point, and determine the data block size according to the ratio of the distance. Make it variable. As shown in FIG.
Although the center of gravity G of the triangle is equidistant from each reference point, there is a distance between the point T other than the center of gravity and the reference point that is shorter than the distance from the center of gravity to the reference point. Thus, the data block at the position of the center of gravity is set to the maximum size, and the area is reduced as approaching each reference point. C1 (x1, y
1), C2 (x2, y2), C3 (x3, y3),
Assuming that T (x, y), the distance of each point from T can be obtained in the same manner from Expressions (13) to (15). Further, the distance of each point from the position of the center of gravity G can be obtained from Expression (16). Then, the smallest one among the distances L1, L2, L3 is found, and the ratio of the distance to the center of gravity is calculated.
Determine the data block size. For example, the center of gravity position G
Let GT be the data block size of the data block, and let CT (the data block of C2 in FIG. 19B) be the data block size of the point at the minimum distance L (L2 in FIG. 19B) from the data block position. L / Lg × (GT−CT) + CT −−− (19)

The vertical and horizontal widths of the data block may be constant without changing the vertical direction, and may be only the horizontal direction, as long as the vertical parallax has been substantially removed by the orientation process. By doing so, the data block is small in the vicinity of the reference point on the assumption that the corresponding point is nearby, and it is uncertain whether it is near as the distance increases. Corresponding points can be obtained.

The method of determining the data block size is not limited to this method. In this way, the search can be performed efficiently and quickly.

5. Setting the Size of Each Data Block Based on the Correlation Value of the Reference Point FIG. 20 is a diagram illustrating the size of the data block. Since the vertices (reference points) of each triangle are measured points, they have already been associated. Therefore, various templates (reference data blocks) at each point
A correlation coefficient with respect to the size is obtained (as shown in FIG. 20A), and the template showing the highest correlation value is set as a template image. In the image of the obtained template size, an area near each reference point is searched. The template size obtained at each reference point is changed according to the search position.

For example, as an example, proportional distribution is performed according to the size obtained at each reference point and the distance from each reference point (the two closest points), and the size is made variable according to the search distance (FIG. 20 (b)). C1 (x1, y1), C2
Assuming that (x2, y2), C3 (x3, y3) and template T (x, y), the template size to be obtained can be obtained from Expressions (13) to (15). Therefore, it is determined from the ratio of the two smaller distances obtained. For example, as shown in FIG. 20B, when L2 <L3 <L1, C
The template size of L2 and CL3 is divided according to the ratio of L2: L3, and the size is T.

The vertical and horizontal widths of the data block may be constant without changing the vertical direction, and may be only the horizontal direction as long as the vertical parallax has been substantially removed by the orientation process.

By performing image correlation processing based on triangles generated from reference points as in the present invention, it is not necessary to prepare a large number of images having different resolutions as in the coarse / dense search image correlation method. In addition, highly reliable image correlation processing can be performed. In addition, since simultaneous measurement can be performed by a surveying instrument, measurement can be performed with high reliability and reliability, and it can be quickly and easily measured even in a place where measurement was conventionally time-consuming and could not be performed.

[0067]

According to the present invention, as described above, by using the non-prism TS and the image correlation processing together, it is impossible to perform the conventional measurement such as an inaccessible place or a dangerous place. In this case, it is also possible to perform the non-contact measurement for the region, and the whole measurement can be further speeded up, made more efficient, and more reliable than the case where each method is used alone.

[Brief description of the drawings]

FIG. 1 is an overall block diagram of a measuring apparatus according to the present invention.

FIG. 2 is a flowchart for online measurement.

FIG. 3 is an explanatory diagram of a stereo image.

FIG. 4 is an explanatory diagram of creating a triangle based on measurement points.

FIG. 5 is an explanatory diagram of an input image and a template image.

FIG. 6 is a diagram showing an example in 3 × 3 pixels.

FIG. 7 is an explanatory diagram of a stereo method.

FIG. 8 is an explanatory diagram of displaying a correlation result.

FIG. 9 is a flowchart for semi-automatic measurement.

FIG. 10 is an explanatory diagram of addition of a measurement point.

FIG. 11 is an explanatory diagram of rearrangement of triangles.

FIG. 12 is a flowchart for graphic display.

FIG. 13 is an explanatory diagram of a graphic display.

FIG. 14 is an explanatory diagram (1) of setting a search area.

FIG. 15 is an explanatory diagram of a search area.

FIG. 16 is an explanatory view (2) of setting a search area.

FIG. 17 is an explanatory diagram of a predetermined search area size.

FIG. 18 is an explanatory view (3) of setting a search area.

FIG. 19 is an explanatory diagram of a center of gravity of a triangle and a data block.

FIG. 20 is a diagram illustrating the size of a data block.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 surveying instrument 2 measuring device 3 camera 10 reference point searching unit 20 search area setting unit 30 calculation unit 40 display unit 50 measuring unit 60 bus

 ────────────────────────────────────────────────── ─── Continuing on the front page (72) Inventor Nobuo Takachi 75-1 Hasunumacho, Itabashi-ku, Tokyo F-term in Topcon Corporation 5B057 AA20 BA02 CA13 CB13 DB03 DC03 DC34

Claims (11)

[Claims] 1. A stereo image including at least three measurement points for which position data is known, at least a part of the measurement points is defined as a division point, and a plurality of predetermined division points are determined. A setting unit that sets a search area based on at least three segmentation points, and an operation that performs a correlation process on an image of a corresponding search area on each stereo image based on the search area set by the setting unit And a measuring unit for measuring the coordinates of a point at an arbitrary position from the correlation result obtained by the calculating unit, wherein the calculating unit determines a measuring area in which a new measuring point is required according to the result of the correlation processing. Device for creating stereo images. 2. The stereo image measuring device according to claim 1, further comprising a display unit for displaying the stereo image, wherein the display unit is added in accordance with the information of the measurement area created by the arithmetic unit. A stereo image measuring device for performing a predetermined display in an area requiring measurement. 3. The stereo image measuring device according to claim 2, wherein the display unit is configured to graphically display an area that requires additional measurement on the stereo image, and the graphic display displayed on the display unit. A stereo image measurement device configured to receive the position data when the position data of a new measurement point in the area is measured by an external surveying device based on the above. 4. The stereo image measuring apparatus according to claim 1, wherein said measuring section outputs information of the measuring area created by said calculating section to a surveying instrument having an automatic collimation function, and outputs the information to the surveying instrument. A stereo image measuring device configured to measure a position of a new measurement point in a region indicated by region data, and to receive the measured position data. 5. The stereo image measuring apparatus according to claim 1, wherein the setting section sets a new measurement point in an area that requires detailed division according to the information on the measurement area created by the arithmetic section. After being selected as a segment point, a new search area is set on the stereo image, and the arithmetic unit is configured to perform a correlation process on the image of the new search areas. Stereo image measurement device. 6. A stereo image measuring apparatus according to claim 1, wherein said setting section searches for an inclusion rectangle including a triangle formed by three adjacent points among the obtained division points or measurement points. A stereo image measuring apparatus, wherein the stereo image is set for each stereo image. 7. The stereo image measuring apparatus according to claim 1, wherein the setting unit stores a reference data block in one search area of the stereo image and a search data block in the other search area of the stereo image. A stereo image measuring device, wherein a position or a moving step of each data block is set according to a distance from a section point. 8. A stereo image measuring apparatus according to claim 1, wherein said setting section stores a reference data block in one search area of the stereo image and a search data block in another search area of the stereo image. A stereo image measuring device, wherein a size of each data block is set according to a distance from a division point. 9. The stereo image measuring apparatus according to claim 1, wherein the setting section provides a reference data block in one search area of the stereo image and a search data block in the search area of the stereo image. A stereo image characterized in that a plurality of size data blocks are set in the vicinity of a segment point, a correlation result is obtained, and a size of each data block is determined according to the correlation result. measuring device. 10. The stereo image measuring apparatus according to claim 1, wherein said setting unit stores a reference data block in one search area of the stereo image and a search data block in another search area of the stereo image. A stereo image measuring device, wherein the size of each data block is determined according to the size of a search area. 11. The stereo image measurement device according to claim 1, wherein the setting unit sets a data block smaller than the set search area based on the set search area, A block corresponding to a data block of one stereo image is used as a template, and the other stereo image is scanned at the same vertical position as the template, and a data block corresponding to the template is searched for based on the calculated correlation value. Stereo image measuring device.