JP3227797B2 - Imaging process identification device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a measuring device in construction and maintenance of a large-scale structure such as a plant.

[0002]

2. Description of the Related Art In order to streamline plant construction, Japanese Patent Application Laid-Open
A construction process using an as-built database described in Japanese Patent Application Laid-Open No. 2-209562 has been proposed. In this process, the following techniques are used.

A technique for estimating the three-dimensional position of an object by dynamically comparing a geometric model with a monocular image is as follows.
There are the following inventions. A two-degree-of-freedom matching algorithm based on the concept of pattern mechanics has been developed. The matching algorithm was then extended to three degrees of freedom.

Therefore, as a method of calibrating a camera for measurement using an image, there are a rating method of a non-measuring camera by Murai et al. And a camera calibration by Sasawa.

In the former rating method, the position of the photographic camera
Calibration was performed on the two coefficients of the polynomial representing the posture, the shift of the principal point at the center of the photograph, the focal length, and the distortion of the lens, and the six coefficients of the polynomial representing the curvature of the film. As a method, linearize the observation equation, find the normal equation by the least squares method based on the position of the reference point on the ground and the position of the reference point on the photograph, and correct the approximate value from the solution. I do. By repeating this operation, the unknown parameters were converged.

In the latter calibration, using a CCD camera, two coefficients of a polynomial representing the position and orientation of the camera, the shift of the principal point at the center of the photograph, the focal length, and the distortion of the lens are calibrated. As a method, based on the position of the reference point on the ground and the position of the reference point on the screen, a normal equation was obtained using the least squares method, and unknown parameters were obtained by solving the normal equation.

[0007]

In construction work, the shape of a structure constantly changes as the work progresses. In fact, when constructing a structure such as a plant, the design information created in advance is materialized as the construction progresses,
Manufacture, delivery and installation are performed. In order to support such dynamic production consistently, it is necessary to constantly manage the situation of a constantly changing structure as design data. Such dynamic design information is called an as-build database.

Now, the accuracy of the as-built database is as follows:
Generally, it is defined by the following two points. That is,
(1) the shape error of the object, and (2) the mounting error of the object. Of these, (1) allows factory management. Therefore, in the present invention, only the problem (2) is considered.

Now, in order to generate and maintain an as-built database, the structure must be frequently measured. Here, focusing on the fact that the design data can be used as prediction data of the position of the object, an interactive system as shown in FIG. 3 can be configured.

This system forms a design database as-built in the following procedure. First, an object is photographed using a camera installed at a work site, and displayed on a screen of a workstation. Next, the design database is searched, position / posture information (object description) of the target object is extracted from the data (class) of the measurement target object, and a wireframe image of the object is generated using a computer graphics technique. This wire frame image is displayed so as to overlap the object image. The operator corrects the position of the object by comparing the wire frame image and the object image on the screen. Using the updated position data, object data (instance) corresponding to reality is created. By repeating this method for each object, the situation of the structure, which changes every moment, is managed as dynamic design information.

When the object model and the object image are collated by the method shown in FIG. 3, the object position is determined based on the camera position. For such a position / posture based on the camera coordinates, an automatic measurement algorithm has been developed and its accuracy has been confirmed.

In the prior art, since the camera is calibrated based on the position of the reference point on the image, the measurement accuracy of the reference point on the screen does not exceed the resolution of the screen.
Therefore, in order to obtain an accuracy higher than the resolution of the screen, many reference points must be measured.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an imaging process identification apparatus and a procedure for accurately determining a position, a magnification, and a shift of the center of a screen of a photographing means by using pattern matching which can be accurately measured by a statistical effect.

[0014]

In order to achieve the above object, an imaging process identification apparatus according to the present invention comprises: an imaging means for imaging an object to generate a shadow image; a measuring means for measuring a posture of the imaging means; Object information storage means having position information, image storage means having image information, wireframe image generation means for generating a wireframe image of an object, an object image shown in an image obtained from the image storage means, A collating unit that measures the position of the photographing unit by collating the wireframe image obtained from the wireframe generating unit, a collation display unit that displays the shadow image and the wireframe image, a position / posture of the photographing unit, An imaging process information storage device having information on a shift between a center of a screen of a collation display unit and an optical axis of the imaging unit, and magnification of the imaging unit; Deviation of the optical axis center of the screen and the image pickup means of the comparison display means obtained from the position and the imaging process information storage unit of the imaging unit resulting from,
Parameter calculating means for calculating the amount of correction of the position of the photographing means, the magnification, and the deviation between the center of the screen of the collation display means and the optical axis of the image taking means, from the magnification of the image taking means.

[0015]

The position of the photographing means with respect to each magnification can be accurately measured by providing the collating means and superposing the wire frame image and the object image. Based on the position data of the photographing means for each magnification, the displacement of the position of the photographing means, the magnification, and the center of the screen are obtained.

[0016]

First, a mathematical model of an imaging process is derived.

Since the position and orientation of the camera change every measurement, an apparatus combining a camera and a surveying instrument was used.

The imaging device produces an image through the following steps. First, in the light receiving stage, the image undergoes a geometric transformation. This conversion is determined by the position and size of the lens and the light receiving element. Next, the image is converted into an electrical signal and displayed on a display. At this stage, an electrical deformation occurs in the image. In the present invention, the above-described image generation process is modeled as follows.

In the present apparatus, the center of the surveying instrument and the center of the camera are structurally different. Using this deviation (camera offset), the conversion from the measurement coordinates to the camera coordinates is represented by the following equation.

[0020]

(Equation 1)

[0021] However, α, β, γ, e mx, e my, e mz is a camera offset. In the present apparatus, the angle offset (α, β) is designed to be zero. Assuming that the lens has no distortion, an image on the light receiving element is formed according to the following equation.

[0022]

(Equation 2)

Here, _crx and _cry are lens magnifications in the x and y directions. Here, considering the shift of the light receiving element with respect to the lens, the image output from the light receiving element can be expressed by the following equation.

[0024]

(Equation 3)

Here, e _sx and e _sy represent the shift of the light receiving element. The output image of the light receiving element is stored in the memory of the workstation through photoelectric conversion and electrical conversion. In such a conversion, a displacement of the center of the image and a change in the magnification occur. Such image distortion can be expressed by the following equation.

[0026]

(Equation 4)

Here, e _ix and e _iy represent the shift between the center of the light receiving element and the image center, and c _ix and c _iy represent the magnification between the light receiving element and the image. The image of the device is stored on the memory of the workstation and compared with a wireframe image generated on the same memory. Therefore, the distortion of the cathode ray tube does not need to be considered in principle. To summarize the above,
The following equation is obtained.

[0028]

(Equation 5)

[0029]

(Equation 6)

[0030] Here, x, y, z: object position relative coordinate system fixed to the surveying instrument _{x m, y m, z m} : object position to a camera coordinate system x _i, y _i: object picture of the screen coordinate system The position of. _{Moreover, e mx, e my, e} mz: position of the camera about the surveying instrument coordinate system alpha, beta, gamma: the attitude c _x of the camera about the surveying instrument coordinate system, c _y: the combined lens, a photoelectric conversion, an electrical conversion magnification e _x, e _y: the displacement of the optical axis relating to the screen coordinate system, all of which are parameters that vary depending on the assembly and installation state of the imaging apparatus. In the construction and maintenance work environment,
Strict control of the image pickup device in consideration that it is not practical, the present invention, these _{e mx, e my, e mz} , α, β,
gamma, handles _{c x, c y, e x} , and e _y as unknown parameters.

Now, e _mx , e in the unknown parameters of _equation (5)
_{Since my} and _emz depend on the maintenance conditions of the measurement system, accuracy control is not easy. In addition, since c _x and cy in _Equation 6 depend on the electrical displacement of the image, it is expected that variations will occur depending on the combination of the camera and the workstation. However, the number 5 alpha, beta, gamma and 6 of e _x, e _y is
Since it is determined only by the assembling accuracy of the camera alone, the error is considered to be relatively small. Therefore, in the present invention,
alpha, a beta e _x, to be identified, including the e _y. The alpha, a beta e _x, the error due to the inclusion in the e _y is
Under normal measurement conditions, the deviation angle between the camera optical axis and the surveying instrument optical axis can be expected to be sufficiently small. Focusing on this fact, replacing the number 5 and 6 of the e _x in e _tx defined by the following equation. Further, γ is obtained by superposing a wire frame on an object image.

When the unknown parameters are arranged as described above, the model of the imaging process is simplified as follows.

[0033]

(Equation 7)

[0034]

(Equation 8)

At this time, the unknown parameter is θ = (e _mx , e
_{my, e mz, c x,} c y, e x, the e _y).

[0036]

(Equation 9)

Here, α is a deviation angle. At this time, there is a measurement error er expressed by
α appears.

[0038]

(Equation 10)

Here, θ _s is the viewing angle of the camera, and z _m is the distance between the camera and the target object. Now, θ _s = 20 degrees, z
_{When m} = 5000 mm and θ = 0.1 degrees, the measurement error is 0.27 mm. Therefore, the error due to the replacement is considered to be within an allowable range in normal construction work.

The equations (7) and (8) can be used to simulate the imaging process. In other words, by superimposing the object image and the wire frame of the object generated using Equations 7 and 8, the relative position and orientation of the object and the imaging device can be detected. In the present invention, using this, the unknown parameters of Equations 7 and 8 are identified.

When the wire frame of the object and the object image are superimposed, the following two principles are established.

(1) An object image whose position and orientation are known matches a wireframe image generated under the same conditions.

(2) Due to a change in the camera position / posture, a unique pattern is generated in the shift between the wire frame image and the object image (see FIG. 4).

Referring to FIG. When the position / posture of the camera changes with respect to the object, a specific image distortion occurs according to the movement mode. This distortion is called an optical flow. If the optical flow is available, the movement amount f of the camera can be calculated backward. However, z is the optical flow, u _m, u _r represents the diffusion image of the image and the object model. Γ is a gain matrix.

Based on this principle, a method for identifying the imaging process models 7 and 8 will be considered. The problem is,
Given x, y, z and x _i , y _i (that is, shape, position,
Using an object image whose posture is known), the parameter θ =
(e _mx , e _my , e _mz , c _x , c _y , e _x , e _y ).

Equations 7 and 8 show that x _i and y _i depend nonlinearly on unknown parameters. In such a situation, parameter estimation becomes difficult. Indeed parameters e _mx and e _x, because wireframe image to changes in e _my and e _y is to slip the same trend can not isolate the time of verification. Therefore, in the present invention, an attempt was made to perform estimation while decomposing each unknown parameter by iterative calculation.

FIG. 6 shows the principle of the iterative calculation introduced in the present invention.

FIG. 5 shows only the relationship of xz for simplicity. The repetitive calculation is performed in the following procedure. First, it is given an initial value 0 in e _x, to measure the object 1 & 2. However, the object 1 is located near and the object 2 is located far away. At this time, each object image should appear at a position represented by the following equation.

[0049]

[Equation 11]

[0050]

(Equation 12)

The following relationship is established between c _x and e _mx and between c _x and e _mz .

[0052]

(Equation 13)

[0053]

[Equation 14]

However,

[0055]

(Equation 15)

[0056]

(Equation 16)

[0057]

[Equation 17]

[0058]

(Equation 18)

Is as follows. Now, as described above, when the collation operation is performed while changing the magnification c _x with respect to the two objects in the distance,
Equations 13 and 14 produce two graphs. Now, the magnification c _x corresponding to the intersection of these two graphs should match if the initially set _ex is a true value. In other words, mismatches c _x represents a mismatch e _x. Focusing on this point, when the c _x do not match, correct the e _x according to the following equation.

[0060]

[Equation 19]

[0061] However, z _m1, z _m2 is the distance from the camera center to the model, n represents represents a number of repetitions of operations for obtaining the e _x. Using the corrected ex, the _equation 11 and below are repeated. If this repetition converges, all parameters have been estimated.

One embodiment of the present invention will be described with reference to FIG.
The example of the present invention includes a photographing unit 1, a measuring unit 2, and a measuring unit 2.
An imaging process information storage device 6 connected to the imaging process information storage device 6, a wire frame image generation unit 3 connected to the imaging process information storage device 6, a collation unit 4 connected to the imaging unit 1 and the wire frame image generation unit 3, A parameter calculation means 5 connected to the means 4 and the wireframe image generation means 3; an object information storage means 7 connected to the wireframe image generation means 3; and an image information storage connected to the imaging means 1 and the collation means 4. A device provided with the device 8 is used.

The photographing means 1 photographs a target object. The measuring means 2 is combined with the photographing means 1 and measures the attitude of the photographing means 1. The imaging process information storage device 6 has information on the position and orientation of the imaging means, the shift of the principal point at the center of the screen, and the focal length. In the wireframe image generating means 3,
A wire frame image is generated from the upper part of the imaging process and the object information. The collating means 4 measures the position of the photographing means by collating the video with the wire frame image. The parameter calculating means 5 calculates the position of the photographing means, the focal length and the correction amount of the center of the screen from the position of the photographing means and the focal length at that time. In the object information storage means 7, the shape of the object
It has location information. The image information storage means 8 has image information.

Using the imaging process identification apparatus shown in FIG. 1, the identification of the imaging process is performed in accordance with the procedure shown in FIG.

Known objects 1 and 2 at different distances from the photographing means
Is photographed (step 1). Next, the attitude of the photographing means is measured by the measuring means (step 2). Then, the matching unit and the matching display unit obtain γ by overlapping the wire frame with the object image (step 3). Further, in the matching means and the matching display means, the object 1 is measured while the displacement e _x of the center of the screen is fixed and the magnification c _x is changed. Specifically, for each magnification c _x ,
Transverse values e _mx position of the imaging means, for measuring the depth direction values e _mz position of the imaging means. This operation corresponds to obtaining the lateral value e _mx of the position of the photographing unit and the value _emz of the position of the photographing unit in the depth direction from the displacement e _{x of} the center of the screen and the magnification c _{x in Expression} (11). Step 4). In step 5, the operation of step 4 is performed on the object 2. As a result of this operation, in Expression 12, the lateral value e _mx of the position of the image capturing means and the value _{emz in} the depth direction of the position of the image capturing means are obtained from the displacement e _x of the screen center and the magnification c _x. .

In step 6, the parameter calculation means uses the least squares method for the objects 1 and 2 to calculate the magnification c _x
And normal values between the values e _{mz in} the depth direction of the position of the photographing means, and solutions of these normal equations are obtained.
This solution is used as an estimated value of the magnification c _x . This operation is performed for object 1
(2) This corresponds to finding a ₁ and b ₁ of Equation 14 for 2 and finding solutions to these equations.

Step 7: The parameter calculation means uses the least squares method for the objects 1 and 2 to calculate the magnification c _x
And a normal equation between the depth value e _mz of the position of the photographing means and the estimated value c of the magnification for the objects 1 and 2
The horizontal values e _mx1 and e _mx2 of the position of the photographing means with respect to _x are obtained. In this work, a ₂ and b ₂ of Expression 13 are obtained for the objects 1 and 2, and the estimated value c _x of the magnification in these equations is obtained.
This corresponds to obtaining a value in the horizontal direction of the position of the photographing means with respect to.

Step 8: The difference between the lateral direction values e _mx1 and e _mx2 of the position of the photographing means with respect to the estimated magnification value c _x is 0.5 mm.
If not, the calculation ends. Otherwise in the case with number 19, corrects the deviation e _x of the screen center, the process returns to step 4.

Regarding the parameters related to the measurement of yz, the imaging process is identified by the procedure from step 4 to step 8.

[0070]

According to the present invention, by pattern matching,
The accuracy of the position of the photographing means, the magnification, and the displacement of the center of the screen can be improved. Further, the number of measurement points required for parameter estimation is small, and the work at the time of photographing is reduced.

[Brief description of the drawings]

FIG. 1 is an explanatory diagram showing an imaging process identification device according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram showing an imaging process identification procedure according to an embodiment of the present invention.

FIG. 3 is an explanatory diagram showing an as-build database generation system.

FIG. 4 is an explanatory diagram showing a structure of an optical flow.

FIG. 5 is an explanatory diagram showing an identification process.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Photographing means, 2 ... Measurement means, 3 ... Wire frame generation means, 4 ... Collation means, 4A ... Collation display means, 5 ... Parameter calculation means, 6 ... Imaging process information storage device, 7 ... Object storage device, 8 ... Image storage.

──────────────────────────────────────────────────続 き Continued on the front page (58) Fields surveyed (Int. Cl. ⁷ , DB name) G01B 11/00-11/30 E04G 21/00

Claims (1)

(57) [Claims] A photographing means for photographing an object to generate an image, a measuring means for measuring a posture of the photographing means, an object information storing means having information on the shape and position of the object, and an image having image information Storage means, a wireframe image generation means for generating a wireframe image of the object, by comparing the object image in the image obtained from the image storage means and the wireframe image obtained from the wireframe generation means, Collating means for measuring the position of the photographing means,
The collation display means for displaying the shadow image and the wireframe image, the position and orientation of the photographing means, a shift between the center of the screen of the collation display means and the optical axis of the imaging means, and information on the magnification of the imaging means. An imaging process information storage device, a position of the imaging device obtained from the collation device, a shift between a screen center of the collation display device obtained from the imaging process information storage device and an optical axis of the imaging device, the imaging device An imaging process identification apparatus, comprising: a parameter calculating unit that calculates a correction amount of a shift between an optical axis of the imaging unit and a position of the imaging unit, a magnification, and a center of a screen of the collation display unit from the magnification of the imaging unit.