JP2005017262A - Three-dimensional surveying system and three-dimensional space object restoring method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a three-dimensional surveying system capable of easily and efficiently performing three-dimensional survey, and to provide a three-dimensional space object restoring method using the same. <P>SOLUTION: Cameras 10L and 10R, on the left side and on the right side, respectively, oriented in the photographing direction shown with an arrow 12, comprise optical systems 14L and 14R, shutters 16L and 16R, and image sensors 18L and 18R. The internal locating factors such as focal distances of the optical systems 14L and 14R and main points of image sensors 18L and 18R are known in advance thanks to calibration. The cameras 10L and 10R are so fixed as to eliminate relative movement by a camera fixing device 22, and the external locating elements such as projection center and attitude angles are uniquely determined. With this three-dimensional surveying system, the left and right image data captured by left and right cameras 10L and 10R, and known internal locating factors and external locating factors, are used to acquire a three-dimensional coordinate of a prescribed point on a target object. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a three-dimensional surveying apparatus and a three-dimensional spatial object restoration method. Specifically, two or three or more cameras are used, and two (three or more) images are simultaneously captured by simultaneously photographing a target. ) Is obtained, and the obtained image is processed by a computer to obtain the three-dimensional coordinates of an arbitrary point of the image.

  Conventional three-dimensional surveying devices measure using a traditional device such as a major, a total station that measures the coordinates of a point with a target, or a laser scanner that measures the distance between the target and itself. It was.

  However, the conventional three-dimensional surveying apparatus is inefficient because the surveying work is complicated and inefficient, the field work time is long, and there is no link means between data effective for creating a three-dimensional model from the survey result. There is a problem that it is difficult to realize a practical three-dimensional surveying apparatus.

  The present invention has been made in view of the above-described conventional problems, and an object of the present invention is to provide a three-dimensional surveying apparatus that can perform three-dimensional surveying simply and efficiently, and a three-dimensional spatial object restoration method using the same. It is in.

  In order to achieve the above object, the present invention is a three-dimensional surveying apparatus, in which two or three or more cameras whose internal orientation elements such as focal length and principal point are known in advance are connected to each other. A camera fixing device that fixes the camera so that there is no relative movement, a camera synchronization device that synchronizes the shooting timing of the camera, and the left image data taken by the left camera in the shooting direction of the camera and the left camera. Stereo image data composed of right image data taken by the right camera, internal orientation elements, and external orientation elements such as camera projection center and posture angle that are uniquely determined by the absence of relative movement of the camera. And an image analysis means for obtaining the three-dimensional coordinates of the target. The camera synchronization device is preferably capable of remote operation.

  The image analysis means obtains a spatial geometric parameter such as the length, width, and height of the target using the three-dimensional coordinates, and displays the three-dimensional image of the target on the display screen using the spatial geometric parameter. It is characterized by doing.

  In addition, a three-dimensional spatial object restoration method is used that uses two or more cameras that are positioned at regular intervals and do not move relative to each other, and that have a known internal orientation element such as focal length and principal point. Shooting at the same time and there is no relative movement of the stereo image data, internal orientation elements, and camera composed of the left image taken by the left camera and the right image taken by the right camera in the shooting direction. 3D coordinates of the target are obtained using external orientation elements such as the camera projection center and posture angle determined uniquely by the three-dimensional coordinates, and the 3D image of the target is displayed on the display screen using the three-dimensional coordinates. It is characterized by displaying.

  Further, the above three-dimensional space object restoration method attaches an identification code to the left image and the right image, recognizes the left image and the right image taken at the same time based on the identification code, and makes a target from a stereo image formed thereby. It is characterized in that three-dimensional coordinates of the object and spatial geometric parameters such as the length, width and height of the target object are obtained. The identification code is configured such that the image data management means attaches to the left image and the right image according to the photographing time, the location, and the camera identification number.

  According to the present invention, when a three-dimensional survey is performed by using a camera that has been calibrated and whose internal orientation element and external orientation element are already known, a known point such as a target is detected in the measurement target space. There is no need to place it, and there is no need to obtain the length of the target in advance. For this reason, three-dimensional surveying can be performed easily and efficiently. The three-dimensional surveying apparatus and the three-dimensional space object restoration method using the same according to the present invention can be used mainly for restoration of a target in a virtual space displayed on a computer display or the like. For example, it can be used for three-dimensional restoration of factories, three-dimensional restoration of buildings and rooms, three-dimensional restoration of archeological survey sites, and location simulation of towns or production sites.

  Hereinafter, the best mode for carrying out the present invention (hereinafter referred to as an embodiment) will be described with reference to the drawings.

  FIG. 1 shows a block diagram of a configuration of a three-dimensional surveying apparatus according to the present invention. In FIG. 1, the three-dimensional surveying apparatus includes two cameras 10L and 10R. In these cameras, the left side is the camera 10L in the shooting direction indicated by the arrow 12, and the right side is the camera 10R. The cameras 10L and 10R include optical systems 14L and 14R configured by lenses and the like, shutters 16L and 16R, and image sensors 18L and 18R configured by a CCD and the like. The shutters 16L and 16R are configured to be simultaneously operated by the camera synchronization device 20 that synchronizes the shooting timings of the cameras 10L and 10R.

  In the two cameras 10L and 10R, the focal lengths of the optical systems 14L and 14R and the internal orientation elements such as principal points at which the image sensors 18L and 18R intersect the optical axes of the cameras 10L and 10R are known in advance. Is used. This internal orientation element is a factor necessary to determine the positional relationship between the projection center and the image, and is used to accurately restore the trajectory of the photographing light beam, that is, the shape of the target. The two cameras 10L and 10R are fixed by the camera fixing device 22 so that there is no relative movement between them. By fixing the left and right cameras 10L and 10R in this way, the relative positional relationship between the cameras 10L and 10R is fixed, so external orientation elements such as the projection center and the posture angle can be uniquely determined. The projection center and posture angle will be described later. These internal orientation elements and external orientation elements are calculated in advance by camera calibration.

  Image data obtained by photographing the target with the cameras 10L and 10R is output as electrical signals from the image sensors 18L and 18R, and the left image data photographed by the left camera 10L is stored in the image memory 24L. The right image data captured by the right camera 10R is stored in the image memory 24R. The left image data and the right image data stored in the image memories 24L and 24R are sent to the processing device 26, and left and right image data photographed at the same time constitute stereo image data. The left image data and the right image data are configured such that the processing device 26 attaches an identification code determined based on the photographing time, the location, and the camera identification number. The processing device 26 in this case corresponds to the image data management means according to the present invention. When the processing device 26 processes the left and right image data constituting the stereo image data to obtain the three-dimensional coordinates, the left and right image data photographed simultaneously by the identification code are identified and used.

  In the processing device 26, the stereo image data, the internal orientation element, and the external orientation element are used to obtain the three-dimensional coordinates of a predetermined point on the target. As described above, since the internal orientation element and the external orientation element are known in advance, these data are stored in the processing device 26. The procedure for obtaining the three-dimensional coordinates will be described later. The processing device 26 in this case corresponds to the image analysis means according to the present invention.

  The processing device 26 uses the obtained three-dimensional coordinates to obtain spatial geometric parameters such as the length, width, and height of the target that is a three-dimensional spatial object. In addition, a three-dimensional image of the target can be generated based on the spatial geometric parameters, and displayed on the display screen of the display device 27 constituted by a computer display or the like for restoration. Thereby, it is possible to restore a three-dimensional image of the target on a computer display or the like and perform various simulations. For this reason, it is possible to very easily study the interior of a factory or building, or the townscape.

  FIG. 2 shows an example of the camera fixing device described above. In FIG. 2, the cameras 10 </ b> L and 10 </ b> R are fixed to both ends of a rod-shaped member 28 that constitutes the camera fixing device 22, and the rod-shaped member 28 is fixed to a column 30. In the present embodiment, the support column 30 is attached to the support portion 32 with casters so that the cameras 10L and 10R can be easily moved. Thereby, 3D surveying can be carried out while moving in a wide place, and the work efficiency of 3D surveying can be improved.

  In the example described above, the case of two cameras (10L, 10R) has been described, but three or more cameras may be used. In this case, two appropriate cameras capable of simultaneously photographing a predetermined point of the target are selected, and the stereo image data is constituted by image data from the selected cameras.

  FIG. 3 shows an example of a coordinate system when a three-dimensional survey is performed by the three-dimensional survey apparatus shown in FIG. The purpose of the three-dimensional survey is to obtain geometric information (three-dimensional coordinates) of the target in the three-dimensional space through geometric transformation from the acquired image information. In order to obtain geometric information of a target in a three-dimensional space from a digital image, it is necessary to create a relational equation between a point on the target and an image point corresponding to the point. The coordinate system shown in FIG. 3 is used in creating this relational equation.

In FIG. 3, A is a point on the target to be measured, and the coordinates (X _A , Y _A , Z with respect to arbitrary coordinate axes X, Y, Z passing through an arbitrary origin O in the real space. _A ). The three-dimensional surveying apparatus according to the present embodiment is an apparatus that obtains these coordinates (X _A , Y _A , Z _A ) from acquired stereo image data. Further, in the present embodiment, there are two cameras that shoot the point A in the shooting direction, and FIG. 3 shows only the image sensors 18L and 18R among the components of the camera. Elements are omitted. Left and right cameras 10L, 10R projection center _S L is an external orientation elements of, _S coordinates of _R, relative to the origin O, respectively _{(X SL, Y SL, Z} SL) and _{(X SR, Y SR,} Z _SR ). The sensor surface of the image sensor 18L has a posture rotated by ω _L with respect to the X axis passing through the origin O around the projection center S _L , φ _L with respect to the Y axis, and κ _{L with} respect to the Z axis. The sensor surface of the image sensor 18R is rotated by ω _R with respect to the X axis passing through the origin O around the projection center S _R , φ _R with respect to the Y axis, and κ _{R with} respect to the Z axis. It is a posture. In this case, the angles ω _L , φ _L , κ _L and ω _R , φ _R , κ _R are posture angles that are external orientation elements of the cameras 10L, 10R. As described above, since the two cameras 10L and 10R are fixed by the camera fixing device 22 so as not to move relative to each other, the projection centers and posture angles of the cameras 10L and 10R are calibrated. Can be determined uniquely by performing in advance.

Next, main points 34L and 34R, which are points where the sensor surfaces of the image sensors 18L and 18R intersect with the optical axis, are slightly shifted from the centers of the image sensors 18L and 18R. This is because it is extremely difficult to make the optical axis of the camera completely coincide with the center position of the image sensors 18L and 18R in manufacturing the camera. At this time, the coordinates of the principal points 34L and 34R on the x _R y _R and x _L y _L coordinate axes on the sensor surface with the center positions of the image sensors 18L and 18R as the origin are (x _0L , y _0L ), (x _0R , y _0R ). Further, when the target is photographed with the left and right cameras, the point A on the target forms an image with the points a _L and a _R on the image sensors 18L and 18R, and the above x _L y of the a _L and a _R. The coordinates on the _L and x _R y _R coordinate axes are (x _L , y _L ) and (x _R , y _R ). The distance between the main point 34L of the left camera and the projection center S _L and the distance between the main point 34R of the right camera and the projection center S _R are the focal lengths f _L and f _R of the optical system of the left camera and the right camera, respectively. It is. The internal orientation elements such as the principal points 34L and 34R and the focal lengths f _L and f _R described above are also obtained in advance by performing calibration. In the example of FIG. 3, negative images are formed on the image sensors 18L and 18R.

ここで、上記各式を次のように変換する。

ただし、

であり、また、

である。 From the geometric relationship shown in FIG. 3 described above, the following relational equation is obtained. This relational equation calculation process is executed by the processing device 26 described above.

Here, the above equations are converted as follows.

However,

And also

It is.

By solving the equation (1) or (2), the coordinates (X _A , Y _A , Z _A ) in the real space of the point A on the target to be measured can be obtained. In this case, the above equation (1) or (2) has four equations, one more than the unknowns to be obtained (X _A , Y _A , Z _A ), so that the accuracy of the solution can be improved by the method of least squares. it can.

Next, using the three-dimensional coordinates (X _A , Y _A , Z _A ) of the point A on the target calculated as described above, the spatial geometric parameter can be obtained by the following equation. This spatial geometric parameter is used to restore the three-dimensional shape of the target as described above.

により、点Ａ１から点Ａ２までの距離ｄを求めることができる。 First,

Thus, the distance d from the point A1 to the point A2 can be obtained.

により、点Ａ１とＡ２とを通過する直線と点Ａ３との距離Ｗを求めることができる。このような、点と点との距離及び点と線との距離を組み合わせることにより、目標物の長さ、幅、高さなどの空間幾何パラメータ決定できる。 next,

Thus, the distance W between the straight line passing through the points A1 and A2 and the point A3 can be obtained. By combining the distance between the points and the distance between the points and the line, the spatial geometric parameters such as the length, width, and height of the target can be determined.

FIG. 4 shows a process flow in the case of performing three-dimensional surveying using the three-dimensional surveying apparatus shown in FIG. In FIG. 4, the L parameter relating to the left camera 10L stored in the processing device 26, that is, the projection center (X _SL , Y _SL , Z _SL ), the attitude angle (ω _L , φ _L , κ _L ), the focal length f _L, and The principal point (x _0L , y _0L ) and the R parameter relating to the right camera 10R, that is, the projection center (X _SR , Y _SR , Z _SR ), the posture angle (ω _R , φ _R , κ _R ), the focal length f _R And the principal point (x _0R , y _0R ) are initialized (S1, S2).

  Next, the processing device 26 acquires the left image data and the right image data captured by the left and right cameras 10L and 10R and stored in the image memories 24L and 24R (S3 and S4). Since the acquired image data is attached with the above-described identification code, it is determined whether or not the identification codes match (S5). If the identification codes match, it can be determined that the left and right image data are captured simultaneously. This determination is important because right and left image data taken at the same time cannot be used unless three-dimensional coordinates can be obtained correctly. If the identification codes do not match, the process returns to steps S3 and S4, and re-executes from the acquisition of the image data.

The coordinates (x _L , y _L ) and the right (R) image sensor of the image point a _L of the measurement point A imaged on the left (L) image sensor 18L by the left image data and the right image data with the matching identification codes. The coordinates (x _R , y _R ) of the image point a _R of the measurement point A imaged on 18R are obtained (S6, S7). Next judges image sensors 18L, whether the image point a _L was determined coordinates on 18R and the image point a _R is the image point if That same measurement point A corresponds (S8). This determination may be performed by human eyes by displaying the left image data and the right image data on the display, or may be performed by software for determining the coincidence between the image point and its vicinity. If the image point does not correspond, the process is re-executed from steps S6 and S7.

If an image point a _L and the image point a _R is determined to correspond, left and right cameras 10L, the 10R parameter, introduces each value stored in the processing unit 26 (S9, S10) The above equations (1) and (2) are established using this and the image points a _L and a _R (S11). This equation is solved to obtain the three-dimensional coordinates of the predetermined measurement point A (S12). By repeating the steps described above, all the predetermined coordinates on the target in the space to be measured can be obtained, and three-dimensional survey can be performed.

  FIG. 5 shows a block diagram of a configuration example of the camera synchronization device described above. In FIG. 5, when a photographing signal is transmitted by radio communication from the photographing signal oscillator 36, the camera synchronization device 20 receives this signal by the receiver 38, and uses this as a trigger to generate a synchronization signal generating device by the signal processor 40. An operation signal is input to 42. When the operation signal is input, the synchronization signal generator 42 generates the operation signals of the shutters 16L and 16R of the left and right cameras 10L and 10R, and the shutters 16L and 16R are simultaneously operated. Thereby, the photographing timings of the left and right cameras 10L and 10R can be matched by remote control by the photographing signal oscillator 36.

  Note that the camera synchronization device 20 does not necessarily have to be configured to operate by remote control. For example, the shooting signal oscillator 36 may be integrated with the camera synchronization device 20 and the camera synchronization device 20 and the shooting signal oscillator 36 may be connected by an electric cord.

It is a block diagram of the structure of the three-dimensional survey apparatus concerning this invention. It is a figure which shows the example of a camera fixing device. It is a figure which shows the example of the coordinate system at the time of implementing three-dimensional surveying with the three-dimensional surveying apparatus shown by FIG. It is a flowchart of the process in the case of performing a three-dimensional survey using the three-dimensional survey apparatus shown in FIG. It is a block diagram of the structural example of a camera synchronizer.

Explanation of symbols

10L, 10R camera, 12 arrows, 14R, 14L optical system,
16R, 16L Shutter, 18R, 18L Image sensor, 20 Camera synchronization device, 22 Camera fixing device, 24R, 24L Image memory, 26 Processing device, 27 Display device, 28 Bar-shaped member, 30 Prop, 32 Support section with casters, 34R, 34L principal point,
1 imaging signal oscillator, 38 receiver, 40 signal processor, 42 synchronization signal generator.

Claims (6)

Two or more cameras with known internal orientation elements such as focal length and principal point,
A camera fixing device for fixing the cameras so that there is no relative movement between them;
A camera synchronization device for synchronizing the shooting timing of the camera;
Among the cameras, stereo image data composed of left image data taken by the left camera in the shooting direction and right image data taken by the right camera synchronized with the left camera, the internal orientation element, Image analysis means for obtaining the three-dimensional coordinates of the target using external orientation elements such as the camera projection center and posture angle that are uniquely determined by the absence of relative movement of the camera;
A three-dimensional surveying apparatus comprising:   2. The three-dimensional survey apparatus according to claim 1, wherein the camera synchronization device can be remotely operated.   3. The three-dimensional surveying apparatus according to claim 1, wherein the image analysis means obtains a spatial geometric parameter such as a length, a width, and a height of the target using the three-dimensional coordinates, and the spatial geometry. A three-dimensional surveying apparatus, wherein a three-dimensional image of a target is displayed on a display screen according to a parameter. Take two or more cameras that are placed at regular intervals and do not move relative to each other, and use a camera with known internal orientation elements such as focal length and principal point,
Among the cameras, there is no stereo image data composed of a left image taken by the left camera and a right image taken by the right camera in the shooting direction, and there is no relative movement of the internal orientation element and the camera. Using the external orientation elements such as the camera projection center and posture angle that are uniquely determined, the three-dimensional coordinates of the target are obtained.
A three-dimensional spatial object restoration method, wherein a three-dimensional image of a target is displayed on a display screen using the three-dimensional coordinates.   5. The three-dimensional space object restoration method according to claim 4, wherein an identification code is attached to the left image and the right image, and based on the identification code, the left image and the right image taken simultaneously are recognized, and configured thereby. A three-dimensional spatial object restoration method, wherein a three-dimensional coordinate of a target and a spatial geometric parameter such as a length, width, and height of the target are obtained from a stereo image.   6. The three-dimensional space object reconstruction method according to claim 5, wherein the identification code is attached to the left image and the right image by an image data management means according to an imaging time, a point, and a camera identification number. Restoration method.

Images