JP2007010419A - Three-dimensional shape of object verifying system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable three-dimensional shape measurement which dispenses with the repetition of movement, setting and calibration. <P>SOLUTION: The system includes cameras 11a, 11b, measurement devices 1a, 1b with total stations (TS) 13a, 13b, and a controller 3. The controller 3 acquires data that reference points A, B are measured by the TS 13a, 13b; conducts calibration that determines the photographing positions of cameras 1a, 1b; acquires image data that the camera 11a, 11b substantially and simultaneously has photographed a predetermined object respectively; produces the three-dimensional data of the object based on the acquired image data and the photographing positions of the cameras 11a, 11b; compares the produced three-dimensional data with reference data; and provides outputs responding a result of the comparison. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a three-dimensional measurement technique, and more particularly to a three-dimensional measurement technique using a camera-integrated total station.

Conventionally, a camera-integrated total station is known as a three-dimensional measuring device (for example, Patent Document 1). If this camera-integrated total station is used, it is possible to easily measure the three-dimensional shape of the object to be photographed.
JP 2005-91298 A

  Conventionally, since measurement is performed using a single camera-integrated total station, it has been necessary to repeatedly move, set, and calibrate the camera-integrated total station. This is not so much a problem when surveying while moving across a large site consisting of multiple measurement areas, but when measuring in a relatively small area only in one place, move, set and calibrate. It had to be repeated and was troublesome.

  Further, when the measurement is repeated while staying at one place, the measurement object itself may be replaced, or the three-dimensional shape of the object may change.

  Accordingly, an object of the present invention is to monitor or inspect the three-dimensional shape of an object using a camera-integrated total station.

  Another object of the present invention is to detect a change in the three-dimensional shape of an object over time using a camera-integrated total station.

  An object three-dimensional shape verification system according to an embodiment of the present invention includes a plurality of cameras, and a plurality of total stations (hereinafter referred to as TSs) each having a predetermined positional relationship with each of the plurality of cameras. Based on data obtained by measuring two or more reference points in each TS, calibration means for determining the shooting position of each camera and image data obtained by shooting the predetermined object substantially simultaneously by the plurality of cameras are acquired. Means for generating three-dimensional data of the object based on the acquired image data and the photographing position of each camera determined by the calibration means, and storing reference data relating to the object to be photographed A storage unit, comparing means for comparing the generated three-dimensional data with reference data stored in the storage unit, and the ratio And means for performing an output corresponding to the comparison result by the means.

  Here, “substantially simultaneous” means not only strictly simultaneous but also within a certain fixed time.

  In a preferred embodiment, when the generated three-dimensional data does not satisfy a predetermined requirement with respect to the reference data as a result of the comparison by the comparison unit, the calibration unit performs recalibration and performs recalibration. When the comparison unit does not satisfy the predetermined requirement even when the comparison unit again compares the reference data with the three-dimensional data of the object generated based on image data captured later, the output unit Information indicating a predetermined alarm may be output.

  In a preferred embodiment, when performing the recalibration, the calibration means determines whether or not the relative positional relationship between the two or more reference points has been displaced from the previous calibration. You may make it perform.

  In a preferred embodiment, the apparatus further comprises means for storing the three-dimensional data generated by the generating means in the storage unit in order to use it as reference data for the three-dimensional data generated by the next and subsequent photographing. Also good.

  In a preferred embodiment, the object to be photographed is an industrial product, and the reference data may be product data relating to a feature point of the industrial product. In this case, the comparison means extracts the feature point of the industrial product that is the object based on the three-dimensional data, and compares it with the product data.

  A three-dimensional shape verification apparatus according to an embodiment of the present invention includes a measuring apparatus including a plurality of cameras and a plurality of total stations (hereinafter referred to as TS) each having a predetermined positional relationship with each of the plurality of cameras. 3D shape verification apparatus for an object connected to Then, data obtained by measuring two or more reference points with each TS is acquired, and calibration data for determining the shooting position of each camera; and image data obtained by shooting the predetermined object substantially simultaneously with the plurality of cameras. A means for obtaining the object, a means for generating three-dimensional data of the object based on the obtained image data and the photographing position of each camera determined by the calibration means; and reference data relating to the object to be photographed. A storage unit for storing; a comparison unit that compares the generated three-dimensional data with reference data stored in the storage unit; and a unit that performs output according to a comparison result by the comparison unit.

  Hereinafter, a three-dimensional shape measurement system according to an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a diagram showing an overall configuration of a three-dimensional shape measurement system according to the first embodiment of the present invention.

  This system includes two or more measuring devices 1 (1a, 1b) and a controller 3 connected to the input device 5 and the display device 7, and measures the three-dimensional shape of the photographing object 9. Although FIG. 1 shows a case where there are two measuring devices 1, three or more measuring devices 1 can be used. In the present embodiment, when the three-dimensional shape of the photographing object 9 changes with the passage of time, the change is detected.

  Each measuring device 1 has the same configuration. First, the configuration of the measuring apparatus 1 will be described.

  The measuring apparatus 1 can use, for example, a motor-driven camera-integrated total station. That is, the measuring apparatus 1 includes the digital still camera (hereinafter referred to as “camera”) 11 (11a, 11b), the total station (hereinafter referred to as “TS”) 13 (13a, 13b), and the orientations of the cameras 11 and TS13. A motor 15 (15a, 15b) for changing and a support frame 17 (17a, 17b) are provided.

  TS13 is a measuring device that measures a distance to a survey target point and a direction represented by a horizontal angle and a vertical angle with respect to a predetermined basic direction where horizontal angle = 0 and vertical angle = 0. The TS 13 outputs data indicating the measured distance and direction (hereinafter referred to as “TS data”).

  The camera 11 is fixed to the TS 13 with a certain positional relationship. Accordingly, the camera 11 and the TS 13 are integrally changed in direction with respect to the support frame 17. Therefore, when the shooting direction of the camera 11 is changed, the shooting position of the camera and the shooting direction with respect to the basic direction can be obtained from the TS data output from the TS 13.

  The direction of the camera 11 and the TS 13 is changed by driving the motor 15. The motor 15 is driven by the controller 3 as will be described later. In addition, when the motor 15 is not mounted in the measuring device 1, the operator may change the directions of the camera 11 and the TS 13 manually.

  The controller 3 acquires data of images taken by the cameras 11a and 11b of the measuring devices 1a and 1b and TS data that are measurement results of the TSs 13a and 13b, and performs predetermined image processing based on these data. The three-dimensional shape of the photographing object 9 is measured and the shape change is detected.

  In order to measure the three-dimensional shape of the photographing object 9 and to detect the shape change in this system, first, the measuring devices 1a and 1b are respectively installed at arbitrary positions where the photographing object 9 can be seen. The positions where the measuring devices 1a and 1b are installed are the shooting positions. Further, two or more reference points A and B that can be seen from the photographing position are determined in advance.

  FIG. 2 is a diagram showing a detailed functional configuration of the controller 3.

  The controller 3 is configured by a general-purpose computer system, for example, and each component or function in the controller 3 described below is realized by executing a computer program, for example.

  First, the controller 3 is set with a posture control unit 311 that controls the direction of the camera 11 and the TS 13, and a division setting unit 312 that sets a divided shooting region when shooting an image of a shooting target divided into a plurality of parts. A division definition storage unit 313 that stores information indicating the divided imaging region;

  The attitude control unit 311 drives the motor 15 to control the direction of the camera 11 and the TS 13 that are integrated. For example, the posture control unit 311 controls the motor 15 based on an input from the input device 5 as described later, or the camera 11 moves in a predetermined direction according to the division definition information stored in the division definition storage unit 313. The motor 15 is controlled so that it faces.

  The division setting unit 312 sets a division imaging region using a setting screen described below when the imaging object 9 is divided into a plurality of images.

  FIG. 3 is a diagram illustrating a divided shooting area, and FIG. 4 is a diagram illustrating an example of a setting screen for the divided shooting area.

  For example, as shown in FIG. 3, a plurality of divided photographing areas 91 to 96 are set when the photographing object 9 is larger than the photographing range of the camera 11 and the whole is not photographed once. Then, as will be described later, the entire three-dimensional data of the photographing object 9 is obtained based on the image data of each of the divided photographing regions 91 to 96.

  A setting screen 100 shown in FIG. 4 is a screen for an operator to set a divided shooting area while referring to the images of the cameras 1a and 1b. When this screen 100 is displayed on the display device 7, an instruction from the operator is received by operating the input device 5.

  The setting screen 100 is provided with an area 110 for displaying an image of the camera 1a, an area 120 for displaying an image of the camera 1b, and direction instruction keys 115 and 125 for changing the directions of the cameras 11a and 11b. Here, for example, the image of the region 110 is a reference image when performing stereo processing described later, and the image of the region 120 is a reference image. When the operator operates the direction instruction keys 115 and 125 with the input device 5, the directions of the cameras 1 a and 1 b change under the control of the attitude control unit 311. Here, when the direction of the camera 1a serving as the standard image is changed, the direction of the camera 1b serving as the reference image may be changed in conjunction with the reference image. When the divided shooting area setting button 130 is pressed, TS data indicating the direction of the cameras 11a and 11b at that time is acquired from the TS 13a and 13b, and the acquired data indicating the direction of the cameras 11a and 11b is defined as the divided shooting area definition. The information is stored in the division definition storage unit 313 as information.

  Referring again to FIG. 2, the controller 3 further includes the following configuration in order to perform image processing of the photographing object 9. That is, the controller 3 includes a calibration processing unit 331 that performs calibration for obtaining the positional relationship between the measuring devices 1a and 1b, a photographing position storage unit 332 that stores the photographing positions of the measuring devices 1a and 1b obtained by calibration, An image data storage unit 333 that stores image data captured by the cameras 11a and 11b, a stereo processing unit 334 that performs stereo processing of the image data captured by the cameras 11a and 11b, and a shooting object 9 calculated by the stereo processing. A three-dimensional data storage unit 335 that stores the three-dimensional data, a shape change detection unit 336 that extracts a shape change of the photographing object 9 based on the measured three-dimensional data, and a captured image and a three-dimensional image on the display device 7. The display control part 337 for displaying etc., and the alarm output part 340 are provided.

  The calibration processing unit 331 acquires TS data when the TSs 13a and 13b measure the reference points A and B, respectively, and obtains a positional relationship between the photographing positions where the measuring devices 1a and 1b are installed. Then, three-dimensional local coordinates at the photographing site are obtained for each photographing position, and are stored in the photographing position storage unit 332 as photographing position data.

  The stereo processing unit 334 performs stereo processing based on the shooting position data stored in the shooting position storage unit 332 and the two pieces of image data of the shooting target 9 taken by the cameras 11a and 11b substantially simultaneously. Then, based on the distance image of the photographing object 9 obtained by the stereo process, three-dimensional data of the photographing object 9 is obtained and stored in the three-dimensional data storage unit 335. When the shooting object 9 is divided and shot, stereo processing is performed on the corresponding image data to obtain three-dimensional data for each of the divided shooting areas, and these are combined to combine the entire shooting object 9. Obtain 3D data.

  The shape change detection unit 336 compares the three-dimensional data (reference data) of the reference photographing target 9 with the three-dimensional data (comparison target data) of the photographing target 9 serving as a reference, and the photographing target 9 Extract shape changes. For example, the reference data may be data of the photographing object 9 photographed at a time different from the comparison target data. That is, in this system, the photographing of the photographing object is repeated regularly or irregularly, and the three-dimensional data based on the image data photographed this time is used as the comparison target data, and the three-dimensional data based on the image data photographed before the previous time is used as a reference. It may be data. The reference data may be fixed based on image data taken at a reference time, or may be image data taken immediately before. Thereby, when the shape of the photographing object 9 changes with time, this can be detected.

  By the way, the position of the camera 11 may be displaced for some reason after the calibration is performed. In such a case, correct three-dimensional data of the photographing object 9 cannot be obtained even if stereo processing is performed using the image data photographed by the camera 11 in the shifted state. Therefore, the shape change detection unit 336 detects the shape change even when such a positional deviation occurs in addition to when the shape of the photographing object 9 changes, even if there is actually no shape change. . Therefore, in order to prevent such erroneous detection, recalibration is performed after detecting the shape change. At this time, the shape change detection unit 336 has a recalibration flag (not shown), and sets this flag at the start of recalibration. In the recalibration, the positions of the reference points A and B used by the calibration processing unit 331 during the previous calibration are stored, and the posture control unit 311 controls the direction of the camera based on this, automatically. You may make it perform.

  The shape change detection unit 336 performs comparison with the reference data again using the three-dimensional data generated based on the image data captured after recalibration as the comparison target data. At this time (the recalibration flag has already been set), if a change in the shape of the photographing object 9 is detected again, it is assumed that the shape of the photographing object 9 has truly changed, and the alarm output unit 340 issues a predetermined alarm. Output.

  At the time of recalibration, the calibration processing unit 331 determines whether or not the relative positional relationship (distance) between the reference points A and B has changed when the reference points A and B are measured by the TS 13. To do. When the relative positional relationship (distance) between the reference points A and B changes, the alarm output unit 340 outputs a predetermined alarm. This is because if the position of one or both of the reference points A and B is deviated, it is impossible to continue measurement in the present system, and this is detected.

  Next, a processing procedure in the system according to the present embodiment having the above configuration or function will be described.

  FIG. 5 is a flowchart relating to preprocessing for performing measurement and shape change detection processing of the photographing object 9 in the present system.

  First, after the operator installs the measuring devices 1a and 1b at the photographing positions, the calibration processing unit 331 performs calibration based on TS data obtained by measuring the reference points A and B with the TSs 13a and 13b (S1). Thereby, the shooting positions of the cameras 11a and 11b at the shooting site are specified. Next, using the setting screen 100 shown in FIG. 4, the division setting unit 312 sets a divided photographing region (S2). The direction of the cameras 11a and 11b at the time of shooting each divided shooting area is acquired from the TS 13a and 13b as the definition information of the divided shooting area determined in this way, and stored in the split definition storage unit 313. Then, at the end of the preprocessing, reference data for determining the presence or absence of a shape change is stored in the three-dimensional data storage unit 335 (S3). That is, the photographing object 9 is photographed by the cameras 11a and 11b, and this image data is stereo-processed by the stereo processing unit 334 to generate three-dimensional data.

  FIG. 6 is a flowchart of the measurement of the object to be photographed and the shape change detection process performed after the pre-processing is completed. This shape change detection process may be performed at a constant cycle or may be performed irregularly (for example, at an arbitrary timing).

  First, the posture control unit 311 acquires the definition information of the divided shooting areas from the division definition storage unit 313, and controls the directions of the cameras 11a and 11b (S11). Then, the cameras 11a and 11b photograph the photographing object 9 substantially simultaneously (S12). The above photographing is performed for all divided photographing regions (S3).

  When shooting of all the divided shooting areas is completed, the stereo processing unit 334 performs stereo processing using the corresponding image data of the cameras 11a and 11b based on the shooting position information. This stereo processing is performed for all divided shooting areas, and three-dimensional data of the entire shooting target 9 is generated (S14).

  The shape change detection unit 336 detects the change in the shape of the photographing object 9 by comparing the three-dimensional data generated here with the reference data already generated in the preprocessing (S15). If there is no change in shape (S16: No), the recalibration flag indicating whether or not recalibration has been executed is cleared, and the process ends (S19).

  On the other hand, when a change in the shape of the photographing object 9 is detected (S16: Yes), it is determined whether or not a recalibration flag is set (S17). If the recalibration flag has been set (S17: Yes), the detected shape change is not an influence of the positional deviation of the cameras 11a and 11b. That is, since the shape of the subject 9 has changed, the alarm output unit 340 outputs an alarm and ends the process (S18).

  Here, all kinds of alarms are possible. For example, an error message is output to the display device 7, an e-mail is sent to a predetermined e-mail address, or a call is made to a predetermined telephone number. May be.

  If the recalibration flag is not set in step S17 (S17: No), this flag is set, and the calibration processing unit 331 is instructed to execute recalibration (S21). When the calibration processing unit 331 detects a change in the relative distance between the reference points A and B by this recalibration, the process shifts to step S18 as a positional deviation of the reference points (S22: Yes). If no reference point displacement is detected (S22: No), the process returns to step S11 to continue the process.

  Thereby, the change of the three-dimensional shape of the photographing object 9 can be detected. In addition, as a suitable example of the photographing object 9 of the present embodiment, there is a terrain such as a steep cliff. By continuously measuring the shape of the cliff with this system and detecting the shape change, it is possible to detect the cliff collapse itself or the subtle cliff shape change that is a precursor.

  Next, a second embodiment of the present invention will be described. In the following description regarding the present embodiment, a description will be given focusing on the configuration different from the first embodiment, and the same configuration as the first embodiment may be denoted by the same reference numeral and description thereof may be omitted.

  In the system of the present embodiment, it is inspected whether or not the object to be photographed has a predetermined shape, and a non-predetermined shape is detected. Therefore, an imaging object suitable for the system of this embodiment is, for example, an industrial product. That is, it is possible to easily inspect whether or not the industrial product is completed according to a predetermined standard by this system.

  The overall configuration of this system is the same as that of the first embodiment shown in FIG.

  FIG. 6 shows a functional configuration of the controller 30 according to the present embodiment.

  Similarly to the controller 3 of the first embodiment, the controller 30 is also configured by a general-purpose computer system, for example, and each component or function is realized by executing a computer program, for example.

  The controller 30 compares the product data storage unit 342 indicating the standard of the product that is the photographic object 9 with the three-dimensional data of the photographic object 9 measured by this system. A product inspection unit 341 for determining whether a certain product conforms to the standard.

  The product data stored in the product data storage unit 342 is data indicating appearance characteristics of the product. For example, the dimensions of the entire product, the positional relationship between predetermined feature points, and the like may be used. More specifically, it may be a positional relationship (for example, distance) between feature points such as protrusions and holes that are apparent from the outside.

  The product inspection unit 341 analyzes the three-dimensional data stored in the three-dimensional data storage unit 335 and extracts information for comparison with the product data described above. For example, the product inspection unit 341 calculates the dimensions of the object 9 to be photographed, and extracts the positional relationship between feature points such as predetermined protrusions and holes. Then, the feature information extracted from the three-dimensional data is compared with the product data to determine whether or not they match.

  If they do not match, the recalibration is performed in the same manner as in the first embodiment, and then the effect of the positional deviation of the camera 11 can be removed by comparing again.

  In the present system, product inspections can be continuously performed by replacing the imaging objects placed in the imaging range of the camera 11 one after another. At this time, it is necessary that the feature point to be extracted of the object to be photographed be installed with the feature pointed toward the camera 11. However, the installation position may be in an area that can be imaged by the camera 11 and the direction only needs to be able to image a feature point, and the strictness thereof is not required.

  The above-described embodiments of the present invention are examples for explaining the present invention, and are not intended to limit the scope of the present invention only to those embodiments. Those skilled in the art can implement the present invention in various other modes without departing from the gist of the present invention.

  For example, three or more camera-integrated total stations may be used, or three or more reference points for calibration may be provided.

1 is a diagram illustrating an overall configuration of a three-dimensional shape measurement system according to a first embodiment of the present invention. 3 is a diagram illustrating a detailed functional configuration of a controller 3. FIG. It is a figure which shows the division | segmentation imaging | photography area | region when dividing | segmenting and image | photographing a to-be-photographed object. It is a figure which shows an example of the setting screen of a division | segmentation imaging area. It is a flowchart regarding the pre-process of measurement of an imaging target and shape change detection. It is a flowchart of a measurement of an imaging object and a shape change detection process. It is a figure which shows the function structure of the controller 30 which concerns on the 2nd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1a, 1b ... Measuring device, 3, 30 ... Controller, 5 ... Input device, 7 Display device, 9 ... Shooting object, 11a, 11b ... Camera, 15a, 15b ... Motor, 17a, 17b ... Support frame, 100 ... Division Area setting screen, 311 posture control unit, 331 ... calibration processing unit, 334 ... stereo processing unit, 335 ... three-dimensional data storage unit, 336 ... shape change detection unit.

Claims (8)

Multiple cameras,
A plurality of total stations (hereinafter referred to as TS) each having a predetermined positional relationship with each of the plurality of cameras;
Calibration means for determining a shooting position of each camera based on data obtained by measuring two or more reference points in each TS;
Means for respectively acquiring image data obtained by photographing a predetermined object substantially simultaneously by the plurality of cameras;
Means for generating three-dimensional data of the object based on the acquired image data and the photographing position of each camera determined by the calibration means;
A storage unit that stores reference data related to the photographing object;
A comparison means for comparing the generated three-dimensional data with reference data stored in the storage unit;
Means for performing output according to a comparison result by the comparison means, and a three-dimensional shape verification system for an object. As a result of comparison by the comparison unit, when the generated three-dimensional data does not satisfy a predetermined requirement with respect to the reference data, the calibration unit performs recalibration,
If the predetermined means is not satisfied even when the comparison unit compares the reference data with the three-dimensional data of the object generated based on the image data captured after recalibration, 2. The three-dimensional shape verification system for an object according to claim 1, wherein the output means outputs information indicating a predetermined alarm.   The calibration means, when performing the recalibration, determines whether or not the relative positional relationship between the two or more reference points has been displaced from the previous calibration. The three-dimensional shape verification system for an object according to claim 2.   4. The apparatus according to claim 1, further comprising means for storing the three-dimensional data generated by the generating unit in the storage unit in order to use the three-dimensional data as reference data for the three-dimensional data generated by the subsequent shooting. The three-dimensional shape verification system for an object described in 1. The object to be photographed is an industrial product,
The reference data is product data relating to feature points of the industrial product,
The said comparison means extracts the said feature point of the industrial product which is the said object based on the said three-dimensional data, and compares with the said product data, The Claim 1 characterized by the above-mentioned. 3D shape verification system for objects. A three-dimensional shape verification device for an object connected to a plurality of cameras and a measuring device having a plurality of total stations (hereinafter referred to as TS) each having a predetermined positional relationship with each of the plurality of cameras. And
Calibration means for acquiring data obtained by measuring two or more reference points in each TS and determining a shooting position of each camera;
Means for respectively acquiring image data obtained by photographing a predetermined object substantially simultaneously by the plurality of cameras;
Means for generating three-dimensional data of the object based on the acquired image data and the photographing position of each camera determined by the calibration means;
A storage unit that stores reference data related to the photographing object;
A comparison means for comparing the generated three-dimensional data with reference data stored in the storage unit;
Means for performing output according to the comparison result by the comparison means, and a three-dimensional shape verification apparatus for an object. A method for verifying the three-dimensional shape of an object using a measuring device including a plurality of cameras and a plurality of total stations (hereinafter referred to as TS) each having a predetermined positional relationship with each of the plurality of cameras. There,
A calibration step of acquiring data obtained by measuring two or more reference points in each TS and determining a shooting position of each camera;
Each of the plurality of cameras acquiring image data obtained by photographing a predetermined object substantially simultaneously;
Generating three-dimensional data of the object based on the acquired image data and the shooting position of each camera determined by the calibration;
Comparing the generated three-dimensional data with reference data relating to the object to be photographed;
And a step of performing output according to the comparison result. For verification of a three-dimensional shape of an object using a measuring device having a plurality of cameras and a plurality of total stations (hereinafter referred to as TS) each having a predetermined positional relationship with each of the plurality of cameras. A computer program,
When executed on a computer,
A calibration step of acquiring data obtained by measuring two or more reference points in each TS and determining a shooting position of each camera;
Each of the plurality of cameras acquiring image data obtained by photographing a predetermined object substantially simultaneously;
Generating three-dimensional data of the object based on the acquired image data and the shooting position of each camera determined by the calibration;
Comparing the generated three-dimensional data with reference data relating to the object to be photographed;
And a step of performing output according to the comparison result.

Images