WO2016135856A1 - Three-dimensional shape measurement system and measurement method for same - Google Patents

Abstract

To provide a three-dimensional shape measurement system and measurement method capable of shape measurement of the entirety of an object of measurement when only a portion of the shape of the object of measurement can be obtained using distance measurement, a three-dimensional shape measurement system according to the present invention is provided with a distance measurement unit (100) for measuring the distance to a plurality of points on an object of measurement, an imaging unit (101) for acquiring image data by imaging a measurement space including the object of measurement, and a shape reconstruction device (21) for determining a plurality of image data feature points corresponding to each of a plurality image data items imaged from different directions by the imaging unit, determining a provisional relative posture and provisional shape of unknown scale from position information for the image data of the plurality of feature points, and reconstructing the shape of the object of measurement by calculating the three-dimensional coordinates of the provisional relative posture and provisional shape of unknown scale on the basis of the distances to the feature points measured by the distance measurement unit.

Description

Three-dimensional shape measurement system and measurement method thereof

The present invention relates to a system and a measurement method for measuring a three-dimensional shape of a space.

As a means for measuring the shape of the space, there is a 3D sensor that can measure the three-dimensional shape of the space by measuring the distance to the space components. By using this 3D sensor, it is possible to acquire the shape of the entire space quickly and with high accuracy compared to hand measurement. In addition, by using this 3D sensor, it is possible to measure the shape in a non-destructive manner from a point away from a high place or a dangerous place that cannot be reached by human hands.

This 3D sensor can be broadly classified into an active method in which light is applied to a measurement object and measurement is performed and a passive method in which measurement is performed without applying light to the measurement object.
The active 3D sensor irradiates the measurement target with laser light or LED light, and calculates the distance to the measurement target from the time until the irradiation light returns. The three-dimensional coordinates of the surface of the object are obtained by adding the position coordinates of the 3D sensor to the calculated distance information of the object.

Patent Document 1 discloses a high-resolution range image in a TOF (Time-Of-Flight) range image sensor that irradiates a measurement object with irradiation light and obtains a range image from the phase difference between the irradiation light and reflected light. And a technique for preventing a decrease in distance accuracy due to saturation of the light receiving element due to the influence of shot noise, ambient light, or the like when realizing a high frame rate. Specifically, the technique disclosed in Patent Document 1 receives reflected light from an object at the time when there is no light emission from a light source, and emits light that is not easily affected by environmental light based on frequency analysis of the reflected light. The illumination light having the frequency is obtained, the irradiation light having the optimum emission frequency is selected from a plurality of prepared light sources, and the reflected light of the illumination light of the selected light source is received to generate a distance image to the object. This suppresses the influence of ambient light during light reception.

As another active type 3D sensor, Patent Document 2 discloses that a measurement target object is rotationally scanned with a pulse laser, receives reflected light from the measurement target object, and irradiates the target with laser light. Technology for calibrating a laser scanner with a calibration target having a reflective part having a high reflectance with a known shape in a laser scanner that measures distance by measuring the time from reflection to return Is disclosed.

The passive method 3D sensor includes, for example, two imaging devices as disclosed in Patent Document 3, and the relative orientation between the imaging devices obtained by calibration in advance, and the both imaging devices. There is a stereo camera system that obtains the three-dimensional coordinates of a predetermined part from the position where the predetermined part to be measured is reflected by the principle of triangulation.

International Publication No. 2010/021090 JP 2010-151682 A JP 2013-253799 A

The laser scanner as described in Patent Document 2 can perform highly accurate three-dimensional distance measurement. However, it takes a long time to measure by scanning one point at a time. Was not suitable to do.
A distance image sensor as described in Patent Document 1 can measure a distance at a high speed in order to detect a two-dimensional distance image. However, when the distance to the object is long because the irradiation light diffuses, May not be able to measure distance because it cannot obtain reflected light.
Since the stereo camera type 3D sensor as described in Patent Document 3 can capture an object shape at a distant distance, a shape at a distant place in space can also be detected. However, in order to obtain high measurement accuracy, it is necessary to reduce the quantization error of the image sensor, and a high-definition image sensor is required. In addition, since a stereo camera type 3D sensor requires two image sensors, a cost problem arises.

In order to solve the above-described problems, a three-dimensional shape measurement system of the present invention captures image data by imaging a measurement space including a distance measurement unit that measures distances to a plurality of points of the measurement object and the measurement object. A plurality of feature points of image data corresponding to each of a plurality of image data captured from different directions by the imaging unit, and a scale is obtained from position information in the image data of the feature points of the plurality of points. An unknown provisional relative posture and provisional shape are obtained, and a three-dimensional coordinate value of the scale-unknown provisional relative posture and provisional shape is calculated based on the distance to the feature point measured by the distance measuring unit, and the measurement object And a shape restoring device that restores the shape.

In order to solve the above problem, the measurement method of the three-dimensional shape measurement system for measuring the shape of the measurement space including the measurement object according to the present invention captures a plurality of image data by imaging the measurement space from different directions. Obtaining a plurality of feature points of image data corresponding to each of the plurality of image data, obtaining a temporary relative posture and a provisional shape whose scale is unknown from position information in the image data of the feature points, Based on the distance to the measurement object corresponding to the feature point, the three-dimensional coordinate values of the temporary relative posture and the temporary shape whose scale is unknown are calculated to restore the shape of the measurement object.

According to the three-dimensional shape measurement system and the measurement method of the present invention, even when only a part of the shape of the measurement target can be obtained by the distance measurement unit, the shape measurement of the entire measurement target can be performed.

It is a figure which shows the functional block of the three-dimensional shape measurement system of an Example. It is a figure which shows an example of the hardware constitutions of the three-dimensional shape measurement system of an Example. It is a figure which shows the processing flow of the three-dimensional shape measurement system of an Example. It is a figure which shows the example of a measurement of the three-dimensional shape measurement system of an Example. It is a figure which shows an example of the feature point which the feature point recognition part extracted. It is a figure which shows the example of the correspondence process of the feature point by a correspondence calculation part. It is a detail of the process of a temporary relative attitude | position and a shape estimation part. It is a figure explaining the detailed process of a scale adjustment part. It is a figure explaining the restoration method of the shape of a measuring object. It is the figure which showed the correspondence of a point and the epipolar geometry of an imaging part when one point of a three-dimensional space is imaged from a different direction. It is a figure explaining the process of a distance data utilization area | region setting part. It is a figure explaining the process of a feature point utilization area | region setting part. It is a figure which shows the apparatus external appearance of the three-dimensional shape measurement system of another Example. It is a figure explaining the example of a measurement of a distance measurement part. It is the figure which showed the scanning surface of the laser beam of FIG. It is a figure which shows the apparatus external appearance of the three-dimensional shape measurement system of another Example. It is a figure which shows the processing flow of the three-dimensional shape measurement system of another Example.

(Example 1)
First, the outline of the measurement method of the three-dimensional shape measurement system of the present embodiment will be described.
The three-dimensional shape measurement system of the embodiment includes an imaging unit such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor that captures two-dimensional image information obtained by viewing a measurement object from different directions. A distance measuring unit that measures the distance to a plurality of points of the measurement object by irradiating laser light or infrared light, and image information of the measurement object viewed from different positions photographed by the imaging unit and the distance A three-dimensional coordinate calculation unit that calculates three-dimensional distance coordinates of the measurement object based on distance information of a plurality of points measured by the measurement unit.

More specifically, the imaging unit shoots the measurement object from different positions by moving a shooting point or the like, or at least two image sensors are provided at different positions like a stereo camera, and the measurement object Shoot objects from different positions. At this time, since the distance between the two image sensors is not used for the distance calculation, the distance between the image sensors may not be fixed like a stereo camera.
The imaging unit may capture an RGB color image, or may capture a monochrome image or an infrared image.

The distance measuring unit measures the distance from the phase lag of the reflected light from the measurement object by detecting the reflected light with a laser scanner that measures the distance with laser light or diffusing and irradiating infrared light with a two-dimensional light receiving element. It is possible to use a distance image sensor that detects a distance by a TOF (Time-Of-Flight) method or an irradiation pattern method.
Further, although details will be described later, since the distance information to all the measurement objects in the measurement space is not required, the laser scanner performs two-dimensional scanning on a plane having the same height as the scanner to detect one of the measurement objects. The two-dimensional shape of the part may not be measured, and a plurality of points may be measured without performing continuous scanning.
A more specific configuration example of the three-dimensional shape measurement system of the present embodiment will be described later.

In the three-dimensional shape measurement system of the embodiment, the imaging unit can capture the entire shape of the measurement object in the measurement space, whereas the distance measurement unit can measure the measurement space because the irradiation light does not reach the measurement object. The distance information of the entire shape of the measurement object may not be measured.
However, the three-dimensional shape measurement system of the embodiment includes a scale obtained from the three-dimensional coordinates of the feature points whose distances can be measured among the feature points of the image of the entire shape of the measurement object in the measurement space imaged by the imaging unit. By using a shape whose scale is unknown from the feature points, the three-dimensional coordinates of the feature points whose distance could not be measured are calculated. Thereby, the three-dimensional shape measurement system of the embodiment restores the measurement object.
Furthermore, in the three-dimensional shape measurement system of the embodiment, the measurement points other than the feature points of the image of the entire shape of the measurement object in the measurement space imaged by the imaging unit are measured only on the epipolar line based on the principle of epipolar geometry. By searching, corresponding points are found and the shape is restored.

Hereinafter, the three-dimensional distance coordinates of the measurement object are calculated based on image information obtained by photographing the measurement object from different positions by the imaging unit and distance information of a plurality of points on the measurement object measured by the distance measurement unit. Details of the three-dimensional coordinate calculation unit will be described.
FIG. 1 is a diagram illustrating functional blocks of the three-dimensional shape measurement system according to the embodiment.

The three-dimensional shape measurement system of the present embodiment includes a distance measurement unit 100 that measures a distance to a measurement object in a measurement space, an imaging unit 101 that measures surrounding color information, and a measurement data storage that stores the measured data. Part 102 is provided. In the measurement data storage unit 102, the distance data 103 measured by the distance measurement unit 100 and the image data 104 captured by the imaging unit 101 are stored as a pair for a plurality of times.
The distance measuring unit 100 and the imaging unit 101 only need to measure at the same timing, and the measurement interval need not be constant.

Furthermore, in the three-dimensional shape measurement system of this embodiment, the feature point recognition unit 105 that extracts the feature points of the image data 104 and the feature points extracted by the feature point recognition unit 105 between a plurality of pieces of image data 104. Is provided.
Here, the feature points of the image data 104 are determined by detecting corners based on changes in hue and luminance values. More specifically, it can be calculated by a Harris corner detection method or the like. Alternatively, a contour line is extracted, and a vertex or an inflection point can be obtained from the discrete curvature.

Further, the three-dimensional shape measurement system according to the present embodiment has an unknown scale between the measurement points imaged by the imaging unit 101 between the plurality of image data 104 based on the feature point correspondence 114 calculated by the correspondence calculation unit 106. A provisional relative posture / shape estimation unit 107 is provided for estimating a provisional relative posture 108 and a provisional shape 109 composed of three-dimensional coordinates with unknown scales related to feature points.
The scale unknown means that the absolute length and size related to the posture and shape are not determined, but the distance relationship and the relative size relationship between the postures and within the shape are correct.

As described above, the three-dimensional shape measurement system according to the present embodiment uses the temporary relative posture / shape calculation unit 120 whose scale is unknown, which includes the feature point recognition unit 105, the correspondence calculation unit 106, and the temporary relative posture / shape estimation unit 107. Based on the feature points of the plurality of image data 104, the temporary relative posture 108 and the temporary shape 109 whose scale is unknown can be obtained corresponding to each of the image data 104.

Further, in the three-dimensional shape measurement system of this embodiment, the distance measurement unit 100 measures the scale-unknown temporary relative posture 108 and the temporary shape 109 of the feature points obtained by the scale-unknown temporary relative posture / shape calculation unit 120. An actual shape calculation unit 121 that obtains the entire scale from the distance data 103 and restores the entire shape data 113 is provided.

Specifically, the actual shape calculation unit 121 includes a scale adjustment unit 110 and a shape restoration unit 112.
The scale adjustment unit 110 obtains the overall scale from the temporary shape 109 of the feature point unknown by the provisional relative posture / shape calculation unit 120 of unknown scale and the distance data 103 measured by the distance measurement unit 100, and the scale. Accordingly, the scale of the temporary relative posture 108 is adjusted.
Then, the entire shape data 113 is restored by the shape restoration unit 112.
In addition, when the feature points are not dense, the shape restoration unit 112 may perform shape restoration also on portions other than the feature points by multipolar epipolar constraints.

When the entire shape is restored by the shape restoration unit 112, the correct relative posture 111 of the scale obtained by the scale adjustment unit 110, the temporary shape 109, and the feature point use region setting unit among the feature point correspondences 114. 116 of feature points and weights within the range set by 116 and distance data and weights within the range set by the distance data utilization region setting unit 115 among the distance data 103 measured by the distance measurement unit 100. The entire shape data 113 is restored from the pair. Details of processing contents of the shape restoration unit 112 will be described later.

FIG. 2 is a diagram illustrating an example of a hardware configuration of the three-dimensional shape measurement system according to the present embodiment. The three-dimensional shape measurement system according to the present embodiment includes a distance measurement unit 100 configured by a laser scanner, a distance image sensor, and the like, and a measurement unit 20 including an imaging unit 101 configured by a CCD, a CMOS image sensor, or the like. The shape restoration device 21 performs shape restoration processing based on the measurement content of the measurement unit 20 to obtain a three-dimensional shape of the measurement space.

The shape restoration device 21 includes a central processing unit (CPU) 202 that controls the entire device, a reception device 203 that receives measurement data from the measurement unit 20 and transfers the measurement data to the distance data 103 and the image data 104 in the memory 23, and a distance An input / display device 204 including a data use region setting unit 115 and a feature point use region setting unit 116, a processing unit 22 that performs the three-dimensional measurement process of the embodiment, measurement data of the distance data 103 and the image data 104, The memory 23 stores processing data of the temporary relative posture 108, the temporary shape 109, the relative posture 111, the shape data 113, and the feature point correspondence 114.

The processing unit 22 includes a feature point recognition unit 105, a correspondence calculation unit 106, a temporary relative posture / shape estimation unit 107, a scale adjustment unit 110, and a shape restoration unit 112 that perform processing with reference to the image data 104 in the memory 23. ing. Each processing unit may have a dedicated hardware configuration or may have a dedicated processor and a software process. Further, the CPU 202 may perform program processing.

FIG. 3 is a diagram illustrating a processing flow of the three-dimensional shape measurement system according to the present embodiment.
First, the three-dimensional shape measurement system according to the present embodiment measures the distance data 103 and the image data 104 (see FIG. 3) by the measurement unit 20 (see FIG. 3), and performs a measurement data acquisition process S300 stored in the memory 23. Do it. At this time, the distance data 103 and the image data 104 are configured in pairs. The distance and image measurement data are measured at different positions or orientations, and the measurement target is measured from different directions. The measurement data may be at least two points. However, as the measurement data increases, the amount of information regarding the position of each feature point increases. Therefore, as the number of measurement data increases, the measurement range and measurement accuracy improve.

In step S301, it is determined whether the number of measurement data necessary for the three-dimensional shape measurement process is obtained. If the number of measurement data is insufficient (No in S301), the process returns to step S300.
If the number of data is sufficient (Yes in S301), the process proceeds to step S302.

In step S302, the feature point recognition unit 105 performs feature point recognition processing for recognizing feature points on the image for each piece of image data 104 obtained by imaging the measurement object from a plurality of directions (S302).
Then, the correspondence calculation unit 106 performs correspondence calculation processing for associating the same feature points among a plurality of image data (S303).

Next, the temporary relative posture / shape estimating unit 107 obtains the temporary relative posture 108 and the temporary shape 109 whose scales are unknown from the position information of the same feature points between the plurality of image data 104 (S304).
Next, using the ratio of the correct distance of the scale obtained from the distance related to the temporary shape 109 whose scale is unknown by the scale adjustment unit 110 and the distance data 103 measured in the measurement data acquisition process S300, the temporary relative posture 108 and the temporary shape are used. The scale of 109 is adjusted (S305).

Finally, in the three-dimensional shape measurement system of the present embodiment, the shape restoration unit 112 restores the shape of the measurement object from the temporary relative posture 108 and the temporary shape 109 and the distance data 103 measured in the measurement data acquisition process S300. (S306).
At this time, if the number of feature points obtained in S302 is small, the shape restoration unit 112 may perform shape restoration on portions other than the feature points by multi-view epipolar constraint.

In the flowchart of FIG. 3, a flow for performing shape restoration processing after measurement / feature point extraction of the measurement object is shown. However, more than the number of image data / distance data necessary for the restoration processing is captured and measured. In this case, during the restoration process, pipeline processing for performing the next measurement / acquisition measurement data acquisition process, feature point recognition process, and corresponding calculation process may be performed.

As described above, the feature point of the image data is obtained, the relative posture and shape of the feature point is obtained, and the shape of the measurement object is restored based on the distance data of the feature point, thereby reducing the calculation cost, Processing time can be shortened.

FIG. 4 shows a measurement example of the three-dimensional shape measurement system of the embodiment. 4A shows distance data 103 that is a measurement result of the distance measurement unit 100, and FIG. 4B shows image data 104 that is a measurement result of the imaging unit 101.
The distance data 103 in FIG. 4 (a) is a distance value represented on the screen as a luminance value, and a part having a longer distance is represented by a color closer to black. The part where the distance could not be measured is indicated by hatching (reference numeral 401).
In FIG. 4A, a region 403 is a portion where the distance can be measured.

FIG. 4B shows image data at the same place as in FIG. 4A, and the distance of a specific part of the image data can be found from the same coordinates in FIG. 4A. For example, the ground on the lower side of FIG. 4B corresponds to a portion where the distance can be measured by reference numeral 403 in FIG. The center of the distance data corresponds to the building at the center of the image data. However, the center of the distance data is a hatched area. That is, the distance to the building cannot be measured.
However, if the correspondence relationship between the distance data and the image data is known in advance, the same coordinates may not correspond to the same part.
Although details will be described later, in this embodiment, the distance between the temporary shape whose scale is unknown and the distance from which the distance is known is obtained from the position information of the feature points of the image such as the white line portion 402 in FIG. Perform shape restoration from information.

Next, details of the feature point recognition unit 105 in FIG. 1 are shown in FIG. 5, details of the correspondence calculation unit 106 are shown in FIG. 6, details of the temporary relative posture / shape estimation unit 107 are shown in FIG. 7, and details of the scale adjustment unit 110 are shown. Details of the shape restoration unit 112 will be described with reference to FIGS.

FIG. 5 is a diagram illustrating an example of feature points extracted by the feature point recognition unit 105, in which a plurality of feature points 501 (black circles) are extracted from one image data 500 of the image data 104 measured by the imaging unit 101. Is shown.
Here, the feature point is a point where the hue or luminance pattern around a predetermined position in the image is peculiar, such as a corner or a color boundary. In the example shown in FIG. 5, the feature point 501 is indicated by a black circle. In addition to the Harris corner detection method described above, features for selecting feature points include the corner features of SIFT (Scale-Invariant Feature Transform), SURF (Speeded Up Robust Features), and FAST (Features from Accelerated Segment Test). Can be mentioned. However, the method is not limited to the above method as long as it records an image pattern around a predetermined coordinate and can be associated with different images.

FIG. 6 is a diagram illustrating an example of feature point correspondence processing by the correspondence calculation unit 106. Image data 600 in FIG. 6A and image data 601 in FIG. 6B indicate image data obtained by imaging the measurement object from different points in the image data 104. For this reason, the same measurement object is imaged at different positions in the image data 600 and the image data 601.
The feature point 602 is one of the feature points extracted from the image data 600 by the feature point recognition unit 105, and the feature point 603 is one of the feature points extracted from the image data 601 by the feature point recognition unit 105. It has become. The feature point 602 and the feature point 603 are feature points of the same object to be measured, but the image data 600 and the image data 601 are imaged at different positions.

The correspondence calculation unit 106 associates the position of each feature point in the photographing target with the image data. In FIG. 6, when the feature pattern of the feature point 602 reflected in the image data 600 recognized by the feature point recognition unit 105 and the feature pattern of the feature point 603 reflected in the image data 601 match, the correspondence calculation unit 106 The feature points are associated with each other.
In FIG. 6A and FIG. 6B, black points connected by dotted lines represent the associated feature points.

The correspondence calculation unit 106 performs the feature point correspondence processing on the feature points between all the image data. However, if this correspondence calculation process is performed between all the image data, the calculation time is increased. Therefore, images used for association may be selected at predetermined intervals. In addition, the correspondence calculation may be performed only between images in which the shooting locations of the images are known to be close.

The details of the process of the temporary relative posture / shape estimation unit 107 will be described with reference to FIG.
FIG. 7 is a diagram illustrating a positional relationship between corresponding feature points and the imaging unit 101 in two pieces of image data captured by the imaging unit 101. The temporary relative posture / shape estimation unit 107 estimates the temporary relative posture 108 and the temporary shape 109 between the imaging points whose scales are unknown from the correspondence relationship between the coordinates of a plurality of feature points between the images.

Here, the relative posture is a difference regarding the position and orientation between the shooting points, and the temporary relative posture 108 indicates a case where the scale is unknown. The relative posture 111 indicates a relative posture with a correct scale. The shape data 113 indicates a set of points from which three-dimensional coordinates are obtained, and the temporary shape 109 indicates shape data whose scale is unknown.

The relative orientation between the three-dimensional coordinates of the feature points whose scale is unknown and the shooting point can be uniquely determined if a plurality of sets of the coordinates of the feature points between the images and their corresponding relationships are obtained. As the number of necessary correspondences, it is only necessary that five or more feature point correspondences per two image pairs are obtained. This calculation is obtained by a 5-point algorithm, an 8-point algorithm, or the following calculation method.

Hereinafter, a more specific calculation method will be described.
FIG. 7A shows image data 700 captured by the imaging unit 101. FIG. 7B shows image data 701 obtained by imaging the same measurement object from different positions. A reference numeral 702 in FIG. 7A represents one of characteristic points of the measurement object, and a reference numeral 703 in FIG. 7B represents a characteristic point of the same measurement object corresponding to the reference numeral 702.

FIG. 7C is a diagram showing the positional relationship between the measurement object 707 and the image data 700 and 701. Reference numeral 704 represents an imaging point when the image data 700 is captured, and reference numeral 705 represents an imaging point when the image data 701 is captured. Reference numeral 706 is the value of the relative orientation of reference numerals 704 and 705 of the imaging points.
A corner portion 708 of the measurement object 707 corresponds to a reference numeral 702 that is a feature point of the image data 700 and corresponds to a reference numeral 703 that is a feature point of the image data 701.
The positional relationship between the corner 708 of the measurement object in FIG. 7C and the reference numerals 702 and 703 of the feature points can be expressed by the following equation.

In equation (1), (fx, fy) is the focal length of the camera, (cx, cy) is the principal point of the screen, and s is a normalization so that the element in the third row of the vector on the left side is 1. Which corresponds to the distance to the entity in the camera coordinate system. The matrix (R, t) is the relative posture between the shooting points, and (X, Y, Z) is the three-dimensional coordinates of the feature points.

According to Equation (1), if the tertiary coordinates of the feature points are known, the coordinates (u, v) on the image data are determined. Therefore, the three-dimensional coordinates (X, Y, Z) of the feature points and the relative postures such that (u, v) matches the coordinates (mx, my) on the image of the feature points actually obtained by photographing. By obtaining the matrix (R, t), the temporary relative posture and the temporary shape can be determined.

However, since (mx, my) and (u, v) do not exactly match due to the influence of the feature point recognition error, (mx, my) and (u, v) shown in the following equation (2): The optimal relative posture and the three-dimensional coordinates of a plurality of feature points are obtained by minimizing the backprojection error E that is the difference of

Thus, by obtaining the three-dimensional coordinates of the feature points, the shape of the measurement object regarding the position of the feature points is determined. Any optimization calculation method may be used as long as the evaluation formula can be minimized, such as a steepest descent method or a Levenberg-Marquardt method.

Next, detailed processing of the scale adjustment unit 110 will be described with reference to FIG.
FIG. 8 is a diagram illustrating a method of adjusting the scale of the temporary shape 109 using the distance data 103 measured by the distance measuring unit 100.

The temporary relative posture 108 and the temporary shape 109 calculated by the temporary relative posture / shape estimation unit 107 using the image data 104 captured by the imaging unit 101 are correct in terms of the magnitude relationship between the distances within the shape, but are the absolute scale of the measurement target. I don't know. On the other hand, regarding the part obtained by the distance measuring unit 100, the absolute scale is known, but in many cases, since the distances of all points cannot be measured, only the distances in a partial region in the screen are obtained. .
Therefore, the scale adjustment unit 110 gives the scale information of the partial region obtained by the distance measurement unit 100 to the region where the distance measurement has not been performed but the unknown shape of the scale has been calculated, and the scale in the region is calculated. Find the ratio to be adjusted so that it can be restored.

Specifically, for the feature point 801 of the image data 104 in FIG. 8A, it is assumed that the coordinate 802 of the correct distance 804 of the scale is obtained by the distance measuring unit 800.
On the other hand, it is assumed that the shape of the feature point 801 is estimated at the position of the distance 805 where the scale is unknown as the calculation result of the temporary relative posture / shape estimation unit 107.

The scale adjustment unit 110 determines the overall scale based on the ratio r between the distance 804 and the distance 805. That is, as shown in FIG. 8B, using this ratio r, the feature point 801 in the temporary shape 109 is moved to the position of the coordinate 808 (same as the coordinate 802). Further, the position of the feature point 806 for which distance measurement could not be performed is moved to the position of the coordinate 810 after the scale adjustment using the ratio r. Thereby, the feature point 801 and the feature point 806 can obtain a correct scale.

In FIG. 8A, in addition to the feature point 801, the feature point 809 also has the ratio r regarding the feature point 809 and the feature point 809 when the distance measuring unit 800 has obtained the coordinates 807 having the correct scale distance. An average with the ratio r for point 801 is taken to determine the overall scale. That is, as shown in FIG. 8B, the feature point 801 in the temporary shape 109 is moved to the position of the coordinate 808 (different from the coordinate 802) by adjusting the scale according to the average ratio. Similarly, the feature points 806, 809, and 811 are also scaled according to the average ratio and moved to the positions of the coordinates 810, 812, and 813 to obtain a correct scale.
Thus, when there are a plurality of feature points whose distances can be measured, the optimum scale is calculated by taking the average of each scale.

The ratio r of the scale obtained by the above, illustrates a method of correcting a provisional relative position t in the correct relative orientation t _r scale in equation (3).

Further, a method for correcting the coordinates (X, Y, Z) of unknown feature points to the correct coordinates (Xr, Yr, Zr) of the scales is shown in Equation (4).

Next, details of processing contents of the shape restoration unit 112 will be described with reference to FIG.
FIG. 9 is a diagram illustrating a method for restoring the shape of the measurement object.
The shape restoration unit 112 further adds scale adjustment as a constraint condition to the back projection error obtained by the temporary relative posture / shape estimation unit 107 using the entire scale adjusted by the scale adjustment unit 110 as an initial value. The correct relative posture and shape are restored.
In the present invention, by combining a scale unknown shape obtained using images taken from a plurality of different positions in the imaging unit and a scale obtained from a part of the shape near the sensor obtained by the distance measurement unit 100, It is possible to measure a three-dimensional shape even for a distant shape that cannot be directly measured by the distance measuring unit 100.

Specifically, in FIG. 9, it is assumed that a distance 901 to the measurement object can be measured by the distance measuring unit 100 and that a feature point 900 of the image data cannot be measured for the feature point 902. At this time, if the three-dimensional coordinate 903 of the feature point 900 of the image data is temporarily set and the three-dimensional coordinate 904 of the feature point 902 is temporarily set, the position of the feature point of the image data is obtained by Expression (1).

Further, for the feature point 900 that satisfies the formula (1) and whose distance can be measured, the tentatively set three-dimensional coordinate 903 and the distance to the sensor should be equal to the distance 901 obtained by the measurement. Thus, the correct shape of the scale can be obtained.

The evaluation formula is shown in the following formula (5).

Here, the weight w in Equation (5) represents the constraint weight regarding the difference between the distance to the feature point and the distance obtained by measurement. This weight controls the balance of constraints relating to the difference between the backprojection error and the distance (this will be referred to as a distance error), and is set according to the ratio between the feature point recognition accuracy and the distance measurement accuracy.
In Expression (5), the first and second terms are the same as in Expression (2), and indicate a backprojection error.
The third term indicates the distance error, d indicates the distance measured by the distance measuring unit, and s indicates the distance from the sensor obtained by Equation (1) to the temporarily set feature point.

Furthermore, the shape restoration unit 112 obtains the posture relationship between the three-dimensional coordinates of the feature points and the distance measurement unit 100. If the feature points are not obtained sufficiently densely, the relative posture of the feature points is used. Then, using the Multi 部位 View な か っ Stereo method, the shape of the part where the feature points could not be obtained is restored. This restoration is performed according to the following procedure based on the principle of epipolar geometry shown in FIG.

FIG. 10 is a diagram showing the correspondence between the point 1001 and the imaging unit 1003 when one point 1001 of the measurement object in the measurement space is imaged from different directions, and shows the principle of epipolar geometry.
The point 1001 in the measurement space corresponds to the point 1000 on the image data 1004 and is located somewhere on the extended line connecting the imaging unit 1003 and the point 1000. A straight line 1002 of the image data 1005 when a line including the point 1001 is imaged from another viewpoint is referred to as an epipolar line. A point on the image data 1005 when the point 1001 in the measurement space is imaged from another viewpoint is limited to the epipolar line 1002. This principle is called epipolar geometry.

With Multi View Stereo using this principle, corresponding points can be found by searching only on the epipolar line even if the image of the measurement object is not a unique pattern and not recognized as a feature point. The shape can be restored. The dense shape restoration method is not limited to Multi View Stereo as long as the dense shape can be restored from the relative posture and the image data.

Next, the distance data utilization area setting unit 115 in FIG. 1 will be described.
In the shape restoration unit 112, the value of the weight w is set according to the balance between the feature point recognition accuracy and the distance recognition accuracy. Of these, the distance recognition accuracy varies depending on the material, color, distance, etc. of the measurement object. There is.
Therefore, the distance data utilization area setting unit 115 sets the weight w according to the distance accuracy according to the distance accuracy, and sets a plurality of types of image data areas satisfying the distance accuracy and the weight w. Then, the shape restoration unit 112 optimizes the equation (5) while determining the scale by the scale adjustment unit 110 according to the weight w in accordance with the region in the image data.

FIG. 11 is a diagram for explaining the processing of the distance data use area setting unit 115.
FIG. 11 illustrates an example in which the distance data usage area setting unit 115 sets three types of weights based on the image data 1101. In the example of FIG. 11, when the part of the feature point 1102 is measured by the distance measuring unit 100, the accuracy is the highest, the part of the feature point 1104 has the next highest precision, and the part of the feature point 1106 has the highest precision. Let it be low.

In this case, as the setting procedure, in the image data 1101, the distance data utilization region setting unit 115 first sets the region 1103 including the feature point 1102 and sets the largest value for the weight w. Subsequently, an area 1105 including the feature point 1104 is set, and a weight w having a smaller value than the weight w set in the area 1103 is set. Finally, an area 1107 including the feature point 1106 is set, and the smallest weight w is set.
It should be noted that a predetermined weight w is set for an area where the weight w is not set.

In accordance with each weight w determined for each area determined by the distance data utilization area setting unit 115, the scale adjustment unit 110 determines a scale by a weighted average, and the shape restoration unit 112 also uses the weight w in Expression (5). The shape is restored according to the value of.
Note that the types of settings are not limited to three. When nothing is set, a predetermined fixed value is adopted as the weight w. It is also possible to use only the distance data in the area by setting the weight of the predetermined area to the predetermined value w and setting the other weights to zero.

Next, the feature point use area setting unit 116 in FIG. 1 will be described.
In the shape restoration unit 112, the value of the weight w is set according to the balance between the feature point recognition accuracy and the distance recognition accuracy. Of these, the feature point recognition accuracy depends on the resolution of the captured image and the sharpness of each feature point. , Each may be different.
Therefore, the feature point use region setting unit 116 sets a plurality of types of the weight w for each region of the image data according to the distance recognition accuracy of the feature point, and the scale adjustment unit 110 adjusts the scale according to the weight w. The shape restoration unit 112 optimizes the equation (5). Note that the set value of the weight w relates to a term relating to the distance recognition accuracy as shown in the equation (5), so the weight w is reduced as the feature point recognition accuracy is higher.

FIG. 12 is a diagram for explaining the processing of the feature point use area setting unit 116.
FIG. 12 shows an example in which the feature point use region setting unit 116 sets three types of weights based on the image data 1201. In the example of FIG. 12, the feature point 1202 has the highest feature point recognition accuracy, the feature point 1204 has the next highest feature point recognition accuracy, and the feature point 1206 has the lowest feature point recognition accuracy. .

As a setting procedure in this case, the feature point use region setting unit 116 first sets the region 1203 including the feature point 1202 in the image data 1201 and sets the smallest value for the weight w. Subsequently, an area 1205 including the feature point 1204 is set, and a value larger than the weight w set in the area 1203 is set. Finally, an area 1207 including the feature point 1206 is set, and the largest weight w is set.
It should be noted that a predetermined weight w is set in an area where the weight w is not set.

In accordance with the respective weights w determined by the feature point use region setting unit 116, the scale adjustment unit 110 determines the scale by the weighted average, and the shape restoration unit 112 uses the weight w in the equation (5). The shape is restored according to the value of.
Note that the types of settings are not limited to three. When nothing is set, a predetermined fixed value is adopted as the weight w. It is also possible to use only the feature points in the region by setting the weight of the predetermined region to the predetermined value w and setting the other weights to zero.

(Example 2)
In the above-described embodiment, the example in which the distance measuring unit 100 is configured by a laser scanner, a distance image sensor, or the like has been described. However, based on the distance data to the feature points that can be measured by the distance measuring unit 100, the overall shape data Is restoring. In other words, it is not necessary for the distance measuring unit 100 to measure the distances to all the points of the measurement object, and it is only necessary to measure the distances to some feature points of the captured image.
The present embodiment uses a 2D distance sensor that measures a two-dimensional shape of a part of the measurement object by scanning the distance measuring unit 100 on a plane, and calculates the shape of the space from the distance measured by the 2D distance sensor. Is to restore.

The functional block of the three-dimensional shape measurement system of this embodiment is the same as the functional block of FIG. 1 except that the distance measuring unit 100 is configured by a 2D distance sensor, and includes a distance data using region setting unit 115 and a feature point using region setting unit 116. It is different.

The three-dimensional shape measurement system according to the present exemplary embodiment uses a feature point recognition unit 105, a correspondence calculation unit 106, and a temporary relative posture / shape estimation unit 107 for image data 104 captured by the imaging unit 101. The shape 109 is obtained. Further, the scale adjustment unit 110 uses the scale information obtained from the distance data 103 and the provisional shape 109 to correct the provisional relative posture 108 to obtain a correct relative posture 111 of the scale. Subsequently, the shape restoration unit 112 restores the three-dimensional shape by mapping the distance data obtained by the distance measurement unit 100 on the three-dimensional space based on the relative posture. This process is the same as in the above-described embodiment.
In this embodiment, the processing is performed by setting the weight w in the equation (5) to a fixed predetermined value.

FIG. 13 is a diagram illustrating an external appearance of the three-dimensional shape measurement system according to the present embodiment.
The measurement apparatus of the present embodiment is configured to place the imaging apparatus 1300 on a distance measurement unit 1301 that is a 2D distance sensor. Then, the measurement apparatus of the present embodiment is moved to perform imaging of the measurement object and distance measurement from different directions.

The distance measuring unit 1301 is a sensor capable of measuring a two-dimensional shape such as surrounding irregularities at a time. For example, it is a laser scanner that scans a laser beam in a straight line, receives the reflected light, and measures the distance according to the time from the irradiation of the laser light to the light reception.

In the apparatus of the present embodiment in FIG. 13, the position scanned by the distance measuring unit 1301 and the imaging position of the imaging device 1300 are associated with each other, and the measurement object corresponding to the coordinate position of the feature point of the image data of the imaging device 1300 is associated. Get distance data.
More specifically, when the feature point of the image data 101 imaged by the imaging device 1300 exists on a horizontal line passing through the center, the laser beam scan of the distance measuring unit 1301 that is a 2D distance sensor is also made to correspond to the horizontal line. . For this reason, the direction and inclination of the distance measuring unit 1301 can be changed, and the distance measuring unit 1301 can change the scanning position of the laser light to measure the distance at an arbitrary position of the image data.
The measurement apparatus of the three-dimensional shape measurement system according to the present embodiment restores the measurement object with the same configuration as the shape restoration apparatus 21 shown in FIG. 2 based on the measurement results of the imaging unit 1300 and the distance measurement unit 1301. To do. For this reason, the description of the apparatus configuration that performs the restoration process in the measurement apparatus of the present embodiment is omitted.

A measurement example of the distance measurement unit 1301 will be described with reference to FIG.
The distance measuring unit 1301 is a 2D distance sensor, scans the measurement target 1401 with the laser beam 1402, receives the reflected light of the irradiated laser beam, and receives the light after irradiating from the phase difference between the irradiated light and the reflected light. The time until the measurement is obtained, and the distance to the measurement object 1401 is calculated. By scanning the laser beam in one direction, the two-dimensional shape of the scanning line is measured. At this time, since there is no measurement object in the irradiation direction of the laser beam 1403, the reflected light is not received. For this reason, distance measurement is impossible.

Next, a shape restoration method based on the measurement result of the distance measurement unit 1301 will be described. FIG. 15 is a view showing a scanning surface of the laser beam of FIG.
The distance measurement unit 1301 sequentially measures n data while changing the measurement direction φ by the angular resolution δφ. Here, the measurement direction of the i-th measurement data is φi, and the measurement result is the distance ri. A combination of distance and direction (ri, φi) at this time is a position represented by a relative polar coordinate system of the measurement object with the distance measurement unit 1301 as the center.

In addition, a dotted line arrow represents the measurement result regarding the irradiation direction of each laser beam 1402, and the end 1500 of the arrow is the position of the point to be measured. A closed solid line indicates a measurement target object in space, and a dotted line arrow 1501 that collides with the object is data that has been successfully measured. A dotted arrow that did not hit the closed solid line indicates that measurement was not possible because there was no reflected light. A set of data that has been successfully measured becomes a two-dimensional shape of the measurement object 1401 expressed in polar coordinates.

Conversion from the position (ri, φi) measured by the distance measuring unit 1301 and expressed in the polar coordinate system to the orthogonal coordinate system (X, Y, Z) is performed by the following equation (6).

Using the coordinates (X, Y, Z) obtained in this way and the relative posture 111, the shape is restored by calculating the three-dimensional coordinate values (Xr, Yr, Zr) by the conversion formula (7).

According to the present embodiment, by combining a scale unknown shape obtained using images taken from a plurality of different positions by the imaging unit and a scale obtained from a part of the shape obtained by the distance measurement unit 100, It is possible to measure the three-dimensional shape of the entire shape of the measurement target that cannot be directly measured by the distance measuring unit 100.
Further, according to the present embodiment, since a three-dimensional shape measurement system can be configured by using an inexpensive 2D distance sensor for the distance measurement unit 1301, a system can be obtained at a low cost.
In addition, since the scanning period of the laser light is shortened, the measurement time of the distance required for one measurement is shortened, and the measurement object can be measured from many directions, and the measurement accuracy of the three-dimensional measurement is improved. .

(Example 3)
As described above, in the three-dimensional shape measurement system according to the present embodiment, the temporary relative posture and the temporary shape whose scale is unknown in the three-dimensional space are obtained from the correspondence between the feature points of the image data captured from a plurality of directions, and the distance between the feature points. The shape of the three-dimensional space is restored by adjusting the temporary relative posture and the scale of the temporary shape using the data.
That is, the distance measuring unit may obtain distance data of feature points of image data.

In the present embodiment, an embodiment will be described in which a distance sensor that can measure the distance to a predetermined point in the measurement space by arbitrarily changing the direction and inclination of the measurement direction is used as the distance measurement unit to restore the shape of the space.
FIG. 16 is a diagram illustrating an external appearance of the three-dimensional shape measurement system according to the present embodiment.
The measurement apparatus according to the present embodiment includes a distance measurement unit 1601 and an imaging unit 1600, and moves to perform imaging and distance measurement of a measurement target from different directions to calculate the shape of the three-dimensional space.

Similar to the above-described embodiment, the imaging unit 1600 captures the entire shape of the measurement object in the measurement space and acquires a plurality of image data with different viewpoints.
The distance measuring unit 1601 is provided with a laser distance meter that irradiates a laser beam 1602 in a certain direction and measures the distance to one point on the measurement object, and changes the irradiation direction and inclination of the laser beam 1602 of this laser distance meter. For this purpose, a pan head 1603 for controlling the direction of the laser distance meter is provided. This pan head 1603 is controlled so that laser light can be irradiated toward a measurement object corresponding to an arbitrary point of image data.
The three-dimensional shape measurement system of the present embodiment restores the measurement object based on the measurement results of the imaging unit 1600 and the distance measurement unit 1601 with the same configuration as the shape restoration device 21 shown in FIG. For this reason, description of the apparatus configuration that performs the restoration process in the present embodiment is omitted.

FIG. 17 is a diagram showing a processing flow of the three-dimensional shape measurement system of the present embodiment.
First, in the three-dimensional shape measurement system of the present embodiment, the imaging unit 1600 images the measurement object in the measurement space, and acquires image data obtained by imaging the measurement object from a plurality of directions (S171).
Next, the image data acquired in step S171 is analyzed, and feature point recognition processing for recognizing feature points on the image is performed (S172).
Then, a correspondence calculation process for associating the same feature points among a plurality of image data is performed (S173).

In step S174, the three-dimensional shape measurement system according to the present embodiment obtains a temporary relative posture and a temporary shape whose scale is unknown from position information of the same feature point between a plurality of image data.
Thereafter, the pan head 1603 is controlled so that the laser beam is irradiated in the direction of one point of the measurement object corresponding to the feature point of the image data determined in step S172, and the distance measurement unit 1600 measures the measurement object corresponding to the feature point. The distance to one point of the object is measured (S175). This distance measurement is performed for the number of feature points of the image data.

Next, the three-dimensional shape measurement system of the present embodiment uses the ratio of the correct distance of the scale obtained from the distance data measured in step S175 and the distance related to the temporary shape of unknown scale obtained in step S174 to make a temporary relative The posture and the scale of the temporary shape are adjusted (S176).
Finally, the shape of the measurement object is restored from the temporary relative posture and the temporary shape and the measured distance data of the feature points (S177).
At this time, if the number of feature points obtained in step S172 is small, shape restoration may also be performed for portions other than the feature points by multipolar epipolar constraint.

The above flow is performed every time the measuring apparatus of the present embodiment performs movement measurement.
By the way, the process of step S173 and step S174 requires many processing resources, such as a processor, and requires processing time. Also, in the distance measurement in step S175, if the number of feature points is large, it takes time to adjust the direction, and the acquisition time of distance data becomes long. For this reason, the processing time of the entire flow of FIG. 17 becomes long, and there may be a case where measurement time becomes long or measurement from many viewpoint directions cannot be performed.

In such a case, since the process of step S175 can be performed any time after the feature point is obtained, the distance data acquisition (S175) process of the feature point is started after the process of step S172. The distance data acquisition process of S175 and the processes executed in the order of steps S173 and S174 may be performed in parallel.

According to the present embodiment, by combining a scale unknown shape obtained using images taken from a plurality of different positions by the imaging unit and a scale obtained from a part of the shape obtained by the distance measurement unit 100, It is possible to measure the three-dimensional shape of the entire shape of the measurement target that cannot be directly measured by the distance measuring unit 100.
Further, according to the present embodiment, since the distance measurement to the point of the measurement object corresponding to the feature point of the image data can be performed with high accuracy, the measurement accuracy of the three-dimensional measurement system can be easily improved.

Further, the present invention is not limited to the above-described embodiments, and includes various modifications. The above-described embodiments have been described in detail for easy understanding in the present invention, and are not necessarily limited to those having all the configurations described. Further, a part of the configuration of an embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of an embodiment.

Taking the above embodiment from another perspective, the three-dimensional measurement system of the present invention
An imaging unit that captures an image of a measurement space including a measurement object and acquires image data;
Temporary relative posture for calculating a plurality of feature points of image data corresponding to each of a plurality of image data captured from different directions by the imaging unit, and calculating a temporary relative posture and a temporary shape whose scales are unknown, A shape calculation unit;
A distance measuring unit for measuring a distance to the measurement object corresponding to the feature point;
The three-dimensional coordinate value of the feature point whose distance could not be measured by the distance measurement unit is calculated from the distance of the feature point whose distance could be measured by the distance measurement unit, the temporary relative posture whose scale is unknown, and the temporary shape, An actual shape calculator that restores the overall shape;
It is characterized by providing.

Moreover, the measurement method of the three-dimensional measurement system of the present invention includes:
A distance measurement unit that measures distances to a plurality of points of the measurement object, and an imaging unit that images the measurement space including the measurement object and acquires image data, and measures the shape of the measurement space A measurement method of a three-dimensional shape measurement system,
Obtaining a plurality of feature points of image data corresponding to each of a plurality of image data imaged from different directions by the imaging unit, and calculating a scale-unknown provisional relative posture and provisional shape composed of the feature points;
The three-dimensional coordinate value of the feature point whose distance could not be measured by the distance measurement unit is calculated from the distance of the feature point whose distance could be measured by the distance measurement unit, the temporary relative posture whose scale is unknown, and the temporary shape, Restoring the overall shape;
It is characterized by having.

DESCRIPTION OF SYMBOLS 100 Distance measurement part 101 Imaging part 102 Measurement data storage part 103 Distance data 104 Image data 105 Feature point recognition part 106 Correspondence calculation part 114 Feature point correspondence 107 Temporary relative attitude | position / shape estimation part 108 Temporary relative attitude 109 Temporary shape 110 Scale adjustment part 111 Relative posture 112 Shape restoration unit 113 Shape data 115 Distance data use region setting unit 116 Feature point use region setting unit 120 Temporary relative posture / shape calculation unit 121 Actual shape calculation unit

Claims (16)

1. A distance measuring unit for measuring the distance to a plurality of points of the measurement object;
   An imaging unit that images the measurement space including the measurement object and obtains image data;
   Obtaining a plurality of feature points of image data corresponding to each of a plurality of image data imaged from different directions by the imaging unit;
   From the position information in the image data of the feature points of the plurality of points, obtain a temporary relative posture and a temporary shape whose scale is unknown,
   Based on the distance to the feature point measured by the distance measuring unit, calculating the three-dimensional coordinate values of the temporary relative posture and the temporary shape with unknown scale,
   A shape restoration device for restoring the shape of the measurement object;
   A three-dimensional shape measurement system comprising:
2. The three-dimensional shape measurement system according to claim 1,
   The three-dimensional shape measurement system, wherein the distance measurement unit is a 2D distance sensor that measures a distance in one direction of a measurement space including a position of the feature point of the image data.
3. The three-dimensional shape measurement system according to claim 1,
   The three-dimensional shape measurement system, wherein the distance measurement unit is a laser distance meter that measures a measurement object in a measurement space corresponding to the position of the feature point of the image data.
4. The three-dimensional shape measurement system according to claim 1,
   The shape restoration device
   A measurement data storage unit that stores a plurality of pairs of distance data measured by the distance measurement unit and image data captured by the imaging unit;
   A feature point recognition unit for recognizing feature points of the image data;
   A correspondence calculator for associating each of the feature points obtained from a plurality of pieces of the image data;
   A temporary relative posture / shape estimation unit for obtaining a temporary relative posture and a temporary shape whose scale is unknown from the correspondence calculated by the correspondence calculation unit;
   The correct scale is obtained by adjusting the temporary relative posture and the temporary shape scale obtained by the temporary relative posture / shape estimation unit from the ratio of the distance obtained by measuring the size and shape of the temporary shape by the distance measuring unit. A scale adjustment unit for determining the relative posture and shape of
   A shape restoration unit for restoring the shape from the shape calculated by the scale adjustment unit;
   A three-dimensional shape measurement system characterized by comprising:
5. The three-dimensional shape measurement system according to claim 1,
   The shape restoration device
   A measurement data storage unit that stores a plurality of pairs of distance data measured by the distance measurement unit and image data captured by the imaging unit;
   A feature point recognition unit for recognizing feature points of the image data;
   A correspondence calculator for associating each of the feature points obtained from a plurality of pieces of the image data;
   A temporary relative posture / shape estimation unit for obtaining a temporary relative posture and a temporary shape whose scale is unknown from the correspondence calculated by the correspondence calculation unit;
   The correct scale is obtained by adjusting the temporary relative posture and the temporary shape scale obtained by the temporary relative posture / shape estimation unit from the ratio of the distance obtained by measuring the size and shape of the temporary shape by the distance measuring unit. A scale adjustment unit for determining the relative posture and shape of
   Back projection error indicating the difference between the screen coordinate value of the feature point obtained from the shape and the relative posture calculated by the scale adjustment unit and the screen coordinate value of the feature point obtained from the image data, and calculated by the scale adjustment unit A shape restoration unit that restores the shape by using a distance error indicating a difference between a distance to the feature point that constitutes the shape and a distance obtained by measurement by the distance measurement unit, as a constraint condition;
   A three-dimensional shape measurement system characterized by comprising:
6. The three-dimensional shape measurement system according to claim 4 or 5,
   The three-dimensional shape measurement system, wherein the shape restoration unit restores the shape of a part other than the feature point using the principle of epipolar geometry using the image data taken by the imaging unit and the relative posture.
7. The three-dimensional shape measurement system according to claim 5 or 6,
   The three-dimensional shape measurement system, wherein the shape restoration unit has a weighting factor for setting a balance between the backprojection error and the distance error.
8. In the three-dimensional shape measurement system according to any one of claims 4 to 7,
   The distance measurement unit measures a distance to a measurement object at a time corresponding to imaging of the measurement object in a measurement space.
9. In the three-dimensional shape measurement system according to any one of claims 4 to 8,
   The said distance measurement part measures the distance to the said feature point for every feature point which the said feature point recognition part recognized, The three-dimensional shape measurement system characterized by the above-mentioned.
10. The three-dimensional shape measurement system according to any one of claims 4 to 9,
    The distance measuring unit measures a two-dimensional shape of a part of a measurement object including a point corresponding to the feature point at a time,
    The shape restoration unit restores the shape by converting the distance of the feature point measured by the distance measurement unit into a three-dimensional coordinate value based on the relative posture.
11. In the three-dimensional shape measurement system according to any one of claims 4 to 10,
    A distance data use area setting unit is provided that sets a weighting coefficient for the image data in accordance with the distance accuracy and sets a plurality of types of image data areas that satisfy the distance accuracy and the weighting coefficient as a pair. And
    The scale adjustment unit and the shape restoration unit perform scale calculation and shape restoration using the weighting factor, and a three-dimensional shape measurement system.
12. The three-dimensional shape measurement system according to any one of claims 4 to 11,
    A distance data use region setting unit that sets a weighting factor for the image data in accordance with the distance accuracy and sets a plurality of types of the corresponding image data region that satisfies the distance accuracy and the weighting factor. Have
    The three-dimensional shape measurement system, wherein the scale adjustment unit and the shape restoration unit perform scale calculation and shape restoration using distance data included in the region for each weight coefficient.
13. The three-dimensional shape measurement system according to any one of claims 4 to 12,
    A feature point use region setting unit that sets a weighting factor corresponding to the feature point recognition accuracy for the image data, and sets a plurality of types of the image data region that satisfies the feature point recognition accuracy and the weighting factor. Have
    The scale adjustment unit and the shape restoration unit perform scale calculation and shape restoration using the weighting factor, and a three-dimensional shape measurement system.
14. The three-dimensional shape measurement system according to any one of claims 4 to 13,
    A feature point use region setting unit that sets a weighting factor corresponding to the feature point recognition accuracy for the image data, and sets a plurality of types of the image data region that satisfies the feature point recognition accuracy and the weighting factor. Have
    The scale adjustment unit and the shape restoration unit use a feature point included in the region for each weighting coefficient.
15. A measurement method of a three-dimensional shape measurement system for measuring the shape of a measurement space including a measurement object,
    Capture the measurement space from different directions to obtain a plurality of image data,
    Obtaining a plurality of feature points of image data corresponding to each of the plurality of image data;
    From the position information in the image data of the feature points, obtain a temporary relative posture and a temporary shape whose scale is unknown,
    Based on the distance from the imaging point to the measurement object corresponding to the feature point, a three-dimensional coordinate value of the temporary relative posture and provisional shape whose scale is unknown is calculated, and the shape of the measurement object is restored. A measurement method of the three-dimensional shape measurement system.
16. A measurement method of a three-dimensional shape measurement system for measuring the shape of a measurement space,
    Multiple pairs of distance data measured by the distance measurement unit and image data captured by the imaging unit are stored in pairs,
    Recognizing feature points of the image data;
    Associating each of the feature points obtained from a plurality of the image data,
    A temporary relative posture and a temporary shape whose scale is unknown are obtained from the associated feature points, and the correct relative posture and shape of the scale are obtained by adjusting the temporary relative posture and the scale of the temporary shape according to the distance measured by the distance measuring unit. Seeking
    Back projection error indicating a difference between the screen coordinate value of the feature point obtained from the relative posture and shape of the correct scale and the screen coordinate value of the feature point obtained from the image data, and the feature point constituting the correct scale shape A method for measuring a three-dimensional shape measurement system, wherein a shape is restored with a distance error indicating a difference between a distance to a distance measured by the distance measurement unit as a constraint condition.

Images