JP2008275366A - Stereoscopic 3-d measurement system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an interactive stereoscopic 3-D measurement system made capable of stereoscopic 3-D measurement of the stationary object at low price simply and highly precisely. <P>SOLUTION: The stereoscopic 3-D measurement system for measuring the 3-D measurement of the stationary object by searching the corresponding positions of the stereoscopic images is provided with an imaging device, an active illuminating means for emitting light changing spatially and time sequentially, a 3-D measurement processing device and a display device, while illuminating a measurement object with the active illumination and taking the time sequential images using imaging devices for stereoscopic images. The taken stereoscopic images are inputted into the 3-D measurement system, in the 3-D measurement processing device the distance to the measurement object is measured by searching the corresponding position of the stereoscopic images by using the plurality of frames in the time direction, furthermore making the measurement results display in a real time on the display by measurement process successively using the latest stereoscopic images. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a stereo three-dimensional measurement technique, and more particularly to a stereo three-dimensional measurement system for measuring a three-dimensional shape of a stationary object.

  Conventionally, in order to measure a three-dimensional shape of a stationary object, many stereo measurement methods using two or more stereo cameras are used (see Non-Patent Document 1 and Non-Patent Document 2).

  In the “passive stereo measurement method” using only a camera, a spatial direction window is generally used in stereo correspondence search (stereo correspondence), that is, stereo matching.

  When performing 3D shape measurement of a stationary object using the “passive stereo measurement method”, when there is no appropriate texture on the surface of the stationary object to be measured, a stereo corresponding point search (hereinafter simply referred to as “corresponding position search”). However, there is a problem that it was not possible to measure.

  In addition, even if there is an appropriate texture on the surface of the stationary object to be measured, it is necessary to use a “region” called “local support”, that is, a “spatial direction window” in order to search for a corresponding position between stereo images. For this reason, the depth cannot be estimated correctly for the part that smoothes the depth or includes the part where the depth changes abruptly in the spatial direction window, that is, the measurement result is inaccurate. There is a problem.

  In order to eliminate the ambiguity of correspondence between stereo images, an “active stereo measurement method” in which one of the plurality of stereo cameras is replaced with a light source is often used. Specific examples of the “active stereo measurement method” include, for example, the “slit light projection method” disclosed in Non-Patent Document 3 and the “pattern light projection method” disclosed in Non-Patent Document 4.

Since the stereo measurement methods disclosed in Non-Patent Document 3 and Non-Patent Document 4 are active stereo measurement methods in which one of the stereo cameras is replaced with a projector, a PC projector that is more expensive than a camera is used. In addition, there is a problem that a simple and inexpensive stereo three-dimensional measuring apparatus cannot be realized by these measuring methods.
M. Okutomi, T. Kanade, "A Multiple-Baseline Stereo", Pattern Analysis and Machine Intelligence, Volume 15 , No. 4, p.353-363, 1993 D. Scharstein and R. Szeliski, “A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms”, International Journal of Computer Vision, 47, No. 1, p.7-42, 2002 J. Salvi, J. Pages, J. Batlle, “Pattern Codification Strategies in Structured Light Systems” , Pattern Recognition, 37, p.827-849, 2004 P. J. Besl, “Active, Optical Range Imaging Sensors”, Machine Vision and Applications, Volume 1, Issue 2, p. .127-152, 1988 J. Davis, R. Ramamoorthi, S. Rusinkiewicz, “Spacetime Stereo: A Unifying Framework for Depth from Triangulation (A) Unifying Framework for Depth from Triangulation ”, In Computer Vision and Pattern Recognition, Volume 2, p.359-366, 2003 L. Zhang, B. Curless, SM Seitz, "Spacetime Stereo: Shape Recovery for Dynamic Scenes," In Computer Vision and Pattern Recognition, Volume 2, p.367-374, 2003 N. Ayache and C. Hansen, “Rectification of Images for Binocular and Trinocular Stereovision”, In 9th International Conference on Pattern Recognition (In 9th International Conference on Pattern Recognition), p.11-16, 1988 A. Fusiello, E. Trucco, A. Verri, “A compact algorithm for rectification of stereo pairs” , Machine Vision and Applications, Vol. 12, No. 1, pp. 16-22, 2000

  On the other hand, as shown in FIG. 1, a time-direction window (hereinafter simply referred to as “time-direction window”) is used using time-series images (base image and reference image) obtained by photographing with a stereo camera. In other words, a stereo measurement method called “spacetime stereo” is proposed in which a corresponding position search between a base image and a reference image is performed by using changes in pixel values in the time direction. (See Non-Patent Document 5 and Non-Patent Document 6).

  In this spacetime stereo measurement method, using a change in temporal pixel value at the corresponding position when changing illumination is applied to a stationary object to be measured, the reference image and the reference image are used. Seek correspondence between. This illumination change is different from “structured illumination” used in the active stereo measurement method disclosed in Non-Patent Document 3 and Non-Patent Document 4, for example, with simple lighting such as a flashlight and a shadow. This can be achieved by changing

  In addition, the spacetime stereo measurement method only needs to be calibrated in the usual “passive stereo measurement method”, and uses simple lighting, so it is structured like the normal “active stereo measurement method”. No illumination or calibration for structured illumination is required.

  However, in the spacetime stereo measurement method, after inputting all the time-series images obtained by photographing the measurement target with a change in illumination, the stereo measurement processing is performed for all the input time-series images at once. Therefore, there is a problem that it has not been possible to determine whether or not an appropriate illumination change has been given to the measurement target when the time-series image is captured.

  That is, in the “spacetime stereo measurement method” disclosed in Non-Patent Document 5 and Non-Patent Document 6, “a lighting change due to simple illumination is given to a measurement object, and a time-series image of the measurement object is taken with a stereo camera, “(1) What illumination change can be given,” it is only described that “a plurality of frames are used to set a region of interest in the spatial direction and the time (frame) direction and search for corresponding positions between stereo images”. (2) Since it is not possible to determine how long the measurement result will be sufficient, there is no guarantee that a satisfactory measurement result will be obtained, and the measurer is forced to take an unnecessarily long measurement. There was a problem with.

  The present invention has been made under the circumstances described above, and an object of the present invention is to easily and accurately provide a stationary object that has fine parts and may not have a texture at a low cost. An object of the present invention is to provide an interactive stereo three-dimensional measurement system that enables stereo three-dimensional measurement.

  The present invention relates to a stereo three-dimensional measurement system that performs a three-dimensional measurement of a measurement target that is a stationary object by searching for a corresponding position between stereo images. Active illumination means for generating active illumination that also changes over time, a three-dimensional measurement processing device, and a display device, while illuminating the measurement object with the active illumination generated from the active illumination means, A time-series stereo image is captured using an imaging device, and the captured stereo image is input to the three-dimensional measurement processing device. The three-dimensional measurement processing device uses a plurality of frames in the time direction to generate a stereo image. By searching for the corresponding position between, the distance to the measurement object is measured, the measurement result obtained by the measurement is displayed on the display device, Performed sequentially measurement process for each of the new stereo image is input, effectively be achieved by displaying the measurement result on the display device.

ただし、（ｕ_ｂ,ｖ_ｂ）は基準画像座標を表し、ｄは視差を表し、また、Ｉ_ｂ（ｕ,ｖ,ｔ）とＩ_ｒ（ｕ,ｖ,ｔ）は、それぞれ基準画像と参照画像の位置（ｕ,ｖ）におけるフレーム番号ｔでの画素値であり、次に、フレーム間差分値が一定値以上のとき、次の数式に基づき、計算されたＳＤ（ｕ_ｂ,ｖ_ｂ,ｄ,ｔ）を、基準画像座標（ｕ_ｂ,ｖ_ｂ）と視差ｄを軸に持つＳＳＤ空間へ積算し、

前記ＳＳＤ空間における視差ｄ方向の最小値探索によって、ステレオ画像間の対応位置を求めることによってより効果的に達成される。 The object of the present invention is that the spatial direction area when searching for a corresponding position between stereo images is 1 × 1 [pixel], or that the three-dimensional measurement processing device is configured to occlude stereo images. For displaying the detected occlusion on the display device, or for the imaging device to take two synchronized stereo cameras for taking a stereo image, that is, for taking a reference image. Each time a new stereo image is input, the three-dimensional measurement processing device removes lens distortion from the input stereo image each time a new stereo image is input. After performing the difference between frames based on the following formula, the square of the difference between the corresponding candidate pixels between the stereo images (SD) is calculated,

Where (u _b , v _b ) represents the standard image coordinates, d represents the parallax, and I _b (u, v, t) and I _r (u, v, t) refer to the standard image, respectively. This is the pixel value at the frame number t at the position (u, v) of the image. Next, when the inter-frame difference value is greater than or equal to a certain value, the calculated SD (u _b , v _b , d, t) is integrated into the SSD space having the reference image coordinates (u _b , v _b ) and the parallax d as axes,

This is more effectively achieved by obtaining a corresponding position between stereo images by searching for a minimum value in the parallax d direction in the SSD space.

ただし、（ｕ_ｂ,ｖ_ｂ）は基準画像座標を表し、ｄは視差を表し、また、Ｉ_ｂ（ｕ,ｖ,ｔ）とＩ_ｒ（ｕ,ｖ,ｔ）は、それぞれ基準画像と参照画像の位置（ｕ,ｖ）におけるフレーム番号ｔでの画素値であり、次に、フレーム間差分値が一定値以上のとき、次の数式に基づき、計算されたＳＤ（ｕ_ｂ,ｖ_ｂ,ｄ,ｔ）を、基準画像座標（ｕ_ｂ,ｖ_ｂ）と視差ｄを軸に持つＳＳＤ空間へ積算し、

前記ＳＳＤ空間における視差ｄ方向の最小値探索によって、ステレオ画像間の対応位置を求めることによってより効果的に達成される。 Further, the above-described object of the present invention is to capture the reference image by the imaging apparatus capturing three or more synchronized stereo cameras for capturing a stereo image, that is, one reference camera for capturing a reference image. Each time a new stereo image is input, the three-dimensional measurement processing device removes lens distortion from the input stereo image and Based on the mathematical formula, after performing the inter-frame difference, calculate the square (SD) of the difference between the pixel values of the corresponding candidate pixels between the stereo images,

Where (u _b , v _b ) represents the standard image coordinates, d represents the parallax, and I _b (u, v, t) and I _r (u, v, t) refer to the standard image, respectively. This is the pixel value at the frame number t at the position (u, v) of the image. Next, when the inter-frame difference value is greater than or equal to a certain value, the calculated SD (u _b , v _b , d, t) is integrated into the SSD space having the reference image coordinates (u _b , v _b ) and the parallax d as axes,

This is more effectively achieved by obtaining a corresponding position between stereo images by searching for a minimum value in the parallax d direction in the SSD space.

  Stereo measurement by the stereo three-dimensional measurement system according to the present invention is a simple three-dimensional measurement of a stationary object with simple active illumination on a stationary object to be measured and two or more synchronized stereo cameras. Interactive stereo measurement using time-series windows using time-series images taken.

  The “simple active illumination” used in the present invention only needs to move, for example, a flashlight or the shadow of a stick on the measurement target. Further, in the present invention, a change in pixel value in the time direction is used to detect corresponding points between stereo images.

  Therefore, according to the present invention, the “spatial direction window”, which is used when obtaining correspondence between stereo images, which is necessary for the conventional “passive stereo measurement method”, becomes unnecessary, and the time direction window is used. By doing so, it is possible to accurately obtain corresponding points between stereo images for each pixel (1 × 1 [pixel] spatial direction window), and thus it is possible to perform stereo three-dimensional measurement of a stationary object with high accuracy. There are excellent effects.

  Further, in the stereo three-dimensional measurement system according to the present invention, an expensive PC projector and “structured illumination”, which are necessary in the conventional “active stereo measurement method”, are not required. There is a remarkable effect that it can be easily performed at a price.

  Furthermore, when performing stereo three-dimensional measurement of a stationary object using the stereo three-dimensional measurement system according to the present invention, that is, during measurement, an input image (stereo image obtained by photographing a stationary object with a stereo camera). ) Is processed sequentially, and the measurement results obtained by processing are displayed in real time. Therefore, in order to obtain a satisfactory measurement result, the measurer can perform a simple active operation while checking the measurement result. It is possible to interactively adjust the position and intensity of illumination, the direction of illumination, and the like, and an interactive three-dimensional measurement system that displays measurement results in real time can be constructed.

  In short, according to the present invention, every time the latest stereo image is input, the latest measurement result is updated to the latest measurement result, and the latest measurement result is displayed on a display unit such as a display. By feeding back the latest measurement results to the measurer, there is an excellent effect that the measurer can interactively adjust how to apply a simple active illumination to the measurement target.

  First, the focus and outline of the present invention will be described.

  In order to search for a corresponding position between stereo images, the corresponding position for each small region is usually searched by simultaneously using pixels around the target position. This small area is called a “space direction window”, and it is assumed that the distance to the measurement target in the space direction window is uniform and that there is no deformation between the corresponding space direction windows.

  For this reason, as described in the background art, there is a problem that the measurement result becomes inaccurate at a portion where the distance is discontinuous such as an edge of the measurement target.

  Therefore, in the present invention, the concept of the “spatial direction window” is abolished, and a search for a corresponding position between stereo images for each pixel has been considered.

  At this time, when performing stereo measurement using two synchronized stereo cameras (base camera and reference camera), a pair of stereo images (that is, one base image obtained by capturing with the base camera and a reference) In a single reference image obtained by photographing with a camera, correspondence between stereo images cannot be uniquely determined, and three-dimensional measurement is impossible.

  However, a plurality of pairs of stereo images (hereinafter simply referred to as “multiple images”) while illuminating the measurement object with illumination that varies spatially and temporally (hereinafter such illumination is also simply referred to as “active illumination”). If the “stereo image” is used, the corresponding pixels between the stereo images should have the same brightness change in the time direction. Therefore, by using a plurality of stereo images in the time direction, an effect similar to that of the “spatial direction window” can be expected.

  Usually, in the “spatial direction window”, it is set as a small region having a size of about 11 × 11 [pixel]. That is, an area in the other image corresponding to the pixel value of 11 × 11 = 121 [pixel] is searched. In order to realize this as a time-series image, when an image is shot at 121 frames, that is, 30 FPS, the size of 11 × 11 [pixel] is simply obtained by using the image shot for about 4 seconds. The same effect as that of the “spatial direction window” can be expected.

  It is desirable that the “active illumination” given to the measurement object at this time can be used as simple as possible and at a low price. Actually, such “simple active illumination” can be realized by a hand-held flashlight, for example. However, it is unclear whether a “three-dimensional” measurement result can be obtained by illuminating “where” and “how much” the measurement target.

  Therefore, in the present invention, when a new (latest) stereo image is captured and input, the latest measurement result obtained by measuring the input latest stereo image is added to the previous measurement result, and the latest measurement is performed. By generating the results and displaying the latest measurement results generated in real time, the measurer adjusts the position and intensity to which "active illumination" is applied while checking the latest measurement results displayed in real time, An interactive stereo three-dimensional measurement system that ends measurement immediately when a satisfactory measurement result is obtained, or ends measurement when a predetermined condition (for example, a predetermined measurement time) is satisfied. Proposed.

  In short, the stereo three-dimensional measurement system according to the present invention performs three-dimensional measurement using simple active illumination on a stationary object to be measured and time-series images taken by two or more synchronized stereo cameras. This is a stereo three-dimensional measurement system that can perform and confirm the measurement result in real time.

  Hereinafter, the best mode for carrying out the present invention will be described in detail with reference to the drawings.

  FIG. 2A is a block diagram showing an example of an implementation configuration of a stereo three-dimensional measurement system according to the present invention. As shown in FIG. 2A, the stereo three-dimensional measurement system 1 of the present invention includes an imaging device 10, active illumination means 20, a three-dimensional measurement processing device 30, and a display device 40. .

  First, the imaging apparatus 10 is an apparatus for capturing a time-series image of a stationary object that is a measurement target. As an example, as shown in FIG. 2A, two synchronized stereo cameras are used. (A reference camera 11 for capturing a reference image and a reference camera 12 for capturing a reference image).

  The active illumination unit 20 is a unit that generates active illumination on a stationary object to be measured. The “active illumination” in the present invention means illumination that changes both spatially and temporally.

  As a specific example of illuminating a stationary object that is a measurement target with the active illumination unit 20, for example, it is only necessary to move a flashlight or a shadow of a stick on the measurement target.

  In the present invention, a change in illumination by the active illumination means 20 is given to the measurement object, and in this embodiment, a time series image (standard) of the measurement object is used by using the standard camera 11 and the reference camera 12 that constitute the imaging device 10. Image and reference image). The captured time-series images (standard image and reference image) are input to the three-dimensional measurement processing device 30.

  On the other hand, the three-dimensional measurement processing device 30 performs three-dimensional measurement processing using the input time-series images (standard image and reference image), and displays the generated measurement result on the display device 40.

  As the three-dimensional measurement processing device 30, for example, a CPU (Central Processing Unit) having a memory can be used. Further, as the display device 40, for example, a display such as a liquid crystal display can be used.

  FIG. 2B is an image conceptual diagram in the case of actually performing stereo three-dimensional measurement of a measurement target using the stereo three-dimensional measurement system 1 having the configuration of FIG.

  As conceptually shown in FIG. 2B, the stereo three-dimensional measurement system of the present invention is an interactive measurement system. When performing three-dimensional measurement, first, the illumination change by the active illumination means 20 is performed. The time series images (base image and reference image) of the measurement target 23 are photographed using the synchronized reference camera 11 and reference camera 12. In the present embodiment, the active illumination by the active illumination means 20 means that the measurer 21 moves while hitting the flashlight 22 held by the measurer 21 against the measurement target 23.

  The captured time-series images (standard image and reference image) are input to the three-dimensional measurement processing device 30, and the measurement results generated by the three-dimensional measurement processing in the three-dimensional measurement processing device 30 are real-time on the display device 40. Is displayed. The measurer 21 confirms the measurement result displayed on the display device 40 in real time, and the portion where the appropriate texture does not exist on the surface of the stationary object to be measured, that is, the illumination of the flashlight 22 is insufficient. Illuminate the position in a concentrated manner.

  Then, a new time-series image (standard image and reference image) captured using the synchronized reference camera 11 and reference camera 12 is input to the three-dimensional measurement processing device 30, and the input new time-series image ( A three-dimensional measurement process is performed based on the reference image and the reference image, and the generated measurement result is displayed on the display device 40 in real time.

  As described above, the stereo three-dimensional measurement by the stereo three-dimensional measurement system of the present invention is a three-dimensional measurement that gives an illumination change to a measurement target while feeding back and using the measurement result. For example, if the measurement result is incomplete (location where there is no appropriate texture on the surface of the stationary object being measured), or the stationary object being measured has a specular reflection component Can interact with the active lighting angle, etc. Further, the measurement can be terminated when the measurer is satisfied with the measurement result.

  In the stereo three-dimensional measurement system of the present invention, in order to search for corresponding positions between captured time-series images, 1 × 1 [pixel] in space and as many images as necessary in time are used. That is, in the present invention, the corresponding position is searched for each pixel of the image. In the present invention, the measurement result is displayed in real time in order to determine that the necessary number of images have been input in time. Further, in order to display in real time, the latest measurement result is updated and displayed every time one set of stereo images is input.

  Next, FIG. 3 shows the three-dimensional measurement processing by the stereo three-dimensional measurement system according to the present invention, that is, the three-dimensional measurement processing device 30 shown in FIG. 2 (hereinafter simply referred to as “three-dimensional measurement processing device”). 6 is a flowchart showing a flow of “three-dimensional measurement processing” performed in step S2.

  As shown in FIG. 3, in the present invention, first, stereo images (standard images and reference images) captured by the synchronized standard camera 11 and reference camera 12 are input to the three-dimensional measurement processing device (step S1). ).

  In the three-dimensional measurement processing device, each time a new stereo image is input, first, lens distortion aberration removal and rectification are performed on the input stereo image (hereinafter simply referred to as “input image”). (Parallelization) (see Non-Patent Document 7 and Non-Patent Document 8) is performed (step S2, step S3).

  Next, as shown in the following formula 1, after performing the inter-frame difference (step S5), the square of the difference (SD: Squared Difference) of the corresponding candidate pixels between the stereo images is calculated. (Step 6).

ただし、（ｕ_ｂ,ｖ_ｂ）は基準画像座標を表し、ｄは視差を表す。また、Ｉ_ｂ（ｕ,ｖ,ｔ）とＩ_ｒ（ｕ,ｖ,ｔ）は、それぞれ基準画像と参照画像の位置（ｕ,ｖ）におけるフレーム番号ｔでの画素値である。

However, (u _b , v _b ) represents reference image coordinates, and d represents parallax. I _b (u, v, t) and I _r (u, v, t) are pixel values at frame number t at the position (u, v) of the standard image and the reference image, respectively.

  Here, the reason for performing the inter-frame difference is to reduce the influence of the difference in brightness between images. Further, in order to perform the inter-frame difference, one frame delay is performed in the frame memory of the three-dimensional measurement processing device (step S4).

Next, as shown in Equation 2 below, the square of the difference between corresponding candidate pixels between stereo images, that is, SD (u _b , v _b , d, t) calculated by Equation 1 is used. As shown in FIG. 4, the integration is performed and arranged in the SSD space having the reference image coordinates (u _b , v _b ) and the parallax d as axes (step S7).

次に、ステレオ画像間の対応位置は、図４に示されたように、このＳＳＤ空間における視差ｄ方向の最小値探索によって求める（ステップＳ８）。

Next, the corresponding position between the stereo images is obtained by searching for the minimum value in the parallax d direction in the SSD space, as shown in FIG. 4 (step S8).

  Here, in actual measurement, there is no change in illumination in all frames of the time-series image. For example, when a hand-held flashlight is considered as a change in illumination by the active illumination means 20, almost no part of the image is illuminated by the flashlight and the pixel value does not change.

  Therefore, since the pixel values that do not change do not contribute to the corresponding point search, in the actual calculation, when calculating Equation 1, the inter-frame difference value of the reference image is compared with a certain threshold value, and the inter-frame difference of the reference image is calculated. Only when the difference value is equal to or greater than the threshold value, the SD calculated by Equation 1 is integrated into the SSD space by Equation 2 (see FIG. 5). By this processing, when the active illumination means 20 is flashlight illumination, the parallax calculation time can be shortened to about 1/3.

  Next, in the stereo three-dimensional measurement system of the present invention, back-matching is used to detect occlusion, that is, obtained by exchanging the parallax obtained by the normal search, the standard image, and the reference image. If the absolute value of the difference in parallax is greater than or equal to a predetermined threshold value, it is determined as occlusion (step S9).

  Next, the measurement result (disparity map and occlusion detection result) obtained by the above processing is displayed on the display device 40 (step S10).

  Next, for example, when the active illumination by the active illumination means 20 is the flashlight 22 held by the measurer 21, the measurer 21 displays the measurement result (disparity map and occlusion detection result) displayed on the display device 40. While confirming, move the flashlight to illuminate the measurement target. It is possible to efficiently measure a portion where the measurement result is incomplete by giving an illumination change intensively. The measurement ends when the measurer 21 is satisfied with the measurement result (step S11).

  As described above, the three-dimensional measurement processing device 30 performs the processing from step S2 to step S10 each time a new stereo image acquired by the imaging device 10 (base camera and reference camera) is input, in other words. The sequential measurement results are updated, and when a satisfactory measurement result is obtained, the stereo three-dimensional measurement process is terminated.

  Here, an example of the measurement result measured by the stereo three-dimensional measurement system of the present invention and displayed on the display device 40 is shown in FIG.

  From FIG. 6 (A) to FIG. 6 (C), it can be clearly seen that measurement results (parallax map and occlusion detection result) are gradually generated.

  The final measurement result is shown in FIG. The lower part of FIG. 6D shows a standard image and a reference image captured by two stereo cameras (a standard camera and a reference camera). 6D shows the estimated parallax map, and the upper right of FIG. 6D shows the detected occlusion region (white). The threshold for occlusion detection can be adjusted while checking the occlusion detection result.

  The measurer illuminates the measurement target by moving a flashlight or the like while checking the parallax map and the occlusion detection result. It is possible to efficiently measure a portion where the measurement result is incomplete by giving an illumination change intensively.

  As described above, the embodiment of the stereo three-dimensional measurement system of the present invention has been described. However, the imaging device 10 in the stereo three-dimensional measurement system of the present invention is not limited to two synchronized stereo cameras, It is possible to use a stereo camera, that is, three or more synchronized stereo cameras (one standard camera for capturing a reference image and two or more reference cameras for capturing a reference image). Needless to say. By using three or more synchronized stereo cameras as the imaging device 10, it is possible to reduce the ambiguity of the corresponding positions between the stereo images and the occlusion area (occlusion).

  Next, using the stereo three-dimensional measurement system of the present invention, a measurement experiment was performed on a stationary object having a complicated shape or a fine shape, and the effectiveness of the present invention was confirmed.

  As an image pickup apparatus, a camera called Flea 2 of Point Gray Research was used, and a 640 × 480 [pixel] gray scale image was taken with the camera. As active illumination, in order to shorten the measurement time, as shown in the photographed image of FIG. 6D, a still image with a random stripe pattern is input to the PC projector, and the measurement target is illuminated with the projector in hand. . A laser pointer was also used. Still image projection can be realized even with low-cost lighting such as a modified flashlight or a slide projector. The parallax estimation was performed on a pixel basis.

  FIG. 7 shows a captured image (standard image and reference image) and a parallax map. A black area in the parallax map shown in FIG. 7C indicates an area detected as occlusion. The measurement object is a flower having a complicated shape and includes many areas with discontinuous depth and occlusion areas. As can be seen from FIG. 7, good measurement results are obtained. In addition, when the plane area on the left side of the flower (corresponding to 159 × 342 [pixel], 90 × 200 [mm]) is used, the plane is fitted to the estimated three-dimensional position, and the standard deviation of the depth is calculated. It was 0.45 [mm]. The baseline length (baseline length) is 254 [mm], and the distance from the camera to the plane is about 640 [mm]. The display speed was about 3 [fps].

  FIG. 8 shows another measurement result. The measurement target includes a very fine shape such as a racket gut, but as shown in FIG. 8, good parallax estimation is performed. In addition, it can be confirmed that an occlusion area due to the gut portion in the back plane area is also detected. In the frame portion of the racket, an area erroneously detected as occlusion is a portion where the color is black and the reflectance is low.

It is a conceptual diagram for demonstrating the conventional "spacetime stereo" stereo measurement. FIG. 2A is a block diagram showing an example of an implementation configuration of a stereo three-dimensional measurement system according to the present invention. FIG. 2B is an image conceptual diagram in the case of actually performing stereo three-dimensional measurement of a measurement target using the stereo three-dimensional measurement system 1 having the configuration of FIG. It is a flowchart which shows the flow of the three-dimensional measurement process by the stereo three-dimensional measurement system which concerns on this invention. It is a schematic diagram for demonstrating SSD space and a minimum value search in the stereo three-dimensional measurement system which concerns on this invention. It is a schematic diagram for explaining selection of an integration frame in the stereo three-dimensional measurement system according to the present invention. It is a figure which shows the example of a display of the measurement result by the stereo three-dimensional measurement system which concerns on this invention. It is a figure which shows the measurement result (flower) by the stereo three-dimensional measurement system which concerns on this invention. It is a figure which shows the measurement result (racquet) by the stereo three-dimensional measurement system which concerns on this invention.

Claims (5)

A stereo three-dimensional measurement system that performs three-dimensional measurement of a measurement target that is a stationary object by searching for a corresponding position between stereo images,
An imaging device, an active illumination unit that generates active illumination that changes both spatially and temporally, a three-dimensional measurement processing device, and a display device,
While illuminating the measurement object with the active illumination generated from the active illumination means, a time-series stereo image is captured using the imaging device, and the captured stereo image is input to the three-dimensional measurement processing device. ,
In the three-dimensional measurement processing device, by searching for a corresponding position between stereo images using a plurality of frames in the time direction, the distance to the measurement target is measured, and the measurement result obtained by the measurement is displayed. A stereo three-dimensional measurement system that displays on a device, sequentially performs measurement processing each time a latest stereo image is input, and displays the measurement result on the display device.   The stereo three-dimensional measurement system according to claim 1, wherein a spatial direction area when searching for a corresponding position between stereo images is 1 × 1 [pixel].   The stereo three-dimensional measurement system according to claim 1, wherein the three-dimensional measurement processing device has a function of detecting occlusion between stereo images, and displays the detected occlusion on the display device. The imaging apparatus is composed of two synchronized stereo cameras for capturing a stereo image, that is, a reference camera for capturing a reference image and a reference camera for capturing a reference image.
Each time a new stereo image is input, the three-dimensional measurement processing device removes lens distortion from the input stereo image, and then performs an inter-frame difference based on the following equation. Calculating the square (SD) of the difference between the pixel values of the corresponding candidate pixels between the stereo images,

Where (u _b , v _b ) represents the standard image coordinates, d represents the parallax, and I _b (u, v, t) and I _r (u, v, t) refer to the standard image, respectively. The pixel value at frame number t at the position (u, v) of the image,
Next, when the inter-frame difference value is greater than or equal to a certain value, the calculated SD (u _b , v _b , d, t) is converted into the reference image coordinates (u _b , v _b ) and the parallax d based on the following formula: To the SSD space with

The stereo three-dimensional measurement system according to any one of claims 1 to 3, wherein a corresponding position between stereo images is obtained by searching for a minimum value in the parallax d direction in the SSD space. The imaging device includes three or more synchronized stereo cameras for capturing a stereo image, that is, one reference camera for capturing a reference image and two or more reference cameras for capturing a reference image. Configured,
Each time a new stereo image is input, the three-dimensional measurement processing device removes lens distortion from the input stereo image, and then performs an inter-frame difference based on the following equation. Calculating the square (SD) of the difference between the pixel values of the corresponding candidate pixels between the stereo images,

Where (u _b , v _b ) represents the standard image coordinates, d represents the parallax, and I _b (u, v, t) and I _r (u, v, t) refer to the standard image, respectively. The pixel value at frame number t at the position (u, v) of the image,
Next, when the inter-frame difference value is greater than or equal to a certain value, the calculated SD (u _b , v _b , d, t) is converted into the reference image coordinates (u _b , v _b ) and the parallax d based on the following formula: To the SSD space with

The stereo three-dimensional measurement system according to any one of claims 1 to 3, wherein a corresponding position between stereo images is obtained by searching for a minimum value in the parallax d direction in the SSD space.

Images