JP2015007637A - Three-dimensional measuring device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a three-dimensional measuring device that has improved the accuracy of three-dimensional measurement by reducing the deviation between a measurement point input into one of viewpoint images and a corresponding point set at the other of the viewpoint images.SOLUTION: A measurement point 7, which is a three-dimensional measurement reference, is input into a left viewpoint image 6L. A corresponding point 8a is set on an extension line extended horizontally from the measurement point 7 toward a right viewpoint image 6R. If a vertical deviation D occurs between the left and right viewpoint images 6L, 6R, and straight lines 24, 27 passing through the measurement point 7 and corresponding point 8a of the respective viewpoint images 6L, 6R intersect with a horizontal line with a small angle θ, the corresponding point 8a is erroneously set at a position far from a true corresponding point 8. As a countermeasure to this, a corresponding point detection direction search unit is provided that, at each viewpoint image 6L, 6R, searches for a new corresponding point from the erroneously set corresponding point 8a along the straight line 27, and replaces the corresponding point 8a with the new corresponding point 8.

Description

  The present invention relates to a three-dimensional measurement apparatus that performs three-dimensional measurement based on a parallax image.

  When each viewpoint image is captured from two different viewpoints and a specific point on the target object can be confirmed in each viewpoint image, the coordinates of the specific point and the distance from the specific point to the imaging position are calculated based on the principle of triangulation. Can lead. Patent Document 1 describes a three-dimensional measuring apparatus using this principle.

  In order to obtain three-dimensional information from a parallax image having a horizontal parallax direction, the horizontal lines of the two viewpoint images constituting the parallax image must be matched with the parallax direction correctly. For example, when two viewpoint images are arranged side by side, it is difficult to obtain correct three-dimensional information if the horizontal lines in the respective viewpoint images are relatively inclined. When taking a viewpoint image from each viewpoint, there is no problem if each imaging screen is maintained in a horizontal posture and the imaging viewpoint is on the same horizontal line.

  However, even in a stereo camera in which two imaging optical systems are provided in one housing, the housing is kept in a horizontal posture, for example, due to accumulation of errors at the time of manufacturing assembly or parts, or due to changes over time. Even if the images are picked up in this way, the individual viewpoint images may be picked up in a slightly inclined posture, or the viewpoint images may be picked up without being arranged on the same horizontal line. In order to cope with such a situation, prior to deriving three-dimensional information including the dimensions of the measurement object and the distance from the imaging position from the parallax image, two-dimensional projective transformation processing (hereinafter referred to as “two-dimensional projection conversion processing”) is performed. (Referred to as calibration), and the horizontal direction of each viewpoint image coincides with the parallax direction (hereinafter referred to as image parallelization).

JP 2000-28355 A

  If the parallax images are parallelized by calibration, the horizontal directions of the two viewpoint images can be made to coincide with each other. However, depending on the vibration of the device and the accuracy of calibration, when two viewpoint images are arranged horizontally on the basis of the screen frame, the position of a specific point in the viewpoint image is not aligned on the horizontal line but shifted in the vertical direction. Often occurs (hereinafter, vertical misalignment).

  When measuring 3D information from a parallax image, for example, a measurement instruction point is set in the left viewpoint image, a horizontal line passing through the measurement instruction point is extended toward the right viewpoint image, and a measurement instruction is included in the right viewpoint image. A process for searching for a corresponding point corresponding to the point is performed. When there is a straight line passing through the measurement instruction point in the left viewpoint image, the intersection of the horizontal line extending from the measurement instruction point to the right viewpoint image and the corresponding straight line in the right viewpoint image is recognized as a corresponding point. . However, when the vertical deviation described above exists between the left and right viewpoint images, and there is a straight line that passes through the measurement instruction point in the left viewpoint image, the smaller the slope of this straight line is, the closer to the horizontal, the more The coordinates of the intersection recognized as a point are likely to fluctuate. The measurement of the three-dimensional information is performed on the assumption that the measurement instruction point of the left viewpoint image and the corresponding point of the right viewpoint image are the same point in the three-dimensional space. Reliability is impaired.

  The object of the present invention is to search for a corresponding point corresponding to a measurement instruction point input to one viewpoint image in the other viewpoint image, even if there is a vertical shift between the respective viewpoint images, The object is to provide a three-dimensional measuring apparatus that minimizes the influence and does not reduce the measurement accuracy.

  The three-dimensional measurement apparatus of the present invention includes a parallax image storage unit, an image display unit, a measurement point input unit, a corresponding point search unit, a three-dimensional measurement unit, a straight line detection unit, an angle detection unit, and a horizontal determination. And a corresponding point detection direction search unit. The parallax image storage unit stores the first viewpoint image and the second viewpoint image. The image display unit displays at least one of the first and second viewpoint images. The measurement point input unit inputs a measurement point to one viewpoint image displayed on the image display unit. The corresponding point search unit searches for a corresponding point corresponding to the measurement point in the other viewpoint image in which the measurement point is not input. The three-dimensional measurement unit derives three-dimensional information of the subject based on the first and second viewpoint images, the measurement points, and the corresponding points. The straight line detection unit detects a straight line passing through the vicinity of the measurement point or the corresponding point. When a straight line is detected, the angle detection unit detects an angle at which the straight line intersects at an acute angle with respect to the horizontal direction of the viewpoint image. The horizontal determination unit determines that the detected angle is smaller than a preset threshold value S when the horizontal line is included, and if the detected angle is larger than the threshold S, the horizontal line is not included. The corresponding point detection direction searching unit searches for a new corresponding point along a straight line passing through the corresponding point from the viewpoint image from which the corresponding point is derived when the horizontal determining unit determines the horizon inclusion state. Do. When a new corresponding point is detected by the corresponding point detection direction search unit, the new corresponding point is used to derive the three-dimensional information instead of the original corresponding point.

  The image display unit may display the three-dimensional information derived by the three-dimensional measurement unit. According to the angle detected by the angle detection unit, the maximum value of the vertical image deviation generated between the first and second viewpoint images is Vmax, and the horizontal parallax error allowed in the three-dimensional measurement is When Umax is set, the value of the threshold S is preferably “S = Arctan (Vmax / Umax)”.

  You may provide the warning generation part which generate | occur | produces the warning from which the reliability of the three-dimensional information derived | led-out by a three-dimensional measurement part differs in steps. In particular, when a warning occurs, after a determination by the horizontal determination unit, when a specific image belonging to a certain area centered on the measurement point of the viewpoint image and a certain area are scanned along a straight line passing through the measurement point Compared with scanned images obtained sequentially from within a certain area during scanning, image similarity is determined, and when the similarity is below a preset reference level, the corresponding point detection direction search unit previously Since there is a high possibility that the searched new corresponding point is an appropriate corresponding point, the derivation of the three-dimensional information using the new corresponding point may be allowed.

  According to the present invention, even when a vertical shift of images occurs between viewpoint images and a linear pattern slightly tilted with respect to the parallax direction is present in the viewpoint image, the measurement input to one of the viewpoint images It is possible to prevent erroneous setting when setting a corresponding point corresponding to a point to the other viewpoint image, and to perform three-dimensional measurement with high accuracy.

It is explanatory drawing which shows the usage condition of the imaging device using this invention. It is explanatory drawing which shows the example of the imaged parallax image. It is a block diagram which shows the outline of a structure of the imaging device using this invention. It is a flowchart which shows an example of the process sequence by the imaging device of this invention. It is explanatory drawing when a vertical shift | offset | difference generate | occur | produces between viewpoint images with a small inclination linear pattern. It is explanatory drawing when the vertical shift generate | occur | produces between viewpoint images with a linear pattern with a big inclination. It is a block diagram which shows the outline of a structure of the imaging device which concerns on other embodiment. It is explanatory drawing which shows the scanning process at the time of determining the similarity of an image. It is a flowchart including the improved process sequence. It is a flowchart including a further improved processing procedure.

  As shown in FIG. 1, the imaging apparatus using the present invention is composed of a stereo camera 2, and a left-eye camera 2a and a right-eye camera 2b are provided side by side. The optical system used for each of the cameras 2a and 2b has the same optical specifications including the focal length. In addition, an image sensor is individually incorporated in the imaging plane of each optical system. These left-eye camera 2a and right-eye camera 2b constitute an imaging unit for parallax images.

  An image display panel 4 made of liquid crystal or the like is provided on the rear surface of the stereo camera 2. Live images captured by the cameras 2a and 2b are displayed side by side on the image display panel 4. When the shutter button 5 is operated, the main imaging is performed following the live image displayed at that time, and the image data of the left viewpoint image and the right viewpoint image are associated with each other and recorded in the same image file. . Note that tag data including shooting conditions such as shooting distance and shooting magnification is also recorded in the same image file.

  FIG. 1 illustrates an example in which furniture is imaged as the subject 3 and its height H is a target for three-dimensional measurement. As shown in FIG. 2, the left viewpoint image 6L and the right viewpoint image 6R obtained as live images are displayed side by side on the image display panel 4. At this time, the stereo camera 2 is provided with a calibration function so that the left viewpoint image 6L and the right viewpoint image 6R can be correctly aligned and displayed in a horizontal orientation. However, the calibration function is also limited, and as shown in FIG. 2, the left viewpoint image 6L and the right viewpoint image 6R may be displayed in a posture in which the screen is shifted up and down and one of them is slightly rotated. .

  In the three-dimensional measurement using the stereo camera 2, when a measurement point 7 serving as a reference is designated in the left viewpoint image 6L displayed on the image display panel 4, the corresponding point 8 corresponding to the measurement point 7 is the other right viewpoint. It is automatically set on the image 6R. In order to prevent an error from entering the corresponding point 8 calculation process as much as possible, the stereo camera 2 is provided with a unique configuration shown by functional blocks in FIG. Since the measurement point 7 is usually specified on the subject 3 to be measured, it is input on the contour line or the shape line as shown in FIG. Often entered on top of.

  The imaging unit 11 includes lenses 2 a and 2 b and image sensors individually provided on the imaging surfaces thereof, and imaging signals sequentially obtained from the image sensors are input to the signal processing unit 12. The signal processing unit 12 performs known image signal processing, and the image data of the left viewpoint image 6L and the right viewpoint image 6R obtained as live images are sequentially rewritten on the VRAM 13 connected to the signal processing unit 12. Further, the left viewpoint image 6L and the right viewpoint image 6R can be displayed on the image display panel 4 by the image data read from the VRAM 13 at an appropriate timing.

  The signal processing unit 12 includes a straight line detection unit 14 and an angle detection unit 15. As shown in FIG. 2, for example, when the measurement point 7 serving as a reference for the three-dimensional measurement is designated on the left viewpoint image 6L, the straight line detection unit 14 displays a part of the image on or near the measurement point 7. It is detected whether or not any straight line segment constituting the line passes. Note that the vicinity in this case is determined according to the number of pixels and the pixel size of the image sensor, and can be determined within a range of five pixels from the measurement point 7 to the periphery, for example. The angle detection unit 15 detects how much the straight line detected by the straight line detection unit 14 is at an angle with respect to the horizontal line. The horizontal reference in this case is based on the pixel array of the image sensor that is capturing the left viewpoint image 6L in which the measurement point 7 is specified.

  If the angle of the straight line detected by the angle detection unit 15 is close to horizontal, there is a risk of adversely affecting the reliability when the three-dimensional information is derived. The horizontal determination unit 16 determines whether or not the angle of the detected straight line is smaller than a preset threshold S. Since the inclination of the straight line with respect to the horizontal is not particularly directional, the angle detection unit 15 detects the degree of the angle that intersects the horizontal line at an acute angle.

  The operation input unit 10 includes a touch panel and cursor operation buttons for inputting the measurement points 7 described above, and a shutter button 5 which is operated when leaving a parallax image for 3D measurement as a record. It is also used for the input part of the measurement point that becomes the measurement reference. The signal processing unit 12 is further connected with a warning generation unit 19 and an image data memory 20. The warning generation unit 19 performs a warning operation when the evaluation by the straight line detection unit 14 and the horizontal determination unit 16, that is, whether or not an appropriate image can be recorded as an image for three-dimensional measurement is NG. As such warning processing by the warning generation unit 19, generation of a notification sound, display of a warning on the image display panel 4, a release lock operation for prohibiting the pressing operation of the shutter button 5, or the like can be used.

  The image data memory 20 can store image data of the left viewpoint image 6L and the right viewpoint image 6R that are appropriately captured by operating the shutter button 5. When the image data is stored, control by the recording control unit 17 may be added. For example, the left viewpoint image 6L and the right viewpoint image 6R captured by the operation of the shutter button 5 are once recorded in a predetermined area of the VRAM 13, but as described above, it may not always be appropriate for derivation of three-dimensional information. . Therefore, in such a case, recording / saving of the image data in the image data memory 20 can be automatically stopped by a command from the recording control unit 17. Note that the image data of the parallax image that is actually captured by the operation of the shutter button 5 can be temporarily written in the image data memory 20. In this case, when it is determined that the image data is inappropriate for the three-dimensional measurement, it is possible to control to automatically erase the written image data. In these cases, it is desirable to perform confirmation display on the image display panel 4.

  The process performed by the stereo camera 2 will be described using the flowchart shown in FIG. In the imaging process S2, in response to the pressing operation of the shutter button 5, the left viewpoint image 6L and the right viewpoint image 6R captured by the left eye camera 2a and the right eye camera 2b are captured. The pair of viewpoint images is recorded and temporarily stored in the image data memory 20 in a form associated with the same image file. Then, the viewpoint images 6L and 6R are displayed on the image display panel 4 by the display process S3. The measurement point input process S4 is a process of designating a measurement point 7 that serves as a reference in the left viewpoint image 6L displayed on the image display panel 4. For this processing, an operation from the operation input unit 10, for example, a cursor operation with a mouse or a touch operation on a touch panel incorporated in the image display panel 4 can be used.

  In the straight line detection process S5, the straight line detection unit 14 detects a straight line passing directly above or near the measurement point 7. When it is detected in the straight line determination process S6 that there is a corresponding straight line, the angle detection unit 15 performs an angle detection process S7, and an angle θ intersecting the horizontal line on the acute angle side is detected. Then, it is determined whether or not the angle θ is larger than the threshold value S by the horizontal determination process S8. If the angle θ is larger than the threshold value S, the horizontal determination unit 16 determines that the horizontal line is not included, and determines that the presence of the straight line has little influence on the derivation of the three-dimensional information. Note that the three-dimensional information includes the entire or partial size of the subject that is the subject of the parallax image, the distance from the imaging position to the subject, and the like.

  Even in the case where no straight line pattern exists in the vicinity of the measurement point 7 in the straight line detection process S5, it is determined that there is no particular problem in the derivation of the three-dimensional information, as in the case where the horizontal line is not included. In these cases, the left viewpoint image 6L and the right viewpoint image 6R picked up this time can be classified as parallax images suitable for three-dimensional measurement, and the recording control unit 17 uses the image recording process S9 to display images related to the parallax images. The data is recorded and stored in the image data memory 20 as data in a common file together with tag data such as shooting conditions. Further, the angle θ detected in the angle detection process S7 may be additionally recorded in one of the tag data.

  When a straight line is detected by the straight line determination process S6 and the straight line is less than or equal to the threshold value S by the horizontal determination process S8, it is determined that the horizontal line is included, and the parallax images are not suitable for three-dimensional measurement. Therefore, the parallax image is preferably classified and handled as a parallax image suitable for three-dimensional measurement. Then, for example, it is possible to perform control such that the recording process by the image recording process S9 is not performed only on a parallax image that is not suitable for three-dimensional measurement. In addition, when recording is performed by the image recording process S9, it is also possible to perform control so that only the parallax image that is not suitable for the three-dimensional measurement is subjected to the recording process with the image being not suitable for the three-dimensional measurement.

  In addition, when a parallax image that is not suitable for three-dimensional measurement is captured, the measurement accuracy of the three-dimensional information is reduced along with the display of the angle θ measured in the angle detection process S7 on the image display panel 4. It is desirable to perform display or display that the saving to the image data memory 20 has been stopped. Of course, regardless of the warning, an operation for forcibly saving the captured parallax image can also be included as an option. In this case, it is preferable to save the content of the warning together with the image data. Furthermore, it is possible to prompt the re-imaging process S11 as shown after the warning process S10 in FIG. In performing the warning process, it is possible to change the warning level step by step according to the angle derived by the angle derivation process S7. Note that the warning may be generated by sound or voice in addition to the screen display of images and marks.

  The above criteria for determining whether or not it is suitable for the measurement of the three-dimensional information is based on the following based on the derivation process of the three-dimensional information. When the captured left viewpoint image 6L and right viewpoint image 6R are parallelized by calibration, two viewpoint images are arranged horizontally. However, depending on variations in assembly accuracy of the stereo camera 2, changes with time, calibration accuracy, and the like, the pair of viewpoint images is not completely horizontally arranged, and vertical misalignment often occurs. The vertical deviation D shown in FIG.

  Further, when the three-dimensional information is derived from the captured left viewpoint image 6L and right viewpoint image 6R, it is set on the subject 3 of the left viewpoint image 6L in order to derive the parallax of the subject 3 to be measured. It is essential to obtain the corresponding point 8 corresponding to the measurement point 7 in the right viewpoint image 6R. The corresponding point 8 is searched and acquired in the horizontal direction from the measurement point 7 on the screen in which the left and right viewpoint images 6L and 6R are arranged and displayed horizontally.

  As shown in FIG. 5A, when the measurement point 7 input to the left viewpoint image 6L is acquired and a substantially horizontal straight line 24 is detected at the center of the measurement point 7 by the straight line detection process S5, the measurement point 7 is directed toward the right viewpoint image 6R. The corresponding point 8 is searched horizontally from the measurement point 7. If a vertical shift D occurs between the two images, the straight line 27 of the right viewpoint image corresponding to the straight line 24 of the left viewpoint image 6L is also displaced upward by the vertical shift D. For this reason, when the corresponding point 8 is searched horizontally from the measurement point 7, the corresponding point 8 a that is larger than the true corresponding point 8 and moves to the left is set with respect to the abscissa of the screen. Then, since the three-dimensional information is derived on the assumption that the measurement point 7 and the corresponding point 8a on the left and right viewpoint images 6L and 6R are the same point of the subject 3, a decrease in accuracy cannot be prevented. .

  On the other hand, as shown in FIG. 5B, when the measurement point 7 input to the left viewpoint image 6L is acquired and only the straight line 31 greatly inclined from the horizontal is detected at the center by the straight line detection processing S5, The setting error of 8 becomes minor. Even if a similar vertical deviation D occurs between the images 6L and 6R, when the corresponding point 8 is searched in the horizontal direction from the measurement point 7, the corresponding point 8b is obtained for the straight line 34 corresponding to the straight line 31. Therefore, it does not displace significantly compared with the true corresponding point 8.

  Therefore, in the situation shown in FIG. 5A, the left and right viewpoint images 6L and 6R are preferably not avoided because they are not suitable for three-dimensional measurement. On the other hand, under the situation shown in FIG. 5B, an inadvertent error is unlikely to occur, and can be regarded as suitable for three-dimensional measurement. By performing such a determination process shown in FIG. 4, it is possible to prevent an inadvertent parallax image from being captured while the stereo camera 2 aims to derive three-dimensional information. Thereby, according to the parallax image appropriately imaged by the stereo camera 2, it is possible to derive three-dimensional information with high accuracy, and the reliability of the stereo camera 2 for three-dimensional measurement can be improved.

A method for setting the threshold value S in the horizontal determination process S8 from the allowable maximum displacement of the corresponding point will be described below. When the magnitude of the vertical deviation described above is V and the angle derived by the angle detection process S7 is θ, the corresponding point U is a displacement U that occurs in the horizontal direction with respect to the true corresponding point.
U = V × cot θ
It is expressed. Assuming that the maximum value of vertical deviation is Vmax and the maximum allowable displacement in the horizontal direction is Umax, Vmax is substituted for V in the above equation, Umax is substituted for U, and it is set for horizontal determination processing S8 by solving for θ. The threshold S to be used is S = Arctan (Vmax / Umax) (1)
Is derived. Thus, it is desirable to set the threshold value S in the horizontal determination process S8. In addition, since the error caused by the imaging magnification changes, it is desirable to change the threshold value S according to the change of the imaging magnification.

  The stereo camera 2 is used as an imaging device for three-dimensional measurement, and the captured image data of the left viewpoint image 6L and the right viewpoint image 6R are input together with tag data to a three-dimensional measuring device configured with a personal computer. You can also. In this case, it is sufficient for the stereo camera 2 to determine whether or not the parallax image captured with reference to the measurement point 7 is appropriate. However, when the stereo camera itself has a function of performing three-dimensional measurement, it is necessary to add the above-described corresponding point search function to the stereo camera.

  The improved stereo camera shown in FIG. 6 includes a corresponding point search unit 35 having a horizontal search unit 36 and a detection direction search unit 37 in the signal processing unit 12 of the stereo camera 2 of the first embodiment, and a similar line determination unit. 38, and a three-dimensional measuring unit 40. As described with reference to FIGS. 5A and 5B, the horizontal search unit 36 searches from the measurement point 7 input to the left viewpoint image 6L to the corresponding point 8 in the right viewpoint image 6R in the horizontal direction. The search action is performed. The detection direction search unit 37 detects a straight line passing directly above or near the input measurement point, and when the angle θ is equal to or smaller than the threshold S, the detection direction search unit 37 newly adds the straight line passing directly above or near the corresponding point. Has the effect of searching for a corresponding point.

  The similarity determination unit 38 functions when a measurement point 7 is input to the left viewpoint image 8L, a straight line passing immediately above or near the detection point 7 is detected, and the angle θ is equal to or less than the threshold value S. As shown in FIG. 7, the function of the similarity determination unit 38 is to set an image confirmation area 45 of a certain size with the measurement point 7 on the detected straight line 44 as the center, and virtually set the measurement point 7 to a straight line. Scan along 44. The image confirmation area 45 moves together with the virtual measurement points 7a and 7b accompanying the scanning as indicated by the broken line, and the similarity of the images in the image confirmation area 45 is determined and evaluated during this movement. Note that scanning is performed in both directions so that the image confirmation area 45 covers the entire length of the straight line 44.

  The criterion for similarity is an image in the image confirmation area 45 centered on the initial measurement point 7, and the image in the image confirmation area 45 obtained for each scan is represented by the similarity or pattern of density distribution or color distribution. It can be determined by recognition or the like. In order to increase the efficiency of the determination process, for example, points corresponding to the level of similarity are set for each evaluation item such as the above-described density distribution, color distribution similarity, and pattern similarity, and the total points are set in advance. A method such as comparison with a threshold can be employed. The size of the image confirmation area 45 may be appropriately adjusted depending on the number of pixels, for example, 50 × 50 pixels.

  The function of the second embodiment will be described using the flowchart shown in FIG. In the stereo camera shown in FIG. 6, in addition to the processing shown in FIG. 4, processing by the similarity determination unit 38, the corresponding point search unit 35, and the three-dimensional measurement unit 40 is possible. Therefore, this stereo camera can be regarded as a three-dimensional measuring apparatus integrated with an imaging unit.

  After performing the measurement point input process S4 as in the previous embodiment, the horizontal search unit 36 performs a corresponding point horizontal search process S11 for the right viewpoint image 7R before the straight line detection process S5. As described above, the corresponding point 8 is acquired for the measurement point 7. Subsequently, a straight line detection process S5 and a straight line determination process S6 are performed. When the straight line angle θ exceeds the threshold value S, it is determined that the captured parallax image is appropriate and the three-dimensional information deriving process S12 is performed. Used. The derived three-dimensional information is displayed on the image display panel 4 and stored in the image data memory 20 together with the image data of the parallax image as tag data.

  When the angle θ of the straight line is equal to or smaller than the threshold value S, there is a concern that an error is included in the search result of the corresponding point 8 as described with reference to FIG. 5A. For this reason, the corresponding point search process S13 by the detection direction search unit 37 is performed following the horizontal determination process S8. In this search process S13, a new corresponding point is searched for along the corresponding point acquired in the previous horizontal search process S11 along a straight line passing through the corresponding point or its vicinity. If it demonstrates using FIG. 5A, it is the process which searches the new corresponding point 8 from the original corresponding point 8a in the right viewpoint image 6R. When determining whether or not it corresponds to a new corresponding point 8, the in-area image of a certain size in the left viewpoint image 6 </ b> L centering on the measurement point 7 and the newly searched corresponding point are the center. Based on the similarity to the in-area image of a certain size in the right viewpoint image 6R, a higher similarity is specified as a new corresponding point.

  As a result of this search processing S13, when the detection of a new corresponding point is confirmed in the detection confirmation processing S14, the original corresponding point is replaced with a new corresponding point by the corresponding point replacement processing S15. As a result, a more reliable corresponding point is searched, and it is possible to measure the displacement of the corresponding point with high accuracy even for a parallax image that is originally not suitable for three-dimensional measurement. Become. If a new corresponding point is not detected, a warning process S16 is performed to the effect that the position of the corresponding point is not reliable. As this warning process 16, it is also possible to prompt a re-imaging operation as in the previous embodiment. When re-imaging is performed, it is preferable to perform imaging in a manner different from the previous imaging. As an aid, it is also effective to provide an additional function in the imaging unit 11.

  As shown in FIG. 6, the left-eye camera 2a and the right-eye camera 2b are combined with a rotation mechanism 42, and the left-eye camera 2a and the right-eye camera 2b are rotated at the same angle around the respective imaging optical axes by rotation of the motor. Be ready to rotate. As the rotation mechanism 42, a gear may be fixed to the outer periphery of the lens barrel of the left eye camera 2a and the right eye camera 2b, and an interlocking gear driven by a motor may be engaged with both. In the warning process S16, re-imaging is prompted, and in response to this, if the left eye camera 2a and the right eye camera 2b are rotated by the rotation mechanism 42, straight lines in the left and right viewpoint images to be imaged are greatly inclined and no horizontal line is included. Since it becomes a state, the probability that the parallax image suitable for derivation | leading-out of three-dimensional information can be imaged increases.

  Further, the corresponding point search processing using the similarity determination unit 38 will be described with reference to the flowchart of FIG. When the straight line angle θ is equal to or smaller than the threshold S in the horizontal determination process S8 and it is determined that the horizontal line is included, the pair of viewpoint images 6L and 6R are classified as unsuitable for three-dimensional measurement. However, the reason why it is not suitable for the three-dimensional measurement is that the reliability of the position of the corresponding point is low. Therefore, if the reliability can be increased, it can be used for the three-dimensional measurement. Of course, the corresponding point search process S13 and the replacement process S15 by the detection direction search unit 37 are effective in increasing the reliability at the time of deriving the corresponding point, but the images near the straight line detected around the measurement point are similar. If it is, it means that there is a high probability that a similar image exists along the vicinity of the straight line on the right viewpoint image 6R side, so that a true corresponding point is very difficult to search.

  In the left viewpoint image 6L shown in FIG. 7, the image in the image confirmation area 45 centered on the measurement point 7 and the measurement points 7a and 7b after being moved along the straight line 44 passing through the measurement point 7 are centered. A case where the image in the image confirmation area 45 is similar corresponds to a case where it is difficult to derive a true corresponding point. On the other hand, a case where there is no similarity between these images corresponds to a case where a true corresponding point can be derived. Therefore, as shown in FIG. 8, when it is determined in the horizontal determination process S8 that the horizon line is included, the image reading process S18 in the image confirmation area and the similarity of the read image with respect to the left viewpoint image 6L. It is effective to perform a similarity determination process S20 that is a combination of the determination process S19.

  If the similarity determination process S20 determines that the similarity is high, it means that the image in the image confirmation area 45 centered on the input measurement point 7 is not characteristic. For this reason, even if a new corresponding point is obtained in the right viewpoint image by performing the search process S13, it is a search result in a similar image, and the probability of being a true corresponding point is low. . Therefore, when it is determined that the similarity is high in the similarity determination process S20, it can be said that it is more efficient to shift the process to the warning process S16.

  Conversely, if it is determined that the similarity is low, it means that the image around the input measurement point 7 has a feature. Therefore, in the process of searching for a new corresponding point in the search process S13, it is possible that an image of a certain size centered on the new corresponding point shows higher similarity to an image of a certain size centered on the measurement point. Increases the nature. When a new corresponding point is searched for in this way, the original corresponding point is replaced with a new corresponding point, and the accuracy of the three-dimensional measurement can be improved.

  In the warning process S16, it is desirable to display on the image display panel 4 including the determination result in the similarity determination process S20. Similarly to the previous embodiments, when the result of the horizontal determination process S8 is displayed or when the image recording process S9 is performed, the result of the straight line determination process S6, the result of the horizontal determination process S8, or the like together with the image data. The result of the sex determination process S20 may be recorded. Further, when the left-eye camera 2a and the right-eye camera 2b are rotated around the optical axis using the rotation mechanism 42 at the time of re-imaging, rotation angle information is added to the image data of the parallax image obtained by re-imaging. It is good to keep.

  The embodiment described above basically relates to an imaging apparatus that includes the imaging unit 11, but the present invention can also be used as a three-dimensional measurement apparatus that does not include the imaging unit 11 as a component. In this case, the operation input unit 10 shown in FIG. 6 corresponds to a keyboard or touch panel type operation input unit, the image display panel 4 corresponds to a monitor for a personal computer, and the image data memory 20 corresponds to an internal memory or an external memory of the personal computer. What is necessary is just to give the function of the signal processing apparatus 12 to the arithmetic processing part of a personal computer.

If parallax image data serving as external data is input to an internal memory or an external memory of a personal computer, a three-dimensional shape can be measured in exactly the same manner using the function of the arithmetic processing unit of the personal computer. Furthermore, since the substantial measurement function by the three-dimensional measuring apparatus of the present invention is executed by each processing unit included in the signal processing unit 12 corresponding to the arithmetic processing unit of the personal computer, the present invention provides the three-dimensional measuring apparatus. It can also be implemented in the form of a functioning measurement program.

  In the various embodiments described above, the left viewpoint image and the right viewpoint image arranged and displayed as the first and second viewpoint images have been described. However, the direction of the parallax does not necessarily have to be the horizontal direction. . In addition, the input of the measurement point is not limited to the left viewpoint image, and may be any one of at least two types of viewpoint images. In order to search for a corresponding point from the measurement point, for example, a corresponding point corresponding to the measurement point may be searched using coordinates from specific fixed points such as four corners in each viewpoint image.

2 Stereo camera 2a Left eye camera 2b Right eye camera 4 Image display panel 6L Left viewpoint image 6R Right viewpoint image 7 Measurement point 8 Corresponding point

Claims (5)

A parallax image storage unit that stores the first viewpoint image and the second viewpoint image;
An image display unit for displaying at least one of the first and second viewpoint images;
A measurement point input unit for inputting a measurement point to one of the first and second viewpoint images;
A corresponding point search unit for searching for a corresponding point corresponding to the measurement point on the other of the first and second viewpoint images;
A three-dimensional measurement unit that derives three-dimensional information about the subject based on the first and second viewpoint images, the measurement points, and the corresponding points;
A straight line detector that detects a straight line passing through the vicinity of the measurement point or the corresponding point;
When a straight line is detected by the straight line detection unit, an angle detection unit that detects an angle at which the straight line intersects at an acute angle with respect to a horizontal direction of the viewpoint image;
A horizontal determination unit that determines that the angle is smaller than a preset threshold S, a horizontal line containing state, and a case where the angle is larger than the threshold S, a horizontal line non-containing state;
A corresponding point detection direction search unit that searches for a new corresponding point along a straight line passing through the corresponding point from the viewpoint image from which the corresponding point is derived when the horizontal determining unit determines that the horizontal line is included; Have
When the corresponding point detection direction searching unit detects a new corresponding point, the corresponding point is replaced with the new corresponding point, and the three-dimensional measuring unit is configured to perform a three-dimensional measurement on the subject based on the new corresponding point. A three-dimensional measuring device that derives information.   The three-dimensional measurement apparatus according to claim 1, wherein the image display unit displays the three-dimensional information derived by the three-dimensional measurement unit. When the maximum value of the vertical image shift amount generated between the first and second viewpoint images is Vmax, and the horizontal parallax error allowed in the three-dimensional measurement is Umax, the threshold S is
S = Arctan (Vmax / Umax)
The three-dimensional measuring apparatus according to claim 1 or 2.   4. The warning generation unit according to claim 1, further comprising a warning generation unit that generates a warning in which the reliability of the three-dimensional information derived by the three-dimensional measurement unit varies stepwise according to the magnitude of the angle detected by the angle detection unit. The three-dimensional measuring apparatus of any one of them. Upon occurrence of the warning, after determination by the horizontal determination unit, along a specific image within a certain area centered on the measurement point of the viewpoint image to which the measurement point is input and a straight line passing through the measurement point When the predetermined area is scanned, the similarity of the image is determined by comparing with the scanned image sequentially obtained from within the constant area being scanned, and when the similarity is below a preset reference level, The three-dimensional measurement apparatus according to claim 4, wherein the three-dimensional measurement unit is allowed to derive three-dimensional information after replacement with a new corresponding point searched by the corresponding point detection direction search unit.

Images