JP2008241491A - Three-dimensional measurement instrument - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To allow precise three-dimensional measurement even when zooming and focusing are varied, in a three-dimensional measuring instrument that uses a stereo camera, having zooming and focusing functions. <P>SOLUTION: This three-dimensional measuring instrument has a plurality of imaging means including one or more of imaging means that are variable in the zooming and focusing; a control means for the zooming and focusing; a stereo image processing means for stereo-image-processing photographed images output from the plurality of imaging means, and for measuring three-dimensional information; and a camera information storage means for storing camera information necessary for carrying out the stereo image processings. The camera information storage means has the plurality of pieces of camera information, in response to conditions regarding the camera information varied, depending on the zooming condition and the focusing condition of each imaging means, and the stereo image processing means acquires the zooming condition and the focusing condition of each imaging means and acquires the corresponding camera information from a camera information storage part to be used for the stereo image processing. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a three-dimensional measurement apparatus including stereo imaging means and stereo image processing means.

  As background art in this technical field, for example, there is JP-A-2004-37270 (Patent Document 1). The gazette aims to “provide a calibration data measurement device that eliminates the effects of lens distortion for cameras that shoot with varying optical conditions”. A calibration data measuring apparatus for obtaining calibration data of the camera 2 using a plurality of chart photographed images obtained by photographing the calibration chart 1 taken by the camera 2 that photographs the optical chart by changing the optical conditions, A mark extracting unit 131 for extracting a mark from the chart photographed image, an internal parameter calculating unit 134 for calculating calibration data under a photographed optical condition from the mark position of the extracted mark, and calibration data; And a plurality of optical conditions in which the chart photographed image is photographed. Technology that and an internal parameter function calculating unit 160 for calculating the calibration data corresponding to the changing photographing optical conditions of the camera 2 "is disclosed.

JP 2004-37270 A

  In a stereo camera or a multi-camera having a plurality of imaging units, it is possible to measure three-dimensional information of a subject by comparing image information of the subject in each captured image. In order to perform accurate three-dimensional measurement at that time, the influence of image distortion caused by the optical characteristics of the zoom lens and the focus lens is removed, and the camera focal length and image center, which are called camera parameters, are removed. It is necessary to accurately grasp geometric information such as the shear and shear (hereinafter referred to as internal parameters), and the position / orientation (hereinafter referred to as external parameters) in the camera space.

  In the above-mentioned Patent Document 1, an internal parameter function indicating a change in internal parameter when the zoom focal length or focus position is changed as a photographing optical condition is created at the time of calibration, and based on the measured camera focal length at the time of operation. The other internal parameters are calculated from the internal parameter function, and the optimal internal parameters are obtained by performing bundle adjustment using them as initial values. However, it is not suitable for real-time operation using a video camera or the like because bundle adjustment is performed, and a general zoom variable camera can change the focus position to adjust the focus after zooming. Since it is necessary to move the focus in a coordinated manner, it is not possible to obtain a sufficiently accurate parameter simply by having a set of internal parameters corresponding to only one change in zoom or focus.

  In addition, since only the lens distortion correction information for the monocular camera and the internal parameter function indicating the change of the internal parameter are stored in the above-mentioned Patent Document 1, the external parameter must be calculated each time when performing stereo image processing. There's a problem.

  Therefore, in the present invention, in a stereo camera including two or more imaging units, a plurality of pieces of information corresponding to a combination of zoom and focus are related to the external parameters between the imaging units for information on lens distortion and internal parameters of each imaging unit. As for information, common information is stored as a table regardless of zoom and focus, so that even when zooming or focusing is performed, 3D measurement is performed with high accuracy using appropriate lens distortion information and camera parameters. With the goal.

  The above object can be achieved by the invention described in the claims.

  According to the present invention, in a three-dimensional measuring apparatus using a stereo camera with variable zoom and focus, even when zooming or focusing is performed, high-precision three-dimensional measurement is performed using appropriate lens distortion information and camera parameters. Is possible.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a schematic diagram showing a three-dimensional measuring apparatus according to a first embodiment of the present invention. In FIG. 1, 0101_1 is a first imaging unit, 0101_2 is a second imaging unit, 0102 is a zoom / focus control unit, 0103 is a stereo image processing unit, and 0104 is a camera information storage unit. Reference numeral 0110 denotes a stereo camera unit.

  In the three-dimensional measurement apparatus shown in FIG. 1, the first imaging unit 0101_1 and the second imaging unit 0101_2 are imaging units with variable zoom and focus, respectively, a lens group including a zoom lens and a focus lens, a shutter, and imaging. It is composed of elements, AGC, AD, camera signal processing DSP, timing generator, etc., and performs imaging by photoelectric conversion and outputs image data. In order to perform stereo image processing with high accuracy, it is preferable that the captured image of the first imaging unit 0101_1 and the captured image of the second imaging unit 0101_2 are captured by simultaneous exposure. The zoom / focus control unit 0102 receives the first image pickup unit 0101_1 or the second image pickup unit 0101_1 based on a user input input from a user input unit (not shown) or an image recognition result acquired from an image recognition unit (not shown). The zoom magnification and focus position of the imaging unit 0101_2 are controlled. In order to perform stereo image processing with high accuracy, it is preferable that the zoom magnification and the focus between the imaging units are matched as much as possible. However, in using this configuration, it is not always necessary that the zoom magnification and focus between the imaging units match. For determining the zoom magnification and focus position of the first imaging unit 0101_1 or the second imaging unit 0101_2, for example, the step position of the stepping motor that operates the zoom lens and the step position of the stepping motor that operates the focus lens are used. Generally, the zoom / focus control unit 0102 can control the zoom magnification and the focus position by controlling the movement amount of the stepping motor from the current step position by serial communication or the like. The stereo image processing unit 0103 acquires the image data output from the first imaging unit 0101_1 and the second imaging unit 0101_2 as a stereo image. Also, the zoom / focus control unit 0102 acquires zoom and focus information of the first imaging unit 0101_1 and the second imaging unit 0101_2, and the camera information storage unit 0104 acquires camera information corresponding to the zoom and focus information. To do. The stereo image processing unit 0103 performs stereo image processing of the stereo image using this camera information, performs 3D measurement of the subject, and evaluates the reliability of parallax information, distance information, 3D position information, and 3D measurement. Outputs values etc. as 3D information. Stereo image processing includes lens distortion correction processing, parallelization processing, preprocessing such as low-pass filter processing for noise removal, feature amount calculation processing such as edge detection, normalized cross-correlation, sum of absolute differences, etc. Post-processing that removes singularities by means of stereo matching processing that retrieves disparity information by searching for corresponding points between stereo images using various correlation calculation processing such as block matching and space-sweep method, rank filter processing and labeling , Distance calculation processing that calculates distance information using parallax information, etc., output information obtained in the middle of processing, that is, for example, coordinates of corresponding points on the image when corresponding point search is performed by stereo matching processing, etc. You can also The camera information storage unit 0104 is a storage device such as an EEPROM or a flash memory, and stores information necessary for performing stereo image processing using image data output from the first imaging unit 0101_1 and the second imaging unit 0101_2. Stored as camera information. The camera information includes, for example, lens distortion correction information and internal parameters for each imaging unit of the first imaging unit 0101_1 and the second imaging unit 0101_2, and external parameters between imaging units. Among these, when storing lens distortion correction information or internal parameters that change depending on the zoom magnification and focus position of the first imaging unit 0101_1 and the second imaging unit 0101_2, a combination of the zoom magnification and the focus position A plurality of lens distortion correction information or internal parameters corresponding to. Details of how to store the camera information will be described later. Camera information can be acquired by performing camera calibration processing in advance, for example, using existing camera calibration hands that use a calibration pattern at the time of factory shipment or initial adjustment after installation, such as the method using Zhang What is necessary is just to measure and to store in a camera information storage part. The stereo camera unit 0110 includes a first image capturing unit 0101_1, a second image capturing unit 0101_2, a zoom / focus control unit 0102, a stereo image processing unit 0103, and a camera information storage unit 0104. It shows that it is building. At this time, it is preferable that the first image capturing unit 0101_1 and the second image capturing unit 0101_2 are fixed as a single housing because external parameters between the image capturing units are less likely to change and resistance to secular change is increased. However, the positional relationship between the imaging units is known and fixed, or panning, tilting, and parallel movement are performed, but the amount of movement and the positional relationship between the imaging units after movement can be obtained by some method. However, it is not always necessary that the two imaging units are fixed as a single casing. Note that the imaging control processing performed by the zoom / focus control unit 0102 and the stereo image processing performed by the stereo image processing unit 0103 are usually performed by a microcomputer, DSP, image processing LSI, or the like in the camera. Further, the zoom / focus control unit 0102 and the stereo image processing unit 0103 may be outside the stereo camera unit 0110, and may be configured to perform imaging control processing and stereo image processing corresponding to a PC.

  Although the case where two imaging units are used is shown here as a configuration, the number of imaging units is not limited to two, and may be three or more imaging units installed at different positions. In addition, the stereo imaging unit 0201 may be configured using a mirror or a prism, and may have one imaging unit configured by one imaging element. One of the two imaging units may be a fixed-focus imaging unit that does not have a zoom lens and a focus lens.

  FIG. 2 is a diagram illustrating an example of a method for storing camera information in the camera information storage unit according to the first embodiment of the present invention. In FIG. 2, the horizontal axis indicates the zoom magnification, the vertical axis indicates the focus position, and the camera information (1, 1) indicates camera information when the zoom magnification is the first zoom magnification and the focus position is at the first focus position. The camera information (2, 1) indicates camera information when the zoom magnification is the first zoom magnification and the focus position is at the second focus position. Similarly, the camera information is obtained for each combination of the zoom magnification and the focus position. Exists. Therefore, the camera information storage unit 0104 stores a plurality of pieces of camera information corresponding to the combination of the zoom magnification and the focus position. Here, the plurality of camera information stored in the camera information storage unit 0104 indicates lens distortion correction information or internal parameters that change depending on the zoom magnification and focus position of the imaging unit, and these values are different for each imaging unit. Therefore, it is necessary to store a plurality of pieces of camera information for each imaging unit. Note that the external parameters between the image capturing units do not depend on the zoom magnification or the focus position of each image capturing unit, and do not need to be stored as a plurality of camera information. As described above, by storing a plurality of lens distortion correction information or internal parameters that change depending on the zoom magnification and focus position of the imaging unit according to the combination of the zoom magnification and the focus position, zoom and focus can be changed. Even when a stereo camera is constructed using a simple imaging unit, it is possible to perform three-dimensional measurement by stereo image processing using optimal camera information suitable for the zoom magnification and focus position at that time.

  FIG. 3 is a diagram showing an example of a method for storing camera information in consideration of the relationship between the zoom lens and the focus lens according to the first embodiment of the present invention. In FIG. 3, the horizontal axis represents the stepping motor step position for operating the zoom lens (hereinafter referred to as zoom control position), the vertical axis represents the stepping motor step position for operating the focus lens (hereinafter referred to as focus control position), and the W end. Indicates the optical wide end, the T end indicates the optical tele end, the F end indicates the far side mechanical end, and the N end indicates the near side mechanical end. The curve shown by the solid line in the figure shows the relationship between the zoom control position and the focus control position when the in-focus position is at infinity. The curve shown by the dotted line is the most in-focus position within the specification range. The relationship between the zoom control position and the focus control position in the case of a close position (xm in the drawing) is shown. Therefore, the effective range in which actual focus control is possible is the portion indicated by the hatching in the figure, and the effective range in which focus control is possible varies depending on the zoom control position. For example, in the example of FIG. 3, the effective range in which focus control is possible is narrower toward the W end side where the depth of field is deeper, and becomes wider as the zoom control position is closer to the T end where the depth of field is shallower. Therefore, the camera information storage unit 0104 may actually store the camera information with a small number of samples when the zoom control position is close to the W end, and store the camera information with a large number of samples when the zoom control position is close to the T end. . The specification value of the position where the in-focus position is closest may be referred to the specification value of the camera that satisfies the required specification of the application target to which the three-dimensional measurement is applied. A commercially available digital camera or the like is called a macro mode, and sometimes has a function of moving the focus control position closer to the N end side than the curve indicated by the dotted line in FIG. . Therefore, when it is desired to perform stereo image processing even in the macro mode, it is only necessary to store camera information in a range between the curve indicated by the dotted line and the N end in the figure. However, since the appearance of the subject varies greatly between the imaging units during close-up photography, it is not suitable for performing three-dimensional measurement by stereo image processing. As described above, by changing the number of stored camera information according to the zoom control position and the focus control position, the number of stored camera information can be optimized and the amount of memory can be reduced.

  FIG. 4 is a first diagram illustrating an example of a method for determining a zoom control position and a focus control position for storing camera information according to the first embodiment of the present invention. 4, (a) is a diagram of the first imaging unit, (b) is a diagram of the second imaging unit, in which the horizontal axis is the zoom control position, the vertical axis is the focus control position, and a is The zoom control position at the W end in the first imaging unit, b is the zoom control position at the T end in the first imaging unit, a ′ is the zoom control position at the W end in the second imaging unit, and b ′ is the second zoom control position. This is the zoom control position at the T end in the imaging unit. At this time, the zoom control position for storing the camera information is moved by the same amount of step position α from the zoom control position at the W end, that is, the zoom control position a + α in the first imaging unit, and the zoom control in the second imaging unit. Determined from position a ′ + α. The zoom control position at the W end and the number of steps between the W end and the T end have individual differences for each camera. By using the relative step position from the W end in this way, the zoom magnification between the imaging units is used. Since the camera information when the difference is small can be stored, stereo image processing can be performed with high accuracy. The α for determining the zoom control position for storing the camera information may be determined based on the zoom magnification, the zoom control position, and the like. For example, when zoom is used only at a predetermined magnification, that is, when a preset value is used, α may be determined by determining a zoom control position at which the zoom magnification is reached. Also, if you want to use at any zoom magnification, take into account that lens distortion information and internal parameters change continuously with changes in the zoom control position so that they are at a fixed interval with respect to the zoom control position. α may be determined. Further, here, the zoom control positions a and a ′ at the W end are considered as references, but it is preferable that the focal lengths between the imaging units are strictly equal in order to perform stereo image processing, and in the vicinity of a and a ′. When a zoom control position where the focal lengths of the first imaging unit and the second imaging unit match can be detected, the zoom control position may be used as a reference position instead of a and a ′. Similarly, with respect to the focus control position, c and c ′ in the figure are the focus control positions when the in-focus positions at the zoom control positions a + α and a + α ′ are infinite, and the focus control position for storing camera information is c. , C ′ and the position moved by the same amount of step position β, that is, the focus control position c + β in the first imaging unit and the focus control position c ′ + β in the second imaging unit. Β for determining the focus control position for storing the camera information may be determined based on the focus position, the focus control position, or the like. For example, when the focus is used in advance only at a predetermined focal position, that is, when a preset value is used, β may be determined by determining the focus control position when the focus is reached. Also, if you want to use it at any focus, consider that lens distortion information and internal parameters change continuously in response to changes in the focus control position. May be determined. FIG. 5 is a second diagram showing an example of a method for determining a zoom control position and a focus control position for storing camera information according to the first embodiment of the present invention. In FIG. 5, as in FIG. 4, the zoom control position for storing camera information and the focus control position are determined as relative positions from a certain reference position. Here, the zoom control position for storing camera information is the zoom control position at the W end. And the interval between the zoom control position at the T end and the coefficient γ are applied. Similarly, a factor δ is applied to the distance between the focus control position for storing camera information when the in-focus position is at infinity and the focus control position when the in-focus position is the closest in the specification. It is decided by. As described above, the zoom control position and the focus control position may be determined based on the relative positions determined from the two reference positions. Thus, by determining the zoom control position for storing camera information and the focus control position as relative positions from a certain reference position, the camera information of each imaging unit when the zoom or focus between the imaging units is as close as possible is obtained. Since it can be stored, accurate stereo image processing becomes possible.

  FIG. 6 is a diagram showing an example of a stereo image processing sequence using camera information according to the first embodiment of the present invention. The stereo image processing sequence in FIG. 6 is executed by the stereo image processing unit 0103.

  In the stereo image processing sequence of FIG. 6, in ST0601, captured images are acquired from the first imaging unit 0101_1 and the second imaging unit 0101_2. In ST0602, the zoom control position and focus control position of the first imaging unit 0101_1 and the zoom control position and focus control position of the second imaging unit 0101_2 are acquired from the zoom / focus control unit 0102. In ST0603, the camera information corresponding to the zoom control position and the focus control position of the first imaging unit 0101_1, the camera information corresponding to the zoom control position and the focus control position of the second imaging unit 0101_2, and depending on the zoom and focus. Camera information not to be acquired, that is, external parameters are acquired. At this time, if the camera information corresponding to the zoom control position and the focus control position exists for each imaging unit, the camera information may be obtained as it is, but if the corresponding camera information does not exist, If the corresponding camera information exists in the zoom control information and the focus control information, the camera information is substituted. Or, by referring to a plurality of zoom control information and focus control information in which there is corresponding camera information in the vicinity, weighting is performed from the relationship with the actual zoom control information and focus control information, and the camera information is linearly interpolated. You may make it the form which acquires highly accurate camera information. In ST0604, correction processing such as lens distortion correction, magnification correction, and parallelization processing is performed using the acquired camera information. In ST0605, pre-processing such as low-pass filter processing for noise removal, feature amount calculation processing such as edge detection, block matching such as normalized cross-correlation and sum of absolute differences, and various correlation calculations such as space-sweep method Perform stereo matching processing that searches for corresponding points between stereo images using processing and acquires disparity information, post-processing that removes singular points by rank filter processing, labeling, etc., and then the camera from the finally acquired disparity information Measure 3D information of the subject using the information. As described above, the zoom control information and the focus control information are acquired from each imaging unit, and the corresponding camera information is acquired from the camera information storage unit 0104 and used, so that the stereo image processing unit 0103 can adjust the zoom and focus of each imaging unit. Even in the event of a change, highly accurate 3D measurement can be performed. Note that the correction process shown in ST0604 and the three-dimensional measurement process shown in ST0605 need only be performed according to the required specifications of the application using the final stereo image processing output result. Therefore, the contents of necessary camera information differ depending on the required specifications of the application. For example, if you want to use 3D information for subject separation, there is no need to acquire the 3D shape of the subject as an absolute position, and it is only necessary to know the relative parallax of the subject. Camera information is not necessary, and it is sufficient to have lens distortion information for correcting lens distortion. Further, when it is desired to measure a highly accurate three-dimensional shape of a subject like a three-dimensional shape measuring apparatus, it is necessary to have all internal parameters and external parameters as camera information in addition to lens distortion correction information. Further, when there is an online calibration device (not shown) or the like, and some camera information can be measured from the moving image, it is not necessary to acquire the information from the camera information storage unit 0104. As described above, the contents of the camera information stored in the camera information storage unit 0104 can be determined according to the use environment of the application.

Next, how to hold each camera information stored in the camera information storage unit 0104 will be described. The camera information stored in the camera information storage unit 0104 includes, in the stereo image processing sequence shown in FIG. 6, the association between the coordinates before correction and the coordinates after correction in the various correction processes in ST0604, and the measurement process of three-dimensional information in ST0605. Is used as a coefficient of calculation in. At this time, the camera information used for the correction process is stored as a correspondence table that associates the coordinates before correction and the coordinates after correction, thereby speeding up the correction process. In addition, it may have a post-correction coordinate corresponding to the pre-correction coordinate or a coefficient of an arithmetic expression for calculating the pre-correction coordinate corresponding to the post-correction coordinate. However, the amount of data to be stored can be reduced, and a plurality of corrections can be combined and performed by a single correction process, so that deterioration in image quality due to repeated corrections can be avoided. As a coefficient of an arithmetic expression for associating the coordinates before correction and the coordinates after correction, for example, the lens distortion coefficient in the radial direction and the lens distortion coefficient in the tangential direction in the lens distortion model of Weng are well known. In addition, the camera information used as the calculation coefficient in the three-dimensional information measurement process may store the coefficient value as it is.

  FIG. 7 is a diagram showing an example of lens distortion correction processing using the correspondence table according to the first embodiment of the present invention. The lens distortion correction process using the correspondence table in FIG. 7 is executed in ST0604 in the stereo image processing sequence shown in FIG.

  In FIG. 7, (a) is image data output from the first imaging unit or the second imaging unit and is a captured image before lens distortion correction, and (b) is the lens distortion correction information from the camera information storage unit 0104 before correction. A correspondence table of coordinates and coordinates after correction is acquired and a captured image (a) is corrected according to the correspondence table, and (c) is a correspondence image used for correction. The dotted lines shown in the pre-correction image (a) and the post-correction image (b) are not the subject actually reflected in the image, but are auxiliary lines for indicating the shape of distortion. The uncorrected image (a) is distorted radially from the center to the periphery of the image as a result of the occurrence of the barrel distortion, which is a typical distortion shape. The stereo image processing unit 0102 refers to the correspondence table, determines the position of the coordinates before correction corresponding to each pixel of the corrected image to be generated, and corrects the luminance data or the color data stored in the coordinates before correction. Save to image coordinates. In this example, the coordinates after correction are described in pixel accuracy in the correspondence table, whereas the coordinates before correction are described in sub-pixel accuracy, so the luminance data or color data of the coordinates before correction are A highly accurate correction process can be performed by performing an interpolation process such as a re-neighbor interpolation method or a collinear interpolation method from the coordinates of pixel accuracy. By previously writing a plurality of interpolation destination coordinates in the correspondence table, the amount of data increases, but it is also possible to reduce the arithmetic processing during interpolation. By storing the correspondence table between the coordinates before correction and the coordinates after correction as camera information in this way, the speed of the correction process can be increased.

  In FIG. 7, the lens distortion correction process is shown as an example of the correction process using the correspondence table, but it is also possible to perform the magnification correction process and the parallelization process between the imaging units. The magnification correction process between the imaging units can be performed by taking the correspondence between the coordinates before correction and the coordinates after correction from the relative ratio of the focal length values for each imaging unit. Further, the parallelization processing for correcting the corresponding points on the image between the imaging units so as to be aligned on the same horizontal line is written in, for example, the document “Xugang, Saburo Tsubasa,“ 3D Vision ”, Kyoritsu Publishing (1998)”. This method can be used by taking the correspondence between the coordinates before correction and the coordinates after correction using the internal parameters for each imaging unit, the external parameters between the imaging units, and the basic matrix that can be calculated from those parameters. It is. Thus, instead of sequentially performing each correction process, by taking the correspondence between the final corrected coordinates and the uncorrected coordinates when each correction process is performed, the actual image correction process is performed at one time, You may reduce the influence of the image quality deterioration accompanying image correction.

  FIG. 8 is a diagram showing an example of lens distortion and magnification correction processing using the correspondence table according to the first embodiment of the present invention. The lens distortion and magnification correction correction processing using the correspondence table in FIG. 8 is executed in ST0604 in the stereo image processing sequence shown in FIG.

In FIG. 8, (a) is the image data output from the first imaging unit and the captured image before the lens distortion correction, and (b) is the lens distortion correction information from the camera information storage unit 0104 as the coordinates before correction and the coordinates after correction. A corrected image obtained by acquiring a correspondence table and correcting the captured image (a) according to the correspondence table, (c) is a lens distortion correction from the camera information storage unit 0104, and between the first imaging unit and the second imaging unit. After obtaining a correspondence table for performing magnification correction for correcting the magnification of and correcting the captured image (a) according to the correspondence table, (d) is the image data output by the second imaging unit as in (b) It is an image after correction corrected to. Image before correction (a) of the first imaging unit, image (b) after correction of lens distortion of the first imaging unit, image after correction of magnification (c) of the first imaging unit, and lens distortion of the second imaging unit The dotted line shown in the corrected image (d) is not a subject actually reflected in the image, but an auxiliary line for clearly indicating the distortion shape and the magnification difference between the imaging units. Here, although the lens distortion correction and the magnification correction between the imaging units are performed as the correction processing, an image after lens distortion correction (b) is created from the image before correction (a), and the image after lens distortion correction (b) is created. Reduce image quality degradation due to image correction by creating magnification-corrected image (c) directly from pre-correction image (a) using one correspondence table instead of creating magnification-corrected image (c) Is realized. Similarly, it is possible to perform parallel processing between cameras at a time. Thus, since the camera information in the two imaging units can be specified from the zoom control position and the focus control position of each imaging unit, not only the lens distortion for each imaging unit is corrected, but also the camera information between the imaging units is used. Thus, the correction process for optimally performing the stereo image processing can be performed as a single correction process.

  Thus, according to the present embodiment, in the stereo camera having the zooming and focusing yesterday, for the camera information that changes depending on the zoom and focus state, a plurality of camera information corresponding to the combination of zoom and focus is stored. By using it for stereo image processing, highly accurate three-dimensional measurement can be performed using appropriate camera information even when zooming or focusing is performed.

  FIG. 9 is a first schematic diagram showing a three-dimensional measuring apparatus according to the second embodiment of the present invention. In FIG. 9, 0101_1 is the first imaging unit, 0101_2 is the second imaging unit, 0102 is the zoom / focus control unit, 0103 is the stereo image processing unit, 0104 is the camera information storage unit, 0110 is the stereo camera unit, and 0911 is pan / tilt. It is a control unit.

  In the three-dimensional measuring apparatus shown in FIG. 9, the stereo camera unit 0110 has the same configuration as the stereo camera unit shown in FIG. 1, and is attached to a turntable that can rotate in the pan direction or the tilt direction. . The pan / tilt control unit 0911 controls the turntable based on a user input input from a user input unit (not shown) or an image recognition result acquired from an image recognition unit (not shown) to thereby control the stereo camera unit 0110. Is operated in the pan or tilt direction. Since the stereo camera unit 0110 includes an imaging unit having a zoom function and a focus function, the stereo camera unit 0110 constitutes a stereo camera having a pan / tilt zoom function like a surveillance dome camera as a whole. FIG. 10 is a schematic view showing an example of a stereo camera having a pan / tilt / zoom function. In FIG. 10, reference numeral 1010 denotes a stereo camera with a zoom / focus function, which has a configuration equivalent to that of the stereo camera unit 0110. Reference numeral 1012 denotes a turntable that can be operated in the pan and tilt directions of the stereo camera 1010 with a zoom / focus function.

  FIG. 11 is a diagram showing an example of the operation of the three-dimensional measuring apparatus capable of pan / tilt according to the second embodiment of the present invention.

  In FIG. 11, 1010 is a stereo camera with a zoom / focus function, 1012 is a turntable, and 1013 is a subject. A stereo camera 1010 with a zoom / focus function is fixed to a turntable 1012 and installed downward from the ceiling. For example, an image of indoors, a garden, or the like can be acquired. The symbol θ in the figure is the depression angle from the ceiling when the stereo camera 1010 with zoom / focus function operates in the tilt direction. In FIG. 10, one stereo camera 1010 with zoom / focus function is viewed from the ceiling. It shows a situation when the depression angle is 45 degrees, 90 degrees, and 135 degrees, respectively. In this manner, when the stereo camera is stationary with different depression angles, even when an equal amount of zooming or focusing operation is performed, the lens distortion and the manner of changing internal parameters may change. Therefore, the camera information storage unit 0104 detects in advance the amount of change in camera information corresponding to the attitude of the stereo camera 1005 with pan / tilt / zoom function and stores it as correction information, and the stereo image processing 0110 receives the information from the pan / tilt control unit 0911. By acquiring attitude information of the stereo camera 1010 with a zoom / focus function and acquiring corresponding correction information from the camera information storage unit 0104 together with the camera information, the change in the camera information due to such an attitude difference is corrected. Therefore, highly accurate three-dimensional measurement processing can be performed. Although the camera information storage unit 0104 stores the correction amount of the camera information according to the posture difference here, the camera information for each posture may be detected and stored in advance. In this case, the amount of data to be stored increases, but it is not necessary to perform correction calculation processing, so that the processing amount can be reduced. Further, when the depression angle is in the range of 0 to 90 degrees and in the range of 90 to 180 degrees, the upper and lower sides of the imaging are reversed. For such a case, for example, the first image capturing unit 0101_1 and the second image capturing unit 0101_2 output an inverted video, or the stereo image processing unit 0103 performs stereo image processing by inverting the coordinates in the vertical axis direction. Although the vertical axis direction of the image used for stereo image processing is inverted, for example, lens distortion correction information and information on the center of the image can be used for information corresponding to the inversion of the image. It is necessary to correct. Correction information based on such image inversion may be stored in the camera information storage unit 0104.

  FIG. 12 is a second schematic diagram showing a three-dimensional measuring apparatus according to the second embodiment of the present invention. In FIG. 12, 0101_1 is the first imaging unit, 0101_2 is the second imaging unit, 0102 is the zoom / focus control unit, 0103 is the stereo image processing unit, 0104 is the camera information storage unit, 0110 is the stereo camera unit, and 1205 is the temperature. It is a detection unit.

  In the three-dimensional measuring apparatus shown in FIG. 11, the stereo camera unit 0110 has the same configuration as the stereo camera unit shown in FIG. The temperature detection unit 1205 can measure the temperature of the casing of the first imaging unit 0101_1 and the second imaging unit 0101_2 of the stereo camera unit 0110 as an absolute temperature or a relative temperature from a certain reference temperature. Similar to the difference in attitude of the stereo camera unit 0110, the lens distortion and internal parameter change method when zooming or focusing operation is changed due to the expansion and contraction of the lens and the deformation of the structure depending on the temperature at the time of imaging. There is a fear. Therefore, the camera information storage unit 0104 detects in advance the amount of change in camera information corresponding to the temperatures of the first imaging unit 0101_1 and the second imaging unit 0101_2, and stores them as correction information, and stereo image processing 0110. By acquiring the temperature information of the first imaging unit 0101_1 and the second imaging unit 0101_2 from the temperature detection unit 1205 and acquiring the corresponding correction information together with the camera information from the camera information storage unit 0104, Changes in camera information due to temperature differences can be corrected, and highly accurate three-dimensional measurement processing can be performed. Although the camera information storage unit 0104 stores the correction amount of the camera information according to the temperature here, the camera information for each temperature may be detected and stored in advance. In this case, the amount of data to be stored increases, but it is not necessary to perform correction calculation processing, so that the processing amount can be reduced.

  In addition to the stereo camera attitude difference and temperature described above, the zoom magnification and focus of the same amount can also be obtained for each imaging unit depending on the zoom lens movement direction during zooming and the focus lens movement direction during focusing. The camera information at the position may be different. In such a case, for example, the zoom control position may be adjusted to the W end once during zooming, and the zoom lens movement direction may be kept constant by zooming from the W end direction to the T end direction. Similarly, the focus lens movement direction is always fixed so that the focus control position is set to the focus control position when the in-focus position is at infinity during focusing, so that focusing is performed from the far side to the near side. do it. Thus, by controlling the movement directions of the zoom lens and the focus lens to be constant, the reproducibility of camera information accompanying zooming and focusing can be improved.

  As described above, according to the present embodiment, the camera information is corrected based on the attitude difference of the stereo camera and the temperature during use, and the moving direction of zooming and focusing is controlled, so that the high reproducibility of the camera information can be used. Three-dimensional measurement by accurate stereo image processing can be performed.

  In addition, this invention is not limited to an above-described Example, Various modifications are included. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. Further, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment.

  The present invention is applicable to surveillance cameras, vehicle-mounted cameras, and stereoscopic cameras.

1 is a schematic diagram showing a three-dimensional measuring apparatus according to a first embodiment of the present invention. FIG. 6 is a diagram showing an example of a camera information storage method in the camera information storage unit according to the first embodiment of the present invention. It is a figure which shows an example of the storage method of the camera information which considered the relationship between the zoom lens and focus lens which concern on 1st Example of this invention. It is a 1st figure which shows an example of the method of determining the zoom control position and focus control position which store the camera information based on 1st Example of this invention. It is a 2nd figure which shows an example of the method of determining the zoom control position and focus control position which store the camera information based on 1st Example of this invention. It is a figure which shows an example of the stereo image processing sequence using the camera information which concerns on 1st Example of this invention. It is a figure which shows an example of the lens distortion correction process using the correspondence table which concerns on 1st Example of this invention. It is an example of a lens distortion and magnification correction process using the correspondence table according to the first embodiment of the present invention. It is a general-view figure which shows an example of the stereo camera which has the pan tilt zoom function based on 2nd Example of this invention. It is a 1st schematic diagram which shows the three-dimensional measuring apparatus which concerns on 2nd Example of this invention. It is a figure which shows an example of the mode of operation | movement of the three-dimensional measuring apparatus which can perform the pan-tilt based on 2nd Example of this invention. It is a 2nd schematic diagram which shows the three-dimensional measuring apparatus which concerns on 2nd Example of this invention.

Explanation of symbols

0101_1 First imaging unit 0101_2 Second imaging unit 0102 Zoom / focus control unit 0103 Stereo image processing unit 0104 Camera information storage unit 0110 Stereo camera unit 0911 Pan / tilt control unit 1010 Stereo camera 1012 with zoom / focus function Rotary table 1113 Subject 1205 Temperature detector

Claims (10)

A plurality of imaging means including one or more imaging means with variable zoom and focus;
Zoom / focus control means for controlling zooming and focusing of the imaging means with variable zoom and focus;
Stereo image processing means for processing three-dimensional information by performing stereo image processing on the captured images output by the plurality of imaging means;
Camera information storage means for storing camera information necessary for the stereo image processing means to perform stereo image processing;
With
Among the camera information stored in the camera information storage unit, for the camera information that changes depending on the zoom state or the focus state of the imaging means whose zoom and focus are variable, a plurality of camera information corresponding to the state is obtained. Have
The stereo image processing means acquires the zoom state and focus state of the imaging means with variable zoom and focus from the zoom / focus control means, and acquires corresponding camera information from the stereo camera information storage means. Used for stereo image processing,
3D measuring device. The three-dimensional measurement apparatus according to claim 1,
When the camera information storage means has a plurality of camera information, if the zoom state and the focus state of the imaging means with variable zoom and focus are in a specific combination, the camera information corresponding to the state is stored. To do,
3D measuring device. The three-dimensional measurement apparatus according to claim 1,
Step information of the stepping motor that controls the zoom lens and step information of the stepping motor that controls the focus lens are used as information indicating the zoom state and the focus state of the imaging means with variable zoom and focus, respectively.
3D measuring device. The three-dimensional measurement apparatus according to claim 1,
Among the camera information stored in the camera information storage means, the camera information that changes depending on the zoom state or the focus state of the imaging means with variable zoom and focus is the lens distortion or focal length or image for each imaging means. The information on the center or shear includes camera information that does not depend on the zoom state or focus state of the image pickup means with variable zoom and focus, and information on rotation or translation between the image pickup means, and the camera information storage means is a camera Have one or more types of information as information,
3D measuring device. The three-dimensional measurement apparatus according to claim 1,
When the stereo image processing unit corrects lens distortion as a pre-process of stereo image processing, the camera information storage unit stores a relative difference between the zoom state and the focus state between the plurality of imaging units. Correcting at the same time using information,
3D measuring device. The three-dimensional measurement apparatus according to claim 1,
When the imaging control device controls zooming or focusing of the imaging means with variable zoom and focus, the zoom lens movement direction and the focus lens movement direction are always controlled to be constant;
3D measuring device. A plurality of imaging means including one or more imaging means with variable zoom and focus;
Zoom / focus control means for controlling the imaging means with variable zoom and focus;
Stereo image processing means for performing stereo image processing on the captured images output by the plurality of imaging means and measuring three-dimensional information;
Camera information storage means for storing camera information necessary for the stereo image processing means to perform stereo image processing;
Imaging environment detection means for detecting an imaging environment when the plurality of imaging means capture images;
Pan-tilt control means for controlling movement in the pan direction and tilt direction of the plurality of imaging means,
Among the camera information stored in the camera information storage unit, for the camera information that changes depending on the zoom state or the focus state of the imaging means whose zoom and focus are variable, a plurality of camera information corresponding to the state is obtained. Have
The stereo image processing means acquires the zoom state and focus state of the imaging means with variable zoom and focus from the zoom / focus control means, and acquires corresponding camera information from the stereo camera information storage means. Used for stereo image processing,
A three-dimensional measuring device characterized by A plurality of imaging means including one or more imaging means with variable zoom and focus;
Zoom / focus control means for controlling the imaging means with variable zoom and focus;
Stereo image processing means for performing stereo image processing on the captured images output by the plurality of imaging means and measuring three-dimensional information;
Camera information storage means for storing camera information necessary for the stereo image processing means to perform stereo image processing;
Imaging environment detection means for detecting an imaging environment when the plurality of imaging means capture images;
A temperature detector for measuring the temperature of the plurality of imaging means;
With
Among the camera information stored in the camera information storage unit, for the camera information that changes depending on the zoom state or the focus state of the imaging means whose zoom and focus are variable, a plurality of camera information corresponding to the state is obtained. Have
The stereo image processing means acquires the zoom state and focus state of the imaging means with variable zoom and focus from the zoom / focus control means, and acquires corresponding camera information from the stereo camera information storage means. Used for stereo image processing,
A three-dimensional measuring device characterized by In the three-dimensional measuring apparatus according to claim 7-8,
The camera information storage means stores correction information of camera information corresponding to the pan / tilt posture or temperature of the plurality of imaging means,
Performing stereo image processing using the camera information corrected by the stereo image processing means in accordance with the posture or temperature of pan / tilt,
3D measuring device. In the three-dimensional measuring apparatus according to claim 7-8,
The camera information storage means stores a plurality of camera information corresponding to a zoom state and a focus state and a pan / tilt posture or temperature of the plurality of imaging means;
The stereo image processing means performs stereo image processing using camera information corrected according to a zoom state and a focus state and a pan / tilt attitude or temperature of the plurality of imaging means;
3D measuring device.

Images