TWI557437B - Stereo camera apparatus, self-calibration apparatus and method for calibrating - Google Patents

Description

Stereo camera device, automatic correction device and correction method

The present disclosure relates to an image pickup apparatus, a correction apparatus, and a correction method, and more particularly to a stereo camera apparatus, an automatic correction apparatus, and a correction method.

With the popularity and popularity of 3D (3 dimensional) stereoscopic movies in recent years, the demand for stereoscopic digital images has also increased. At present, in addition to computer synthesis and 2D to 3D conversion, the main stereoscopic digital image has become one of the mainstream trends in stereoscopic imaging. In general, the actual shooting of a stereo camera is to take a stereo image by two or more cameras. With the increasing consumer demand for stereoscopic digital images, a wide variety of stereo cameras and shooting equipment have sprung up. The current common stereo cameras can be divided into two major architectures: side-by-side rig and mirror rig.

In detail, the stereoscopic effect of the stereoscopic image is generated based on the fact that the two eyes of the person have a certain parallax, since the two eyes of the person are mainly arranged in the horizontal direction, in other words, the effective stereoscopic image parallax is the parallax in the horizontal direction, therefore, the artificial There should be no obvious vertical or other directions of parallax in the stereoscopic image, so as to avoid the stereoscopic image quality caused by the blindness of the horizontal direction due to the parallax outside the horizontal direction when the human eye views the artificial stereo image, and may even affect the quality of the stereo image. Produces dizziness or discomfort. Currently, shooting stereo Before the image, the stereo camera needs to undergo a manual correction process so that the plurality of photographic lenses in the stereo camera are first aligned well, and then the distance and direction between the photographic lenses are adjusted correspondingly according to the shooting requirements. However, since the current mainstream method is to use manual correction, in general, an experienced calibration technician takes about 2 to 3 hours to correct once, and a stereo movie usually has more than 200 shots. In other words, the time and money spent on a movie is very impressive. Therefore, the longer the correction is, the more likely it is to cause significant financial damage, and it is more likely to cause pressure on the crew (such as the movie star’s schedule factor or Allowable shooting time limits for special countries). Therefore, how to provide a fast and accurate stereo camera correction method has become one of the urgent problems to be solved.

The present disclosure provides a stereo camera device including an image capture device, an optical axis control module, and an arithmetic module. The image capturing device is adapted to obtain a stereoscopic image, and the image capturing device comprises a plurality of image capturing units. The optical axis control module is coupled to the image capturing device. The operation module is coupled to the image capturing device and the optical axis control module, wherein the computing module calculates a correction condition according to the stereo image, and the optical axis control module adjusts the image capturing light of the image capturing units in the image capturing device according to the correction condition. In the axial direction, after the adjustment of the optical axis control module, the image axes of the image capturing units are aligned.

The present disclosure provides a method for correcting a stereo camera device, including: capturing a plurality of image signals by using multiple image capturing units; performing one on the image signals Aligning the error operation and calculating an error value between the image signals; determining whether the error value is higher than a threshold; and if the error value is higher than the threshold, performing an alignment step of correcting the error value to correct the position of the image capturing unit And the direction of the image optical axis, and then perform an alignment error operation on the image signals until the error value is lower than the threshold. If the error value is lower than the threshold, the correction is completed.

The present disclosure provides an automatic correction device adapted to automatically correct the position of a plurality of image capturing units and the direction of the image capturing optical axis. The automatic correcting device includes an optical axis control module and an arithmetic module. The optical axis control module is coupled to the image capturing units. The optical axis control module includes a plurality of stages to mount the image taking units. The computing module is coupled to the image capturing unit and the optical axis control module. The calculation module calculates a correction condition according to the plurality of images captured by the image capturing unit, and the optical axis control module adjusts the stages according to the correction condition and drives the direction of the image capturing optical axis of the image capturing unit to be adjusted. After being adjusted by the optical axis control module, the image axes of the image capturing units are aligned.

In order to make the above features of the present disclosure more apparent, the following embodiments are described in detail with reference to the accompanying drawings.

1 is a schematic diagram of a stereo camera device according to an embodiment of the present disclosure. Referring to FIG. 1 , in the embodiment, the stereo camera device 100 can include an image capture device 110 , an optical axis control module 120 , and an operation. Module 130. The image capturing device 110 is adapted to acquire a stereoscopic image. And the image capturing device 110 can include a plurality of imaging units. For example, as shown in FIG. 1 , the image capturing device 110 includes an image capturing unit 110A and an image capturing unit 110B , and the image capturing unit 110A and the image capturing unit 110B are side-by-side. However, the disclosure is not limited thereto, and in other embodiments, the image capturing device 110 may include more image capturing units. In this embodiment, the optical axis control module 120 can be coupled to the imaging device 110. For example, as shown in FIG. 1 , the number of optical axis control modules 120 may also be two. The optical axis control module 120A and the optical axis control module 120B can respectively couple and control the image capturing unit 110A and the image capturing unit 110B, thereby controlling the image capturing unit 110A and the image capturing unit 110B, respectively. However, in other embodiments, the number of optical axis control modules 120 may also be matched to the number of image units, and is not limited to two. Moreover, the computing module 130 can be coupled to the image capturing device 110 and the optical axis control module 120. The calculation module 130 can calculate a correction condition according to the stereo image, and the optical axis control module 120 can automatically adjust the imaging optical axis XA of the image capturing unit 110A and the image capturing unit 110B in the image capturing device 110 according to the correction condition. The direction of the image optical axis XB is taken until the correction condition is satisfied. Among them, the details of the correction conditions will be described later. For example, in the embodiment, the optical axis control module 120 can control the front and rear position (X axis), the high and low position (Z axis), the roll angle, and the tilt angle of the image capturing unit 110A and the image capturing unit 110B ( Pitch). Moreover, the optical axis control module 120 can also control the horizontal spacing (inter-axial, that is, along the Y-axis direction in FIG. 1) and the yaw angle of the image capturing unit 110A and the image capturing unit 110B.

After the adjustment by the optical axis control module 120, the image capturing optical axis XA of the image capturing unit 110A and the imaging optical axis XB of the image capturing unit 110B are taken. Align. For example, in the embodiment, the optical axis alignment described in the embodiment of FIG. 1 is that the imaging optical axis XA and the imaging optical axis XB are parallel to each other in space and at least on the same horizontal plane (for example, FIG. 1 In the XY axis plane of the image capturing unit 110A, in other words, the image capturing optical axis XA of the image capturing unit 110A and the image capturing optical axis XB of the image capturing unit 110B are corrected by the optical axis control module 120, and the image capturing unit 110A and the image capturing unit are taken. The errors in the directions of the pitch, the yaw, and the roll of the units 110B are minimized. Further, the image taking optical axis XA and the taking optical axis XB are aligned in the horizontal direction and the vertical direction. Further, in the present embodiment, the image capturing optical axis XA and the imaging optical axis XB are located on the same horizontal plane. In the present embodiment, the tilt angle is, for example, an angle change of the image taking optical axis XA generated when the image capturing unit 110A rotates on the Y axis. The deflection angle is, for example, generated when the image capturing unit 110A rotates on the Z axis. The angle change of the image optical axis XA is taken, and the flip angle is, for example, an angle change of the image taking optical axis XA generated when the image capturing unit 110A is rotated about the X axis. Therefore, in the actual shooting, since the image capturing optical axis XA is parallel to the image capturing optical axis XB and located on the same horizontal plane, the mechanism of the two eyes is similar to that of the human eye, and therefore, the photographer can simply follow the stereoscopic image. The required image depth size (space sense) and the concavity feeling (stereoscopic sense) correspond to the distance d and the horizontal convergence degree of the inter-axial distance between the image capturing unit 110A and the image capturing unit 110B (ie, the convergence). The deflection angle θ) avoids the phenomenon that the vertical direction or the angle (inclination angle and flip angle) causes the shooting to fail or the image is uncomfortable to the viewer. At the same time, since the image capturing unit 110A can be automatically controlled by the optical axis control module 120, the manual correction can be saved. It takes a lot of time, which saves shooting costs and improves shooting efficiency.

In detail, the optical axis control module 120 may include a drive controller 122 and a plurality of drivers 125, and the drive controller 122 may drive the drivers 125 to change the position of the image capturing unit 110A and the image capturing unit 110B and the image capturing optical axis XA. And the direction of the image optical axis XB. For example, in the embodiment, the driving controller 122 and the driver 125 may respectively include a driving controller 122A and a driver 125A corresponding to the image capturing unit 110A, and a driving controller 122A and a driver 125B including the corresponding image capturing unit 110B. . Moreover, these drivers 125 may also include a precision motor capable of linear motion in forward or reverse directions and associated driver components, which may be driven to change the image taking unit 110A and the image capturing unit 110B under the control of the drive controller 122. The position and angle (such as the position in the X-axis, the Y-axis, and the Z-axis stereo coordinate space, and the inclination direction, the deflection angle, and the angular direction of the flip angle, in other words, in the embodiment, each image capturing unit can be, for example, six. The driver 125 controls the respective positions and angular directions), but the disclosure is not limited thereto. In addition, the optical axis control module 120 may further include a plurality of stages such as the stage PTA and the stage PTB in FIG. 1 to respectively respectively mount the image capturing unit 110A and the image capturing unit 110B. In this embodiment, the stages PTA and PTB are, for example, platforms for locking the image capturing unit 110A and the image capturing unit 110B. However, in other embodiments, the stages PTA and PTB may be implemented in other types such as support. The arm or the bracket is suitable for mounting the device for fixing the stage PTA and the PTA, and the invention is not limited thereto.

Further, the optical axis control module 120 and the computing module 130 included in the stereo camera device 100 in the embodiment of FIG. 1 may also form an automatic The calibration device 100' can be equipped with various different imaging devices according to the shooting requirements. For example, the stereo camera device can be a stereo camera, a dual camera, a dual digital camera or any stereo signal generating device, but the present invention does not Limited. Therefore, in actual shooting, different camera devices can be replaced according to the shooting site or environment (such as polar, deep sea, desert or space), which can have excellent shooting flexibility and also has an automatic similar to the stereo camera device 100. Corrective effect.

In more detail, in the present embodiment, the drivers 125 are movable back and forth to position the positions of the image capturing unit 110A and the image capturing unit 110B and the direction of the image capturing optical axis XA and the image capturing optical axis XB. For example, FIG. 2 is a schematic diagram showing a positioning process of the driver in the embodiment of FIG. 1. Referring to FIG. 1 and FIG. 2, taking the image capturing unit 110A as an example, when the image capturing unit 110A has not been corrected, the image capturing unit is taken. The optical axis XA is located in the direction of the optical axis XA ₀ . If the image capturing optical axis XA of the image capturing unit 110A is to be positioned in the X-axis direction according to the aforementioned correction condition, the driver 125 rotates the image capturing unit 110A with the Z axis as the rotating axis, thereby driving the image capturing optical axis XA from the optical axis. The direction of XA ₀ is first moved to the direction of the optical axis XA ₁ , and then the driver 125 rotates the image capturing unit 110A to drive the optical axis XA ₁ and then move to the direction of the optical axis XA ₂ , wherein the first rotation The angle θ 1 is larger than the second rotation angle θ 2 , and after the two rotations, the direction of the image optical axis XA and the X axis is closer to parallel. Then, the driver 125 drives the image capturing unit 110A to rotate, thereby driving the optical axis XA _{2 to} move in the direction of the optical axis XA ₃ , and then the driver 125 drives the image capturing unit 110A to rotate, thereby driving the optical axis XA ₃ and then moving to the X. In the direction of the axis, wherein the third rotation angle θ 3 is greater than the fourth rotation angle θ 4 , and after the two rotations, the direction of the image optical axis XA and the X axis is compared with the previous The rotation is closer to parallel, so that the above process can be repeated until the direction of the image optical axis XA and the X axis are parallel. In other words, the driver 125 can be repeatedly rotated back and forth several times such that the image capturing optical axis XA is substantially parallel to the X axis. In this embodiment, the number of rotations is only used to illustrate the embodiment, and the disclosure is not limited thereto. In addition, each image capturing unit in the image capturing device 110 can be positioned closer by this mechanism. Further, the optical axis control module 120 can fix the positions of the image capturing unit 110A and the image capturing unit 110B after adjusting the image capturing unit 110A and the image capturing unit 110B. Thereby, it can be further ensured that the corrected image capturing unit 110A and the image capturing unit 110B can be maintained without being easily changed.

In addition, referring to FIG. 1 , in the embodiment, the computing module 130 further includes a signal capturing unit 132 , an operation unit 134 , and an instruction signal output unit 136 . The signal capturing unit 132 can receive the images respectively obtained by the image capturing units 110A and 110B. The computing unit 134 is coupled to the signal capturing unit 132. For example, the computing unit 134 can be a computer, a Field Programmable Gate Array (FPGA), or an embedded system board. However, the disclosure is not limited thereto. The calculation unit 134 can calculate the correction condition according to the feature and the correspondence between the images respectively acquired by the image capturing units 110A and 110B. The command signal output unit 136 can convert the correction condition into a control signal output to the optical axis control module 120.

FIG. 3 is a schematic diagram showing the correction of the stereo camera device in the embodiment of FIG. 1. 4A and FIG. 4B respectively illustrate images captured by each of the image capturing units of FIG. 3 before correction, FIG. 4C illustrates difference images of the images of FIGS. 4A and 4B, and FIG. 4D illustrates corrections. Referring to FIG. 1 to FIG. 4B, in the present embodiment, when the image capturing unit 110A and the image capturing unit 110B in the image capturing device 110 have not been corrected, the image capturing unit 110A and the image capturing unit 110B are not corrected. The image capturing unit 110A and the image capturing unit 110B can be corrected by using a near object NJ and a far object FJ disposed in front of the image capturing device 110. Wherein, the distance between the near object NJ and the image capturing device 110 is closer than the distance between the far object FJ and the image capturing device 110, and the near object NJ may include a plurality of feature points RG for correcting the image capturing device 110, and the far object FJ may It is a reference object containing obvious features, and may not include the near object NJ and the feature image of the feature point RG (that is, the features of the near object NJ and the far object FJ may not be too similar to avoid a correction error). For example, in the present embodiment, the near object NJ is, for example, a nine-point correction plate composed of a plurality of feature points RG placed on a holder M, and the far object FJ may have a checkerboard pattern. The image capturing unit 110A and the image capturing unit 110B can capture the images of the plurality of feature points RG on the near object NJ, respectively, and can also capture the near objects through the gap between the near-object NJ nine-point calibration plates. The far object FJ behind the NJ is as shown in FIG. In the present embodiment, the image taken by the image capturing optical axis XA of the image capturing unit 110A and the image capturing optical axis XB of the image capturing unit 110B are corrected as shown in FIGS. 4A and 4B, respectively. 4A is subtracted from the image of FIG. 4B to obtain a difference image as shown in FIG. 4C.

Since the image capturing unit 110A and the image capturing unit 110B have not been corrected, There may be differences between the position and the angular direction in all directions, and thus in the captured image. Referring to FIG. 4C, the feature points RG are not aligned with the checkerboard pattern (ie, the image of the far object FJ) and have vertical and horizontal errors. In other words, in the present embodiment, the difference in the image of the difference image caused by the difference between the position and the angular direction of the image capturing unit 110A and the image capturing unit 110B in FIG. 4C can be The correction conditions in this embodiment are derived. For example, in the embodiment, the computing module 130 can calculate the difference image (that is, as shown in FIG. 4C ) according to the difference image of the stereo image (that is, the image shown in FIG. 4A and FIG. 4B ). When the degree of the speckle in the difference image (for example, the thickness of the speckle) is minimized, the degree of the image optical axis XA and the image capturing optical axis XB need to be changed in the positional and angular directions, that is, the correction condition in this embodiment. . Then, the optical axis control module 120 can automatically adjust the direction of the imaging optical axis XA and the imaging optical axis XB of the image capturing unit 110A and the image capturing unit 110B in the image capturing device 110 according to the correction condition until the correction condition is satisfied. Also, even if the speckle of the difference image in FIG. 4C is almost disappeared, as shown in FIG. 4D. In this way, it can be ensured that the image capturing unit 110A and the image capturing unit 110B are well corrected in both direction and angle, and the shooting failure can be avoided, as described in the embodiment of FIG. 1 , and details are not described herein again.

5A to FIG. 5E are schematic diagrams showing changes in the optical axes of the imaging images in the embodiment of FIG. 3, and further, referring to FIG. 5A, in the embodiment, in the calibration process, firstly, The object NJ aligns the image capturing optical axis XA with the image capturing optical axis XB, in other words, the image capturing unit 110A and the image capturing unit 110B are respectively adjusted until the features in the separately captured images. Point RG overlaps. Then, referring to FIG. 5B again, the far object FJ is used to align the image capturing optical axis XA with the image capturing optical axis XB, in other words, the image capturing unit 110A and the image capturing unit 110B are respectively adjusted until the chessboard in the separately captured image. The plaid overlaps. Then, the near image NJ is used to align the image capturing optical axis XA with the image capturing optical axis XB, as shown in FIG. 5C, and then the far object FJ is used to align the image capturing optical axis XA with the image capturing optical axis XB. In this way, the near object NJ is aligned with the far object FJ to take the image optical axis XA and the image capturing optical axis XB until the feature points RG in the image captured by the image capturing unit 110A and the image capturing unit 110B overlap and the checkerboard pattern is also overlapping. At this time, the difference image generated by the subtraction of the images respectively captured by the image capturing unit 110A and the image capturing unit 110B may be an almost black image as illustrated in FIG. 4D. The correction can be determined by the operation module 130, and then the optical axis control module 120 automatically adjusts the image capture device 110 according to the correction condition, so that the stereo camera device 100 can be quickly and accurately corrected. Save on shooting costs and improve shooting efficiency. However, the near-black difference image shown in FIG. 4D is only used to illustrate the embodiment. In other embodiments, other different correction conditions and error thresholds may also be used (eg, the thickness of the speckle in the difference image is less than To some extent).

In detail, in this embodiment, the image captured by the image capturing unit 110A (or the image capturing unit 110B) may be further designated as a reference image, and then the block matching method may be used by the computing module 130. The calculated correction condition is adjusted by the optical axis control module 120 to the position and angular direction of the image capturing unit 110B (or the image capturing unit 110A). Moreover, the operation module 130 can calculate the position of the image capturing unit 110B (or the image capturing unit 110A). Or the amount of movement required in the direction, and the higher of the values of these movements is prioritized until the correction condition is satisfied to complete the calibration procedure.

Further, referring to FIG. 4A to FIG. 5E, in the embodiment, each image capturing unit (such as the image capturing units 110A and 110B) in the image capturing device 110 can have six degrees of freedom (ie, X, Y, and Displacement in the Z-axis direction and inclination, deflection angle, and flip angle). However, since the horizontal error is related to the displacement in the Y-axis direction and the two degrees of freedom of the deflection angle, the vertical error is related to the displacement and the inclination in the Z-axis direction. Therefore, in the correction, the following correction process can be used. Further saving the correction time. Taking the horizontal correction as an example, the horizontal positions of the image capturing units 110A and 110B may be first adjusted to align the images of the plurality of close objects NJ of the near object NJ captured by the image capturing units 110A and 110B. Then, the deflection angles of the image capturing units 110A and 110B are adjusted to align the images of the plurality of far objects FJ of the far object FJ captured by the image capturing units 110A and 110B. Then, the near object NJ image and the far object FJ image are repeatedly aligned until the near object NJ image is aligned with the far object FJ image. The alignment is as described in FIG. 4A to FIG. 5E, and details are not described herein again. Thereby, the time required for the horizontal correction can be further saved. For example, in the present embodiment, the time required to complete the calibration once is about 10 minutes, whereby the correction time and money saved are considerable.

In addition, the correction in the vertical direction is similar to the correction in the horizontal direction, and the vertical positions of the image capturing units 110A and 110B may be adjusted first to make the near objects NJ of the near object NJ captured by the image capturing units 110A and 110B. Image alignment. Then, the tilt angles of the image capturing units 110A and 110B are adjusted to make the far objects FJ captured by the image capturing units 110A and 110B far away. The image of the object FJ is aligned. Then, the near object NJ image and the far object FJ image are repeatedly aligned until the near object NJ image is aligned with the far object FJ image. Thereby, the stereo camera 100 can be automatically corrected, and the shooting efficiency can be improved, and the shooting cost can be saved.

FIG. 6 is a schematic diagram of a stereo camera device according to another embodiment of the present disclosure. Referring to FIG. 1 and FIG. 6 , it is similar to the embodiment of FIG. 1 . However, in this embodiment, the difference lies in the stereo camera device 200 . The lens half-reflecting unit BS may be further disposed on the image capturing path of the image capturing unit 210A and the image capturing unit 210B, wherein the lens optical axis XA′ of the image capturing unit 210A and the image capturing unit 210B and the lens optical axis XB′ The directions are not on the same horizontal plane (for example, not on the XY plane in FIG. 6), in other words, the stage PTA and the stage PTB are not located on the same horizontal plane and the lens optical axis XA of the image capturing unit 210A and the image capturing unit 210B are caused. 'The lens optical axis XB' is oriented in different directions. In detail, the stereo camera 200 in FIG. 6 is a mirror architecture. Moreover, the image taking optical axis XA formed by the image capturing unit 210A and the image capturing unit 210B after passing through the transflective unit BS is located on the same horizontal plane as the image capturing optical axis XB, thereby achieving similarity to the embodiment of FIG. efficacy. In addition, similar to the embodiment of FIG. 1, in the embodiment, the two optical axis control modules 220A and 220B included in the optical axis control module 220 are similar to the optical axis control modules 120A and 120B in FIG. For example, the two drive controllers 222A and 222B included in the drive controller 222 also have similar functions to the drive controllers 122, 122A and 122B in the embodiment of FIG. 1, and the plurality of drivers 225 (including the drive in FIG. 6) 225A and 225B) also have similar merits as drivers 125, 125A, and 125B in FIG. Effect, no longer repeat here. Therefore, the stereo camera device 200 in FIG. 6 can be corrected by a similar calibration process as shown in the embodiment of FIG. 1 and can have similar functions. The detailed description of the related device and the calibration process can be referred to the figure. 1 to FIG. 5E, and details are not described herein again. Moreover, in other embodiments, the stereo camera device may be a stereo camera, a dual camera, a dual digital camera, or any stereo signal generating device. Moreover, the generated stereoscopic video signal format may also be side by side, up-and-down, left and right dual signals, or multi-view signals, etc., and the disclosure is not limited thereto.

In addition, as shown in FIG. 1 , the optical axis control module 220 and the computing module 130 included in the stereo camera device 200 of the embodiment of FIG. 6 may also form an automatic calibration device 200 ′, and may be implemented with FIG. 1 . The automatic correction device 100' in the example has similar effects and will not be described again here.

FIG. 7 is a flowchart of a method for correcting a stereo camera device according to an embodiment of the present disclosure. Referring to FIG. 1 and FIG. 7 , in the embodiment, the calibration method may be used to correct the stereo camera device in the embodiment of FIG. 1 . 100. The calibration method may include: capturing a plurality of image signals by using a plurality of image capturing units (such as the image capturing unit 110A and the image capturing unit 110B in FIG. 1) (step S100). An alignment error operation is performed on the image signals, and an error value between the image signals is calculated (step S200). It is judged whether or not the error value is higher than a threshold (step S300). Wherein, if the error value is higher than the threshold value, an alignment step of correcting the error value is performed to correct the position of the image capturing unit and the direction of the image capturing optical axis (step S400), and then perform an alignment error operation on the image signals, and Calculating an error value between the image signals (step S200) until the error The difference is below the threshold, and if the error is below the threshold, the correction is completed.

In detail, the alignment error calculation in step S200 can be performed by the operation module 130 in the embodiment of FIG. 1 by subtracting the images illustrated in FIG. 4A and FIG. 4B to obtain the difference image of FIG. 4C. The error value may be the difference in position caused by the misalignment between the checkerboard pattern and the feature point RG in FIG. 4C. Moreover, the threshold value in step S300 can be set according to the shooting requirements of the photographer. In this embodiment, the corrected difference image that is nearly all black as shown in FIG. 4D can be used. Since the threshold in FIG. 4D is small, the speckle in the difference image is less noticeable than the preset threshold, and the difference image looks like all black, in other words, the image capturing unit 110A and the image capturing unit at this time. The image captured by the unit 110B is almost coincident. However, the disclosure is not limited thereto. For details, refer to the description in the embodiment of FIG. 4A to FIG. 4D, and details are not described herein again.

In more detail, the alignment error operation may further include the step of performing a feature extraction step to capture the image signals and find an image feature between the image signals. A feature calculation step is performed to calculate the position and information of the image features. A feature mapping step is performed to compare the position and information of the image features and find a correspondence between the image features to estimate an error value. Moreover, the correction method may further include using an image of the plurality of feature points RG on a calibration plate as the near object NJ for alignment. And using one of the image signal blocks as the far object FJ for alignment. For example, in the embodiment, the image signals can be captured by the signal capturing unit 132 in the embodiment of FIG. 1, and the image features included in the captured image signals are included. For example, the checker pattern of the feature point RG and the far object FJ described in FIGS. 4A to 4D. The operation unit 134 may perform a feature calculation step and a feature correspondence step to derive an error value (that is, a position difference generated by the misalignment between the checkerboard pattern and the feature point RG in the foregoing FIG. 4C) to facilitate subsequent correction. .

Further, the aligning step of step S400 may include: a horizontal alignment step, a vertical alignment step, and an image alignment step. The horizontal alignment step is to adjust the angular error and the distance error between the image capturing units in the horizontal direction until less than a horizontal error threshold. The vertical alignment step is to adjust the angular error and the distance error between the image capturing units in the vertical direction until it is less than a vertical error threshold. The image alignment step is to adjust the size error and the flip error of the image captured by the optical axes of the image capturing units until less than an image alignment threshold and an error threshold. The alignment steps can be performed by the optical module control module 120 controlled by the computing module 130 in the embodiment of FIG. Also, in the present embodiment, the alignment step in the horizontal direction is related to the Y-axis and the deflection angle, the alignment step in the vertical direction is related to the Z-axis and the tilt angle, and the image alignment step is related to the X-axis and the flip angle.

Furthermore, since each image capturing unit (such as the image capturing units 110A and 110B) in the image capturing device 110 can have six degrees of freedom (that is, displacement in the X, Y, and Z directions, and inclination, deflection angle, and flipping) angle). Since the horizontal error is related to the displacement in the Y-axis direction and the two degrees of freedom of the deflection angle, and the vertical error is related to the displacement and the inclination in the Z-axis direction, the correction can be as described with reference to FIGS. 4A to 5E. Level correction and vertical calibration The correct calibration process steps will not be described here.

In summary, the stereo camera device and the automatic calibration device in the embodiment of the present disclosure can calculate a correction condition by using the stereo image obtained by the image capture device, and adjust the optical axis control module to automatically correct the image capture device. The position in the X-axis, Y-axis, and Z-axis stereo coordinate space of each image capturing unit, and the angles of inclination, deflection angle, and flip angle. Moreover, the correction method in the embodiment of the present disclosure can perform an alignment error operation on the feature points of the near object and the feature image of the far object in the captured image signal to obtain an error value between the image signals, when the error value When the threshold value is greater than the threshold value, the alignment step is performed to lower the error value below a threshold value, thereby achieving the function of correction.

The present disclosure has been disclosed in the above embodiments, but it is not intended to limit the disclosure, and any one of ordinary skill in the art can make some changes and refinements without departing from the spirit and scope of the disclosure. The scope of protection of this disclosure is subject to the definition of the scope of the patent application.

100, 200‧‧‧ stereo camera

100', 200'‧‧‧ automatic correction device

110, 210‧‧‧ image capture device

110A, 110B, 210A, 210B‧‧‧ image capture unit

120, 120A, 120B, 220, 220A, 220B‧‧‧ optical axis control module

122, 122A, 122B, 222, 222A, 222B‧‧‧ drive controller

125, 125A, 125B, 225, 225A, 225B‧‧‧ drive

130, 230‧‧‧ computing module

132‧‧‧Signal capture unit

134‧‧‧ arithmetic unit

136‧‧‧Command signal output unit

BS‧‧‧transflective unit

FJ‧‧‧ far objects

M‧‧‧ bracket

NJ‧‧‧ near objects

RG‧‧‧ feature points

S100, S200, S300, S400‧‧‧ steps

PTA, PTB‧‧‧ stage

XA, XB‧‧‧ take image optical axis

XA’, XB’‧‧‧ lens optical axis

XA ₀ , XA ₁ , XA ₂ , XA ₃ ‧‧‧ optical axis

θ 1, θ 2, θ 3, θ 4‧‧‧ rotation angle

1 is a schematic diagram of a stereo camera device in an embodiment of the present disclosure.

2 is a schematic view showing a positioning process of the driver in the embodiment of FIG. 1.

FIG. 3 is a schematic diagram showing the correction of the stereo camera device in the embodiment of FIG. 1. FIG.

4A and 4B illustrate that each of the image capturing units in FIG. 3 is corrected. Previously captured images.

FIG. 4C illustrates a difference image of the images of FIGS. 4A and 4B.

FIG. 4D illustrates a difference image between images captured by each image capturing unit after being corrected.

5A to 5E are schematic diagrams showing changes in the optical axes of the imaging images in the embodiment of FIG. 3 during the calibration process.

FIG. 6 is a schematic diagram of a stereo camera device in another embodiment of the present disclosure.

FIG. 7 is a flow chart showing a method of correcting a stereo camera device in an embodiment of the present disclosure.

100‧‧‧ Stereo camera

110‧‧‧Image capture device

110A, 110B‧‧‧ image capture unit

120, 120A, 120B‧‧‧ optical axis control module

122‧‧‧Drive Controller

125‧‧‧ drive

130‧‧‧ Computing Module

132‧‧‧Signal capture unit

134‧‧‧ arithmetic unit

136‧‧‧Command signal output unit

PTA, PTB‧‧‧ stage

XA, XB‧‧‧ take image optical axis

Claims (26)

A stereo camera device includes: an image capture device adapted to obtain a stereoscopic image, the image capture device includes a plurality of image capture units; an optical axis control module coupled to the image capture device; and an operation module The operation module is coupled to the optical axis control module, wherein the operation module calculates a correction condition according to the stereoscopic image, and the optical axis control module automatically adjusts the image capture device according to the correction condition. The image capturing optical axis direction of the image capturing unit is aligned with the image capturing optical axis of the image capturing unit after the adjustment of the optical axis control module, wherein the optical axis control module adjusts the level of the image capturing unit Aligning a plurality of near-object images of a close object captured by the image capturing unit, and adjusting a deflection angle of the image capturing units to obtain a far object captured by the image capturing units The plurality of far object images are aligned, and the optical axis control module repeatedly aligns the near object image and the far object image until the near object image is aligned with the far object image; the optical axis control module adjusts the image capturing units Vertical position Aligning a plurality of near-object images of a close object captured by the image capturing unit, and adjusting the tilt angles of the image capturing units to obtain a plurality of far object images of a far object captured by the image capturing units Aligning, the optical axis control module more repeatedly aligns the near object image with the far object image until the near object image is aligned with the far object image. The stereo camera device of claim 1, wherein the image taking optical axes of the image capturing units are aligned in a horizontal direction and a vertical direction after the adjustment by the optical axis control module. The stereo camera device of claim 1, wherein the image capturing optical axes of the image capturing units are located on the same horizontal plane after being adjusted by the optical axis control module. The stereo camera device of claim 1, wherein the image capturing units of the image capturing device are side-by-side. The stereo camera device of claim 1, further comprising a transflective unit disposed on the image capturing path of the image capturing unit, wherein the lens optical axes of the image capturing units are not in the same direction On the horizontal surface, the image capturing optical axes formed by the image taking units after passing through the transflective unit are located on the same horizontal plane. The stereo camera device of claim 1, wherein the optical axis control module comprises a driving controller and a plurality of drivers, the driving controller driving the drivers to change the position and the capturing of the image capturing units. Like the direction of the optical axis. The stereo camera device of claim 6, wherein the drivers move back and forth to position the positions of the image capturing units in proximity to the image capturing optical axis direction. The stereo camera device of claim 7, wherein the optical axis control module fixes the positions of the image capturing units after adjusting the image capturing units. The stereo camera device of claim 1, wherein the optical axis control module controls front and rear positions, high and low positions, flip angles, and tilt angles of the image taking units. a stereo camera device according to claim 3, wherein The optical axis control module controls a horizontal interval of the image capturing optical axes of the image capturing units and the deflection angle. The oscillating device of claim 1, wherein the computing module further comprises: a signal capturing unit that receives the images respectively obtained by the image capturing units; and an arithmetic unit coupled to the signal 撷Taking the unit, wherein the operation unit calculates the correction condition according to the feature and the correspondence between the images respectively obtained by the image capturing units; and an instruction signal output unit, converting the correction condition into a control signal output to the Optical axis control module. A method for correcting a stereo camera device includes: capturing a plurality of image signals by using a plurality of image capturing units; performing an alignment error operation on the image signals; and calculating an error value between the image signals; determining the error Whether the value is higher than a threshold; and if the error value is higher than the threshold, performing an alignment step of correcting the error value to automatically correct the position of the image capturing unit and the direction of the image capturing optical axis, and then re- The image signal performs an alignment error operation until the error value is lower than the threshold. If the error value is lower than the threshold, the calibration is completed. The alignment step includes: performing a horizontal alignment step to adjust the image capturing units. The angular error and the distance error in the horizontal direction are less than a horizontal error threshold, wherein the horizontal alignment step includes: Adjusting the horizontal positions of the image capturing units to align the plurality of near object images of a close object captured by the image capturing units; adjusting the deflection angles of the image capturing units to make the image capturing units Aligning a plurality of far object images of a far object captured; and repeatedly aligning the near object image with the far object image until the near object image is aligned with the far object image; and performing a vertical alignment step to Adjusting an angular error and a distance error between the image capturing units in a vertical direction until less than a vertical error threshold, wherein the vertical alignment step includes: adjusting vertical positions of the image capturing units to make the Aligning a plurality of near-object images of a near object captured by the unit; adjusting the tilt angles of the image capturing units to align the plurality of far object images of a far object captured by the image capturing units; And repeatedly aligning the near object image with the far object image until the near object image is aligned with the far object image. The calibration method of claim 12, wherein the alignment error operation comprises the following steps: performing a feature extraction step to capture the image signals and find an image feature between the image signals; Performing a feature calculation step to calculate the position and information of the image features; and performing a feature correspondence step to compare the position and information of the image features, and find a correspondence between the image features to calculate The error value. The calibration method of claim 12, wherein the aligning step further comprises: performing an image alignment step to adjust an image size error and a flip error of the image captured by the optical axes of the image capturing units until: Less than one image alignment threshold and error threshold. The method of claim 12, further comprising: using an image of a plurality of feature points on a calibration plate as a near object for alignment; and utilizing an image feature block of the image signals as a far Objects are aligned. An automatic correction device is adapted to automatically correct the position of the plurality of image capturing units and the direction of the image capturing optical axis, the automatic correcting device comprising: an optical axis control module coupled to the image capturing units, the optical axis control The module includes a plurality of stages for mounting the image capturing units, and a computing module coupled to the image capturing unit and the optical axis control module, wherein the computing module is configured according to the image capturing units Calculating a correction condition according to the plurality of images, the optical axis control module adjusts the stages according to the correction condition and drives the direction of the image capturing optical axis of the image capturing unit to be adjusted after being adjusted by the optical axis control module. The image capturing optical axes of the image capturing units are aligned, wherein the optical axis control module adjusts the horizontal positions of the image capturing units to obtain a plurality of near object images of a near object captured by the image capturing units. Aligning and adjusting the deflection angles of the image capturing units to align a plurality of far object images of a far object captured by the image capturing units, the optical axis control module being heavier Aligning the near object image with the far object image until the near object image is aligned with the far object image; the optical axis control module adjusts vertical positions of the image capturing units to enable the image capturing units to capture Aligning a plurality of near-object images of a near object, and adjusting the tilt angles of the image capturing units to align a plurality of far object images of a far object captured by the image capturing units, the optical axis control module The near object image and the far object image are more repeatedly aligned until the near object image is aligned with the far object image. The automatic calibration device of claim 16, wherein the image taking optical axes of the image capturing units are aligned in a horizontal direction and a vertical direction after being adjusted by the optical axis control module. The automatic calibration device of claim 16, wherein the image capturing optical axes of the image capturing units are located on the same horizontal plane after being adjusted by the optical axis control module. The automatic calibration device of claim 16, wherein the image capturing units mounted on the stages are side by side. The automatic correction device of claim 16, further comprising a transflective unit disposed on the image taking path of the image capturing unit, wherein the image capturing units are mounted on the plurality of stages The optical axis directions of the lens are not located on the same horizontal plane, and the optical axes formed by the image capturing units after passing through the transflective unit are located on the same horizontal plane. The automatic calibration device of claim 16, wherein the optical axis control module comprises a drive controller and a plurality of drivers, and the drive controller drives the drivers to drive the carriers to adjust the acquisitions. The position of the image unit and the direction of the image optical axis. The automatic correcting device according to claim 21, wherein the drivers move back and forth to position the positions of the image capturing units in proximity to the image capturing optical axis direction. The automatic calibration device of claim 22, wherein the optical axis control module fixes the positions of the carriers after adjusting the carriers. The automatic calibration device of claim 16, wherein the optical axis control module controls front and rear positions, high and low positions, flip angles, and tilt angles of the stages. The automatic calibration device of claim 19, wherein the optical axis control module controls the horizontal spacing and the deflection angle of the image capturing optical axes of the loading stations. The automatic calibration device of claim 16, wherein the computing module further comprises: a signal acquisition unit that receives the images respectively obtained by the image capturing units; and an arithmetic unit coupled to the signal 撷Taking the unit, wherein the operation unit calculates the correction condition according to the feature and the correspondence between the images respectively obtained by the image capturing units; and an instruction signal output unit, converting the correction condition into a control signal output to the Optical axis control module.