JP6734994B2 - Stereo measuring device and system - Google Patents

Description

The present invention relates to a stereo measurement device and system including a monocular camera and a plurality of cameras.

Conventionally, stereo measurement devices and systems that measure an object or environment using a plurality of cameras have been put into practical use. For example, an in-vehicle stereo camera of an automobile is used to perform various measurements such as intruder detection and dangerous behavior detection, and control of voice notification and danger avoidance behavior is performed based on the measurement results. In the automatic driving of a car, there is an example in which the front-back direction is measured by a stereo camera and the car is controlled. In recent years, stereo cameras have been applied in the surveillance field and marketing.

As a method of three-dimensional measurement using a stereo camera, first, the same object is simultaneously photographed by two cameras, distortion correction and parallelization are performed on the two obtained images, and the same three-dimensional measurement is performed. The difference and parallax on the two captured images are calculated. Next, the shape and the like of the three-dimensional object are measured using the calculated parallax.

In the case of such three-dimensional measurement, the measurement areas of the two cameras need to be almost the same. The parameters of the camera, the angle of view, the parallax depth, etc. handled in the above series of processing depend on the specifications of the two cameras, and the measurement range is fixed.

Regarding the measurement range, for example, in the case of a vehicle-mounted camera for automatic vehicle driving, the range from 2 m to 40 m ahead is said to be the measurement range. Therefore, there is a problem that measurement cannot be performed within 2 m or 40 m or more.

As a simple solution to the above point regarding the measurement range, it is conceivable to provide a plurality of sets of two cameras and switch between them. The method is to use a nearby camera at a distance of 2 m or less and switch to a distant camera for measurement at a distance of 40 m or more. However, this method, which includes a plurality of sets of two cameras, has a problem of high cost. Further, in the case of an on-vehicle camera for automatic driving of a vehicle, it is impossible to switch the vehicle while it is in progress. Furthermore, when measuring the ultra-distant view, for example, in the case of measuring 200 m or more, there is a problem that a combination of a plurality of cameras is required and the cost becomes high.

The following methods are further known as methods for solving the above-mentioned problem regarding the measurement range.

Non-Patent Document 1 proposes a stereo camera with variable parameters, and by controlling the camera zoom, it is possible to measure the vicinity and long-distance with one unit, and it is possible to expand the measurement range.

Patent Document 1 proposes to widen the measurement range by combining a plurality of sets of stereo cameras.

Similar to Patent Document 1, Patent Document 2 also proposes to widen the measurement range by combining a plurality of sets of stereo cameras. However, in Patent Document 2, the measurement target is determined based on the focus value of the camera. The position is estimated, calibration data prepared in advance is selected for the estimated position, and three-dimensional measurement is performed. When estimating the position, it is recommended to use image information such as a person's face.

JP, 2006-033282, A JP 2012-058188 A

“Development of self-calibration technology for variable parameter stereo cameras”, 22nd Image Sensing Symposium SSII2016, SI-04, 2016

However, the above-mentioned known methods have the following problems to be solved.

First, in Non-Patent Document 1, it is difficult to appropriately control and measure a camera in a wide range from a near region to a super-distant region according to a measurement target.

Patent Document 1 has a problem that the cost of the measurement system is high. In addition, since the combined camera's sbecks are fixed in advance, it cannot be changed thereafter. There is also the problem of low versatility.

In Patent Document 2, it is difficult to dynamically control the camera and perform appropriate measurement for an actual moving measurement target. Further, as in Patent Document 1, calibration data is prepared in advance, and there is a problem that versatility is low. In addition, there are cases in which the measurement range cannot be focused due to light or disturbance, and at that time, the estimated position is unclear, and the measurement is affected.

From the above, in the present invention, a stereo measurement device capable of appropriately controlling and measuring the camera with a single stereo camera in a wide area that is desired to be distant from the vicinity according to the measurement target. And to provide a system.

From the above, in the present invention, "a stereo measurement device that gives left and right image information from the left and right cameras and processes left and right image information from a stereo camera unit including a camera control unit of the left and right cameras, The stereo measurement device includes a feature point selection unit that selects feature points from image scenes of left and right image information, a calibration calculation unit that performs calibration using the selected feature points, and calculates calibration data, and a calculation unit. A distance calculation unit that calculates the distance to the target object to be sensed using the corrected calibration data and the left and right image information, and controls the camera control units of the left and right cameras from the calculated distance and the image information. And a control amount calculating unit for calculating a control amount for controlling the control amount for applying the control amount to the camera control units of the left and right cameras of the stereo camera unit.

Further, the present invention is a “stereo measurement system including a stereo measurement device and a stereo camera unit that provides left and right image information from the left and right cameras and includes a camera control unit of the left and right cameras”.

According to the present invention, a single stereo camera can be used to measure a wide area that is desired to be far away from the vicinity.

Further, according to the embodiments of the present invention, it is possible to expect to reduce the cost of the camera in the measurement system. Further, when the object to be measured is chased and is sensed, the svecks of the camera are not controlled, so that appropriate sensing can be performed. For example, if a person is far away, the system can zoom in the camera and measure. The zoom ratio can be controlled based on the currently measured position. Also, when a person approaches, it is possible to zoom out and measure. As a result, in the field, it is possible to measure widely and dynamically by intelligently controlling such a stereo camera without considering the cost aspect. Can contribute to.

1 is a diagram showing a configuration example of a stereo camera device 100 according to a first embodiment of the present invention. FIG. 3 is a diagram showing a configuration example of a camera control unit 12. FIG. 6 is a diagram showing an example of a processing flow of the feature point extraction unit 13. The figure which shows the structural example of the distance calculation part 16. The figure which shows the flow of a process in the zoom control part 21 and the baseline control part 22. The figure which shows the flow of the process which controls rotation using the pan control value and tilt control value in the pan control part 24 and the tilt control part 25. The figure which shows the control processing flow in an asynchronous stereo camera. The figure which shows the structural example of the stereo camera apparatus 100 which concerns on Example 2 of this invention. The figure which shows the stereo camera apparatus 100 which concerns on Example 3 of this invention. The figure which shows the stereo camera apparatus 100 which concerns on Example 4 of this invention. The figure which shows the stereo camera apparatus 100 which concerns on Example 5 of this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the following description of the embodiments of the stereo measurement apparatus and system of the present invention, a stereo camera apparatus composed of two cameras will be described as an example. It should be noted that the stereo measurement device is intended to have the configuration of only the processing unit shown below, and the stereo measurement system is intended to have the configuration of the processing unit and the stereo camera unit shown below.

FIG. 1 is a diagram showing a configuration example of a stereo camera device according to a first embodiment of the present invention. The stereo camera device 100 is roughly composed of a stereo camera unit 101 and a processing unit 102 composed of a computer. As the computer, a device having a processing function such as a personal computer, a small built-in computer or a computer connectable to a network can be considered.

Of these, the stereo camera unit 101 includes a real video input unit 10 including left and right real video input units 10R and 10L including a lens and an imaging unit, and an image acquisition unit including left and right image acquisition units 11R and 11L. 11 and a camera control unit 12 including left and right camera control units 12R and 12L. The stereo camera unit 101 can be referred to as a mechanical mechanism unit mainly including a camera.

The actual image input unit 10 in the stereo camera unit 101 is configured to include two sets of image pickup units and lenses, and inputs images from the left and right cameras of the stereo camera.

The image pickup unit in the real image input unit 10 is a mechanism including an image pickup device such as a CMOS (Complementary Metal Oxide Semiconductor) or a CCD (Charge Coupled Device) which is an image sensor. The lens has a zoomable lens. By zooming, it is possible to image a distant view area, and it is also possible to image a near area.

The image acquisition unit 11 has a function of converting an electronic signal input from the real video input unit 10 into an image. There are various image formats here. BMP, JPEG, PNG, etc. The image format can be determined by the system.

The camera control unit 12 can adjust the parameters of the camera such as the angle of view, the orientation, and the distance between the cameras in the actual image input unit 10.

In the first embodiment, the scene to be photographed is not specified. In the case of a stereo camera for surveillance functions, moving objects and surveillance areas are used as shooting scenes. If it is a vehicle-mounted stereo camera, the shooting scene is a road or a vehicle ahead. The shooting scene can be a complicated scene or a simple scene, and the shot video is input. Further, the camera of the first embodiment may be configured as a camera capable of changing the focal length, the rotation angle, the distance between the cameras and the like. Thus, the stereo camera device 100 can be operated in various devices and various environments.

FIG. 2 is a diagram showing a configuration example of the camera control unit 12. The camera control unit 12 includes a zoom control unit 21, a baseline control unit 22, a rotation control unit 23, a tilt control unit 24, and a pan control unit 25. The zoom control unit 21 controls the lens mechanism by using the zoom control value 26. The zoom control unit 21 changes the focal length and the angle of view, which is the imaging range of the camera, by moving the lens. The baseline control unit 23 uses the baseline control value 27 to adjust the distance between the cameras. The tilt control unit 24 uses the tilt control value 28 to control the angle of the camera. The pan control unit uses the pan control value 29 to control the pan angle of the camera. The tilt control unit 23 and the pan control unit 24 respectively control the angle of view and the horizontal direction, and the angle of view and the vertical direction of the camera. The rotation control unit 23 can change the posture after rotating the camera.

Note that specific processing contents of the zoom control unit 21 and the baseline control unit 22 will be described later with reference to FIG. 5, and specific processing contents of the rotation control unit 23 will be described later with reference to FIG. 6.

The above adjustments in the configurations of FIGS. 5 and 6 can be performed independently or simultaneously. Here, it is possible to adjust according to the system specifications and the needs of the scene to be photographed.

Regarding the stereo camera unit 101 in the stereo camera device 100 according to the first embodiment shown in FIG. 1, the actual image input unit 10, the image acquisition unit 11, and the camera control unit 12 each have an example in which left and right camera functions are integrally formed. Although shown, these may be separated. When it is a separate body, it has a right video input unit 10R and a left video input unit 10L, a right image acquisition unit 11R and a left image acquisition unit 11L, a right camera control unit 12R and a left camera control unit 12L, respectively. ..

The processing unit 102 configured by a computer in the stereo camera device 100 includes a feature point selection unit 13, a calibrating unit 14, a calibrating data database DB1, a distance calculating unit 16, and a control amount calculating unit 17. The stereo camera unit 101 is a mechanical mechanism unit mainly composed of a camera, whereas the processing unit 102 configured by a computer can be said to have an electronic processing function.

In the processing unit 102 of FIG. 1, the feature point selection unit 13 executes a process of selecting a feature point for calibration using the image information input from the left and right image acquisition units 11R and 11L. .. The feature point selection is a process of extracting feature points from a shooting scene and selecting effective points from them, and can be realized by an existing method.

FIG. 3 shows an example of the flow of the feature point extraction processing. In the conventional example of feature point extraction, feature points are often extracted from one scene shot at one position, but the first embodiment shows an example of extracting feature point groups from a plurality of scenes.

In the specific example of FIG. 3, as illustrated in process 31, shooting is performed while controlling the rotation of the camera, and as illustrated in process 32, images I1, I2, and I3 in a plurality of scenes are obtained in time series. .. Next, in process 33, feature points are selected for each of the images I1, I2, and I3 in the plurality of scenes, and in process 34, effective feature points are selected from the plurality of extracted feature points.

Although various types of feature point extraction methods are known, examples of highly versatile methods include SIFT (Scale-Invariant Feature Transform) and HOG (Histograms of Oriented Gradients).

In the feature point group Px extracted in the process 33, there may be a feature point that is erroneously matched. When the wrong feature point is used, the calibration calculation may be adversely affected and the calibration data may include a large error. Therefore, a process of removing valid feature points and selecting valid feature points Pr like the process 34 is required. RANSAC (Random sample consensus) is often used as an effective feature point selection method. In the process 34, the point group Px is a wrong feature point, and Pr is a correct feature point showing a predetermined regularity. By the RANSAC process, the wrong feature point Px can be removed.

The calibrator 14 performs camera calibration using the feature points output from the feature point selector 13. Camera calibration is the process of calculating camera internal parameters and camera external parameters.

As internal parameters of the camera, f is the focal length, α is the aspect ratio, s is the skew, and (u ₀ , v ₀ ) is the center coordinate of the coordinates. Further, as external parameters, R is a rotation matrix indicating the orientation of the cameras, and T is a translation matrix indicating the positional relationship between the cameras. In the 3×3 (r11, r12,...r33) matrix, R, which indicates the orientation of the camera, is defined by Euler angles, and is represented by three parameters, which are the camera installation angle, pan θ, tilt φ, and roll ψ. To be done. Calibration techniques are required to calculate the internal and external parameters, respectively. Conventionally, “Z. Zhang, “A flexible new technology for camera calibration”, and IEEE Transactions on Pattern Analysis and Machine Intelligence, which are often used, are described in 13 (2000), 13 (2000), 13 (2000), 13 (13), 13 (2000), 13 (2000), 13 (2000), 13(13): Also, many methods for automatically estimating camera parameters using images have been proposed. Further, the automatic calibration method described in Non-Patent Document 1 has been proposed.

The calculated camera parameters are recorded in the calibration data database DB1 as the calibration data D1.

In the first embodiment of the present invention, it is necessary to recalibrate every time the camera configuration is changed. For example, in the first embodiment, when any of the zoom, the baseline, the rotation angle, etc. of the camera is changed, the feature point selection unit 13 selects the feature points from the re-input scene. After that, the calibrator 14 also calculates the calibration data D1 again, and the latest calibration data D1 is registered in the calibration database DB1. Then, the calibration data is used according to the parameters of the camera controlled camera.

FIG. 4 is a diagram showing a configuration example of the distance calculation unit 16. The distance calculation unit 16 calculates the distance data P1 using the left and right images from the left and right image acquisition units 11R and 11L and the calibration data D1. The calculation of the distance data P1 is obtained by sequentially executing the processes of the distortion correction unit 44, the parallel processing unit 45, and the distance calculation unit 46.

A specific processing procedure for calculating the distance will be described below using mathematical expressions. As a premise of the description in the mathematical expressions below, the internal parameter matrix K and the external parameter matrix D of the camera are respectively expressed by equations (1) and (2). Can be expressed as In the equation (1), K is an internal parameter matrix, f is the focal length, a is the aspect ratio, s is the skew, and (vc, uc) is the center coordinate of the image coordinates. Further, in the equation (2), D is an external parameter matrix, (r11, r12, r13, r21, r22, r23, r31, r32, r33) indicates the orientation of the camera, and (tx, ty, tz) is the camera. The position coordinates between are shown.

When these two parameter matrices K and D and the constant λ are used, the image coordinates (u, v) and the world coordinates (X, Y, Z) are associated with each other by the expression (3).

In addition, in the external parameter D indicating the orientation of the camera (r11, r12,...r33), when defined by the Euler angle, it is represented by three parameters that are the installation angle of the camera: pan θ, tilt φ, and roll ψ. It Therefore, the number of camera parameters necessary for associating the image coordinates with the world coordinates is 11, which is the total of 5 internal parameters and 6 external parameters.

The distortion correction unit 44 that performs the first process in the distance calculation unit 46 in FIG. 4 corrects the distortion of the camera image using the camera parameters. Then, the parallel processing unit 45 processes the parallelization of the left and right cameras. The three-dimensional measured value including the distance can be calculated by (4).

In the equation (4), (xl, yl) and (xr, yr) are pixel values on the left and right camera images. After the parallelization processing, yl=yr=y. f is the focal length, B is the baseline, and the distance between the cameras. d is the difference between the images projected on the left and right cameras, respectively, in the same three-dimensional form. The difference is the three-dimensional parallax. The relationship between the world coordinates (X, Y, Z) and the image coordinates (u, v) is as shown in equation (5). Obtaining the world coordinates (X, Y, Z) means obtaining the distance data P1.

In FIG. 1, the control amount calculator 17 calculates a control value using the calculated distance data P1 and image data. Then, the camera control unit 12 controls using the control values, respectively.

FIG. 5 shows a flowchart relating to control amount calculation and control processing in the zoom control unit 21 and the baseline control unit 22.

In process step S51 which is the first process in the zoom control unit 21 and the baseline control unit 22 shown in FIG. 5, an image is input from the left and right cameras. In processing step S52, the distance data P1 measured by the distance calculation unit 16 is stored in the distance database DB2. In processing step S54, the stability of the distance data P1 is evaluated. For example, when the distance data P1 is stored several times (here, five times), it is determined to be stable. In processing step S55, the average distance is calculated using the distance data determined to be stable, and in processing step S56, the zoom and baseline control amounts of the zoom control unit 21 and the baseline control unit 22 are calculated.

A simple calculation method of the control amount of the zoom and the baseline can be obtained by zoom ratio a=distance d/reference s. The reference s can be set according to the zoom of the camera. If the distance is short, the zoom rate a is decreased, and if the distance is long, the zoom rate a is increased. Further, it can be obtained as a baseline control value b=(total baseline width f/reference s)×n (n=0, 1, 2,... N), where N is a control granularity. N is an index for controlling the number of rounds. In process step S57, the control process is repeatedly executed until the image acquisition is completed. If there is no image, the image acquisition is ended and the control process is ended in processing step S58.

FIG. 6 is a diagram showing a flow of processing of rotation control using the pan control value and the tilt control value in the pan control unit 24 and the tilt control unit 25.

In the first processing step S61 in the rotation control shown in FIG. 6, the left and right images from the left and right image acquisition units 11R and 11L are acquired, and in the processing step S62, the inter-frame difference is calculated. If there is a difference, it indicates that there is movement such as person movement in the image. A processing step S63 extracts the motion area. For example, the image area is divided into a plurality of areas, and it is detected that a person or the like moves between the divided small areas. In the processing step S64, when there is a region movement, the movement amount is calculated in the processing step S65, and the pan/tilt rotation control is performed in the processing step S66. In process step S67, the control process is repeated until the image acquisition is completed. If there is no image, the process proceeds to processing step S68, the image acquisition is ended, and the control process is also ended.

For example, if it is the current position (x0, y0) and is the movement position (x1, y1), the pan control value p and the tilt control value t use the moving distances in the triangular relationship, and according to the equation (6), It is possible to calculate the angle.

Through the series of processes described above, the ideal stereo cameras should be synchronized and the calculated distance should be stable. However, in reality, the left and right cameras have the problem of being asynchronous. In that case, the distance during movement is unstable, and it is difficult to control the distance based.

From this, the present invention further proposes to implement as shown in FIG. FIG. 7 is a diagram showing a flow of control processing in the asynchronous stereo camera.

In S71, which is the first processing step in FIG. 7, left and right camera images are acquired. In process step S72, distortion correction of the camera is performed. In processing step S73, parallax calculation processing is performed. Then, in processing step S74, difference processing between frames of the left and right cameras is performed. Then, if there is no difference in the processing step S75 (Y side), it is determined to be stationary and the process proceeds to the processing of step S710, and if there is a difference (N side), it is determined to be the motion and processing in the processing step S76 is performed. Move.

If it is determined that there is a difference (N side) in the determination processing of processing step S75, it is determined that the movement is a person or the like. In the case of motion, a motion region on the image is calculated in processing step S76. Then, in processing step S77, the current values of the pan and tilt of the camera are calculated. In processing step S78, the calculated motion area is compared with the current value, the difference is calculated, and the position of the person or the like is determined. In processing step S79, the pan control value and the tilt control value are calculated based on the difference, and the rotation control is performed.

In the case of stationary (Y side) in the determination processing of processing step S75, the abnormal distance value is determined in processing step S710, and only normal data is stored in processing step S711 as distance data of five consecutive frames at rest. The stability is evaluated in process step S712. If it is stable (Y side), the average distance is calculated in processing step S713, and the zoom control value and the baseline control value are calculated. Using the control value, zooming and baseline control are performed in processing step S714. In process step S715, the control process is repeated until the image acquisition is completed. If there is no image, in step S716 the image acquisition is ended and the control processing is also ended.

Example 2 will be described with reference to FIG. FIG. 8 is a diagram showing a configuration example of the stereo camera device 100 according to the second embodiment of the present invention.

In the second embodiment, the functions from the real image input unit 10 to the control amount calculation unit 17 in the configuration shown in FIG. 1 are the same as those in the first embodiment, and therefore the description thereof will be omitted.

In the second embodiment, the stereo camera unit 101, which is a mechanical mechanism unit, and the processing unit 102, which is an electronic processing function, are communicably connected to each other via a network 81. Here, the cameras on the network 81 are calibrated. Then, the distance of the object measured by the camera is calculated. After that, the camera system configuration is such that the camera is controlled via the network 81 by using the calculated distance, which enables remote processing. As a result, it is possible to measure remotely regardless of the installation location or measurement location.

The third embodiment will be described with reference to FIG. FIG. 9 is a diagram showing a stereo camera device 100 according to the third embodiment of the present invention.

According to the third embodiment, the accuracy of the scanned distance can be improved when controlling a camera and scanning a wide range.

In the case of one camera in the related art, for example, there is a relationship that the accuracy of a near distance is high and the accuracy decreases at a far distance. In the third embodiment, one camera first scans the near area 94 with the camera svecks 91. Then, the camera svecks are changed to 92, and the far area 95 is scanned. Similarly, the camera svecks are changed to 93, and a farther area 96 is scanned. Then, the results of the entire region 97 can be obtained by integrating the scan results. In the image processing results obtained for the entire area 97, the accuracy of the distance can be high for all areas. By doing so, various applications such as map construction and environmental scanning can be applied for automatic driving in the future.

Example 4 will be described with reference to FIG. FIG. 10 is a diagram showing a stereo camera device 100 according to the fourth embodiment of the present invention. The fourth embodiment is an embodiment in which the sensing object is tracked and appropriate measurement is performed without controlling the camera. Here, a moving object M such as a person or a car is a target object to be measured. The movement path of the object M to be sensed is like g, and it is assumed that the object M gradually moves away from the short distance position. The sensing system does not control the pan, tilt, zoom, and baseline of the camera, but tracks the target object M. As a result, it becomes possible to measure a wide area at a super far distance from a close position by using one stereo camera. Moreover, it becomes possible to perform appropriate sensing according to the movement of the target.

In the example of FIG. 10, the zoom is 1x at the close position, the baseline is 15 cm, and the angle is 10°, and the zoom is 5x at the intermediate position, the baseline is 20 cm, and the angle is 20°. Shows an example in which the zoom is 10 times, the baseline is 25 cm, and the angle is 10°.

Example 5 will be described with reference to FIG. FIG. 11 is a diagram showing a stereo camera device 100 according to the fifth embodiment of the present invention. Example 5 is an example of a sensing system.

In this system, the stereo camera 101 and the computer 111 are connected by a network 81. The image is sent to the computer 111 via the network 81. The computer 111 uses the received image to perform the processing shown in FIG. The zoom control unit 115 controls the focal length of the camera. The baseline control unit 116 controls the baseline stage 113. The rotation control unit 117 controls the rotation stage 114. These units are connected to the computer 111 and perform control according to the processing results.

Next, the temporal relationship of the processing of each part in the configuration of FIG. 1 will be described. Here, the image acquisition unit 11 of the camera acquires an image at, for example, 30 frames/second, and at the present time, it is assumed that the image processing of the second frame will be performed.

The image of the second frame is simultaneously given from the image acquisition unit 11 to the feature point selection unit 13. At this time, the distance calculation unit 16 detects the distance from the left and right images to the object and the movement of the target object in the image, and if there is movement, a new camera is detected via the control amount calculation unit 17 and the camera control unit 12. It works to position. At this time, the distance calculation unit 16 refers to the calibration data database DB1 to obtain the calibration data D2 used for correction when calculating the distance. The latest calibration data D2 is at least the first frame. This is obtained by the processing of the feature point selection unit 13 and the carry unit calculation unit 14 in the image processing of the eye. That is, the distance calculation unit 16 calculates the distance from the image at the current time and the latest calibration data D2 obtained by the processing at the past time. Note that in the processing of the second frame, the calibration data database DB1 stores the calibration data D2 obtained from the image acquired in the second frame, and the processing of the distance calculation unit 16 in the third frame Will be used in.

The calibration data is calculated at the timing when the camera is controlled. Thereby, it becomes possible to calculate highly accurate distance data. Then, the camera can be appropriately controlled and further measurement can be performed according to the current distance of the measurement target.

Through the above, the stereo camera device 100 can calibrate the latest situation and reflect it in the distance control.

10: real image input unit, 11: image acquisition unit, 12: camera control unit, 13: feature point selection unit, 14: calib calculation unit, DB1: calib data database, 16: distance calculation unit, 17: control amount calculation unit. , 21: zoom control unit, 22: baseline control unit 23: rotation control unit, 24: tilt control unit, 25: pan control unit, 26: zoom control value, 27: baseline control value, 28: tilt control value, 29: Pan control value, 44: Distortion correction unit, 45: Parallel processing unit, 46: Distance calculation unit, P1: Distance data, 81: Network, 91, 92, 93: Camera spec, 94, 95, 96: Scan area , 97: integrated scan result, 100: stereo camera device, M: measurement target, g: movement path, 111: computer, 101: stereo camera, 113: baseline stage, 114: rotary stage, 115: zoom control unit, 116 : Baseline control unit, 117: Rotation control unit

Claims (12)

A stereo measurement device provided with left and right image information by the left and right cameras, and a processing unit for processing the left and right image information from a stereo camera unit including a camera control unit of the left and right cameras,
The processing unit of the stereo measurement device uses a feature point selection unit that selects a feature point from an image scene of left and right image information, and a calibration calculation that performs calibration using the selected feature point and calculates calibration data. Section, the calculated calibration data and the left and right image information, a distance calculation section for calculating a distance to a target object of a sensing target, and the calculated distance and image information, the left and right camera and a control amount calculating section for calculating a control amount for controlling the camera controller gives the control amount to the camera control unit of the left and right camera of the stereo camera unit,
The camera control units of the left and right cameras of the stereo camera unit include at least a control unit for zooming, a baseline, and a rotation, and the control amount calculating unit determines at least a control amount for zooming, a baseline, and rotation. Calculated and given to the camera control unit,
The control amount calculation unit that calculates the control amount for the zoom and baseline evaluates the stability of the measured distance data, and uses the distance data determined to be stable, using the average distance to the target object of the sensing target. To calculate the control amount of zoom and baseline,
The control amount calculation unit that calculates the control amount for the rotation controls the rotation by calculating the amount of movement of the target object of the sensing target when the region moves of the target object between the small regions obtained by dividing the image region into a plurality of areas in advance. The stereo measurement device , which calculates the control amount of the left and right cameras and selects the control amount calculation process of the zoom and baseline and the control amount calculation process of the rotation control according to the difference between the frames of the left and right cameras . The stereo measurement device according to claim 1 , wherein
The stereo measurement apparatus, wherein the control unit for the rotation includes a tilt control unit and a pan control unit. The stereo measurement device according to claim 1 or 2 , wherein
A feature point selection unit that selects feature points from image scenes of left and right image information, extracts feature points from a plurality of image scenes, and selects valid feature points from the extracted feature points. .. The stereo measurement device according to any one of claims 1 to 3 , wherein:
The distance calculation unit that calculates the distance to the target object of the sensing target includes a distortion correction unit that corrects the distortion of the camera image and a parallel processing unit that processes the parallelization of the left and right cameras. A stereo measurement apparatus, which calculates a distance to a target object to be sensed from subsequent image information. The stereo measurement device according to claim 1 or 2 , wherein
The control amount calculation unit that calculates the control amount for the zoom and the baseline, using the distance data measured by the distance calculation unit, calculates the average distance to the target object of the sensing target, and the zoom and the baseline. A stereo measurement device characterized by calculating a control amount. The stereo measurement device according to claim 1 or 2 , wherein
The control amount calculation unit that calculates the control amount for the zoom and baseline evaluates the stability of the distance data measured by the distance calculation unit and determines that the control amount is stable if the accuracy of the calculated control amount is increased. A stereo measurement device, characterized in that an average distance to a target object to be sensed is calculated using the distance data, and a control amount of zoom and baseline is calculated. The stereo measurement device according to claim 1 or 2 , wherein
The control amount calculation unit that calculates the control amount for the rotation calculates the amount of movement of the target object of the sensing target when there is a region movement of the target object between the small regions obtained by preliminarily dividing the image region into a plurality of regions, and controls the rotation. A stereo measuring device characterized by performing. The stereo measurement device according to any one of claims 1 to 7 ,
The stereo measurement device, wherein the stereo camera unit and the processing unit are connected via the Internet. The stereo measurement device according to any one of claims 1 to 8 , wherein:
The stereo camera unit holds data of a plurality of camera specifications at a plurality of different distances and changes the camera specifications according to the distance to obtain left and right image information. The stereo measurement device according to any one of claims 1 to 9 , wherein the left and right cameras are asynchronous cameras.
Regarding the images from the left and right cameras, if there is no difference between the frames of the left and right cameras, it is determined to be still and the control amount for zoom and baseline is calculated, and if there is a difference between the frames of the left and right cameras, A stereo measuring device characterized by calculating a control amount for rotation by judging it as movement. The stereo measuring device according to any one of claims 1 to 10 ,
A stereo characterized by tracking the moving object by calculating control amounts for the zoom, baseline, and rotation of the moving object within the visual fields of the left and right cameras and giving the calculated control amounts to the camera control unit. Measuring device. A stereo measurement device according to any one of claims 1 to 11 , and a stereo camera unit that provides left and right image information from left and right cameras and includes a camera control unit of the left and right cameras. Stereo measurement system.

Images