EP2630632A1

EP2630632A1 - Determination of the base width of a stereo image capture system

Info

Publication number: EP2630632A1
Application number: EP11770406.4A
Authority: EP
Inventors: Markus Hueser; Paul-Sebastian Lauer
Original assignee: Robert Bosch GmbH
Current assignee: Robert Bosch GmbH
Priority date: 2010-10-22
Filing date: 2011-10-07
Publication date: 2013-08-28
Also published as: DE102010042821A1; DE102010042821B4; CN103180877A; WO2012052308A1

Abstract

The invention relates to a method for determining a base width of a stereo capture system (102), in particular a stereo camera system. The method comprises a step of determining the base width on the basis of a recorded dimension of an object (104) captured by the stereo capture system (102), wherein the recorded dimension represents a value determined by the stereo capture system, a reference dimension of the object, which is predetermined for the object, with a reference distance between the object and the stereo capture system, the reference distance and a disparity value of the stereo capture system, which value is based on the object.

Description

Description title

DETERMINING THE BASE WIDTH OF A STEREO IMAGE RECORDING SYSTEM

State of the art

The present invention relates to a method for determining a basic width of a stereo detection system, to a corresponding device and to a corresponding computer program product.

The video-based environment perception for driver assistance systems is becoming increasingly important. U.a. Sensors consisting of one or more cameras. External influences, such as temperature changes or mechanical forces, make it possible to decalibrate a previously calibrated camera system. In this case, both the orientation of the cameras to each other and the base width of a stereo camera system, hereinafter referred to as "stereo base width", can be adjusted. The now wrong parameters of the camera model lead to a faulty depth measurement based on the stereo triangulation and all derived variables.

DE 10 2005 039 561 A1 describes a method for temperature compensation of optical systems.

Disclosure of the invention

Against this background, the present invention provides a method for determining a basic width of a stereo detection system, furthermore a device which uses this method and finally a corresponding computer program product according to the independent patent claims presented. Advantageous embodiments emerge from the respective subclaims and the following description.

The method according to the invention can be based on known methods for determining the intrinsic and extrinsic camera parameters with the aid of a reference object. In this case, a reference object whose geometric features are known, detected in one or more video images. From the detections of the geometrical features, it is possible to use known methods, as described, for example, in SCHREER, O .: Stereoanalysis and Image Synthesis. Springer Berlin Heidelberg, 2005. ISBN 978-3-540-23439-5 (Print) are described, the camera parameters are calculated. With known camera parameters, the distance from world points, ie points from the recorded scene, to the camera system can be determined with the aid of stereo triangulation.

The inventive approach is based on the fact that the stereo base width is determined based on a detected object. For the field of application of a stereo camera system, the stereo base width can be determined by measuring known geometric quantities in an image. Known geometrical quantities may be e.g. the height of license plates, the diameter of traffic signs or the width of traffic posts. These values are generally standardized and therefore known. A system according to the invention can determine these variables by the stereo measuring principle and correct the camera parameters in case of deviations. The relative orientation of the two cameras to each other can be determined by known methods. By contrast, the stereo base width can be redetermined separately at the same time or also in a second step by the inventive approach.

Known methods for automatically determining the orientation of the cameras do not include the correction of the stereo base width. Advantageously, by means of the approach according to the invention, a hitherto required implementation of a complete recalibration of the camera system with the reference object can be avoided. This time-consuming step had to be done so far "offline". Advantageously, the inventive approach allows an automatic correction of the depth measurement during operation of the camera system ("online" mode). So can during the operation of the camera system the stereo base width of a possibly modified system structure according to the invention be measured by an automatic determination of the basic width of the stereo camera system. Both in offline mode and in online mode, the cameras work. The vehicle can be in each case in motion. Thus, the determination of the base width while driving the vehicle can be performed. In offline mode, however, unlike the online mode, the downstream image processing algorithm eg for detecting vehicles or pedestrians does not work.

The present invention provides a method for determining a base width of a stereo detection system, in particular a stereo camera system, comprising the following step:

Determining the base width based on a detected dimension of an object detected by the stereo detection system, the detected dimension representing a value determined by the stereo detection system, a reference dimension of the object predetermined for the object at a reference distance of the object from the stereo detection system, the reference distance and an object related disparity value of the stereo detection system.

Thus, determining the base width may be based on an extent in the image, such as width, height, or depth, of an object detected by the stereo detection system at a certain distance from the stereo detection system and an object related disparity value of the stereo detection system , The distance may advantageously be determined from the extent in the image, but may alternatively be e.g. also via a separate distance measuring system, e.g. a radar, laser or PMD sensor.

The stereo detection system may comprise two detection devices arranged adjacent to one another, for example two cameras with objectives. The two detection devices can each generate an image of the same area of an environment of the stereo detection system. A distance between optical axes of the two detection devices can correspond here to the base width. The object may be a or marking located in the vicinity of the stereo detection system and located within a detection range of the stereo detection system. In a step of determining, the detected dimension may be based on a picture taken by the stereo detection system. The detected dimension of the object may, for example, correspond to a distance or a number of pixels between two image points characterizing the edges of the object on a sensor element of the stereo detection system. When working with sub-pixels, the distance may not be even. In a corresponding manner, the reference dimension can be specified in sub-pixels. A sub-pixel can be understood as a fraction of a pixel. When the object is detected at a reference distance, the detected dimension corresponds to the predetermined reference dimension. The object may have been previously measured so that the reference dimension at the corresponding reference distance is known. The reference dimension and the reference distance can be stored in a memory and read out of it. Information about a distance to the object at the time of image acquisition, from which the detected dimension of the object is determined, may be received via a suitable interface. The removal can be done by evaluating information provided by the stereo detection system, eg by the extent of the object in the image. Alternatively, the distance can be determined in another suitable manner, for example with a radar, laser or PM D sensor. The disparity value may be determined from images or image information provided by the two detectors. Disparity can be understood to mean an offset between two corresponding pixels of the two detection devices, each of which maps the same real point in the surroundings of the stereo detection system. It is also possible to use a plurality of disparity values relating to the object or a disparity information determined with respect to the entire object. The stereo detection system may, for example, be arranged on a vehicle and be designed to detect the surroundings of the vehicle. In online mode, continuous recordings of the surroundings can be generated by the stereo recording system. One or more images taken during online operation may be used as the basis for determining the basic width of the stereo detection system. To determine if correction of the base width is required, a current distance to the detected object can be determined in two different ways. If both distance values match, no correction is required. If the two distance values do not match, then a correction is required. On the one hand, the distance can be determined based on the dimension of the object detected at a particular time, the reference dimension of the object and the reference distance of the object on which the reference dimension is based. On the other hand, the distance can be determined based on the currently set base width and a disparity value determined at the specific time.

The base width may also be determined based on a focal length and a size of a sensor element of the stereo detection system. The sensor can represent an image recorder. In particular, the base width can be determined from a quotient of the product of the distance, the disparity value and the size of the sensor element and the focal length. According to an embodiment, the base width may be determined based on a distance of the detected object to the stereo detection system and the disparity value, wherein the distance is determined based on the sensed dimension, the reference dimension, and the reference distance. Advantageously, therefore, no further device for determining an actual distance to the object is required.

Furthermore, the method may include a step of detecting the object in a captured by the stereo detection system recording the environment of the stereo detection system and a step of assigning the reference dimension and the reference distance to the object. Advantageously, the object can be detected by means of an object recognition method in a captured during online operation of the stereo detection system recording. In a lookup table corresponding reference dimensions and reference distances can be stored for several reference objects. For each reference object, a plurality of different reference dimensions and associated reference distances can be stored. Furthermore, can suitable features of the reference object or objects may be stored, which allow recognition of the reference object.

In this case, in the step of recognizing, the object can be recognized as one of a plurality of known reference objects, to each of which at least one reference dimension and at least one reference distance are known. Thus, one or more reference pairs can be stored per reference object, wherein each reference pair comprises in each case a value for a reference dimension and a value for a reference distance. If the stereo detection system is arranged on a vehicle, the reference objects can typically be objects occurring in the surroundings of the vehicle, for example a traffic infrastructure or areas of other vehicles. Thus, the reference objects may be objects that are detected by the stereo detection system anyway during normal operation.

According to an embodiment, the step of determining the base width may be performed in response to the detection of the object in the shot. Thus, the method can be automatically triggered by the detection of a suitable object.

The method may include a step of determining the disparity value based on a stereo algorithm associated with the stereo detection system. By means of the stereo algorithm, a depth information of the recording of the stereo detection system can be determined. The disparity value of the object may be related to a point or edge associated with the object.

The present invention further provides an apparatus for determining a base width of a stereo detection system, in particular a stereo camera system, comprising: means for determining the base width based on a detected dimension of an object detected by the stereo detection system, wherein the detected dimension represents a value determined by the stereo detection system, a reference dimension of the object predetermined for the object at a reference distance of the object from the stereo. Detection system, the reference distance and an object related disparity value of the stereo detection system

The device may be part of the stereo detection system. The device is designed to carry out or implement the steps of the method according to the invention in corresponding devices. Also by this embodiment of the invention in the form of a device, the object underlying the invention can be solved quickly and efficiently.

In the present case, a device can be understood as meaning an electrical device which processes sensor signals and outputs control signals in dependence thereon. The device may have an interface, which may be formed in hardware and / or software. In the case of a hardware-based embodiment, the interfaces can be part of a so-called system ASIC or a so-called FPGA, for example, which contains a wide variety of functions of the device. However, it is also possible that the interfaces are own, in integrated circuits or at least partially consist of discrete components. In a software training, the interfaces may be software modules that are present, for example, on a microcontroller in addition to other software modules.

A computer program product with program code which can be stored on a machine-readable carrier such as a semiconductor memory, a hard disk memory or an optical memory and is used to carry out the method according to one of the embodiments described above if the program is on a computer corresponding to a computer is also of advantage Device is running.

The invention will be explained in more detail below with reference to the accompanying drawings. Show it:

Fig. 1 is an overview view of an embodiment of the present invention; Fig. 2 is a flowchart of an embodiment of the present invention; and Fig. 3 is a schematic representation of a stereo detection system, according to an embodiment of the present invention. In the following description of preferred embodiments of the present invention, the same or similar reference numerals are used for the elements shown in the various figures and similarly acting, wherein a repeated description of these elements is omitted. FIG. 1 shows a vehicle 100 with a stereo camera system 102, according to FIG

Embodiment of the present invention. The stereo camera system 102 can be used as a stereo measurement system. The stereo camera system 102 is designed to record images of the surroundings of the vehicle 100 while the vehicle 100 is in motion, as well as when the vehicle 100 is stationary. For example, the vehicle may be mounted on a vehicle

Fahrbahn and a traffic sign 104 may be located in a detection range of the stereo camera system 102. Thus, the stereo camera system 102 may detect the traffic sign 104. The images acquired by the stereo camera system can be provided, for example, to a driver assistance function and evaluated by it. For example, the traffic sign 104 can be detected by means of an object recognition. Further, the images or discrete images of the stereo camera system 102 may be provided to a device 106 for determining a base width of the stereo camera system 102. The device 106 may be part of the stereo camera system 102 or may be implemented as a separate unit interfaced to the stereo camera system 102. The device 106 is designed to determine, for example, based on the traffic sign 104, the base width. To this end, the device 106 may process disparity information regarding the traffic sign 104 determined by the stereo camera system 102. The base width determined by the device 106 may be used to update an existing value for the base width of the stereo camera system 102.

2 shows a flowchart of a method for determining a basic width of a stereo detection system, in particular a stereo camera system, according to an embodiment of the present invention. The Ver- Driving can be used to determine the base width of the camera system shown in Fig. 1.

In a step 210, a recording recorded by the stereo detection system is analyzed with respect to one or a plurality of predetermined objects. For this purpose, a suitable object recognition can be performed. If a corresponding object is detected, for example the traffic sign shown in FIG. 1, a distance to the object can be determined in a step 212. This can be done, for example, by measuring a dimension of a specific feature of the object and comparing it with a stored reference dimension. The dimension may correspond to an extension of the feature in the shot or image of the stereo detection system. The feature may represent, for example, a height, width or other size of the traffic sign. The reference dimension is related to the corresponding feature. The reference dimension can for example be stored in a memory and be read out in the case of recognition of the object. In a step 214, a disparity value assigned to the object can be determined. Subsequently, in a step 216, a currently existing base width can be determined based on the disparity value and the distance. In a step 218, a previously assumed base width of the stereo detection system may be replaced by the currently determined base width.

3 shows a plan view of an axially parallel stereo geometry of a stereo detection system, according to an exemplary embodiment of the present invention. The stereo detection system has a first detection system with a first lens 321 or a first lens consisting of one or more lenses and a first sensor element 323 and a second detection system with a second lens 325 or a second lens consisting of one or more lenses and a second sensor element 327. The optical axes of the first and second detection systems are parallel to each other, pass through the points 331, 333 and are spaced apart by the base width B. The first and second detection systems each have the same focus f. Shown is a point 341 of an object detected by the stereo detection system, which is incident on a first pixel 343 of the first sensor element 323 and on a second pixel 345 of the second sensor. sorelements 327 is mapped. The vectors ui, u ₂ indicate a distance between the pixels 343, 345 and the corresponding optical axes. Also, a distance p of the point 341 from the stereo detection system, and in particular from a plane in which the sensor elements 323, 327 are arranged, is shown.

According to one embodiment, for a method of automatically determining the base width of a stereo camera system, it is a prerequisite that the extent of known objects, such as a license plate or a traffic sign, is known in the image at one or more given reference distances. For the description of the method, h ₀ in the following course exemplarily designates the height of number plates in the camera image, indicated in pixels, at a reference distance of p ₀ = 10 m. As already described above, other objects with known geometric features are possible. Other reference distances are also possible.

Thus, the current distance of an object p ^ on the ratio of current height hi and reference height h ₀ and the reference distance p ₀ describe.

A =? - Po (1)

Here p ^ represents the current distance, hi the currently measured altitude, p ₀ the reference distance and h ₀ the reference altitude.

At the same time, the distance p [m] of an object to the stereo camera system (in the case of the axis-parallel stereo geometry) is described by the following formula.

B - f

δ - ά,

Where B is the stereo base width in meters, f is the focal length of a camera in meters, δ is the disparity in pixels and d _{u is} the width of a sensor element (scale factor) in meters per pixel. Fig. 3 outlines this relationship for the case of the axis-parallel stereo geometry. The formula (2) applies to the case of the axis-parallel stereo geometry. In the case of the general stereo geometry, δ is replaced by 51 and 52, each of which is a function of the orientation of the reference object in the camera 1 and the camera 2.

Except for the stereo base width, which may change due to mechanical or temperature-induced influences, for example, the further parameters from equation (2), ie the focal length f and sensor element size d _u , are assumed to be constant or 5 in the case of disparity correctly determines a stereo algorithm. As a result, an erroneous measurement of the distance p 1 of an object with equation (2) can be attributed to an errored value of the stereo base width B.

In order to determine the true value of the stereo base width B, equation (2) is now solved for B p - δ - d "

B = "- (3)

/ and by equation (1) the actual distance p ^ of a reference object, e.g. a license plate, determined. At the same time, a stereo algorithm provides the current disparity 5 for the object whose distance p ^ has just been determined. Then, the current disparity 5 and the current distance p ^ to the reference object in equation (3) is used, from which then the current stereo base width B can be calculated.

The embodiments described and shown in the figures are chosen only by way of example. Different embodiments may be combined together or in relation to individual features. Also, an embodiment can be supplemented by features of another embodiment. Furthermore, method steps according to the invention can be repeated as well as carried out in a sequence other than that described.

Claims

claims

Method for determining a basic width of a stereo detection system (102) arranged on a vehicle, in particular a stereo camera system, comprising the following steps:

Detecting a dimension of an object (104) detected by the stereo detection system (102),

wherein the detected dimension represents a value determined by the stereo detection system,

Ratio of this dimension with a object predetermined reference dimension of the object at a reference distance of the object from the stereo detection system and

the reference distance

for determining a distance of the detected object (104) to the stereo-detection system and

Determining the base width due to the distance and

an object-related disparity value of the stereo detection system. continuously recording images of the surroundings from the stereo during operation.

Detection system are generated, which are used as a basis for determining the base width and

determining the base width while driving the vehicle.

2. The method of claim 1, wherein the baseband width is further determined based on a focal length and a size of a sensor element of the stereo detection system. 3. A method according to any one of the preceding claims, wherein the base width is corrected when a current distance to the detected Object can be determined in two different ways and these do not match, taking

determining a distance based on the dimension of the object detected at a particular time, the reference dimension of the object and the reference distance of the object on which the reference dimension is based, and

the other distance is determined based on the currently set base width and a disparity value determined at the particular time.

Method according to one of the preceding claims, wherein the distance alternatively via a separate system for distance measurement, e.g. a radar, laser or PM D sensor is obtained.

Method according to one of the preceding claims, comprising a step of recognizing the object (104) in the recording and a step of associating the reference dimension and the reference distance to the object.

Method according to one of the preceding claims, wherein in the step of recognizing the object (104) is recognized as one of a plurality of known reference objects, to each of which at least one reference dimension and at least one reference distance is known.

A method according to any one of claims 5 or 6, wherein the step of determining the base width is performed in response to the detection of the object (104) in the shot.

Method according to one of the preceding claims, comprising a step of determining the disparity value based on a stereo algorithm associated with the stereo detection system (102).

Method according to one of the preceding claims, wherein the reference objects are typically objects occurring in the environment of the vehicle, for example a traffic infrastructure or areas of other vehicles, eg a license plate or a traffic sign.

0. The method according to claim 1, wherein the reference dimension is stored in a memory and read out in the case of recognition of the object.

Device for determining a basic width of a stereo detection system (102) arranged on a vehicle, in particular a stereo camera system, comprising the following feature: means (106) for detecting a dimension of an object detected by the stereo detection system (102) ( 104)

which measures this dimension with a reference dimension of the object predetermined for the object at a reference distance of the object from the stereo detection system and

ratio of reference distance,

for determining a distance of the detected object (104) to the stereo detection system with the following

Determining the base width due to the distance and

an object related disparity value of the stereo detection system which in use continuously generates images of the surroundings of the stereo detection system used as a basis for determining the base width and

determining the base width while driving the vehicle.

12. Computer program product with program code for carrying out the method according to one of claims 1 to 1 1, when the program is executed on a device.