WO2019048205A1

WO2019048205A1 - Multi-camera system for a vehicle

Info

Publication number: WO2019048205A1
Application number: PCT/EP2018/072208
Authority: WO
Inventors: Christian KLIER; Navid Nourani-Vatani
Original assignee: Siemens Mobility GmbH
Priority date: 2017-09-07
Filing date: 2018-08-16
Publication date: 2019-03-14
Also published as: DE102017215810A1

Abstract

The invention relates to a multi-camera system (10, 20, 30, 40, 50) for a vehicle. The multi-camera system (10, 20, 30, 40, 50) has at least three cameras (K1, K2, K3) for capturing image data (BD) from the surroundings of the vehicle. The cameras are arranged such that each of the at least three cameras (K1, K2, K3), paired with any other of the at least three cameras (K1, K2, K3), has a base line with a different position and/or a different length and/or a different orientation. The multi-camera system (10, 20, 30, 40, 50) comprises an evaluation unit (1) for determining a three-dimensional image (BD-3D) on the basis of the image data (BD) captured by the at least three cameras (K1, K2, K3), wherein in each case two image recorded images from two cameras (K1, K2; K1, K3; K2, K3) are combined with each other for depth determination. The invention also relates to a vehicle. Furthermore, a method for monitoring the surroundings of a vehicle is described.

Description

description

The invention relates to a multi-camera system for a vehicle. Furthermore, the invention relates to a vehicle. Moreover, the invention relates to a method for monitoring the environment of a vehicle. An automated or even autonomous driving to ermögli ^¬ chen, it is important to accurately and robust perceive the environment in front of a vehicle and around the vehicle. For this purpose, various sensors, such as cameras are used. Cameras have advantages because their resolution is high and they provide the ability to detect colors.

With a single camera, however, the distance to objects can not be detected and determined reliable Tiefeninfor ^¬ mation and redundancy are generated. To generate depth information, either a second

Used sensor that can measure distances, such as a RADAR system or LIDAR, and their information is combined with the camera image, or two cameras are combi ^¬ ned to produce a stereoscopic image. Problems with the use of LIDAR and RADAR are the low resolution of these sensors and the calibration. The limitation of the stereoscopic images is that they can provide accurate depth information only in a limited field of view or recording area. This area depends on the distance between the two cameras and the Brennwei ^¬ te of the lenses.

To obtain a larger recording area, several camera pairs are used at the same time. One camera pair has a shorter baseline, where the baseline is the distance between the two cameras, and another pair has a longer baseline. In this way the Gesamtauf ^¬ receiving area is the sum of the two receiving regions ones shown, provides. As baselines, a connection line between the cameras should be understood in this context.

However, the use of a large number of cameras is complex and expensive.

Thus, there is the task of realizing a stereoscopic image acquisition ^¬ me with an enlarged recording area and a good depth mapping, but less effort.

This object is achieved by a multi-camera system for a vehicle according to claim 1, a vehicle according to claim 10 and a method for monitoring the environment of a vehicle according to claim 11.

The multi-camera system according to the invention for a vehicle has at least three cameras for recording image data from the surroundings of the vehicle. The cameras are such angeord- net that each of the at least three cameras paired with ir ^¬ vicinity of another of the at least three cameras having a base line at a different position and / or a different length and / or a different orientation. That is, two different pairs of cameras have a baseline with a different one each

Position and / or a different length and / or a different orientation. A baseline is formed by links between each camera.

The multi-camera system comprises an evaluation unit for He ^¬ convey a three-dimensional image based on the information detected by the at least three cameras image data, wherein in each case two picture images are combined with each of two cameras Tie ^¬ fenerfassung. Advantageously, a

Compared to the use of a camera pair and / or redundant image data are taken, which make the system more robust by can be compared by two different cameras.

The vehicle according to the invention, preferably a rail vehicle, has the multi-camera system according to the invention. The vehicle according to the invention has the advantages of the camera system ^¬ fiction, contemporary.

In the inventive method for monitoring the ambient of a vehicle, a surrounding area of the vehicle with the aid of at least three cameras, which are so angeord ^¬ net that each of the at least three cameras paired with any other of the at least three cameras, a Basisli ^¬ never with each having a different position and / or having a different length and / or with a different orientation, captured. In addition, a three-dimensional image is determined on the basis of the image data acquired by the at least three cameras, with two image recordings from two cameras in each case being combined with each other for depth ^detection . The method according to the invention has the advantages of the camera system according to the invention.

Some components of the multi-camera system according to the invention can be formed predominantly in the form of software components. This relates in particular to parts of the ^{evaluation unit} . In principle, however, these components may also partly, particularly when it comes to very fast loading ^¬ bills, in the form of software-supported hardware such as FPGAs or the like to be realized. Likewise as the required interfaces, for example, be when it's all about a takeover of data from other Softwarekompo ^¬ components, designed as a software interface can. However, they can also be configured as hardware-based interfaces, which are controlled by suitable software.

A largely software-based implementation has the advantage that even in a vehicle existing computer Systems can be retrofitted after a possible supplementation by additional hardware elements in a simple way by a software update to work in the manner according to the invention. Insofar as the object is also achieved by a corresponding computer program product with a Computerpro ^¬ program, which is directly in a storage device ei ^¬ nes such a computer system can be loaded, ^¬ lead with Programmabschnit ^¬ th order for For all steps of the inventive method when the computer program in the computer system is executed.

Such a computer program ^product , in addition to the computer ^¬ program optionally additional components, such as documentation and / or additional components, including hardware components, such as hardware keys (dongles, etc.) for using the software

For transport to the storage device of the computer system and / or for storage on the computer system, a computer-readable medium, for example a memory stick, a

Hard disk or another portable or permanently installed data carrier serve, on which are stored and readable by a computer unit executable program sections of the computer program. The computer unit may be for example a purpose or more cooperating micropro ^¬ processors or the like.

The dependent claims and the following description each contain particularly advantageous embodiments and further developments of the invention. The claims of claim category may further be formed analogously to the dependent claims ^¬ from a different category of claim and the description parts in particular. In addition, in the context of the invention, the various features of different embodiments and claims can also be combined to form new embodiments. In one embodiment of the multi-camera system according to the invention, the at least three cameras each have the same focal length. The image sections of min ^¬ least three cameras are comparable beneficial and can be easy to three- dimensional images combined.

In a variant of the multi-camera system according to the invention, the at least three cameras are arranged colinearly. Before ^¬ geous different long baselines can be used to to capture different broad areas of the image between different cameras.

In one embodiment of the multi-camera system according to the invention, the distances between adjacent cameras are the same in each case. Advantageously, the recorded image areas are the same, so that a combination of images for generating three-dimensional images is facilitated.

In a variant of the multi-camera system according to the invention, the multi-camera system has three cameras, which are arranged in a triangle.

For example, the triangle may include an isosceles triangle or even an equilateral triangle. Can advanta- geous thus in the case of the equilateral ^¬ generate gon redundant image recordings and in the case of the isosceles triangle extra extensive different receiving areas arise as a result of under ^¬ different union base lines of different length.

In one embodiment of the multi-camera system according to the invention, the multi-camera system comprises four cameras, which are arranged in a quadrangle. Preferably, the four ^¬ eck comprises a rectangle. Advantageously, there are three different base lines with different lengths and possibilities for redundantly recording images. In a variant of the method according to the invention for monitoring the surroundings of a vehicle, depth information which was respectively obtained on the basis of the image data of the different pairs of cameras is compared on the basis of the image data of different pairs of cameras and checked for deviations. That is, it is checked whether the depth values that were determined based on the different pairs of cameras, and how much they may differ.

Based on the different information sources, depth data can be determined more accurately and / or erroneous image information can be detected and discarded.

The invention will be explained in more detail below with reference to the beige ^¬ added figures using ^exemplary embodiments. Show it:

1 shows a schematic representation of a Multikamerasys- tems according to a first exemplary embodiment of the invention ^¬,

2 shows a schematic representation of a Multikamerasys- tems according to a second exemplary embodiment of the invention ^¬, a schematic representation of a Multikamerasys- tems according to a third exemplary embodiment of the invention,

FIG 4 is a schematic illustration of an invention Multikamerasys- tems according to a fourth exemplary embodiment of the invention ^¬, a schematic representation of a Multikamerasys tems according to a fifth embodiment of the E, 6 shows a flowchart which illustrates a method for monitoring the surroundings of a vehicle according to an ^exemplary embodiment of the invention. 1 shows a multi-camera system 10 according to a first embodiment of the invention is shown schematically. The first embodiment may be considered as a special case of the illustrated in Figure 2 second embodiment, wherein the cameras shown in Figure 2 are so far away from each other until they lie in a straight line. The

Multi-camera system 10 has three cameras Kl, K2, K3, wherein a first camera Kl is arranged colinearly between a second camera K2 and a third camera K3 and has the same distance d to the second camera K2 and the third camera K3. The cameras are in the first exporting ^¬ approximately example as well as in all further in Figures 2 to 5 of the ^¬ art oriented such that images can be taken substantially perpendicular to the sheet plane. That is, the representation of the cameras Kl, K2, K3 can each be understood as a top view of the lenses or the outer objective lens of the cameras Kl, K2, K3. The orientation of the cameras may also differ. For example, different cameras K2, K3 are inclined inwards, that is, are aligned not quite paral lel ^¬ each other to receive a receiving area from slightly different directions or angles. It is also possible for all three cameras K1, K2, K3 to be oriented in a different direction than perpendicular to the plane of the page. The cameras Kl, K2, K3 are provided with a Auswer ^¬ processing unit 1 via data transmission cable (solid Li- nien) connected. The evaluation unit 1 determines based on the different image data of the cameras Kl, K2, K3 SD image data from an image area which is limited by the baseline of the three cameras Kl, K2, K3. Be the male is ^¬ rich by the length 2d baseline twice as large as in a stereoscopic recording with two cameras.

2 shows a multi-camera system 20 according to a second embodiment of the invention is shown schematically. In this variant, the three cameras Kl, K2, K3 are arranged in an equilateral triangle to each other. In this variant, although the shooting range is not larger than that of a camera pair, the third camera K3 provides redundant information that can be used to check and ensure that all information about the monitored environment is identical.

Cameras do not provide 3D information directly, but 2D images of it. Here, often, it happens that a ermittel ^¬ tes depth image is not correct. Reasons for this can be, for example, small errors in the triangulation due to shadows, occlusions or the lack of recognizable structures. A multi-camera system makes it possible to compare the determined depth of a first image pair with that of a second image pair. The same scene should be imaged in the second pair as in the first pair, and the second pair should have the same image depth as the first pair. If there is a deviation between the two depths, this should prompt corrective action.

The mentioned effect of a multi-camera system can be applied to ^¬ connexion with other image processing operations in. For example, in the case of object detection and object classification, two images of differently positioned cameras can be used to check whether the two images provide the same object, the same type and the same size. The additional redundancy makes the system reliable, which is crucial for an ADAS system and highly automated or even fully automated driving systems.

FIG. 3 schematically shows a multi-camera system 30 according to a third exemplary embodiment of the invention. In this embodiment, three cameras Kl, K2, K3 disposed in an isosceles triangle, wherein the first and the second camera Kl, K2, the base of the triangle bil ^¬ which only half as long as the legs, which by first and third camera Kl, K3 and the second and third camera K2, K3 are formed. Advantageously, in addition to the width d of the receiving area, a second width 0.5 d, which is only half as wide. In addition, Re ^¬ dancy is obtained for the base line with the length d.

4 shows a multi-camera system 40 according to a fourth embodiment of the invention is shown schematically. In this embodiment, three cameras Kl, K2, K3 disposed in an isosceles triangle, wherein the first and the second camera Kl, K2, the base of the triangle bil ^¬ which times as long 1.5 as the legs, which by first and third camera Kl, K3 and the second and third camera K2, K3 are formed. Advantageously, in addition to the width d of the receiving area, a second width of 1.5 d, which is 1.5 times as wide. Therefore, can also in this embodiment receiving areas with un ^¬ terschiedlichen dimensions realized. In addition, redundancy for the baseline with the dimension d is also achieved here.

In FIG 5, a multi-camera system 50 according to a fifth embodiment of the invention is shown schematically. In this embodiment, four cameras K1, K2, K3, K4 are arranged in a rectangle, with the first and second cameras K1, K2 and the third and fourth cameras K3, K4 forming the longer sides, and the first and fourth cameras Kl, K4 and the second and the third camera K2, K3 each form the shorter sides. In the exemplary embodiment shown in FIG. 5, the longer sides are twice as long as the shorter sides, so that different-width receiving areas are possible. In addition, the diagonals of the rectangle provide an additional third pick-up area with a width of

, Moreover, redundancies result from the cameras arranged parallel to one another. It is noted that the specified length behaves ^¬ nisse between different baselines in different embodiments may also be chosen differently. FIG. 6 shows a flow chart 600 which illustrates a ^method for monitoring the surroundings of a vehicle according to an exemplary embodiment of the invention. In step 6.1, image data BD from the surroundings of a vehicle is acquired with the aid of three colinearly arranged cameras. 6. The image data are combined in step to dreidi ^¬ dimensional image data BD-3D II. This is possible because the different cameras record image data from different perspectives. The two outer cameras are each positioned equidistant from a centrally located camera of the co-linearly arranged cameras. The Ka ^¬ meras each have the same focal length. In this way, images of comparable dimensions are taken from different positions and can be easily compared and / or combined with one another.

It is finally pointed out once again that the above-described methods and devices are merely preferred embodiments of the invention and that the invention can be varied by a person skilled in the art without departing from the scope of the invention, as far as it is specified by the claims is. It is the Fully ^¬ ness sake also noted that the use of the indefinite article "a" and does not exclude "a" that the features in question can also be present more than once. Similarly, the term "unit" does not exclude that it consists of several components, which may also be spatially distributed.

Claims

claims

A multi-camera system (10, 20, 30, 40, 50) for a vehicle, comprising:

at least three cameras (Kl, K2, K3) for recording image data (BD) from the surroundings of the vehicle, wherein the at least three cameras (Kl, K2, K3) are arranged such that each of the at least three cameras (Kl, K2, K3) paired with any other of the at least three Kame ^¬ ras (Kl, K2, K3) having a base line with a differing ^¬ chen position and / or a different length and / or a different orientation, an evaluation unit (1) for determining a three-dimen ^¬ dimensional image (BD-3D) detected on the basis of the at least three cameras (Kl, K2, K3) image data (BD), where the depth detection in each case two image exposures (of each of two cameras Kl, K2, Kl, K3; K2, K3) are combined with each other.

2. multi-camera system (10, 20, 30, 40, 50) according to claim 1, wherein the at least three cameras (Kl, K2, K3) each have the same focal length.

3. multi-camera system (10, 20, 30, 40, 50) according to claim 1 or 2, wherein the at least three cameras (Kl, K2, K3) are arranged colinear.

4. Multi-camera system (10, 20, 30, 40, 50) according to any one of the preceding claims, wherein the distances between adjacent cameras (Kl, K2, K2, K3) are the same.

5. multi-camera system (10, 20, 30, 40, 50) according to any one of the preceding claims, comprising three cameras (Kl, K2, K3), which are arranged in a triangle.

The multi-camera system (10, 20, 30, 40, 50) of claim 5, wherein the triangle comprises an isosceles triangle.

The multi-camera system (10, 20, 30, 40, 50) of claim 6, wherein the triangle comprises an equilateral triangle.

8. multi-camera system (10, 20, 30, 40, 50) according to any one of claims 1 to 7, comprising four cameras (Kl, K2, K3, K4), which are arranged in a quadrangle.

The multi-camera system (10, 20, 30, 40, 50) of claim 8, wherein the quadrilateral comprises a rectangle.

10. vehicle, preferably rail vehicle, comprising a multi-camera system (10, 20, 30, 40, 50) according to any one of the preceding claims.

11. A method of monitoring the environment of a vehicle, comprising the steps of:

image capture of an environmental area of the vehicle by means of at least three cameras (Kl, K2, K3) for recording image data (BD), wherein the at least three cameras (Kl, K2, K3) are arranged such that each of the at least three paired with ANY SIZE ^¬ ner other of the at least three cameras (Kl, K2, K3) has cameras (Kl, K2, K3) a base line at a different position and / or a different length and / or a different orientation loan,

Determining a three-dimensional image (BD-3D) on the basis of the image data (BD) acquired by the at least three cameras (Kl, K2, K3), two images of two cameras (Kl, K2, Kl, K3, K2 , K3) are combined with each other.

12. The method of claim 1, wherein on the basis of the image data (BD) of different pairs of cameras (Kl, K2, K3) depth information, each based on the image data (BD) of the different pairs of cameras (Kl, K2, K3) obtained, compared and checked for deviations.

13. Computer program product with a computer program, wel ^¬ ches directly into a memory unit of a computer unit ei ^¬ nes vehicle is loadable, with program sections to perform all the steps of a method according to claim 11 or 12, when the computer program is executed in the computer unit.

14. The computer-readable medium on which executable program portions are stored by a computer unit in order for For all steps of the method according to claim 11 or 12 ^¬ result when the program segments are executed by the computer unit.