EP3821598A2 - Stereokameravorrichtung sowie verfahren zum betrieb der stereokameravorrichtung - Google Patents
Stereokameravorrichtung sowie verfahren zum betrieb der stereokameravorrichtungInfo
- Publication number
- EP3821598A2 EP3821598A2 EP19740306.6A EP19740306A EP3821598A2 EP 3821598 A2 EP3821598 A2 EP 3821598A2 EP 19740306 A EP19740306 A EP 19740306A EP 3821598 A2 EP3821598 A2 EP 3821598A2
- Authority
- EP
- European Patent Office
- Prior art keywords
- image
- stereo
- mono
- camera
- evaluation device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
- H04N13/239—Image signal generators using stereoscopic image cameras using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
- G06T7/593—Depth or shape recovery from multiple images from stereo images
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
- H04N13/246—Calibration of cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
- H04N13/25—Image signal generators using stereoscopic image cameras using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
- G06T2207/10012—Stereo images
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
- G06T2207/30261—Obstacle
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N2013/0074—Stereoscopic image analysis
- H04N2013/0081—Depth or disparity estimation from stereoscopic image signals
Definitions
- Stereo camera device and method for operating the stereo camera device
- the invention relates to a stereo camera device according to the preamble of claim 1.
- the invention further relates to a motor vehicle with the stereo camera device and a method for operating the stereo camera device.
- Driver assistance systems in vehicles often use one or more cameras to record the vehicle surroundings, for example to detect a wide variety of objects in the vehicle surroundings.
- the cameras are usually located inside the vehicle behind the windshield and look through the window in the direction of travel.
- vehicle cameras for detecting obstacles, road markings, traffic signs and other traffic participants are known.
- stereo camera systems When determining object distances using cameras, stereo camera systems are currently preferably used. These consist of two generally identical camera modules (single cameras), which are constructed as separate optical systems, each with an image recording element and an imaging system.
- a stereo camera system is known from the publication DE 10 2014 206 227 A1, comprising two autonomous mono cameras and a stereo image evaluation unit.
- the first and second mono cameras are each designed for image evaluation using a mono camera algorithm and the stereo image evaluation unit for stereoscopic image evaluation of the image data of the first and second mono cameras using a stereo image algorithm.
- stereo evaluation is no longer possible, but mono information is still available, so that the reliability against failure is increased due to the duplicate environmental information.
- this type of verification does not solve the problem that the correct functionality of the stereo camera device can be identified as such. For example, it cannot be ruled out that individual components of the stereo camera device will become mechanically detached due to aging. An example is a shifted camera lens or a shifted image sensor due to a detachable adhesive point or a loosened screw. In such cases, the mechanically incorrectly arranged individual parts now result in incorrect stereo evaluation.
- the object is achieved by a stereo camera device with the features of claim 1, a motor vehicle with the features of claim 4 and by a method with the features of claim 5.
- a stereo camera device for a motor vehicle is proposed.
- the stereo camera device can be integrated into a driver assistance device of the motor vehicle or can be connected to it.
- the driver assistance device includes z. B. a (emergency) braking assistant, a (emergency) steering assistant, a parking assistant, a cruise control assistant and / or a driving assistant track assistant.
- Such driver assistance functions are essential for example for highly autonomous driving.
- the stereo camera device comprises a first and a second mono camera, the first and the second mono camera each having at least one image sensor and an optical system for capturing images.
- the first and second mono cameras are laterally offset, i. H. arranged offset on a line parallel to the vehicle's transverse axis.
- the first and second mono cameras are constructed identically.
- the stereo camera device comprises an evaluation device, the image sensor of the first mono camera and the image sensor of the second mono camera being connected to the evaluation device for transmitting the image data generated by the respective image sensor.
- the image data transmitted to the evaluation device include in particular at least one image or an image sequence with at least two images from the respective mono camera.
- the evaluation device is designed to evaluate a first stereo image from the image data transmitted to the first and second mono cameras using a stereo camera algorithm.
- data is merged from image information from the image data transmitted by the first and second mono cameras.
- the stereo image makes it possible to determine object distances from one's own motor vehicle.
- the evaluation device is designed to evaluate a second stereo image from the image data transmitted to the first mono camera using a mono algorithm, in particular a structure-from-motion algorithm.
- a mono algorithm in particular a structure-from-motion algorithm.
- the Structure-from-motion algorithm is, in particular, a reconstruction of structures such as sizes, distances, angular similarities and extensions from mutually offset images generated by only one of the two mono cameras.
- the creation of a stereo image from serially acquired image data of a mono camera takes place.
- the evaluation device is designed, from the first and second stereo image, in particular redundant image depth information such as. B. to determine the distance of the stereo camera device to an object.
- the evaluation device is designed to check the determined image depth information from the first and second stereo image for agreement. In this way it is possible to determine a mechanical defect or an offset or slipped arrangement of individual components in the stereo camera system and consequently to avoid incorrect evaluation of image information. This ensures vehicle safety.
- the evaluation device is designed to evaluate a third stereo image from the image data of the second mono camera, the evaluation device being designed to check the image depth information determined from the first and third recorded stereo images for correspondence.
- the test is thus advantageously specified in a cost-effective manner.
- the evaluation device is designed to check the image depth information determined from the second and third recorded stereo image for correspondence. So there is another Verification and thus more precise monitoring of a correct arrangement of the stereo camera device.
- Another object of the invention relates to a motor vehicle with a stereo camera device, the motor vehicle having a driver assistance device which is designed to output a warning to the driver and / or to switch off in the event of a discrepancy in the checked image depth information.
- FIG. 1 shows a schematic representation of the sequence of the method according to the invention.
- image sensors of the first and second mono cameras acquire images, the images being transmitted to the evaluation device as image data.
- the first stereo image is evaluated from the image data transmitted by the first and second mono cameras, and the second stereo image is evaluated from the image data transmitted from the first mono camera.
- the image depth information from the first and second stereo image is then transmitted and checked for agreement. In particular, the determined image depth information or a determined distance to an object or obstacle is thus verified.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Measurement Of Optical Distance (AREA)
- Studio Devices (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
- Traffic Control Systems (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| DE102018211268.1A DE102018211268A1 (de) | 2018-07-09 | 2018-07-09 | Stereokameravorrichtung sowie Verfahren zum Betrieb der Stereokameravorrichtung |
| PCT/DE2019/200070 WO2020011314A2 (de) | 2018-07-09 | 2019-07-03 | Stereokameravorrichtung sowie verfahren zum betrieb der stereokameravorrichtung |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| EP3821598A2 true EP3821598A2 (de) | 2021-05-19 |
Family
ID=67303339
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| EP19740306.6A Pending EP3821598A2 (de) | 2018-07-09 | 2019-07-03 | Stereokameravorrichtung sowie verfahren zum betrieb der stereokameravorrichtung |
Country Status (5)
| Country | Link |
|---|---|
| US (1) | US20210243421A1 (de) |
| EP (1) | EP3821598A2 (de) |
| CN (1) | CN112136323A (de) |
| DE (1) | DE102018211268A1 (de) |
| WO (1) | WO2020011314A2 (de) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US12183028B2 (en) * | 2019-12-04 | 2024-12-31 | Beijing Smarter Eye Technology Co. Ltd. | Object-based short range measurement method, device and system, and storage medium |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7630806B2 (en) * | 1994-05-23 | 2009-12-08 | Automotive Technologies International, Inc. | System and method for detecting and protecting pedestrians |
| JP5979500B2 (ja) * | 2011-04-07 | 2016-08-24 | パナソニックIpマネジメント株式会社 | 立体撮像装置 |
| DE102011101246A1 (de) * | 2011-05-11 | 2012-11-15 | Conti Temic Microelectronic Gmbh | Redudante objektdetektion für fahrerassistenzsysteme |
| JP5645769B2 (ja) * | 2011-08-01 | 2014-12-24 | 株式会社日立製作所 | 画像処理装置 |
| DE102013208106A1 (de) * | 2012-05-24 | 2013-11-28 | Ifm Electronic Gmbh | Tiefenbildkamerasystem |
| DE102014206227A1 (de) | 2014-04-02 | 2015-10-08 | Conti Temic Microelectronic Gmbh | Stereokamerasystem |
| US10618467B2 (en) * | 2016-03-22 | 2020-04-14 | Research & Business Foundation Sungkyunkwan University | Stereo image generating method using mono cameras in vehicle and providing method for omnidirectional image including distance information in vehicle |
| DE102016205471A1 (de) * | 2016-04-01 | 2017-05-04 | Conti Temic Microelectronic Gmbh | Stereokameravorrichtung sowie ein Verfahren zum Betrieb der Stereokameravorrichtung |
| JP6515881B2 (ja) * | 2016-06-28 | 2019-05-22 | 株式会社デンソー | 車両用衝突検知装置 |
-
2018
- 2018-07-09 DE DE102018211268.1A patent/DE102018211268A1/de not_active Withdrawn
-
2019
- 2019-07-03 US US17/052,588 patent/US20210243421A1/en active Pending
- 2019-07-03 WO PCT/DE2019/200070 patent/WO2020011314A2/de not_active Ceased
- 2019-07-03 CN CN201980033019.7A patent/CN112136323A/zh active Pending
- 2019-07-03 EP EP19740306.6A patent/EP3821598A2/de active Pending
Also Published As
| Publication number | Publication date |
|---|---|
| WO2020011314A3 (de) | 2020-03-05 |
| DE102018211268A1 (de) | 2020-01-09 |
| WO2020011314A2 (de) | 2020-01-16 |
| CN112136323A (zh) | 2020-12-25 |
| US20210243421A1 (en) | 2021-08-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP2684070B1 (de) | Verfahren zum detektieren einer parklücke, parkhilfesystem und kraftfahrzeug mit einem parkhilfesystem | |
| DE102018205322A1 (de) | Verfahren und Steuervorrichtung zum Erkennen einer Fehlfunktion zumindest eines Umfeldsensors eines Kraftfahrzeugs | |
| DE102017206123A1 (de) | Verfahren und Vorrichtung zur Fusion von Daten verschiedener Sensoren eines Fahrzeugs im Rahmen einer Objekterkennung | |
| DE102010049093A1 (de) | Verfahren zum Betreiben zumindest eines Sensors eines Fahrzeugs und Fahrzeug mit zumindest einem Sensor | |
| DE102011105074A1 (de) | Verfahren und Vorrichtung zur Bestimmung einer Sichtweite für ein Fahrzeug | |
| DE102014201158A1 (de) | Verfahren und Vorrichtung zum Überprüfen eines von einer Objekterkennung erkannten relevanten Objekts | |
| DE102020200650A1 (de) | Verfahren und Vorrichtung zum Betreiben einer automatisierten Spurführung eines Kraftfahrzeugs | |
| DE102016009327A1 (de) | Vorrichtung und Verfahren zur Kalibrierung einer Kamera | |
| DE102008047007B4 (de) | Vorrichtung und Verfahren zur Durchführung einer Bremsenprüfung | |
| EP4423458B1 (de) | Überwachung und detektion von sensorfehlern in inertial-mess-systemen | |
| DE102013220487A1 (de) | Fahrspurerkennungssystem umd Spurhaltesystem | |
| EP3821598A2 (de) | Stereokameravorrichtung sowie verfahren zum betrieb der stereokameravorrichtung | |
| DE102007017293A1 (de) | Fahrspurdetektion mit Schwarz-Weiß-Kamera und Farbkamera | |
| EP3032517B1 (de) | Vorrichtung und verfahren zur unterstützung eines fahrers eines fahrzeugs, insbesondere eines nutzfahrzeugs | |
| WO2022184341A1 (de) | Verfahren zum betrieb eines fahrzeugs | |
| DE102012023980B4 (de) | Verfahren und Vorrichtung zur Verifizierung einer ersten Relativlage mittels einer zweiten Relativlage | |
| DE102016205471A1 (de) | Stereokameravorrichtung sowie ein Verfahren zum Betrieb der Stereokameravorrichtung | |
| DE102017201620A1 (de) | Vorrichtung und Verfahren zum automatisierten, teilautomatisierten oder assistierenden Rangieren eines Kraftfahrzeugs | |
| EP2565580B1 (de) | Verfahren zur Bestimmung eines Ausmaßes eines Objekts in einer Umgebung eines Fahrzeugs sowie eine entsprechende Vorrichtung und ein Fahrzeug mit einer derartigen Vorrichtung | |
| DE102014206227A1 (de) | Stereokamerasystem | |
| DE102022201199B3 (de) | Verfahren und Vorrichtung zur Überwachung eines Spurerkennungssystems, Spurerkennungssystem | |
| DE102012022857A1 (de) | Vorrichtung und Verfahren zum Erfassen von Fahrwerkssymmetrien Basierend auf 3D-Reifendaten und Kraftsensoren | |
| DE102017010307B4 (de) | Verfahren zur Erfassung einer Umgebung eines Fahrzeugs | |
| DE102019008091A1 (de) | Verfahren zum redundanten Ermitteln einer Eigenbewegung eines Fahrzeugs, Vorrichtung eingerichtet zum Durchführen eines solchen Verfahrens, und Fahrzeug mit einer solchen Vorrichtung | |
| DE102009033855A1 (de) | Verfahren und Vorrichtung zur Funktionsprüfung eines Fahrerassistenzsystems für ein Fahrzeug |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
| PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
| 17P | Request for examination filed |
Effective date: 20210209 |
|
| AK | Designated contracting states |
Kind code of ref document: A2 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
| DAV | Request for validation of the european patent (deleted) | ||
| DAX | Request for extension of the european patent (deleted) | ||
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
| 17Q | First examination report despatched |
Effective date: 20230706 |
|
| RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: AUMOVIO AUTONOMOUS MOBILITY GERMANY GMBH |