EP3619913A1 - Verfahren und vorrichtung zur verbesserung von kameraaufnahmen für fahrerassistenzsysteme - Google Patents
Verfahren und vorrichtung zur verbesserung von kameraaufnahmen für fahrerassistenzsystemeInfo
- Publication number
- EP3619913A1 EP3619913A1 EP18733504.7A EP18733504A EP3619913A1 EP 3619913 A1 EP3619913 A1 EP 3619913A1 EP 18733504 A EP18733504 A EP 18733504A EP 3619913 A1 EP3619913 A1 EP 3619913A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- image
- scattered light
- vehicle
- intensity
- lower limit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
- H04N25/61—Noise processing, e.g. detecting, correcting, reducing or removing noise the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/76—Circuitry for compensating brightness variation in the scene by influencing the image signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/71—Circuitry for evaluating the brightness variation
Definitions
- the invention relates to a method and a device for removing a lightening generated by scattered light from an image of a vehicle camera.
- Camera systems for vehicles are known.
- camera systems are known in which a front camera for detecting a variety of objects in an external environment of the vehicle is used.
- Typical applications for such camera systems are, for example, the detection of traffic signs,
- the vehicle camera of a camera system is usually arranged behind a windshield of the vehicle and looks in the direction of travel through it.
- stray light funnels or stray light diaphragms are typically used, which are intended to hide these disturbances.
- straylight apertures may tend to scatter possible backlight directly into the vehicle camera.
- Mechanical countermeasures are limited effective and can be associated with higher production costs. In addition, these mechanical countermeasures can also have aesthetic effects, which can adversely affect a design of the interior of the vehicle. The opening angle of today produced vehicle cameras is typically about 50 °. There is a need, other road users, especially crossing
- an opening angle of an inserted camera of the camera system is at least 100 °.
- stray light apertures are larger than the aperture angle of the camera so as not to restrict the field of view of the camera.
- Opening angle of the camera and the distance between a camera lens and the windshield and the inclination of the windshield define a necessary width of the lens hoods.
- the required increasing geometric size of the straylight diaphragms can be an all-round view of the
- the present invention proposes a computational solution for the removal of a lightening produced by scattered light from an image of a vehicle camera, wherein this computational solution can be realized with known hardware, e.g. with local contrast enhancement integrated in the ISP (Image Signal Processor) or tone mapping (dynamic compression to reduce the ISP (Image Signal Processor) or tone mapping (dynamic compression to reduce the ISP (Image Signal Processor) or tone mapping (dynamic compression to reduce the ISP (Image Signal Processor) or tone mapping (dynamic compression to reduce the ISP (Image Signal Processor) or tone mapping (dynamic compression to reduce the ISP (Image Signal Processor) or tone mapping (dynamic compression to reduce the ISP (Image Signal Processor) or tone mapping (dynamic compression to reduce the ISP (Image Signal Processor) or tone mapping (dynamic compression to reduce the ISP (Image Signal Processor) or tone mapping (dynamic compression to reduce the ISP (Image Signal Processor) or tone mapping (dynamic compression to reduce the ISP (Image Signal Processor) or tone mapping (dynamic compression to reduce the ISP (Image Signal Processor) or tone mapping
- a solution according to the invention enables several parameters to be monitored and made plausible in order to stabilize and secure an associated system.
- the invention takes advantage of the fact that there are several types of artifacts caused by backlighting in a camera.
- the present invention counteracts in particular a lightening, the existing through
- an estimation of stray light can always be influenced by the content of the respective image.
- the present invention allows parameters that are time-invariant or predicted to be estimated, thereby making it possible to plausibilize an estimate over time, in particular.
- a method of removing a lightening produced by stray light from an image of a vehicle camera is provided.
- an image of an external environment of a vehicle is taken by a vehicle camera disposed in or on the vehicle.
- the viewing direction of the vehicle camera can be chosen arbitrarily.
- the vehicle camera for example, in the field of
- the vehicle camera can be arranged, for example, in the region of a rear window of the vehicle and look through it in the backward direction.
- the vehicle is, for example, a motor vehicle, such as a car, Bus or truck. Within the picture is a progression of the total intensity (including the unwanted
- the scattered light intensity is caused by scattered light, which enters the detection range of the vehicle camera.
- the scattered light is caused by light from the sun, which is in the image or near the image, with a significant amount of sunlight is scattered through the lens hood in the vehicle camera.
- the determination (estimation) of the course of the scattered light intensity can take place, for example, by means of a minfilter, erosion or similar methods, in particular over the entire image, line by line or block by block.
- An approximate function of a course of the lower limit of the scattered light intensity is estimated.
- an approximation function is determined which describes the lower limit of the course of the scattered light intensity.
- the term "lower limit of the scattered light intensity” can be understood to mean, in particular, an intensity profile of the scattered light intensity in the horizontal direction which is not undershot from the determined overall intensity The closer the estimated approximation function is to the approximate one
- the estimated approximation function is subtracted or subtracted from the determined course of the (total) intensity, wherein the subtracting or subtracting takes place in particular line by line.
- the determined light intensity can be locally reduced, in particular at a horizontal position in a line of a relevant image, by a value which the estimated approximation function assumes at the relevant point.
- This image is provided, it may e.g. output, further processed and / or displayed, e.g. on a display inside the vehicle. From the stray-light-reduced or stray-corrected image, image processing can be used, for example. Objects in the vehicle environment can be detected and classified.
- a Gaussian distribution is chosen as the approximation function of the lower limit of the scattered light intensity.
- a Gaussian distribution particularly accurately describes the scattered light intensity within the image caused by sunlight.
- Proximity function of the determined course of the intensity can also be done accordingly over the whole picture, line by line or block by block.
- the lower limit of the scattered light intensity is interpolated in regions. This is especially advantageous in situations where an estimate in blocks or rows is missing or fails.
- a representation of the sky or an environment of the sun can particularly disturb an estimate.
- the estimation of the course of the proximity function of the lower limit of the scattered light intensity can be achieved by several
- the estimated parameters i.e., the estimate of the
- Scattered light intensity can also be tracked from image to image.
- the estimated approximation function of the course of the lower limit of the scattered light intensity within a the first image is tracked in at least one subsequently recorded second image of the external environment of the vehicle. It is exploited that a position of the sun, taking into account the proper motion of the vehicle or the
- Self-movement of the vehicle camera in particular by means of a filter, e.g. by means of a Kalman filter, a position of the sun which causes the scattered light, in which at least one subsequently recorded second image of the external environment of the vehicle is estimated.
- a filter e.g. by means of a Kalman filter
- This is particularly advantageous when the sun disappears and then reappears, which is e.g. in the presence of tree shadows, buildings or underpasses.
- This additionally solves a problem occurring with known methods: not all parameters have to be re-estimated from image to image, but only the intensity, if only from the previous first image a contribution has to be estimated.
- a vignetting of the scattered light is taken into account. This makes it possible to make the estimation of the lower limit of the scattered light intensity more robust.
- an apparatus for removing a lightening produced by stray light from an image of a vehicle camera comprises a vehicle camera and a processor unit, wherein the vehicle camera can be arranged on a vehicle such that the vehicle camera can take at least one image of an external environment of the vehicle, and wherein the
- Processor unit is adapted to the process steps according to the method according to the first aspect of the invention.
- Fig. 1 is a side view of a vehicle with a
- Fig. 2 is a graphical comparison of a determined
- Fig. 1 shows a vehicle 1, in the embodiment shown, an automobile.
- the vehicle 1 comprises a device 2 for removing a lightening generated by scattered light from an image of a vehicle camera 3.
- the device has a vehicle camera 3 and a processor unit 4 in the embodiment shown.
- the vehicle camera 3 can be arranged in an interior space 5 of the vehicle 1, in particular in an area behind a windshield of the vehicle 1
- Vehicle camera 3 can be taken through the windshield, an image or a sequence of images from an external environment 6 of the vehicle 1, wherein the image sequence may include a plurality of outdoor shots, and wherein the outdoor shots in time
- Fig. 2 shows a diagram, wherein a horizontal position is indicated on a horizontal axis, and wherein on a horizontal axis vertical axis is given a linear intensity.
- the following three graphs are plotted for a row of an image recorded by means of the vehicle camera 3, which images the external environment 6 of the vehicle 1:
- the intensity is caused by light, which enters the detection range of the vehicle camera 3.
- the determination of the course of the proximity function 8 of the scattered light intensity can be carried out by means of the processor unit 4, e.g. by means of a minfilter, erosion or similar methods.
- Scattered light intensity (second graph 8) is based in the illustrated embodiment on a Gaussian curve and represents a line by line intensity curve of the scattered light intensity in the horizontal direction, the estimated
- Scattered light intensity can be interpolated in certain areas, especially where an estimate is missing or fails.
- the estimation of the approximation function can be done by several
- Proximity function 8 are checked whether the position of the maximum of the total intensity 7 and the determined stray-light corrected intensity 9 is constant along a vertical course of the image. If this test gives a positive result, i. that the position of the maximum remains constant along the vertical course of the image, it can be assumed that not too much has been subtracted.
- the graph 9 in FIG. 2 is relatively constant in the horizontal ranges 0 to 250, 550 to 750, and 800 to 900
- Stray light within the captured image has the same color.
- This image can be output, processed and / or displayed, e.g. on a display inside the vehicle.
- Scattered light intensity 8 can also be tracked from image to image.
- the estimated approximation function of the lower limit of the scattered light intensity 8 within a first track image in at least one subsequently captured second image of the external environment of the vehicle to be tracked is exploited that a position of the sun, taking into account the proper motion of the vehicle or the
- Tree shadows, buildings or an underpass may be the case.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
- Transforming Light Signals Into Electric Signals (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE102017207376.4A DE102017207376A1 (de) | 2017-05-03 | 2017-05-03 | Verfahren und Vorrichtung zur Verbesserung von Kameraaufnahmen für Fahrerassistenzsysteme |
PCT/DE2018/200044 WO2018202261A1 (de) | 2017-05-03 | 2018-05-02 | Verfahren und vorrichtung zur verbesserung von kameraaufnahmen für fahrerassistenzsysteme |
Publications (1)
Publication Number | Publication Date |
---|---|
EP3619913A1 true EP3619913A1 (de) | 2020-03-11 |
Family
ID=62712700
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP18733504.7A Withdrawn EP3619913A1 (de) | 2017-05-03 | 2018-05-02 | Verfahren und vorrichtung zur verbesserung von kameraaufnahmen für fahrerassistenzsysteme |
Country Status (5)
Country | Link |
---|---|
US (1) | US11025834B2 (de) |
EP (1) | EP3619913A1 (de) |
JP (1) | JP7219710B2 (de) |
DE (1) | DE102017207376A1 (de) |
WO (1) | WO2018202261A1 (de) |
Family Cites Families (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2004015597A (ja) | 2002-06-10 | 2004-01-15 | Minolta Co Ltd | 電子カメラ |
JP2007028540A (ja) | 2005-07-21 | 2007-02-01 | Nissan Motor Co Ltd | 車両外状況提示システム及び車両外状況提示方法 |
WO2008127685A1 (en) * | 2007-04-13 | 2008-10-23 | John Kesterson | Filter assembly and image enhancement system for a surveillance camera and method of using the same |
WO2011078199A1 (en) * | 2009-12-25 | 2011-06-30 | Ricoh Company, Ltd. | Object identifying apparatus, moving body control apparatus, and information providing apparatus |
US8730356B2 (en) * | 2011-03-07 | 2014-05-20 | Sony Corporation | System and method for automatic flash removal from images |
DE102011016652A1 (de) * | 2011-04-09 | 2012-10-11 | Conti Temic Microelectronic Gmbh | Automatische Sonnenblende |
EP3149656B1 (de) | 2014-05-27 | 2020-01-15 | Robert Bosch GmbH | Nachweis, identifizierung und verringerung von der linsenkontamination für fahrzeugmontierte kamerasysteme |
US20160252905A1 (en) | 2014-08-28 | 2016-09-01 | Google Inc. | Real-time active emergency vehicle detection |
DE102014217749A1 (de) * | 2014-09-04 | 2016-03-10 | Conti Temic Microelectronic Gmbh | Streulichtblende |
US9953210B1 (en) * | 2017-05-30 | 2018-04-24 | Gatekeeper Inc. | Apparatus, systems and methods for improved facial detection and recognition in vehicle inspection security systems |
-
2017
- 2017-05-03 DE DE102017207376.4A patent/DE102017207376A1/de not_active Withdrawn
-
2018
- 2018-05-02 WO PCT/DE2018/200044 patent/WO2018202261A1/de unknown
- 2018-05-02 EP EP18733504.7A patent/EP3619913A1/de not_active Withdrawn
- 2018-05-02 JP JP2019540448A patent/JP7219710B2/ja active Active
- 2018-05-02 US US16/479,677 patent/US11025834B2/en active Active
Also Published As
Publication number | Publication date |
---|---|
JP2020520486A (ja) | 2020-07-09 |
US11025834B2 (en) | 2021-06-01 |
DE102017207376A1 (de) | 2018-11-08 |
US20200195856A1 (en) | 2020-06-18 |
JP7219710B2 (ja) | 2023-02-08 |
WO2018202261A1 (de) | 2018-11-08 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
DE112014005160T5 (de) | Verfahren zum Erkennen eines verdeckten Zustands einer Kamera, Kamerasystem und Kraftfahrzeug | |
DE102010030044A1 (de) | Wiederherstellvorrichtung für durch Wettereinflüsse verschlechterte Bilder und Fahrerunterstützungssystem hiermit | |
EP1706773A1 (de) | Nachtsichtsystem für kraftfahrzeuge mit partiellem optischem filter | |
WO2014094941A1 (de) | Kraftfahrzeug mit kamera-monitor-system | |
DE102013114996A1 (de) | Bildsuperauflösung für dynamischen Rückspiegel | |
DE102016121755A1 (de) | Verfahren zum Bestimmen eines zusammengesetzten Bilds eines Umgebungsbereichs eines Kraftfahrzeugs mit Anpassung von Helligkeit und/oder Farbe, Kamerasystem sowie Krafzfahrzeug | |
DE102014006153A1 (de) | Verfahren zur Bilddarstellung eines indirekten Sichtfeldes im Fahrzeugumfeld eines Fahrzeugs, insbesondere eines Nutzfahrzeugs, mit einem Kamera-Monitor-System | |
DE102008042537A1 (de) | Verfahren zum Kalibrieren eines Bilderfassunssystems in einem Kraftfahrzeug | |
DE102009001122A1 (de) | Kameraanordnung und Verfahren zum Ermitteln von Bildsignalen mit Farbwerten | |
DE102013012778A1 (de) | Verfahren zum Detektieren eines sich bewegenden Fußgängers anhand von charakteristischen Merkmalen und optischen Flussvektoren eines Bilds, Kamerasystem und Kraftfahrzeug | |
DE102013222304A1 (de) | Verfahren zur Bestimmung von Objektentfernungen mit einer in einem Kraftfahrzeug verbauten Kamera | |
WO2018202261A1 (de) | Verfahren und vorrichtung zur verbesserung von kameraaufnahmen für fahrerassistenzsysteme | |
DE102008045007A1 (de) | Verfahren zur bildgestützten Überwachung einer Fahrzeugumgebung, insbesondere einer Umgebung eines Nutzfahrzeuges | |
DE102006040349B4 (de) | Verfahren und Vorrichtung zur Erkennung von defekten Pixeln eines Bildaufnahmesensors in einem Fahrerassistenzsystem | |
DE102015217265A1 (de) | Stereokameravorrichtung zur Umgebungserfassung eines Kraftfahrzeugs, Kraftfahrzeug mit einer solchen Stereokameravorrichtung sowie ein Verfahren zur Erkennung von Regentropfen oder Ablagerungen | |
DE102021004337A1 (de) | Vorrichtung und Verfahren zur Filterung von Spiegelungen in Bilddaten | |
DE102012024650A1 (de) | Verfahren zur Reduktion einer HDR-Bewegungsunschärfe bei einem HDR-Bild, HDR-Kameraeinrichtung und Kraftfahrzeug | |
DE102013011844A1 (de) | Verfahren zum Anpassen einer Gammakurve eines Kamerasystems eines Kraftfahrzeugs, Kamerasystem und Kraftfahrzeug | |
WO2022122240A1 (de) | Verfahren und vorrichtung zum detektieren einer störpflanzenabbildung in einem kamerarohbild und bildverarbeitungseinrichtung | |
DE102015216907A1 (de) | Digitaler Rückspiegel | |
DE102012016865B4 (de) | Verbessertes Alpha-Blending von Bildern eines Kamerasystems eines Kraftfahrzeugs | |
DE102013020949A1 (de) | Verbessertes Alpha-Blending von Bildern eines Kamerasystems eines Kraftfahrzeugs | |
DE102016112483A1 (de) | Verfahren zum Reduzieren von Störsignalen in einem Draufsichtbild, das ein Kraftfahrzeug und einen Umgebungsbereich des Kraftfahrzeugs zeigt, Fahrerassistenzsystem sowie Kraftfahrzeug | |
DE102018212179A1 (de) | Bildverarbeitungsvorrichtung und verfahren zur bildverarbeitung | |
DE102019003562A1 (de) | Verfahren zur Auswertung von mittels einer Kamera eines Fahrzeugs erfassten Bildern |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20191203 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) | ||
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
17Q | First examination report despatched |
Effective date: 20201116 |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: CONTI TEMIC MICROELECTRONIC GMBH |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION HAS BEEN WITHDRAWN |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: CONTINENTAL AUTONOMOUS MOBILITY GERMANY GMBH |
|
18W | Application withdrawn |
Effective date: 20220706 |