WO2006024247A1 - Method for detecting precipitation on a windscreen - Google Patents

Method for detecting precipitation on a windscreen Download PDF

Info

Publication number
WO2006024247A1
WO2006024247A1 PCT/DE2005/000676 DE2005000676W WO2006024247A1 WO 2006024247 A1 WO2006024247 A1 WO 2006024247A1 DE 2005000676 W DE2005000676 W DE 2005000676W WO 2006024247 A1 WO2006024247 A1 WO 2006024247A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
camera
vehicle
windscreen
target area
Prior art date
Application number
PCT/DE2005/000676
Other languages
German (de)
French (fr)
Inventor
Stefan Heinrich
Thomas Fechner
Michael Walter
Matthias Zobel
Original Assignee
Adc Automotive Distance Control Systems Gmbh
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Adc Automotive Distance Control Systems Gmbh filed Critical Adc Automotive Distance Control Systems Gmbh
Priority to DE112005001898T priority Critical patent/DE112005001898A5/en
Publication of WO2006024247A1 publication Critical patent/WO2006024247A1/en

Links

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60S—SERVICING, CLEANING, REPAIRING, SUPPORTING, LIFTING, OR MANOEUVRING OF VEHICLES, NOT OTHERWISE PROVIDED FOR
    • B60S1/00—Cleaning of vehicles
    • B60S1/02—Cleaning windscreens, windows or optical devices
    • B60S1/04—Wipers or the like, e.g. scrapers
    • B60S1/06—Wipers or the like, e.g. scrapers characterised by the drive
    • B60S1/08—Wipers or the like, e.g. scrapers characterised by the drive electrically driven
    • B60S1/0818—Wipers or the like, e.g. scrapers characterised by the drive electrically driven including control systems responsive to external conditions, e.g. by detection of moisture, dirt or the like
    • B60S1/0822—Wipers or the like, e.g. scrapers characterised by the drive electrically driven including control systems responsive to external conditions, e.g. by detection of moisture, dirt or the like characterized by the arrangement or type of detection means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60S—SERVICING, CLEANING, REPAIRING, SUPPORTING, LIFTING, OR MANOEUVRING OF VEHICLES, NOT OTHERWISE PROVIDED FOR
    • B60S1/00—Cleaning of vehicles
    • B60S1/02—Cleaning windscreens, windows or optical devices
    • B60S1/04—Wipers or the like, e.g. scrapers
    • B60S1/06—Wipers or the like, e.g. scrapers characterised by the drive
    • B60S1/08—Wipers or the like, e.g. scrapers characterised by the drive electrically driven
    • B60S1/0818—Wipers or the like, e.g. scrapers characterised by the drive electrically driven including control systems responsive to external conditions, e.g. by detection of moisture, dirt or the like
    • B60S1/0822—Wipers or the like, e.g. scrapers characterised by the drive electrically driven including control systems responsive to external conditions, e.g. by detection of moisture, dirt or the like characterized by the arrangement or type of detection means
    • B60S1/0833—Optical rain sensor
    • B60S1/0844—Optical rain sensor including a camera
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/42—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
    • G06V10/431—Frequency domain transformation; Autocorrelation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle

Definitions

  • the invention relates to a method for the detection of precipitation on a disk.
  • State of the art are methods with active illumination of the disc, as can be found for example in DE 197 40 364 A1 or DE 198 46 969 A1.
  • light is fed via one or more light sources in the windshield. Since raindrops influence the reflection behavior, the difference between the amount of light coupled in and reflected on the pane can be measured by means of a photosensor and thus rain detected.
  • the detection of dry / dusty precipitation, icing and condensation fitting on the inside of the disc is not possible with the previously known active method.
  • a separate sensor unit with active lighting is required.
  • the object of the invention is to introduce an alternative method for the detection of precipitation, which can preferably detect all types of precipitation.
  • camera sensors for environmental observation e.g. used for lane detection, object recognition, with a view in the direction of travel or in the return area.
  • a camera sensor for environmental observation is focused on a target area in the vicinity of the vehicle, that is outside the dimensions of the vehicle is arranged on the road in a range of typically several meters in front of the front of the vehicle.
  • the camera is currently not focused on the windshield and therefore can not detect precipitation particles directly, as it can not image them sharply.
  • cameras can not only be optical receiver chips on CMOS or PIN diode arrays, but also scanning systems which image the image as lines per line onto a receiver and thus match the image pixels in the swivel position of the scanner, as can be seen in FIG DE 10135107 A1 can be seen.
  • the invention is based on the fact that precipitation on the pane changes the image in its sharpness and local contrast. Raindrops as well as condensation fitting on the inside of the disk form scattering lenses, which create a soft focus effect in the picture. According to the invention, it has been recognized that even with a camera arrangement for environmental observation, which is not focused on the windshield of the vehicle, it is possible to extract information about rainfall on the windshield from the captured image or corresponding sections. For this purpose, image processing algorithms are applied, which are based on the basic effect of softening during precipitation.
  • the analysis in the local area by contrast measurement in particular in predetermined image sections with relatively constant background image or analysis in the time domain by differential image comparison as well as in the frequency domain by means of a Fourier transform (FT) of the entire image or predetermined image sections to the one-dimensional, ie line by line discrete FT.
  • FT Fourier transform
  • the evaluation of local contrast differences of adjacent pixels, with line-by-line detection, for example, limited to the two adjacent pixels or the 9 pixels around a middle pixel in a matrix arrangement already shows significant sensitivity.
  • the measured contrast depends on the background image and the size and position of the image section used.
  • the assessment of the local contrast differences of adjacent pixels averaged over the entire image in typical environmental situations in traffic has a lower dependence on the respective ambient and lighting situations than the purely local assessment.
  • the use of the wiper and the comparison of two images before and after the wiping process are particularly preferred. If the spectrum of the image is changed by a wiping process, it is wet on the glass. If the wipe does not change the spectrum of the image, there is no moisture on the glass.
  • a heating coil can also be arranged in the pane or on the pane in order to create comparative images.
  • S (u, v) is the 2D Fourier transform of the image.
  • f_Low, f_High are the lower and upper cutoff frequencies.
  • Delta is the threshold to be crossed and f is the local spatial frequency.
  • the horizontal / vertical frequency components (u, v) and thus the orientations of 2D structures are neglected. Only their absolute frequency f [1 / pixel pitch] and thus their resolution is considered.
  • the optimal thresholds fJLow and f_High can be determined from training sequences, but are relatively uncritical.
  • the threshold value delta This depends on the scene dynamics (e.g., desert, forest) as well as the camera parameters (e.g., the exposure control gain).
  • the Fourier spectrum (and thus the integral over the high spectral components) is directly proportional to the amplitude of the input signal.
  • the thresholds can be determined offline from sequences containing scenes with or without fogging.
  • the thresholds can be dynamically adapted to the current scene.
  • the amplitude of the Fourier spectrum and thus the energy of the high - frequency spectral components depends on the contrast of the image. (eg the amplification factor of the camera and thus the time of day (day / night)). However, this does not apply to the ratio of high to low frequencies. This results in a second, more robust classifier.
  • the classifier may deteriorate as details with high-frequency content may be lost (eg headlights, stars at night).
  • the ratio of the high and low frequency spectral components can be obtained by the quotient of the sums of the intensities of high (Hi) and low-pass filtered (Ti) pixels of a picture line.
  • the low-pass filtered pixels Oi can be e.g. through a recursive filter from the
  • T 1 Qr - O 1 H - (I - Or) - O 1-1
  • the high-pass filtered pixels can be calculated from the differences between the original pixels and the low-pass filtered pixels.
  • the above equation corresponds to a high pass. Summing only values with abs (O (i + 1) -O (i-1))> delta, the camera noise can be suppressed, resulting in a very effective classifier.
  • the invention thus provides a multiple benefit of a camera that is used for environmental detection and as a rain sensor.
  • the camera sensor is focused environment observation on a target area that is located outside the dimensions of the vehicle on the road, for example, in a range of 5 to 30 meters in front of the front of the vehicle.
  • the information obtained from the environmental observation may e.g. be used in driver assistance functions such as Lane Depature Warning or Nightvision.
  • the present invention proposed method for detecting rainfall on the windshield is based precisely on this focus property, and differs from that of a conventional rain sensor, where the focus of a camera is directed directly at the windshield.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Automation & Control Theory (AREA)
  • Mechanical Engineering (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)
  • Investigating Or Analysing Materials By Optical Means (AREA)

Abstract

The invention relates to a method for detecting precipitation, especially rain and fog, on a windscreen, which is not based on an active illumination of the windscreen or a measurement of the reflectivity, but on a camera using a plurality of adjacent pixels for detecting an image of a target area. Said target area is located in the surroundings of the vehicle, and the windscreen is thus represented in a blurred manner. In the event of a blurred representation, raindrops or fog on the windscreen produce a soft-focus effect in the image. The sharpness of the image and/or the difference in contrast of adjacent pixels is evaluated and the presence of precipitation is detected therefrom, the image is then preferably subjected to a two-dimensional Fourier transformation and the spatial frequencies are evaluated. A first image is recorded especially for a low contrast target area, a windscreen wiper passing over the windscreen in the visual region of the camera or a heating process is activated, and a second image is then recorded, and both images are evaluated according to changes.

Description

Verfahren zur Detektion von Niederschlag auf einer Scheibe Method for detecting precipitation on a pane
Die Erfindung betrifft ein Verfahren zur Detektion von Niederschlag auf einer Scheibe. Stand der Technik sind dabei Verfahren mit aktiver Beleuchtung der Scheibe, wie diese beispielsweise der DE 197 40 364 A1 oder DE 198 46 969 A1 zu entnehmen sind. Dabei wird über eine oder mehre Lichtquellen Licht in die Windschutzscheibe eingespeist. Da Regentropfen das Reflektionsverhalten beeinflussen, kann mittels eines Photosensors, die Differenz zwischen eingekoppelter- und reflektierter Lichtmenge an der Scheibe gemessen und damit Regen detektiert werden. Die Erkennung von trockenen/stäubförmigen Niederschlag, Vereisung sowie Kondensationsbeschlag auf der Innenseite der Scheibe ist mit den bisher bekannten aktiven Verfahren nicht möglich. Zudem ist eine separate Sensoreinheit mit einer aktiven Beleuchtung erforderlich. Aufgabe der Erfindung ist es, ein alternatives Verfahren zur Detektion von Niederschlag vorzustellen, der vorzugsweise alle Niederschlagsarten erfassen kann.The invention relates to a method for the detection of precipitation on a disk. State of the art are methods with active illumination of the disc, as can be found for example in DE 197 40 364 A1 or DE 198 46 969 A1. In this case, light is fed via one or more light sources in the windshield. Since raindrops influence the reflection behavior, the difference between the amount of light coupled in and reflected on the pane can be measured by means of a photosensor and thus rain detected. The detection of dry / dusty precipitation, icing and condensation fitting on the inside of the disc is not possible with the previously known active method. In addition, a separate sensor unit with active lighting is required. The object of the invention is to introduce an alternative method for the detection of precipitation, which can preferably detect all types of precipitation.
In Kraftfahrzeugen werden zudem zukünftig in vermehrtem Umfang Kamerasensoren zur Umfeldbeobachtung, z.B. für Fahrspurerkennung, Objekterkennung, mit Blick in Fahrrichtung oder in den Rückfahrtbereich eingesetzt. Diese sind häufig im Inneren des Fahrzeugs hinter der Windschutzscheibe angeordnet. Ein Kamerasensor zur Umfeldbeobachtung ist auf ein Zielgebiet in der Umgebung des Fahrzeugs fokussiert, also ausserhalb der Abmessungen des Fahrzeugs auf die Fahrbahn in einem Bereich von typischerweise einigen Metern vor der Front des Fahrzeugs angeordnet ist. Die Kamera ist also gerade nicht auf die Windschutzscheibe fokussiert und kann daher nicht direkt Niederschlagspartikel erkennen, da er diese nicht scharf abbilden kann.In motor vehicles, moreover, camera sensors for environmental observation, e.g. used for lane detection, object recognition, with a view in the direction of travel or in the return area. These are often located inside the vehicle behind the windshield. A camera sensor for environmental observation is focused on a target area in the vicinity of the vehicle, that is outside the dimensions of the vehicle is arranged on the road in a range of typically several meters in front of the front of the vehicle. The camera is currently not focused on the windshield and therefore can not detect precipitation particles directly, as it can not image them sharply.
Kameras können im Sinne dieser Anmeldung nicht nur optische Empfängerchips auf CMOS- oder PIN-Dioden-Arrays sein, sondern auch scannende Systeme, welche das Urngebungsbild als zeilenweise auf einen Empfänger abbilden und die Bildpixel somit Schwenkstellung des Scanners entsprechen, wie dies bspw. aus der DE 10135107 A1 zu entnehmen ist.For the purposes of this application, cameras can not only be optical receiver chips on CMOS or PIN diode arrays, but also scanning systems which image the image as lines per line onto a receiver and thus match the image pixels in the swivel position of the scanner, as can be seen in FIG DE 10135107 A1 can be seen.
Die Erfindung beruht darauf, dass Niederschlag auf der Scheibe das Bild in seiner Schärfe und lokalen Kontrasthaltigkeit verändern. Regentropfen wie auch Kondensationsbeschlag auf der Innenseite der Scheibe bilden Streulinsen, die im Bild einen Weichzeichner-Effekt erzeugen. Erfindungsgemäß wurde erkannt, dass auch mit einer Kameraanordnung zur Umfeldbeobachtung, die nicht auf die Windschutzscheibe des Fahrzeugs fokussiert ist, die Möglichkeit besteht, aus dem erfassten Bild oder entsprechenden Ausschnitten Informationen über Niederschlag auf der Scheibe zu extrahieren. Dazu werden Bildverarbeitungsalgorithmen angewendet, die sich an dem Grundeffekt der Weichzeichnung bei Niederschlag orientieren.The invention is based on the fact that precipitation on the pane changes the image in its sharpness and local contrast. Raindrops as well as condensation fitting on the inside of the disk form scattering lenses, which create a soft focus effect in the picture. According to the invention, it has been recognized that even with a camera arrangement for environmental observation, which is not focused on the windshield of the vehicle, it is possible to extract information about rainfall on the windshield from the captured image or corresponding sections. For this purpose, image processing algorithms are applied, which are based on the basic effect of softening during precipitation.
Hierfür bieten sich grundsätzlich die Analyse im Ortsbereich durch Kontrastmessung, insbesondere in vorgegebenen Bildausschnitten mit relativ konstantem Hintergrundbild oder die Analyse im Zeitbereich durch Differenzbildvergleich sowie im Frequenzbereich mittels einer Fourier-Transformation (FT) des gesamten Bildes oder vorgegebener Bildausschnitte bis hin zur eindimensionalen, also zeilenweisen diskreten FT. insbesondere zeigt die Bewertung lokaler Kontrastunterschiede benachbarter Pixel, bei zeilenweiser Erfassung bspw. beschränkt auf die jeweils zwei benachbarten Pixel oder die 9 Pixel um einen mittleren Pixel bei einer Matrixanordnung schon eine signifikante Empfindlichkeit. Der gemessene Kontrast ist jedoch abhängig vom Hintergrundbild und der Größe und Position des verwendeten Bildausschnitts. Jedoch hat sich gezeigt, dass gerade die Bewertung der lokalen Kontrastunterschiede benachbarter Pixel gemittelt über das gesamte Bild bei typischen Umgebungssituationen im Straßenverkehr eine geringere Abhängigkeit von den jeweiligen Umgebungs- und Beleuchtungssituationen hat als die rein lokale Bewertung.For this purpose, the analysis in the local area by contrast measurement, in particular in predetermined image sections with relatively constant background image or analysis in the time domain by differential image comparison as well as in the frequency domain by means of a Fourier transform (FT) of the entire image or predetermined image sections to the one-dimensional, ie line by line discrete FT. In particular, the evaluation of local contrast differences of adjacent pixels, with line-by-line detection, for example, limited to the two adjacent pixels or the 9 pixels around a middle pixel in a matrix arrangement already shows significant sensitivity. However, the measured contrast depends on the background image and the size and position of the image section used. However, it has been found that the assessment of the local contrast differences of adjacent pixels averaged over the entire image in typical environmental situations in traffic has a lower dependence on the respective ambient and lighting situations than the purely local assessment.
Urn Einflüsse aus der Szene weiter zu eliminieren ist der Einsatz des Wischers und der Vergleich zweier Bilder vor und nach dem Wischvorgang besonders bevorzugt. Wird durch einen Wischvorgang das Spektrum des Bildes stark verändert, so handelt es sich um Nässe auf der Scheibe. Wird durch den Wischvorgang das Spektrum des Bildes nicht verändert befindet sich auf der Scheibe keine Nässe. Alternativ oder ergänzend kann auch eine Heizspirale in der Scheibe bzw. an der Scheibe angeordnet werden, um Vergleichsbilder zu schaffen.In order to further eliminate influences from the scene, the use of the wiper and the comparison of two images before and after the wiping process are particularly preferred. If the spectrum of the image is changed by a wiping process, it is wet on the glass. If the wipe does not change the spectrum of the image, there is no moisture on the glass. Alternatively or additionally, a heating coil can also be arranged in the pane or on the pane in order to create comparative images.
Aus der Bildverarbeitung sind dabei zahlreiche Bewertungsmethoden für die Schärfe und lokale Kontraständerung bekannt. Durch zweidimensionale Fourier-Transformation und Auswertung der Ortsfrequenzen, insbesondere der hochfrequenten Anteile oder die Beschränkung auf Helligkeits- und/oder Farbunterschiede von Pixeln innerhalb eine Zone deutlich kleiner als die Gesamtgröße des Bildes sind auch die Einflüsse des Gesamtbildes des Zielgebiets und dessen Kontrastreichtum für die Auswertung deutlich geringer. Beschlag und Vereisung führen also auf einer nicht fokussierten Scheibe zu einer lokalen Reduktion von Kontrast und Schärfe eines Bildes. Dies lässt sich im Frequenzbereich durch die Dämpfung hoher Frequenzanteile nachweisen. Hierfür seien nochmals beispielhaft einige geeignete Klassifikationsmöglichkeiten aufgeführt:Numerous evaluation methods for the sharpness and local contrast change are known from image processing. By two-dimensional Fourier transformation and evaluation of the spatial frequencies, in particular the high-frequency components or the restriction to brightness and / or color differences of pixels within a zone significantly smaller than the total size of the image are also the influences of the overall image of the target area and its contrast richness for the evaluation much lower. Fogging and icing thus lead to a local reduction of the contrast and sharpness of an image on an unfocused glass. This can be demonstrated in the frequency domain by the attenuation of high frequency components. For this purpose, once again some suitable classification options are listed:
1 ) Integriert man die hochfrequenten Spektralanteile und vergleicht sie mit einem Schwellwert, erhält man den folgenden Klassifikator.1) If one integrates the high-frequency spectral components and compares them with a threshold value, one obtains the following classifier.
f - Hιghf - Hgh
1/ΔF - jS(f)df < A f = Ju2 +v2 f -Low1 / ΔF - jS (f) df <A f = Ju 2 + v 2 f -Low
S(u,v) ist die 2D Fourier - Transformierte des Bildes. f_Low, f_High sind die untere und obere Grenzfrequenz. Delta ist der zu überschreitende Schwellwert und f die lokale Ortsfrequenz. Die horizontalen/vertikalen Frequenzanteile (u, v) und damit die Orientierungen von 2D Strukturen werden dabei vernachlässigt. Es wird nur deren absolute Frequenz f [1 /Pixelabstand] und damit deren Auflösung betrachtet.S (u, v) is the 2D Fourier transform of the image. f_Low, f_High are the lower and upper cutoff frequencies. Delta is the threshold to be crossed and f is the local spatial frequency. The horizontal / vertical frequency components (u, v) and thus the orientations of 2D structures are neglected. Only their absolute frequency f [1 / pixel pitch] and thus their resolution is considered.
Die optimalen Schwellen fJLow und f_High lassen sich aus Trainingssequenzen ermitteln, sind jedoch relativ unkritisch.The optimal thresholds fJLow and f_High can be determined from training sequences, but are relatively uncritical.
Ähnliches gilt für den Schwellwert Delta. Dieser ist von der Szenendynamik (z.B. Wüste, Wald) sowie den Kameraparametern abhängig (z.B. den Verstärkungsfaktoren für die Belichtungssteuerung). Das Fourier-Spektrum (und damit das Integral über die hohen Spektralanteile) ist direkt proportional zu Amplitude des Eingangssignals. Die Schwellen lassen sich aber offline aus Sequenzen, die Szenen mit und ohne Beschlag enthalten, ermitteln.The same applies to the threshold value delta. This depends on the scene dynamics (e.g., desert, forest) as well as the camera parameters (e.g., the exposure control gain). The Fourier spectrum (and thus the integral over the high spectral components) is directly proportional to the amplitude of the input signal. However, the thresholds can be determined offline from sequences containing scenes with or without fogging.
Befindet sich das System in einem Trackingmode (und damit in einem beschlagsfreien Zustand) lassen sich die Schwellen dynamisch an die momentane Szene anpassen. Während des Trackingmodes wird ein Mittelwert für die hochfrequenten Spektralanteile berechnet. Ist der gesetzte Schwellwert größer als der im Trackingmode (und damit im beschlagsfreiem Zustand) ermittelte Wert, lässt sich dieser reduzieren (z.B. neuer Schwellwert = (minimal zulässiger Schwellwert + Mittelwert )/2). 2) Die Amplitude des Fourier - Spektrums und damit die Energie der hochfrequenten Spektralanteile ist vom Kontrast des Bildes abhängig. (z.B. vom Verstärkungsfaktor der Kamera und damit der Tageszeit (Tag/Nacht)). Dies gilt jedoch nicht für das Verhältnis der hohen zu den tiefen Frequenzen. Daraus folgt ein zweiter, robusterer Klassifikator.If the system is in a tracking mode (and thus in a fog-free state), the thresholds can be dynamically adapted to the current scene. During the tracking mode, an average value for the high-frequency spectral components is calculated. If the set threshold value is greater than the value determined in the tracking mode (and thus in the fog-free state), this value can be reduced (eg new threshold value = (minimum permissible threshold value + mean value) / 2). 2) The amplitude of the Fourier spectrum and thus the energy of the high - frequency spectral components depends on the contrast of the image. (eg the amplification factor of the camera and thus the time of day (day / night)). However, this does not apply to the ratio of high to low frequencies. This results in a second, more robust classifier.
ΔΔ
Figure imgf000005_0001
mit dem 2D Fourier Spektrum S(u,v), der Schwelle Delta, der unteren, mittleren und oberen Grenzfrequenz f_Low, f_Medium und f_High.
Figure imgf000005_0001
with the 2D Fourier spectrum S (u, v), the threshold delta, the lower, middle and upper limit frequencies f_Low, f_Medium and f_High.
Steht das gesamte Bild (zeilenweise Bildabtastung) und damit die 2D Fourier- Transformierte S(u, v) nicht zur Verfügung lässt sich diese durch Addition der 1 D Fourier- Transformierten der einzelnen Zeilen nachbilden. Stehen nicht alle Bildzeilen zur Verfügung, kann es zu einer Verschlechterung des Klassifikators kommen, da Details mit hochfrequenten Anteilen verloren gehen können, (z.b. Scheinwerfer, Sterne bei Nacht.)If the entire image (line by line image scanning) and thus the 2D Fourier transform S (u, v) are not available, they can be simulated by adding the 1 D Fourier transforms of the individual lines. If not all image lines are available, the classifier may deteriorate as details with high-frequency content may be lost (eg headlights, stars at night).
3) Um Rechenzeit zu sparen lässt sich das Verhältnis der hoch- und niederfrequenten Spektralanteile durch den Quotienten aus den Summen der Intensitäten aus hoch- (Hi) und tiefpassgefilterten (Ti) Bildpunkten einer Bildzeile gewinnen.3) In order to save computing time, the ratio of the high and low frequency spectral components can be obtained by the quotient of the sums of the intensities of high (Hi) and low-pass filtered (Ti) pixels of a picture line.
Die tiefpassgefilterten Bildpunkte Oi lassen sich z.B. durch ein rekursives Filter aus denThe low-pass filtered pixels Oi can be e.g. through a recursive filter from the
NN
-τr < Δ-τr <Δ
original Bildpunkten Oi bestimmen. T1 = Qr- O1 H- (I - Or) - O1-1 original pixels Oi determine. T 1 = Qr - O 1 H - (I - Or) - O 1-1
Die hochpassgefilterten Bildpunkte lassen sich aus den Differenzen zwischen den Originalbildpunkten und den tiefpassgefilterten Bildpunkten berechnen.The high-pass filtered pixels can be calculated from the differences between the original pixels and the low-pass filtered pixels.
H = O -T 4) Einer der einfachsten Klassifikatoren ergibt sich, falls man nur die hochfrequenten Bildanteile betrachtet.H = O -T 4) One of the simplest classifiers results if one considers only the high-frequency image components.
Figure imgf000006_0001
Figure imgf000006_0001
Die obige Gleichung entspricht einem Hochpass. Summiert man nur Werte mit abs(O(i+1)-O(i-1))>delta, lässt sich das Kamerarauschen unterdrücken und es ergibt sich ein äußerst effektiver Klassifikator. (Anmerkung: Die obige Formel entspricht einer nicht normierten Kontrastmessung über die Bildpunkte einer ganzen Zeile. (Es gilt für den Kontrast: K=(lmax-lmin)/(lmax+lmin). Sind nur zwei Werte vorhanden erhält man K=(H- I2)/(I1 +I2) oder [l(i)-l(i+1)]/[ l(i)+l(i+1)])).The above equation corresponds to a high pass. Summing only values with abs (O (i + 1) -O (i-1))> delta, the camera noise can be suppressed, resulting in a very effective classifier. (Note: The above formula corresponds to a non-normalized contrast measurement over the pixels of a whole line. (It applies to the contrast: K = (lmax-lmin) / (lmax + lmin) If only two values are available, K = (H - I2) / (I1 + I2) or [l (i) -l (i + 1)] / [l (i) + l (i + 1)])).
Die Erfindung bietet somit einen Mehrfachnutzen einer Kamera, die zur Umgebungserfassung und als Regensensor verwendet wird. Der Kamerasensor ist dabei Umfeldbeobachtung auf ein Zielgebiet fokussiert, dass außerhalb der Abmessungen des Fahrzeugs auf die Fahrbahn beispielsweise in einem Bereich von 5 bis 30 Metern vor der Front des Fahrzeugs angeordnet ist. Die aus der Umfeldbeobachtung gewonnenen Informationen können z.B. in Fahrerassistenzfunktionen wie Lane Depature Warning oder Nightvision eingesetzt werden. Das hier erfindungsgemäß vorgestellte Verfahren zur Erkennung von Niederschlägen auf der Windschutzscheibe beruht genau auf dieser Fokuseigenschaft, und unterscheidet sich von dem eines konventionellen Regensensors, wo der Fokus einer Kamera direkt auf die Windschutzscheibe gerichtet ist. The invention thus provides a multiple benefit of a camera that is used for environmental detection and as a rain sensor. The camera sensor is focused environment observation on a target area that is located outside the dimensions of the vehicle on the road, for example, in a range of 5 to 30 meters in front of the front of the vehicle. The information obtained from the environmental observation may e.g. be used in driver assistance functions such as Lane Depature Warning or Nightvision. The present invention proposed method for detecting rainfall on the windshield is based precisely on this focus property, and differs from that of a conventional rain sensor, where the focus of a camera is directed directly at the windshield.

Claims

Patentansprücheclaims
1 ) Verfahren zur Detektion von Niederschlag auf einer Scheibe, dadurch gekenn¬ zeichnet, daß a) eine Kamera mit einer Mehrzahl benachbarter Pixel zur Erfassung eines Bildes eines Zielgebiets verwendet wird, wobei das Zielgebiet von der Kamera deutlich weiter entfernt ist als die Scheibe und die Kamera auf das Zielgebiet in der Umgebung des Fahrzeugs fokussiert ist und somit die Scheibe unscharf abbildet, b) und die Schärfe und/oder Kontrastunterschiede zumindest von Teilbereichen des Bildes bewertet und daraus auf ein Vorhandensein von Niederschlag auf der Scheibe geschlossen wird.1) A method for detecting precipitation on a disc, gekenn¬ characterized in that a) a camera is used with a plurality of adjacent pixels for capturing an image of a target area, the target area of the camera is significantly farther away than the disc and the Camera is focused on the target area in the vicinity of the vehicle and thus images the disc out of focus, b) and evaluates the sharpness and / or contrast differences of at least portions of the image and it is concluded that there is a precipitate on the disc.
2) Verfahren nach Anspruch 1 , dadurch gekennzeichnet, daß das Bild einer Fourier- Transformation unterzogen und die Ortsfrequenzen ausgewertet werden.2) Method according to claim 1, characterized in that the image is subjected to a Fourier transformation and the spatial frequencies are evaluated.
3) Verfahren nach Anspruch 1 , dadurch gekennzeichnet, daß die Helligkeits- und/oder Farbunterschiede von Pixeln innerhalb einer Zone deutlich kleiner als die Gesamtgröße des Bildes bewertet werden.3) Method according to claim 1, characterized in that the brightness and / or color differences of pixels within a zone are evaluated significantly smaller than the total size of the image.
4) Verfahren nach Anspruch 3, dadurch gekennzeichnet, daß die4) Method according to claim 3, characterized in that the
Helligkeitsunterschiede unmittelbar benachbarter Pixel erfasst und über das Gesamtbild summiert und bewertet werden 5) Verfahren nach einem der vorangehenden Ansprüche, dadurch gekennzeichnet, a) daß ein erstes Bild aufgenommen wird, b) nachfolgend ein die Scheibe im Sichtbereich der Kamera überstreichender Scheibenwischer aktiviert wird c) und danach ein zweites Bild aufgenommen und beide Bilder auf Änderungen hin bewertet werden.Brightness differences of immediately adjacent pixels are captured and summed over the overall picture and evaluated 5) Method according to one of the preceding claims, characterized in that a) that a first image is taken, b) subsequently activates a windshield wiper sweeping the disk in the field of view of the camera c) and then taking a second image and evaluating both images for changes become.
6) Verfahren nach Anspruch 5, dadurch gekennzeichnet, daß a) ein erstes Bild aufgenommen und mit Vorgabewerten für die Schärfe des Bildes und/oder Kontrastunterschiede benachbarter Pixel verglichen wird und b) nur bei Unterschreitung von Mindestwerten der Schärfe und/oder Kontrastunterschieden der Scheibenwischer aktiviert wird.6) Method according to claim 5, characterized in that a) a first image is taken and compared with default values for the sharpness of the image and / or contrast differences of adjacent pixels and b) activated only if the minimum values of the sharpness and / or contrast differences of the windshield wipers becomes.
7) Verwendung einer Kamera zur Erfassung der Umgebung eines Fahrzeugs durch eine Scheibe des Fahrzeugs hindurch zur Detektion von Regen gemäß dem Verfahrens nach einem der vorangehenden Ansprüche7) Use of a camera for detecting the environment of a vehicle through a window of the vehicle for the detection of rain according to the method of any one of the preceding claims
8) Kamera zur Erfassung der Umgebung eines Fahrzeugs durch eine Scheibe des Fahrzeugs hindurch, welche einen regenabhängiges Signal gemäß dem Verfahrens nach einem der vorangehenden Ansprüche zur Steuerung der Scheibenwischer erzeugt.8) A camera for detecting the surroundings of a vehicle through a window of the vehicle, which generates a rain-dependent signal according to the method of one of the preceding claims for controlling the windshield wipers.
9) Kraftfahrzeug mit einer Kamera und einer Auswerteeinheit zur Durchführung eines Verfahrens nach einem der vorangehenden Ansprüche 9) Motor vehicle with a camera and an evaluation unit for carrying out a method according to one of the preceding claims
PCT/DE2005/000676 2004-09-03 2005-04-14 Method for detecting precipitation on a windscreen WO2006024247A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
DE112005001898T DE112005001898A5 (en) 2004-09-03 2005-04-14 Method for detecting precipitation on a pane

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
DE102004042597 2004-09-03
DE102004042597.3 2004-09-03
DE102005004220 2005-01-28
DE102005004220.1 2005-01-28

Publications (1)

Publication Number Publication Date
WO2006024247A1 true WO2006024247A1 (en) 2006-03-09

Family

ID=34972116

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/DE2005/000676 WO2006024247A1 (en) 2004-09-03 2005-04-14 Method for detecting precipitation on a windscreen

Country Status (2)

Country Link
DE (1) DE112005001898A5 (en)
WO (1) WO2006024247A1 (en)

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR2901218A1 (en) * 2006-05-22 2007-11-23 Valeo Vision Sa METHOD FOR DETECTING RAIN ON A SPRINKLER
WO2010038223A1 (en) * 2008-10-01 2010-04-08 Hi-Key Limited A method and a system for detecting the presence of an impediment on a lens of an image capture device to light passing through the lens of an image capture device
WO2011127895A1 (en) 2010-04-16 2011-10-20 Conti Temic Microelectronic Gmbh Method and device for assisting a driver while driving a vehicle by detecting weather-related visibility limitations
WO2013163991A1 (en) * 2012-05-03 2013-11-07 Conti Temic Microelectronic Gmbh Detecting rain droplets on a glass pane by means of a camera and illumination
WO2014032933A1 (en) * 2012-08-29 2014-03-06 Bayerische Motoren Werke Aktiengesellschaft Method for operating a vehicle
US8774546B2 (en) 2012-03-08 2014-07-08 Industrial Technology Research Institute Method and apparatus for single-image-based rain streak removal
US9224052B2 (en) 2012-12-19 2015-12-29 Industrial Technology Research Institute Method for in-image periodic noise pixel inpainting
US9335264B2 (en) 2010-11-30 2016-05-10 Conti Temic Microelectronic Gmbh Detection of raindrops on a pane by means of a camera and lighting
US9508015B2 (en) 2011-12-05 2016-11-29 Continental Teves Ag & Co. Ohg Method for evaluating image data of a vehicle camera taking into account information about rain
DE102015218500A1 (en) 2015-09-25 2017-03-30 Conti Temic Microelectronic Gmbh Illumination and camera-based detection of raindrops on a pane
EP3336748A1 (en) 2016-12-13 2018-06-20 Continental Automotive GmbH Detection of heavy rain by temporally measuring blur from edges
EP3336747A1 (en) 2016-12-13 2018-06-20 Continental Automotive GmbH Rain detection with a camera
US10106126B2 (en) 2014-05-15 2018-10-23 Conti Temic Microelectronic Gmbh Apparatus and method for detecting precipitation for a motor vehicle
US10137842B2 (en) 2011-06-03 2018-11-27 Conti Temic Microelectronic Gmbh Camera system for a vehicle
CN114714865A (en) * 2022-04-12 2022-07-08 重庆金康赛力斯新能源汽车设计院有限公司 Demisting system and demisting method for automobile glass

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH10111249A (en) * 1996-10-09 1998-04-28 Fujitsu Ten Ltd Detecting equipment of visibility state
DE19740364A1 (en) 1997-09-13 1999-03-25 Bosch Gmbh Robert Device for operating an optoelectronic rain sensor
DE19846969A1 (en) 1998-04-08 1999-10-14 Bosch Gmbh Robert Rain sensor, especially for motor vehicles
DE10135107A1 (en) 2001-07-19 2003-02-06 Adc Automotive Dist Control Method for optically scanning a scene
DE10201522A1 (en) * 2002-01-17 2003-07-31 Bosch Gmbh Robert Method and device for detecting visual impairments in image sensor systems
WO2003097420A1 (en) * 2002-05-18 2003-11-27 Elmos Semiconductor Ag Rain sensor
EP1418089A2 (en) * 2002-11-05 2004-05-12 C.R.F. Società Consortile per Azioni Multifunctional integrated optical system with CMOS or CCD technology matrix
DE10303047A1 (en) * 2003-01-24 2004-08-05 Daimlerchrysler Ag Method and device for improving the visibility and for determining the weather situation
DE10303046A1 (en) * 2003-01-24 2004-10-21 Daimlerchrysler Ag Quantitative estimation of visibility in motor vehicle, by using e.g. measurement of sharpness or contrast of image obtained from intensity differences of adjacent pixels

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH10111249A (en) * 1996-10-09 1998-04-28 Fujitsu Ten Ltd Detecting equipment of visibility state
DE19740364A1 (en) 1997-09-13 1999-03-25 Bosch Gmbh Robert Device for operating an optoelectronic rain sensor
DE19846969A1 (en) 1998-04-08 1999-10-14 Bosch Gmbh Robert Rain sensor, especially for motor vehicles
DE10135107A1 (en) 2001-07-19 2003-02-06 Adc Automotive Dist Control Method for optically scanning a scene
DE10201522A1 (en) * 2002-01-17 2003-07-31 Bosch Gmbh Robert Method and device for detecting visual impairments in image sensor systems
WO2003097420A1 (en) * 2002-05-18 2003-11-27 Elmos Semiconductor Ag Rain sensor
EP1418089A2 (en) * 2002-11-05 2004-05-12 C.R.F. Società Consortile per Azioni Multifunctional integrated optical system with CMOS or CCD technology matrix
DE10303047A1 (en) * 2003-01-24 2004-08-05 Daimlerchrysler Ag Method and device for improving the visibility and for determining the weather situation
DE10303046A1 (en) * 2003-01-24 2004-10-21 Daimlerchrysler Ag Quantitative estimation of visibility in motor vehicle, by using e.g. measurement of sharpness or contrast of image obtained from intensity differences of adjacent pixels

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
PATENT ABSTRACTS OF JAPAN vol. 1998, no. 09 31 July 1998 (1998-07-31) *

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR2901218A1 (en) * 2006-05-22 2007-11-23 Valeo Vision Sa METHOD FOR DETECTING RAIN ON A SPRINKLER
EP1860426A3 (en) * 2006-05-22 2010-06-02 Valeo Vision Method of detecting rain on a windscreen
US7860275B2 (en) 2006-05-22 2010-12-28 Valeo Vision Method for detecting rain on a windscreen
WO2010038223A1 (en) * 2008-10-01 2010-04-08 Hi-Key Limited A method and a system for detecting the presence of an impediment on a lens of an image capture device to light passing through the lens of an image capture device
US9128813B2 (en) 2010-04-16 2015-09-08 Conti Temic Microelectronic Gmbh Method and device for assisting a driver while driving a vehicle by detecting weather-related visibility limitations
DE102010015214A1 (en) 2010-04-16 2011-10-20 Conti Temic Microelectronic Gmbh Method and device for driver assistance when driving a vehicle by detecting weather-related visual restrictions
WO2011127895A1 (en) 2010-04-16 2011-10-20 Conti Temic Microelectronic Gmbh Method and device for assisting a driver while driving a vehicle by detecting weather-related visibility limitations
US9335264B2 (en) 2010-11-30 2016-05-10 Conti Temic Microelectronic Gmbh Detection of raindrops on a pane by means of a camera and lighting
US10137842B2 (en) 2011-06-03 2018-11-27 Conti Temic Microelectronic Gmbh Camera system for a vehicle
US9508015B2 (en) 2011-12-05 2016-11-29 Continental Teves Ag & Co. Ohg Method for evaluating image data of a vehicle camera taking into account information about rain
US8774546B2 (en) 2012-03-08 2014-07-08 Industrial Technology Research Institute Method and apparatus for single-image-based rain streak removal
WO2013163991A1 (en) * 2012-05-03 2013-11-07 Conti Temic Microelectronic Gmbh Detecting rain droplets on a glass pane by means of a camera and illumination
US9702818B2 (en) 2012-05-03 2017-07-11 Conti Temic Microelectronic Gmbh Detection of raindrops on a windowpane by means of camera and light
WO2014032933A1 (en) * 2012-08-29 2014-03-06 Bayerische Motoren Werke Aktiengesellschaft Method for operating a vehicle
US9224052B2 (en) 2012-12-19 2015-12-29 Industrial Technology Research Institute Method for in-image periodic noise pixel inpainting
US10106126B2 (en) 2014-05-15 2018-10-23 Conti Temic Microelectronic Gmbh Apparatus and method for detecting precipitation for a motor vehicle
DE102015218500A1 (en) 2015-09-25 2017-03-30 Conti Temic Microelectronic Gmbh Illumination and camera-based detection of raindrops on a pane
EP3336748A1 (en) 2016-12-13 2018-06-20 Continental Automotive GmbH Detection of heavy rain by temporally measuring blur from edges
EP3336747A1 (en) 2016-12-13 2018-06-20 Continental Automotive GmbH Rain detection with a camera
CN114714865A (en) * 2022-04-12 2022-07-08 重庆金康赛力斯新能源汽车设计院有限公司 Demisting system and demisting method for automobile glass
CN114714865B (en) * 2022-04-12 2024-05-14 重庆赛力斯新能源汽车设计院有限公司 Defogging system and defogging method for automobile glass

Also Published As

Publication number Publication date
DE112005001898A5 (en) 2007-08-09

Similar Documents

Publication Publication Date Title
WO2006024247A1 (en) Method for detecting precipitation on a windscreen
EP1506108B1 (en) Rain sensor
EP1027235B1 (en) Method and device for detecting objects on a windshield
DE69835175T2 (en) System for detecting moisture on the windshield of a vehicle
WO2019174682A1 (en) Method and device for detecting and evaluating roadway conditions and weather-related environmental influences
DE60214593T2 (en) STEREO ILLUSTRATION WITH MOISTURE SENSOR USING AN IMAGE SENSOR
DE112008001384B4 (en) Method for contamination detection in a TOF distance camera
EP2542881B1 (en) Method and apparatus for detecting mist by means of spectroscopy
WO2011127895A1 (en) Method and device for assisting a driver while driving a vehicle by detecting weather-related visibility limitations
DE102014209197B4 (en) Device and method for detecting precipitation for a motor vehicle
EP1470706A1 (en) Method and device for image processing, in addition to a night viewing system for motor vehicles
WO2012171834A2 (en) Method and device for detecting impairment of visibility through a pane
EP1468401A1 (en) Method and device for recognizing obstructions of view in image sensor systems
WO2005066684A1 (en) Night vision system for motor vehicles, comprising a partial optical filter
DE10303046A1 (en) Quantitative estimation of visibility in motor vehicle, by using e.g. measurement of sharpness or contrast of image obtained from intensity differences of adjacent pixels
EP2844529A1 (en) Detecting rain droplets on a glass pane by means of a camera and illumination
DE102011105074A1 (en) Method for determining visual range for vehicle, involves determining surrounding of vehicle by camera, where contrast dimensions are determined for object depicted in images, where visual range is determined from contrast dimensions
DE102013022076A1 (en) Method for determining a width of a target vehicle by means of a camera system of a motor vehicle, camera system and motor vehicle
DE102004010908B4 (en) Image pickup device and image recording method
EP1440856A2 (en) Method and apparatus for improving visibility and for determining the situation of the weather
DE10219788C1 (en) Method and device for measuring visibility with image sensor systems
WO2017041795A1 (en) Stereo camera device for monitoring the surroundings of a motor vehicle, motor vehicle with such a stereo camera device and method for detecting rain drops or deposits
DE102006004770B4 (en) Method for image-based recognition of vehicles in the vicinity of a road vehicle
DE102015218500A1 (en) Illumination and camera-based detection of raindrops on a pane
WO2012119627A1 (en) Method and device for detecting water on a vehicle windscreen

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KM KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SM SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): BW GH GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LT LU MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
WWE Wipo information: entry into national phase

Ref document number: 1120050018989

Country of ref document: DE

REF Corresponds to

Ref document number: 112005001898

Country of ref document: DE

Date of ref document: 20070809

Kind code of ref document: P

122 Ep: pct application non-entry in european phase