EP2720912A1

EP2720912A1 - Method and apparatus for identifying directional structures on a window of a vehicle

Info

Publication number: EP2720912A1
Application number: EP12721289.2A
Authority: EP
Inventors: Michael Huelsen; Annette Frederiksen; Stephan Simon; Roland Schmid; Werner Uhler; Jad Halimeh
Original assignee: Robert Bosch GmbH
Current assignee: Robert Bosch GmbH
Priority date: 2011-06-17
Filing date: 2012-05-15
Publication date: 2014-04-23
Also published as: CN103857567A; DE102012210116A1; WO2012171737A1; US9616851B2; US20140321701A1; CN103857567B

Abstract

The invention relates to a method for identifying directional structures on a window of a vehicle, which method comprises a step of carrying out (308) an evaluation of pixels of an image on the window, said pixels being arranged along an evaluation path, wherein a profile of the evaluation path is dependent on an expected orientation of the directional structures on the window. The method further comprises a step of identifying (310) a directional structure based on the evaluation.

Description

description

title

Method and device for detecting directional structures on a window of a vehicle

State of the art

The present invention relates to a method and a device for detecting directional structures on a window of a vehicle and to a corresponding computer program product.

Rain sensors are used in vehicles to detect vehicle windows contaminated with raindrops.

US 2004/0201483 A1 deals with an automatic control system of a vehicle. It can be used to activate a cleaning system of a windscreen of the vehicle.

In order to detect an aging condition of a windshield wiper, the printing of a color point on the wiper blade takes place. The color changes with the state of aging. A useful measure of the particularly relevant real wear of the wiper edge is thus not given.

To produce an optical image of a pane, an arrangement such as the front camera arrangement described in DE 10 2009 000 004 A1 can be used. Such a camera can be used for the recognition of a disk state, in particular with regard to the question of whether it is wetted or contaminated by rain, as described in DE 10 2009 000 003 A1. DE 102 54 684 A1 teaches a system for detecting the state or degree of wear of wiper blades of a windshield wiper on the basis of recognized streaks in an image taken with a camera. The writing, however, owes the technical doctrine, as the streaks are recognized in the image.

DISCLOSURE OF THE INVENTION Against this background, the present invention proposes a method and a device for detecting directional structures on a windshield of a vehicle, and finally a corresponding computer program product according to the independent patent claims. Advantageous embodiments emerge from the respective subclaims and the following description.

Directed structures, such as streaks, can be more easily detected on a window of a vehicle by taking into account a typical course of the directed structures on the pane. In particular, the course of the directed structures in an evaluation of, for example, a

Camera recorded image of the disc are taken into account.

According to one embodiment of the invention, the detection and quantification of stripe-like water streaks on a pane, in particular on a motor vehicle front window, is made possible. Cause of the emergence of such

Water streaking is usually the windshield wiper. The water streaks can lead to a visual impairment when looking through the glass. To capture the water streaks is a camera assembly that detects at least a portion of the disk surface.

The quantification of stripe-like water streaks may be useful for automatic control of the windshield wiper.

The stripe-like water streaks occur in the ordinary wiping process, because the windshield wiper is not able, the entire amount of water with a wiping cycle, ie a trace and a return flow back to the rest position to move completely from the disc. There remains a small amount of residual water on the disk in the form of small water droplets or fine streaks of water. These usually evaporate within a short time, eg. Eg 300ms. However, this time depends greatly on the amount of water, the ratio of water surface to water volume, the temperature, the humidity, the wind speed on the disk surface and the water contamination, eg. B. by dissolved road salt in winter, and possibly from other factors.

The longer the amount of residual water in the form of streaks remains after the wiping process on the disc, the more it is perceived disturbing.

Even if a classic rain sensor or a known camera-based rain sensor can still detect the residual amount of water by measurement, it is not advisable to restart the windshield wiper cycle for the remaining amount, since then again a new residual amount remain and the disturbing streaks would form again.

Therefore it makes sense to record and quantify the streaks separately. With the ability to differentiate the streaks from newly encountered raindrops, the wiper control can be designed to be less affected by streaks and more responsive to raindrops.

The approach of the invention can be applied to other wheels that are mechanically wiped, z. As in ships, aircraft, an airport tower or protective screens for surveillance cameras.

According to one embodiment of the invention, a quick and objective assessment of the condition of the wiper blade, e.g. B. in the context of vehicle inspection, allows. In this case, an objective and reproducible measure of the wear state of the wiper is provided. In order to obtain a true-to-life statement, the state of wear is automatically determined during the wiping operation. The vehicle may then prompt the user to change the wiper. Alternatively, the information can be stored in the error memory and retrieved at the next inspection. A further embodiment of the invention is concerned with a slice state, wiper state and visual disturbance determination with a driver assistance camera.

This can be understood as an automatic and objectified determination of the state of wear of the windscreen wiper or windscreen wipers and, in particular, of the wiper blades. This can help to increase traffic safety. As the windshield wiper wear process is very slow, many drivers struggle to determine the correct time to change the wiper. Such drivers will presumably gratefully accept a correct and objective indication of a necessary wiper blade change, especially if the warning occurs for the first time at a time when wear is particularly detrimental to the driver, e.g. when driving in the rain at night. Garages can rely on automatic wear detection when using e.g. as part of the inspection, the wiper blade change to declare necessary.

Furthermore, this can be understood as the automatic and objectified determination of the wear state of the windshield. Here, the same statements on traffic safety and the advantages of objectivity apply as in the determination of wear. Particularly in countries with a higher proportion of whirled up sand or dust, a strong wear of the windscreen can be noticed. This wear is amplified by the use of the windshield wiper.

Another embodiment of the invention addresses the problem of wiper blade condition assessment both in the case where wiping marks are visually apparent and in the case where the wiping streaks are due to either large darkness in the image or a small number of distinguishing features exhibited by the wiping streaks , are not visible to a DAS driver (DAS) sensor assistance system (DAS).

This also includes the problem of a windshield condition assessment in operation, based on detecting the presence of glass scratches, regardless of whether they are visually strong enough for the DAS Vision camera or not. In one embodiment, a system is based on a thorough understanding of the photometric properties of wiping strips on vehicle windshields and is directed to photometric effects caused by wiping strips. This is a significant improvement over the state of the art

Technique in the sense that even if a wiper strip is not visible in an image whose photometric effects are still visible and, due to their uniqueness, indicate the presence of such a wiper strip. For example, a wiper strip may not be visible in an image, either because it is too thin in daylight or because the image was taken at night when a wiper strip, even if it is very thick, can not be detected.

Similarly, the detection of the presence of glass scratches is made possible by directly detecting their optical effects, which are mainly diffraction patterns aligned perpendicular to the direction of movement of a wiping wiper blade. It should be noted that glass scratches caused by rubbing quartz particles in dusty air through the wiper blades over a windshield for an extended period of time are not visible in a DAS camera because their thickness is in the microscopic range and because of them their proximity to the DAS camera, which is generally focused on the infinity area, are out of focus. The characteristic diffraction patterns caused by these glass scratches, on the other hand, are reproduced well in DAS images and can thus indicate the presence of glass scratches.

Another improvement is that no focusing on the windshield is required. Usually, wiper strips provide weak edges unless the camera of the driver assistance system (DAS) focuses on the windshield. However, this would lead to further confusion due to internal edges within the wiper strip, which in turn are caused by breakage of the scene by the wiper strip. One embodiment of the invention, while maintaining focus on infinity or infinity, as in most DAS applications, deals with detecting the photometric effects of wiping streaks and glass scratches, which are visible regardless of whether the Wiping stripes and glass scratches that cause them, even in the picture are visible or not. As a result, such an approach is very suitable for DAS applications and enables reliable and robust detection of wiping marks and glass scratches on the vehicle windshield, thereby enabling assessment of wiper blade or windshield conditions.

Thus, the diffraction and refraction properties of wiping strips on a vehicle windshield are exploited to detect such wiping strips and also to obtain information about what condition a wiper blade is in. The diffraction pattern caused by a wiper strip or a group of wiping strips, as is the case with a wiper blade after wiping, is characteristic and directly dependent on the average thickness of each individual such strip and the spacing therebetween. This allows an accurate determination of how worn a wiper is based on the assumption that an immaculate wiper causes an infinite variety of wiping strips having an average thickness approaching zero, while a poor wiper provides thicker wiper strips having a greater spacing between them - see it caused.

Also, a detection of the diffraction figure caused by glass scratches is built up to detect such scratches and to make a corresponding judgment on the condition of the vehicle windshield. The more scratches are on a vehicle windshield, the stronger and more pronounced are the resulting diffraction figures. This suggests the conclusion that a vehicle windshield wears due to extensive driving in an area having a high density of dust in the air.

The present invention provides a method for recognizing directional structures on a windshield of a vehicle comprising the following steps:

Performing an evaluation of image pixels of an image of the slice arranged along an evaluation path, a course of the evaluation path being deviated from. is dependent on an expected orientation of the directed structures on the disk; and

Recognize a streak based on the rating.

The vehicle may be a motor vehicle and the windshield may be a windscreen or a rear window of the vehicle. In this case, only a partial area of an entire pane can be understood by pane. In particular, it can be a disk whose surface is swept by a wiper device for wiping the disk and can thus be cleaned. A directional structure may be understood to mean an impurity located on the surface of the disk which has an elongate shape. The directional structure may be a streak and may be water, for example, or hydrous. A streak can not be amorphous and have a long-range effect. Under the image can a

Recording taken by an optical detection device. Also, the image can be determined from one or more such images. For example, the image may have been recorded on a camera arranged in the interior of the vehicle near the pane with an image sensor. The image may be before the step of

Performing the evaluation of an image processing in order to strengthen for the determination of a directed structure relevant structures in the image and to reduce disturbing structures, which are assigned for example to the background. The image may be in digital form and have a plurality of pixels. Each pixel may include image information representing, for example, a brightness of an area of the disc associated with the pixel. By evaluating a pixel, it can be determined whether there is an impurity, for example in the form of moisture, on the surface of the disc in the region of the disc assigned to the pixel. Suitable evaluation algorithms or image recognition algorithms can be used for the evaluation.

The evaluation path may be a streak path, ie a course of a streak. The evaluation path can also define an evaluation direction. The evaluation path can be determined by a method which may be described by software as follows: First set an angle or a slope of a straight line, the angle or the slope is measured in the image by the choice of two points in the image. The two points are determined according to the wiping direction of the wiper. The wiping geometry is known. Since the wiper wipes on a curved line, the directed structures are also correct arcuate. They appear almost straight through the small image section of the camera on the windshield. That is, a fixed slope as a straight line is a possible approximation. Thus, in one image, the slope of the directed structures on one side of the image may be stronger than the slope of the directional structures on another side of the image. The line slope is approximated locally. The approximation may be refined by circular arc segments, polygons, or the like, rather than by straight lines.

In general, the evaluation path may be a trajectory on or parallel to which typically a directional structure extends. The evaluation path can represent a projection of a real evaluation path on the surface of the disc into the image. Thus, the pixels arranged along the evaluation path map those areas of the pane over which typically a directional structure extends, provided that a directional structure is present on the pane. By analyzing the scores of the pixels along the evaluation path, it can be determined whether or not there is a directed structure along the evaluation path. The method can be carried out for a plurality of parallel, approximately parallel, concentric or approximately concentric evaluation paths.

Thus, in the step of performing the evaluation, it may be determined whether a pixel arranged along the evaluation path represents an impurity of a portion of the disc represented by the pixel. A corresponding evaluation can be carried out for all or for selected pixels lying on the evaluation path.

According to one embodiment, a step of performing a further evaluation of pixels of the image of the slice arranged along a further path may be performed. In this case, a course of the further path can be approximately perpendicular to a course of the evaluation path. in the

Step of recognizing may be the directional structure based on the rating and the further evaluation. In this case, a strip-wise scanning can be performed. The two paths may be in a different direction, but not through the same point. The further path and the evaluation path thus do not run parallel to one another. For example, the further path and the evaluation path can be oriented orthogonally to one another.

In this way, an elongate portion of the disk over which a highly oriented, contiguous structure is likely to extend may be compared to another elongated portion of the disk over which, with a relatively much lower probability, a contiguous directed structure extends. As a result, a recognition accuracy can be significantly increased.

In this case, a further first pixel arranged along the evaluation path can be assigned to a further first pixel situated on the further path. In the step of determining, a comparison of the evaluation of the first pixel with the further evaluation of the further first pixel can be performed. The directional structure can be detected based on the comparison. It can be assigned to each pixel along the Auswertepfads a pixel along the wider path, so that a plurality of pixel pairs are formed, each comprising different pixels. For example, it can be determined by the comparison on which of the two paths a larger number of impurities are detected. If more impurities are detected on the evaluation path than on the further path, it can be concluded that the impurities on the evaluation path can be assigned to a directional structure.

According to one embodiment, the comparison of the pixels can be designed such that refraction and / or diffraction effects or such figures are recognized and analyzed in the image. Weak diffraction may indicate thick directional structures (e.g., wiping stripes), weak refraction may indicate thin directed structures. The refraction or diffraction can be analyzed depending on the evaluation path.

According to one embodiment, the directed structures may be u (fine) scratches or streaks. It can thus the disk state, also fine scratches, to be recognized. In particular, without focusing on the disc, scratches or streaks can be detected.

According to one embodiment, the extent, in particular the thickness, of the directed structures is determined.

If not a pure analysis in the direction of the evaluation path, or vice versa for determining / determining the evaluation path, the orientation of the directed structures can be determined.

Furthermore, the distance of the directed structures between each other can be determined, for example the distance of the striae stripes from one another. Likewise, differences may be extracted, for example, components of the streaks, e.g. the thickness of each drop of streak.

Thereby, a classification of the directed structures can be carried out, e.g. fine streaks, big streaks, scratches, solid contamination.

According to one embodiment, a course of the evaluation path is dependent on a wipe trajectory of a wiping edge of a wiper device for wiping the window. By wiping, for example, streaks are usually expected in the wiping direction. This knowledge can be used for the image analysis.

According to an embodiment, in a step of determining a state of the wiping edge may be determined based on information about the directional structure. The wiper device may be a windshield wiper and the wiper edge may be an edge of a wiper blade resting on the surface of the disk during operation. The directional structure information may include, for example, data about a length, width, consistency, or frequency of the directed structure. The information about the directed structure can be compared with predetermined comparison values. From this, it can be determined whether the directional structure has been caused by the wiper edge being in a poor condition and should therefore be renewed.

For this purpose, a subsequent evaluation of pixels arranged along the evaluation path of a subsequent image of the slice can also be carried out in a step of performing. In a step of the recognition In addition, a subsequent directed structure can be recognized based on the subsequent evaluation. In the step of determining, the state of the wiping edge may be further performed based on information about the subsequent directional structure. The subsequent image can image the same area of the disk as the original image, but at a later time. The two images may have been detected between two directly successive wiping operations of the wiper device. In this way, a temporal change of a recognized directional structure in the determination of the state of the wiping edge can be taken into account. This is advantageous because directed structures caused by a wiping edge in good condition typically disappear faster than directed structures caused by a wiping edge in a poor condition.

According to one embodiment, the method may include a step of analyzing the pixels of the image of the disk to detect fogging on an inside of the disk. Both the pixels located on the evaluation path and other pixels can be analyzed. The fitting may be moisture located on an inner surface of the disk, that is, a surface of the disk facing the interior of the vehicle. Depending on whether a fitting is detected, for example, a ventilation of the disc can be controlled.

The present invention further provides a method of detecting directional structures on a windshield of a vehicle comprising the steps of:

Performing an evaluation of pixels arranged along an evaluation path of an image of the slice, wherein a course of the evaluation path is dependent on an expected orientation of the directed structures on the slice;

Making a determination as to whether a pixel arranged along the evaluation path represents a light diffraction effect in a region of the disc represented by the pixel; and Detecting a directed structure based on the rating and the determination.

According to this embodiment, in the step of determining, it may be determined whether a pixel arranged along the evaluation path represents a light diffraction effect in an area of the disc represented by the pixel. The light diffraction effect may be a diffraction. Diffraction effects on the disk are often caused by directional structures. In particular, light diffraction effects are also caused by directed structures that are invisible per se. By taking account of light diffraction effects, the recognition accuracy with respect to directed structures can thus be improved. Diffraction occurs in thin streaks. In contrast, a refraction effect occurs with thick streaks, which act like cylindrical lenses.

The present invention further provides a device for detecting directional structures on a pane of a vehicle, having the following features: a feed-through device for carrying out an evaluation of pixels of an image of the pane arranged along an evaluation path, a course of the evaluation path being dependent on an expected orientation of the pane directed structures on the disc is; and a recognizer for recognizing a directional structure based on the evaluation.

By means of the device, the steps of the method according to the invention can be carried out or implemented in corresponding devices. Also by this embodiment of the invention in the form of a device, the object underlying the invention can be solved quickly and efficiently. In the present case, a device can be understood as meaning an electrical device which processes sensor signals and outputs control signals in dependence thereon. The device may have an interface, which may be formed in hardware and / or software. In a hardware-based training, for example, the interfaces can be part of a so-called system ASICs be, which includes a variety of functions of the device. However, it is also possible that the interfaces are their own integrated circuits or at least partially consist of discrete components. In a software training, the interfaces may be software modules that are present, for example, on a microcontroller in addition to other software modules.

A computer program product with program code which can be stored on a machine-readable carrier such as a semiconductor memory, a hard disk memory or an optical memory and is used to carry out the method according to one of the embodiments described above if the program is on a computer corresponding to a computer is also of advantage Device is running.

The invention will now be described by way of example with reference to the accompanying drawings. Show it:

Fig. 1 is a schematic representation of an embodiment of the present invention;

Figure 2 is a schematic representation of a disc of a vehicle.

Fig. 3 is a flowchart of an embodiment of the present invention;

4 shows a section of an image of a pane;

5 shows a detail of a detection image;

6 shows a section of a further detection image;

FIG. 7 shows a representation of a processing of a detection image; FIG.

8 shows an illustration of a further processing of a detection image;

9 shows a detail of a detection image; 10 shows a section of a further detection image;

Fig. 1 1 shows a section of a further detection image;

FIG. 12 shows a section of a further detection image; FIG.

FIG. 13 shows a section of a further detection image; FIG.

Fig. 14 is a graph showing a rain amount on a disk;

Fig. 15 is a graph showing a decay behavior of streaks;

Fig. 16 is a graph showing another decay behavior of streaks;

17 shows a representation of a processing of a detection image;

FIG. 18 shows a representation of a further processing of a detection image; FIG.

Fig. 19 is a schematic diagram of a system according to an embodiment of the present invention;

Fig. 20 is an image of a disk;

Fig. 21 is another image of a disk;

FIG. 22 shows another image of a pane; FIG.

Fig. 23 is another image of a disk;

Fig. 24 is an illustration of a disc with highlighted edges;

Fig. 25 is a schematic diagram of a system according to an embodiment of the present invention; FIG. 26 shows another image of a pane; FIG.

FIG. 27 shows another image of a pane; FIG. Fig. 28 is another image of a disk; FIG. 29 shows another image of a pane; FIG.

Fig. 30 is a schematic diagram of a system according to an embodiment of the present invention;

Fig. 31 is a schematic diagram of a system according to an embodiment of the present invention; and

FIG. 32 is a graph showing a decay behavior of light diffraction effects. FIG.

In the following description of preferred embodiments of the present invention, the same or similar reference numerals are used for the elements shown in the various figures and similarly acting, wherein a repeated description of these elements is omitted.

1 shows a schematic representation of a vehicle 100, according to an embodiment of the present invention. The vehicle 100 has a disk 102 on which a water droplet 104 is located. The vehicle has an image capture device 106, a passage device 108 and a recognition device 110. Furthermore, the vehicle 100 may have a lighting device 12 for illuminating the window 102. According to this exemplary embodiment, the lighting device 1 12 is arranged inside the vehicle 100 and designed to illuminate the drops 104 on the outside of the pane 102.

The image capture device 106 is designed to create an image of a portion of the slice 102 and to provide the image to the execution device 108 via an interface. The execution device 108 is designed to evaluate the image. These can be individual pixels of the image taken alone or evaluated in relation to other pixels of the image. The execution device 108 is designed to provide a result of the evaluation or the evaluations to the recognition device 110 via a further interface. The recognition device 110 is designed to detect a streak on the disc 102 based on the information provided by the passage device 108. It can be determined, for example, whether it is the drop 104 is a raindrop, or above the drop 104 is part of a streak.

According to one exemplary embodiment, part of the pane 102 can be detected by the camera arrangement 106 in a camera image or a camera image detail. Drops 104 on disk 102, even very small drops 104, are visible in this image. The focus is advantageously chosen so that drops 104 or drop edges are displayed sharply while the farther background scene appears blurred. This camera arrangement 106 can optionally be supplemented by one or more active radiation sources 112 in order to facilitate or improve the visibility of the drops 104 or to ensure them even in the dark. By means of a method according to the invention, an analysis of the image (follow) signal of the aforementioned camera arrangement 106, for example in the devices 108, 110, can be carried out.

2 shows a schematic representation of a window 102 of a vehicle, for example of the vehicle 100 shown in FIG. 1. An outer surface of the window 102 is cleaned by a wiper device 214 having a wiping edge 216. When sweeping the disk 102 by the wiping edge 216, a streak 204 remains on the surface of the disk 102 in the area just swept by the wiping edge.

3 shows a flowchart of a method for detecting streaking on a windscreen of a vehicle, according to an embodiment of the present invention. For example, the steps of the method may be performed by the devices shown in FIG. By means of the method, for example, the streaks shown in FIG. 2 can be recognized. For this purpose, in a step 308 based on a received image of the disk image evaluated points that are arranged along a Schlieren path. Information about a course of the Schlieren path may have been determined in advance and made available to the method. In a step 310, a streak, if any, is detected based on the score performed in step 308. Information about the streak can be output or stored in a memory device for later evaluation. Steps 308, 310 may be performed repeatedly. Also, multiple steps 308, 310 may be performed in parallel, for example, for different Schlieren paths.

An embodiment of the present invention will be described with reference to Figs.

FIG. 4 shows an original image section 402 of a camera arrangement between the forward and reverse run of a wiper of the pane. The image section 402 may have been detected, for example, with the camera arrangement shown in FIG. 1.

5 shows a detection image 502 which has been determined by suitable image processing from the original image detail 402 or a plurality of original image details. In the detection image 502, detected drops are shown in gray scale. Black means reliable detection of drops and knows that no drop is detected.

For comparison, FIG. 6 shows a detection image 602 which is 2.40 seconds older than the detection image 502 and in which only newly encountered raindrops but no streaks are visible.

The images 402, 502, 602 shown in FIGS. 4 to 6 may each be composed of a plurality of pixels.

The image 402 was taken between the return and the wiper arm

It can be seen that the droplets of the streaks are arranged predominantly like a chain along oblique lines. The shape of this line is determined by the motion trajectories of the points on the wiper blade lip. In windshield wipers with a simple twist mechanism, ie rotation around a point, As is the normal case, a point on the wiper lip makes a substantially arcuate motion. Due to the disk inclination relative to the camera orientation and the geometrical distortions along the optical path of disk, refocusing optics and camera optics, the arcuate path is generally imaged in a somewhat more complex path. The

Curvature direction of the arc can also reverse.

However, Fig. 5 shows that due to the relatively small sensitive area in this embodiment, the strips 204 are approximately straight. However, the angle of the straight line is not constant, on the left they are a bit steeper here than on the right.

FIG. 6 also shows an image 602 without streaks for comparison. Here all the streaks had dried off and new raindrops 104 had fallen on the sensitive surface.

In accordance with this embodiment, the detection and quantification of stripe-like water streaks is performed on the disk, but drops 104 as in FIG. 6 are not allowed to cause striae detection.

A seemingly obvious difference based on droplet size is unsuitable, as even very small droplets occur with drizzle and fog. Therefore, the orientation of the stripes 204 is used as a distinguishing criterion.

The comparison of images 502, 602 reveals that purely naturally incident drops 104 are randomly randomly distributed locally. There is usually no statistical dependency between the position of a first drop and the position of a second drop unless they are connecting and overflowing.

However, this is different for a drop belonging to a water streak 204. Along the previously known direction there is a high compound probability. If there is a drop, the probability is high that along the known direction more drops are encountered. Conversely, if there is no drop at one location, there is also a low probability of dropping along the known direction.

An inventive algorithm processes the image 502, 602 based on a criterion in two directions, a first direction approximately along the expected Schlieren orientation and a second direction approximately perpendicular thereto.

Fig. 7 shows the detection image 502 being processed along the Schlieren orientation. In this case, a path which is representative of further indicated by arrows paths of a group 1, provided with the reference numeral 704.

Fig. 8 shows the detection image 502, which is approximately perpendicular to the

Schlieren orientation is processed. In this case, a path which is representative of further indicated by arrows paths of a group 2, provided with the reference numeral 804.

The results or intermediate results obtained by the two processes are then related to each other. One can also speak here of two groups of paths 704, 804, along which, for example, as far as possible all the pixels of the detection image 502 are processed.

The stripe-like streaks lead to two distinctly different results on the two groups of paths 704, 804, while raindrops lead to approximately the same result regardless of the path group. This achieves the desired distinctness.

According to one embodiment, the following criterion can be used for this purpose. Along the respective path 704, 804, for example from the group 1 shown in FIG. 7, the detection result is compared with the detection result of the pixel removed in the path direction for each pixel. If a drop is detected at both pixels, an accumulator value ai for the paths 704 of group 1 is incremented, for example, weighted with the distance of the two pixels considered. This is done for values from n = 1 to eg n _max = 20. The choice of n _max depends on the

Schlier droplet size in picture 502. This approach works both with streaks of individual water droplets and with streaks of elongated water strips or interconnected droplets.

In the same way, a second accumulator value a _{2 is} formed on the paths 804 of the group 2 shown in FIG. 8.

The two accumulator values are then related. The difference between the two accumulator values is a measure of the

Schlierenhaftigkeit. In order to obtain a meaningful normalization, the following definition for the streakiness s offers itself:

a + a ₂ where ai represents the path 704 of group 1 along the Schlieren orientation. If s is in the clearly positive range, then the typical stripe streaks are present, and the more pronounced the larger s. Values of s less than or near zero can be ignored. Furthermore, care should be taken that the denominator is not too small, z. B. because no drops are present, because then an evaluation does not make sense.

Thus, s is thus a suitable measure for the quantification of stripe-like water streaks, the orientation of the streaks corresponding to an expected orientation, which results from the geometry of the arrangement.

The geometry of the arrangement can be assumed to be known. In a motor vehicle, the orientations of the paths 704, 804 may be calculated in advance from the design data, for example, in a pre-calibration. It is also possible to automatically determine the orientations from data measured during operation, for example during an online calibration.

According to one embodiment, the orientations were determined manually from detection images on which streaks were visible. Incidentally, in the case of the paths 704, 804, it does not matter whether they are processed forwards or backwards, since the results in the exemplary embodiment proposed here are identical. Is an automatic wiper control in addition to a signal to

Quantifying the amount of water on the disk nor a second signal to quantify the streaks available, so can z. B. in light rain after wiping for at least as long to wait until the streaks have subsided until it is wiped again. Since the time course of the decay depends on many factors and is hardly predictable, it is advantageous to measure it. By quantifying the streaks and their time course, the quantification of the newly added amount of water can be improved by new drops, so that there is the possibility to realize a more acceptable for the user or driver wiping behavior.

According to this embodiment, a solid geometric arrangement is assumed, so that the strip-like streaks always have a previously known direction. In particular, the position of the windshield wiper rotation axis with respect to the camera is relevant. If this arrangement were changed, the streaks would have a different direction than the one previously known and would therefore no longer be readily detectable.

The approach of the present invention can be applied to a camera-based rain sensor that is extended to detect and quantify stripe-like water streaks as compared to a known rain sensor without incurring significant additional costs.

A further exemplary embodiment of the present invention will be described below with reference to FIGS. 9 to 18. The embodiment is based on a camera-based indirect wiper state analysis that provides a viable solution for quickly and objectively determining the wiper wear condition.

Decisive for the wiping quality is, among other things rubber quality, pressure force and angle of wiper arm to the disc, the integrity of the wiper edge. The wear causes fine cracks at the wiper edge and the originally sharp edges are rounded off. The water is then no longer completely and evenly removed in a wiping cycle of the disc. It creates streaks on the disc, which lead to a visual impairment.

The wear is allegedly considerably accelerated by insects on the disc, the hard Chitin tanks can hurt the micro-edge, or by operating the wiper with over-frozen slice or frozen wiper blade.

This embodiment allows a quantitative determination of a measure of the wear state of the wiper, which relates to the extent of streaking and thus to the size relevant to the driver.

The description is based on the camera-based detection and quantification of strip-like water streaks on a pane described with reference to FIGS. 4 to 8.

The wiping cycle consists of a trace and a return of each wiper arm. In the trace, the disc is overwhelmed by the first edge of the wiper blade, when returning from the second edge. Since the two edges generally have different damages, theirs are

Streaks images accordingly different.

Accordingly, it makes sense to analyze both streak images separately. At least the streaking image should be analyzed after rewind, since it is much more important for the perceived visual impairment, since the time interval between return and next trace is usually much longer than the time between trace and rewind.

For a better understanding, different detection images at successive points in time are shown in FIGS. 9 to 13, which were generated with a suitable image processing method. The detected drops are shown in gray values (black = reliable detection, white = no drops). FIG. 9 shows, corresponding to FIG. 5, a detection image 502 which was determined by suitable image processing from an original image detail. Schlieren 204 can be seen in the detection image 502. Specifically, one is

Streaking occurs 400 ms after the wiper has traced, caused by the first wiper edge. On the left in the image 502 the slope of the streaks 204 is stronger than the slope of the streaks 204 on the right in the image 502. This results from the arc of the wiper, but if necessary a linear approximation is possible. FIG. 10 shows another detection image 1002 with streaks 204, 120 ms later

(as Fig. 9). Shingling is shown 40 ms after the return of the wiper caused by the second wiper edge.

Fig. 11 shows a further detection image 1 102 with streaks 204. It is the same situation as shown in Fig. 10, but 200 ms after the wiper returns.

FIG. 12 shows another detection image 1202 with streaks 204. It is the same situation as shown in FIG. 11, but 600 ms after the wiper returns. A dominant streak 204 is still present.

Fig. 13 shows another detection image 1302 with a streak 204 and raindrops 104. It is a situation at a different time. The detection image 1302 is determined by newly encountered raindrops. Although one second has elapsed since the last wipe cycle, there is still a dominant streak at the same location as in FIG. 12.

The detection image 502 shown in FIG. 9 was taken after the winder's trace, while the detection images 1002, 1102, 1202 shown in FIGS. 10 to 12 were detected after the immediately following return. It is obvious that the streak images differ from the return flow.

With close inspection of the images 1002, 1, 102, 1202 taken at intervals of 40 ms, 200 ms and 600 ms after the wiper returns, it can be seen how the small droplets gradually evaporate, while the larger ones Droplets persist longer. The volume of water that evaporates from one drop per unit time is approximately proportional to the air-water surface of the drop. Thus, the time to complete evaporation is approximately proportional to the drop thickness.

A good, new wiper leaves only very small droplets on the disc, which evaporate accordingly quickly. The view through the disc is thus undisturbed again shortly after the wiping process.

A windscreen wiper with cracks and other injuries at the wiper edge leaves more or less large droplets. These preferably arrange themselves in lines on the disk, along the movement trajectory of the respective crack across the disk. Sometimes the droplets are also connected to each other and form water strips.

The image 1202 shown in Fig. 12 indicates that even after 600 ms, a dominant streak 204 has hardly evaporated yet, while most of the smaller streaks have already disappeared.

At another time, with more severe precipitation, the image 1302 shown in Figure 13 was recorded, one second after the wiper returned. It can be seen that here too there is a dominant streak 204, in the same place as in picture 1202.

Damage to the wiper thus leads to locally persistent streaks. This circumstance can optionally be used in the analysis.

Fig. 14 shows a time course of the amount of rain 1418 on the disc. It can be a trace. The abscissa shows the time and the ordinate a measure of the amount of rain. Issued is a wipe cycle 1420 bounded by a trace 1422 and a subsequent return 1424. After trace 1422, rainfall 1418 increases from a residual amount 1426 that remains constant over time to be reduced to remainder 1426 by return 1424. After return 1424, the amount of rain 1418 increases from the residual amount. ge 1426 again to be finally reduced by the trace 1422 back to the residual amount 1426.

Fig. 14 shows a typical time course of a rainfall 1418 on the disk. In the wiping cycle 1420, the disc is wiped twice in succession, in the trace 1422 and in the return 1424. There remains a respective amount of residual water 1426 on the disc, which is formed from more or less small droplets and perceived as streaks. Because of this residual amount 1426, the dashed marking is not exceeded. In the other times, the amount of rain 1418 on the disc increases approximately linearly, with precipitation assumed to be constant.

Fig. 15 shows a schematic representation of the decay behavior of the streaks after the return and the wiper blade. The curve 1530 indicates a good case with one or more rapidly fading streaks.

Shown in addition to the amount of rain 1418 nor a measure of the

Schlierenhness in the form of the curve 1530, which can be determined according to the method described with reference to FIGS. 4 to 8. It can be seen that in the trace and return each streaks occur, but then decay quickly. The visual impairment thus lasts only briefly. The remainder 1426 after the overflow is low. The decay occurs approximately linearly when the droplets show approximately uniform sizes.

16 shows a schematic representation of the decay behavior of the streaks after the return and the wiper blade. The curve 1630 identifies an unfavorable case with one or more slowly decaying streaks.

In FIG. 16, the situation in terms of visual impairment, as compared to the situation shown in FIG. 15, is considerably less favorable because the streaks formed during the wiping process decay much more slowly. Also, the residual amount 1426 is larger. The cause is the presence of larger drops in the streaks, which take longer to dry. With a worn wiper blade, smaller and larger droplets emerge simultaneously with correspondingly different evaporation times. This explains the slightly concave curve of the decay: Initially, large and small droplets are simultaneously present, resulting in a steeper subsidence, later only larger droplets that lead to a flatter decay.

The comparison of individual Abklingverläufe 1530, 1630 of Figures 15 and 16 can thus very quickly a first conclusion on the state of wear of the wiper blade to.

For example, the temporal decay constant can be used for this, ie the time or the number of image cycles that are required until a measure of streakiness has decreased by a specific factor.

Because the increase in wear is a very slow process that can go on for many wiper hours, it makes sense to use the available time and measure over a reasonable period of time.

This is especially useful because the changing environmental conditions influence the decay constant. This depends e.g. temperature, humidity, wind speed on the disk surface and / or the degree of water contamination, e.g. by dissolved road salt in winter, from.

For example, the average decay constant may be periodically, e.g. all n wiping cycles are stored in a permanent memory. Plausibility of the data and long-term analyzes can then be carried out on the data thus stored, e.g. sliding averaging and comparison with at least one threshold for the wear limit in order to determine recommendations for the driver or the workshop.

In addition, the driver can optionally be given feedback on how his behavior, e.g. Manual cleaning or misuse during icing, affects the condition of the wiper blades.

According to one embodiment, diversification of the streaks may be performed. With increasing wear, a situation arises in which the resistance of the streaks increases on average and the difference between the streaks also increases.

In the method described with reference to FIGS. 4 to 8, the streaks are evaluated on paths approximately along the expected orientation (paths group 1) and approximately perpendicular thereto (paths group 2), as shown in FIG. 17.

Fig. 17 shows the detection image 502 being processed along the Schlieren orientation. In this case, a path which is representative of further indicated by arrows paths of a group 1, provided with the reference numeral 704. The arrows 704 represent an evaluation direction, but not a streaking path in the sense of actual physical streaks as a visible path.

FIG. 18 shows the detection image 502, which is approximately perpendicular to the

Thus, the detection image 502 is processed in two directions. Once along the Schlieren orientation and once approximately perpendicular to it.

For wiper blade state analysis, it may be advantageous to divide the Group 1 paths into subgroups, for example, into a first group of paths 804 along which the streaks rapidly decay, for example, where a decay constant is less than a threshold, and a second group of paths 1704, along which the streaks slowly fade away, where, for example, a decay constant is greater than the threshold. Of the first group, only one path is provided with the reference numeral 804 for the sake of clarity.

The state of wear could then be determined by counting, ie the number of paths 1704. Instead of two groups, k groups can also be selected and the evaluation can be done with a histogram with k intervals. The person skilled in the art is able to specify further solutions for evaluation with diversification of the streaks. A further embodiment of the present invention will be described below with reference to FIGS. 19 to 22. The exemplary embodiment comprises additional functions of a video-based rain sensor focused on the pane.

Fig. 19 shows an overall information flow system according to an embodiment of the present invention. Shown are a disk 102 and a camera 106 which is completely or partially focused on the disk. In a detection area of the camera 106, there is a fouling 104 or streak on the outside of the pane and a fog 1904 on the inside of the pane. Images of the disk 102 are created by the camera 106 and output to an image processing unit. FIGS. 20 to 22 show examples of corresponding images. The image processing unit includes a rain detection unit 1907, a streak detection unit 108, a dirt detection unit 1941, and a strike detection unit 1943.

The streak detection unit 108 is configured to receive data from the rain detection unit 1907. Rain detection unit 1907 is configured to determine whether raindrops 104 are on disk 102 based on the images of camera 106.

The streak detection unit 108 is configured to determine whether the stains 104 on the disc 102 are part of a streak based on the data of the rain detection unit 1907 and additionally or alternatively based on the images of the camera 106. The unit 108 for

Schlieren detection is further configured to determine, based on information about detected streaks on the disk 102, information for a wiper change and to output to a display instrument 1945. Alternatively or additionally, the information about the wiper change is also stored in the fault memory of the vehicle. The display element 1945 may be configured to respond to a received information for a Wi-Fi. change a signal. The unit 108 for

Schlieren detection may be configured to detect streaks according to the method described with reference to FIGS. 4 to 8 and to determine the information for the wiper change which indicates a state of the wiper in accordance with the method described with reference to FIGS. 9 to 18.

The contamination detection unit 1941 is configured to determine, based on the images of the camera 106, whether there is a contaminant 104 on the disk 102 and to output a control signal for a wipe-wash function to a wiper controller 1947 in response to a detected contaminant 104 , The wiper controller 1947 is in turn configured to provide information about a wiper position and a wiper passage to the image processing unit.

The fog detection unit 1943 is configured to determine, based on the images of the camera 106, whether there is a fog 1904 on the inside of the disc 102 and, in response to a detected fog 1904, a control signal for an air conditioner or blower to a controller 1949 the air conditioner or the blower.

FIGS. 20 to 22 show images captured by a camera shown in FIGS. 1 or 19, for example. Fig. 20 shows a recording with streaks, Fig. 21 is a recording of a soiled disc and Fig. 22 is a receptacle with dried drops.

The system shown in Fig. 19 may be a windshield-borne video system in which image processing of sharply focused slice areas occurs.

Video systems are used to implement driver assistance systems, such as night-vision systems or warning video systems. These systems usually work with rigidly mounted fixed focus cameras focusing on infinity. For certain applications, such as rain sensor or traffic sign recognition, it is desirable to achieve a different focus from the normal focal plane, namely infinite. For this purpose a fixed focus camera, which focuses at a short distance, can be used. Also, partial areas with different focusses can be imaged with a supplementary or additional optics.

The image of the entire image or partial areas of the captured image area with a short focal plane, typically a few centimeters, allows the use of this imaging information for additional functions. In particular, the insertion of a very short focused area, which allows the surface of the windshield sharply, can be used in addition to the rain detection for additional functions.

The sharp image of a section of the disk surface on an image sensor area allows an evaluation of the sharp details of the disk surface. In addition to rain detection, other parameters such as soiling, streaking or fogging can be evaluated by image processing. On the basis of these results, additional functions such as detection of the windscreen wiper state or automatic control of the wipe-and-wash function can be realized in order to ensure optimal visibility for the driver.

Focussing on the pane allows a sharply imaged slice on the image sensor or part of the image sensor. In addition to evaluating the wetting state of the pane, for example caused by raindrops, snow or ice, this sharply imaged pane area can also be used to acquire further information and functions derived therefrom, as described by way of example with reference to FIG. 19.

After taking the images, the images can be examined for different characteristics in a further processing step. The further image processing can either be integrated directly into the evaluation unit of the camera or executed with an additional control unit. The possible additional functions, which can be realized with the aid of the evaluated image data, are described in more detail below.

After the passage of the windshield wiper, the images can be evaluated in terms of Schlieren detection. The passage of the windshield wiper can be taken from the image data and / or provided as information from the wiper control unit. Due to the shape, frequency, size and behavior of the streaks over time, the evaluation of the streaks makes it possible to make a statement about the condition of the windshield wipers. For a more detailed analysis, the amount of rain on the disc, which provides the gene detection function, can also be included as additional information. In addition, the characteristic features of the streaks can be stored, so that the development and change over time is included in the evaluation. The result of the Schlieren evaluation can be used for a recommendation for a wiper change. The driver can then be informed of the poor condition of the windshield wiper by a display in the instrument cluster / head-up display and by an additional audible warning and asked to change the wiper. Also, the driver can be informed about the condition of the windshield wipers via the on-board diagnostic interface of the vehicle, which can be read in the workshop or by the savvy customer. Such information can also be transmitted to a smartphone, for example. Furthermore, contamination of the disc (due to dried drops, salt residues, insects, etc.) can be detected. For this, the recorded images are evaluated with respect to sharply focused objects on the disc, which are not drops or snowflakes. If the size and / or number of these objects exceeds a certain limit which limits the driver's vision, an automatic control of the wipe-wash function can be realized to ensure good visibility for the driver. For this, the corresponding information is forwarded to the wiper control unit. The success of this wiping-washing process should be checked by image processing in order to start a second wiping-washing process if stubborn soiling is present. In order to prevent an unwanted triggering of the wiping-washing function, the images must be evaluated on sharply focused objects on the disc, which do not change even after the passage of the windshield wiper. These structures, caused for example by stone chips or cracks, must be recognized and their position and size stored so that they are not mistakenly recognized as rain or pollution. Another function can realize the detection of a fog on the inside of the windshield. For this purpose, the images taken with the camera are evaluated in terms of a fog, for example in the form of small droplets in the focused area, which superimposes the images. If such a condition is detected, the air conditioner or the blower can be controlled accordingly to eliminate the fogging and to provide a free driver's view. For this purpose, the corresponding command must be sent to the controller of the air conditioner or the blower. By constantly checking the images for the development of the fitting, the air conditioner or the blower can be optimally controlled.

A disk status, wiper state and visual disturbance determination with a driver assistance camera according to an exemplary embodiment of the present invention will be described below with reference to FIGS. 23 to 33.

Figures 23 and 24 show images 502, 2402 taken, for example, by a camera shown in Figure 1. The image 502 shown in Fig. 23 was obtained after a wiping movement of the wiper blade. The swipe has left a thick swipe that is invisible to the DAS-Vision camera, as the Canny algorithm confirms, as seen in image 2402 shown in FIG. Such invisible wiping strips can be detected according to the exemplary embodiments described below.

In particular, the detection of the presence of invisible wiping marks or wiping marks which are not visually perceptible in an image of a DAS-Vision sensor is made possible. This is a significant improvement over the prior art. The prior art is based solely on detecting visible wiping marks for the wiper blade condition assessment, on the weak and often incorrect assumption that wiping marks are visible. That this is clearly a weak assumption is shown in FIG. 23, where a thick wiper strip formed immediately after a wiping movement of a wiper blade in a bad state is invisible to a DAS-Vision camera, since such a camera is generally in focus to the infinite region. whereby any targets on the windshield Be too close and therefore blurred and visually not pronounced enough for a collection. The image 502 is subjected to a Canny algorithm and, as seen in Figure 24, no wiping edge edges are detected. Such an invisible wiper strip can nevertheless be detected, since an inventive algorithm indirectly detects wiper strips by detecting the optical

Effects that cause them are captured directly instead of often

uncharacteristic or invisible wiper strip edges in a universal DAS image.

Similarly, glass scratches caused by airborne dust wiped over the windshield over time by the wiper blades are a clear indication of vehicle windshield wear. Such scratches are generally not detectable by a conventional DAS camera because their edges are out of focus because conventional DAS cameras are focused on the infinity area and not on the windshield. According to the embodiments, however, the characteristic optical effects caused by such glass scratches can be detected, whereby their presence can be indicated.

Fig. 25 shows a camera arrangement according to an embodiment of the present invention. Shown is a windshield 102, a DAS camera 106, and a plurality of beam paths through the windshield 102, which are partially deflected by wiping strips 204. Shown is a sketch of light rays refracted by wiping strip 204, looking sideways at the windshield 102. Refraction causes waves at each optical wavelength to propagate in different directions, which in turn results in interference and hence diffraction. For simplicity, the light rays are drawn perpendicular to the windshield 102.

A wiper strip 204 on a windshield 102 causes light to break through the wiper strip 204 and thus phase shift between different points on a lightwave front impinging on the windshield 102. Based on the Huygens-Fresnel principle and the interference theory [HECHT, Eugene: Optics. 4th edition. Reading, Mass. : Addison-Wesley Pub. Co, 2002 and BORN, M.; WOLF, E .: Principles of Optics: Electromagnetic Theory of Propagation, Interference and Diffraction of Light. 7th edition. Cambridge University Press, October 1999], this results in a diffraction as seen in FIG. Since the wiper strips are aligned in the wiping direction of the wiper blade, a diffraction figure that is at a certain

POI (point of interest), perpendicular to this direction of movement and to the wiping stripes that cause it. The strength of this diffraction figure depends on the thickness of the wiping strips, the spacing between them and the position of the DAS camera with respect to the POI. As is the case in [DYAR, Melinda; GUNTER, Mickey E.; TASA,

Dennis: Mineralogy and Optical Mineralogy. Mineralogical Society of America, 2007], when wiping a wiper blade in a very good condition over the windshield immediately thereafter many parallel microscopic wiping strips resembling a lattice structure, as shown in Fig. 26, where wiper strips which are in a DAS camera image are not visible, are only visible because for purposes of demonstration on the windshield has been focused, resulting in a one-dimensional diffraction figure, which is present at the POI perpendicular to the wiping direction. Figure 26 shows an image 502 of wiping strip caused by a good wiper blade on the windshield, focusing on the windshield. It is important to note the constant parallel alignment of the wiper strips with respect to the wiping direction. A proper wiper blade causes a very large number of very fine wiping strips, each having an average thickness of almost zero and between which there is a spacing of nearly zero, with the tangent to a wiper strip at any POI on that wiper strip parallel to the wiping direction of the wiper blade at this point runs. On the other hand, when wiping, a poor wiper blade also causes thick wiping strips having a larger spacing therebetween. Generally, a good wiper blade causes microscopically thin wiping stripes with microscopic spacing between them during wiping. Wiping marks that are caused by a wiper blade in a bad condition are too thick and too widely spaced to show diffraction effects with respect to the optical center of the DAS camera. The better the condition in which a wiper blade is located, the more the wiping stripes it causes will be microscopically thin and microscopically spaced from each other. Accordingly, the wiper stripes behave based on the diffraction theory [HECHT, Eugene: Optics. 4th edition.

Reading, Mass. : Addison-Wesley Pub. Co, 2002 and BORN, M.; WOLF, E .: Principles of Optics: Electromagnetic Theory of Propagation, Interference and Diffraction of Light. 7th edition. Cambridge University Press, October 1999], all the more like a lattice structure, making the corresponding diffraction patterns more pronounced. However, thick wiping strips lead to weaker diffraction figures, but provide greater refraction effects, especially around the edges, and they result in a hazy representation of the scene in its middle section. A blurred representation can be observed in FIG. It should be noted that there is always a refraction in a wiping strip, and it is due to the refraction that a phase shift occurs at different points in the occurring wavefront, which in turn leads to a diffraction. However, microscopically small wiping strips are too thin to cause the DAS camera to detect refraction, and thick wiping strips can not be diffractive figures at the close distance of the DAS camera from the windshield. In other words, both refraction and diffraction occur regardless of how thick a wiper strip is, however, any optical effect based on the thickness of the wiper strip is characteristically pronounced, thus making it possible to properly evaluate the wiper blade state.

Quartz particles in airborne dust, when rubbed off the wiper blades on a vehicle windshield, cause glass scratches that are more pronounced over time, as shown in [DYAR, Melinda; GUNTER, Mickey E .; TASA, Dennis: Mineralogy And Optical Mineralogy. Mineralogical Society of America, 2007], and as shown in Figure 27, where the image is focused on the windshield for demonstration purposes so that invisible glass scratches may otherwise be seen in a DAS camera image. Fig. 27 shows an image 502 of glass scratches on the windshield of an older vehicle. The glass scratches are aligned in the same direction as the wiper strips, namely parallel to the wiping direction. Such glass scratches run parallel to the wiping direction of the wiper blade and thus cause at a POI perpendicular to the wiping direction of the wiper blade extending diffraction patterns. Such diffraction figures are visible in a DAS camera, even though the glass scratches that cause them are not visually pronounced enough to show sharp edges in one of the DAS camera images.

Based on these statements, it is understood that if one or more such photometric effects are detected in an image, they may be caused by wiping marks or glass scratches. A refractive-diffractive wiper streak model (ReDiWiSt = refractive

Diffraction wiping strip model) detects and detects such photometric effects and determines whether or not they were caused by wiping marks or glass scratches. ReDiWiSt is based on the diffraction and refraction properties of wiping marks on the windshield and is also due to the diffraction properties due to glass scratches on the windshield

Windshield applicable.

When a wiper blade is in good condition, it leaves behind each wiping movement microscopically small spaced apart microscopic wiping marks which, at any given POI, in the region in which the

Wiper blade as in [DYAR, Melinda; GUNTER, Mickey E .; TASA, Dennis:

Mineralogy And Optical Mineralogy. Mineralogical Society of America, 2007] wipes parallel to the wiping direction of the wiper blade. Without focussing the optics, such microscopic wiping strips are invisible to the DAS camera, as can be seen in FIG. However, their optical effects, in this case diffraction patterns, are nevertheless visually very pronounced, as can also be seen in FIG. 28. And this effect can also be observed at night, as shown in FIG. Such diffraction figures are located at a given POI perpendicular to the wiping stripes that cause them. Fig. 28 shows an image 502 of a windshield. A wiper blade, even when in good condition, leaves microscopic wiping marks when wiping. Although such strips are invisible to a Universal DAS camera, the diffraction patterns that cause them are visible.

Fig. 29 shows an image 502 of a windshield. A wiper blade in good condition leaves diffraction figures at night.

FIG. 30 shows a camera arrangement according to an exemplary embodiment of the present invention. Shown are a DAS camera 106, microscopic

Wiper strip 204, a light source 3012 and a gap diffraction 3004 of light from the source 3012. The light source 3012 is here outside the vehicle. Thus, Figure 30 shows a gap bend 3004 caused by microscopic wiping strips 204 caused by a wiper blade in good condition.

The type of diffraction caused by the microscopic wiping strip by a good wiper blade resembles (but is not identical to) a grating diffraction where each wiper strip acts as a transparent grid tooth through which light refracts. resulting in a phase shift at different points in the incident lightwave front, which in turn leads to diffraction. A conceptual model of this behavior is shown in Figure 30, in which the microscopically small spaced apart microscopic wiping strips 204 are shown in the form of a grid geometry, and also in Figure 25. It should be noted that in Figure 30, the diffraction figure 3004 is attached to the Points at which it intersects the wiper strip 204 (POIs), is perpendicular to the wiper strip 204. It should be noted that the wiping strips 204 are not straight lines but parallel arcs due to the arcuate wiping movement of the wiper blade. It should be noted that the microscopic wiping strips 204 illustrated in FIG

DAS-Kamera106 captured image would not be visible, but here for demonstration purposes nevertheless are shown. This has no adverse effect on this system, as this invention would nevertheless detect the diffraction figure 3004 caused by such wiping strip 204, and would in turn intuitively grasp the wiping strip 204 itself, albeit indirectly. In a picture of a windshield leaves a wiper blade in a bad condition thick wiping stripes when wiping. Such stripes are too thick for diffraction patterns to be visible, and the refraction becomes the dominant optical effect, as shown by the refraction figures in FIGS

Wiper strip of a bad wiper is visible.

A wiper blade in poor condition causes thick wiping marks at points in the area it passes over. At such wiping the diffraction is not conspicuous, and the refraction becomes the more pronounced optical effect. In this case, thick wiper strips break light from a plurality of light sources, which lie opposite the field of view (FOV) of the camera arranged inside the vehicle. If the windshield is focused on for demonstration purposes, the edges of the wiper strips are visible. In a Universal DAS, the windshield is not focused and these edges are visually uncharacteristic, but the refractive figures would still be visually detectable by the DAS and this invention. Refraction in thick wiping strips creates optical (refractive) figures that extend along and are disposed within the wiper strips causing them, as opposed to diffraction patterns that are perpendicular to the very thin wiper strips that cause them.

Fig. 31 shows a camera arrangement according to an embodiment of the present invention. Shown are a DAS camera 106, a light source 3012, thick wiping strips 204 caused by a wiper blade and refracting figures 3004 in a bad state. Shown are refractive figures 3004 caused by thick wiper strips due to a wiper blade in a poor condition. It should be noted how the refractive figures 3004, in contrast to diffraction figures due to microscopically small wiping strips, run parallel to the thick wiping strips 204 which cause them and lie within them in the image plane.

Thus, Fig. 31 shows a model of refraction of light from a source 3012 due to thick wiping strips 204 caused by a wiper blade in a poor condition. The edges of a thick wiper strip 204 are curved, causing the refracted light at this boundary line on Snell's law of refraction [KLINGSHIRN, Claus: Semiconductor Optics. 3rd edition. Berlin: Springer Verlag, February 2007], which results in the refractive figure on such wiping strips 204 in the image, while the wiper strip edges are visually uncharacteristic due to the focusing of the DAS camera 106 in the infinity region. Although the wiper strip 204 may not be visible itself, especially at night. However, this does not limit the scope of performance of this embodiment, since a device according to this embodiment merely searches for the optical effects due to wiping stripes 204 caused by the wiping mechanism of the wiper blades.

ReDiWiSt is based on detecting light streaks in the image. Based on an analysis, such strips, if of interest, i.e., if they are diffraction or refraction figures due to wiping strips, must be either perpendicular or parallel to the wiper strip at the corresponding POIs. When this condition is satisfied, ReDiWiSt then determines whether the figure is bowed or broken by determining whether it is perpendicular or parallel to the wiper strip at the particular point where the figure and the wiper strip intersect. It should be noted that the wiper strip may or may not be visible and its direction is determined only as that which is perpendicular to the wiper blade at the POI.

Since the direction of a wiper strip at a particular POI is parallel to the wiping direction of the wiper blade at this point, it is possible to generate a look-up table or mathematical function based on the mechanical configuration of the wiper mechanism and its motor function. The specifications for achieving the latter two may be obtained from the manufacturer of the wiper blade system. It also takes into account several wiper blades that cover the same area. Optionally, a temporal equivalent may be included in the ReDiWiSt, in the form of looking for patterns of light immediately after a swipe, since optical effects caused by wiper blades are most apparent and pronounced immediately after the wiping mechanism has taken place. By using a time control unit which wipes the wiper blades with the image synchronization rate, so that it is possible to take an image immediately after a wiping movement of a wiper blade, a temporal correspondence can be achieved. This image can then be used by ReDiWiSt to detect optical effects caused by swiping. This is optimal, though not required, due to the fact that over time between two consecutive swipes, the light pattern either disappears and becomes invisible, as the wiping stripes that cause it evaporate due to wind, especially in the case of diffraction, but also in the Case of refraction, or the thick wiping strips, in the case of refraction, deviate from their authoritative orientation, for example, slide down, and thus do not follow the above-discussed geometric laws. Such thick wiping strips generally remain over almost the entire time interval between two consecutive strips along the same arc. Considering surface tension and a distribution of windshield adhesion, as described in [CLIFT,

Roland; GRACE, John; WEBER, Martin E .: Bubbles, Drops, and Particles. Dover Publications, January 1978], this is logical. In addition, the diffraction figures produced by wiping a wiper blade in a non-synchronized wiper blade system pass over a plurality of individual images, whereby a temporal correspondence is merely an option for the functionality of the wiper blade

Represents ReDiWiSt.

Thick wiping strips caused by a wiper blade in a poor state are sufficiently large that refraction effects may be pronounced enough to be detected in an image obtained by the DAS camera.

Beyond the refraction, thick wiping stripes cause a blurred image in the image 502, as shown in Fig. 23, mainly due to its proximity to the optical center of the DAS camera, since the camera is in focus on the infinity region. This makes the blurred representation a characteristic looking for a search for thick wiping marks.

Although a blurred representation can be caused by other objects in the image, such as any smooth surface. However, a wiper strip on the windshield remains mainly in the same position, while other objects causing a blurred image move across the image. Thus, tracking over a few consecutive frames can be used to exclude them.

Using this fuzzy representation collection scheme, ReDiWiSt can detect thick swipes even if no significant light source is present in the image at a time when a frame sequence is captured and processed in real time. Moreover, from the image 2402 in FIG. 24, where a Canny algorithm has been applied, it can be clearly seen that no wiping edge edges are detected in the image 2402. Experiments with other types of first and second order edge detectors, each of which is applied to different thresholds, give the same result. This shows that it is not possible in the prior art to detect such a thick wiper strip and properly indicate that the wiper blade causing this thick wiper strip is in a poor condition. In turn, according to this embodiment, such a thick wiper strip can be grasped, and this in a rather robust manner, by building upon another optical effect caused by a wiper strip on a windshield.

The hardware of an inventive system may include a universal monocular DAS camera with an additional optional control unit for synchronizing the wiper blade wiping mechanism. The synchronization may be performed such that the wiper blade passes over the area opposite the FOV of the DAS camera immediately prior to frame capture, but not necessarily every single frame. Although not required, it is optimal because, as already mentioned above, the optical effects caused by a wiper blade emerge maximally and most clearly directly after a wiping movement of the wiper blade.

In the following, it is assumed that a windshield area is swept by a wiper blade in a good condition. FIG. 32 shows a diagram in which the time t / t _{0 is} plotted on the abscissa and the diffraction figure intensity l / l _{0 is} plotted on the ordinate. Shown is a moment 1422 of the forward wiping movement and a moment 1424 of the backward wiping movement. Between the times 1422, 1424 runs the

Diffraction figure intensity with a slope - | E | 3351 and from the time 1424 with a slope - | E | 3353. Slope 3351 begins at a

Diffraction pattern intensity of 1 and drops to a value of 0.3. Slope 3353 begins at a diffraction pattern intensity of 1 and drops to a value of 0. Thus, a deviation of the diffraction pattern intensity with respect to the wiping interval is shown.

FIG. 32 describes the behavior of the intensity of the caused diffraction figure over time. TO is the time it takes for a wiper blade to move over and back over the POI, ie the point at which the diffraction figure is observed in the image. At t = 0, the time is when the wiper blade has just moved past the POI, and the intensity of the diffraction figure is at l _{0 at most} , because the microscopic wiping strips have just formed here and have not yet evaporated. According to the manual of the Association of HVAC, HVAC, HVAC and HVAC professionals [PARSONS, Bob (ed.): ASHRAE Handbook: Heating , Ventilating, and Air-Conditioning Applications, American Society of Heating and Refrigerating, 1991], is the evaporation rate E of a geometric body of water through

(a + ß - V _w ) ( _Pw - P _a ) Ι Λ \

given, where

E the evaporation rate in kg / m2 ^■ hr,

V _{w is} the air velocity over a water surface in m / s,

P _{w is} the saturation vapor pressure at the water temperature in mm Hg

P _{a is} the saturation vapor pressure at the air dew point in mm Hg

P _{w is} the latent heat of water at the water body temperature in kJ / kg and α and β are constants that change from one geometric body to another, but can generally be roughly estimated to be (α, β) = (42.6,37.6). Thus, E is a scalar for a set of approximations for the above parameters. Thus, as time passes, the intensity will decrease linearly as the microscopic wiping strips evaporate in a similar manner until the wiper blade sweeps back over the POI at t = t ₀ and the intensity of the diffraction figure caused jumps back to 1 ₀ as a result of the Backscattering new microscopic wiping strips are formed. Thereafter, and assuming no further wiping motions are performed, the intensity pattern also drops linearly to zero as the wiping strips evaporate in a linear fashion (rate of - | E |). It should be noted, however, that even in daylight, a diffraction pattern exists over several frames before becoming visually uncharacteristic, leaving the ReDiWiSt more than enough time to without a control unit for synchronizing the wiper blade with the image acquisition rate of the DAS camera get along. In cities and areas with a high dust density in the air cause

Quartz particles in the dust cause the formation of scratches on the windshield glass resulting from the wiper mechanism of the wiper blades [DYAR, Melinda; GUNTER, Mickey E.; TASA, Dennis: Mineralogy And Optical

Mineralogy. Mineralogical Society of America, 2007]. These scratches form into a permanent multiple-gap structure that causes a one-dimensional diffraction figure positioned at a POI, which is perpendicular to the structure at that POI, even when it is not raining and the wiper blades are not activated. If such diffraction figures are detected while the wiper blades are not activated, ReDiWiSt concludes that this is similar to diffraction patterns caused by wiper blades during wiping, ie, at any given POI perpendicular to the wiping direction, such diffraction figures are due to a windshield which are in poor condition due to wiping of the wiper blades in an area having a high dust density in the air for a long period of time. In one embodiment, a valuable function for the DAS is also provided beyond the functionality of the wiper blade and windshield condition assessment. ReDiWiSt captures visual phenomena caused by targets on the windshield and not caused by the scene ahead. For example, due to wiper blades in good condition or due to glass scratches on a windshield in a bad condition, microscopic wiping marks in a stereo vision DAS cause diffraction figures to have a poor disparity estimation on said diffraction figures. A stereo vision DAS, upon detection of an image containing such microscopic wiping strip-induced or glass scrape-induced diffraction patterns, for example from oncoming vehicle headlights as shown in Figs. 28 and 29, may associate them with disparities indicative of the system Two giant piles are attached to an oncoming vehicle. This wrong assumption can lead to confusion in a DAS and to an abnormal behavior of an otherwise well thought-out system. ReDiWiSt provides these systems with a basic understanding of the aforementioned optical effect when detecting these diffraction patterns caused by the microscopic wiping stripes or glass scratches on the windshield, which can eliminate the confusion and resulting errors.

The approach described is suitable in principle for use in any camera-based driver assistance system. It can be implemented with comparatively little effort, since no additional hardware costs arise, since implementation in software is possible.

The embodiments described and shown in the figures are chosen only by way of example. Different embodiments may be combined together or in relation to individual features. Also, an embodiment can be supplemented by features of another embodiment. Furthermore, method steps according to the invention can be repeated as well as carried out in a sequence other than that described.

Claims

claims

1 . A method of detecting directional structures on a windshield of a vehicle (100), comprising the steps of:

Carrying out (308) an evaluation of pixels of an image (402; 502; 602) of at least part of the slice (102) arranged along an evaluation path (704; 804), wherein a course of the evaluation path depends on an expected orientation of the directed structures on the Disc is; and

Detecting (310) a directed structure (204) based on the score.

The method of claim 1, wherein, in the step of performing (308) the evaluation, it is determined whether a pixel disposed along the evaluation path (704; 804) represents an impurity (104) of a portion of the disc (102) represented by the pixel ,

3. The method according to claim 1, comprising a step of carrying out a further evaluation of pixels of the image of the pane arranged along a further path, wherein a profile of the further path is approximately perpendicular to a profile of the evaluation path 804), and wherein in the step of recognizing (310) the directional structure (204) is determined based on the score and the further score.

4. The method according to any one of the preceding claims, are taken into account in performing the evaluation diffraction and / or gap diffraction and / or refraction effects.

A method according to any one of the preceding claims, wherein the directional structures are scratches or streaks, or the directional structures are classified as such.

6. The method according to any one of the preceding claims, wherein the

Expansion, in particular the thickness, and / or the orientation of the directed structures and / or the distance and / or differences of the directed structures between each other is determined.

7. Method according to one of the preceding claims, in which a profile of the evaluation path (704; 804) is dependent on a wipe trajectory of a wiping edge (216) of a wiper device (214) for wiping the disk (102).

8. The method according to claim 7, which comprises a step of determining a

State of the wiper edge based on information about the directional structure (204).

The method of claim 8, further comprising the step of performing a subsequent evaluation of pixels of a subsequent image (602) of the wafer (102) along the evaluation path (704; 804) and a step of recognizing a subsequent directed structure based on the following Evaluation, and in which, in the step of determining, the state of the wiping edge (216) is further performed based on information about the subsequent directional structure.

A method according to any one of the preceding claims, comprising a step of analyzing the pixels of the image (402; 502; 602) of the disk

(102) to detect fogging on an inside of the disc.

1 1. A method of detecting directional structures on a windshield of a vehicle (100), comprising the steps of:

Performing (308) an evaluation of pixels of an image (402; 502; 602) of the slice arranged along an evaluation path (704; 804) (102), wherein a course of the evaluation path is dependent on an expected orientation of the directed structures on the disc;

Making a determination as to whether a pixel disposed along the evaluation path (704; 804) represents a light diffraction effect in a region of the disc (102) represented by the pixel; and

Recognizing (310) a directed structure (204) based on the score and the determination.

12. A device for detecting directional structures on a pane of a vehicle (100), comprising: a feedthrough device (108) for performing an evaluation of pixels of an image (402; 502; 602) arranged along an evaluation path (704; 804) the slice (102), wherein a course of the evaluation path is dependent on an expected orientation of the directed structures on the disc; and a recognizer (110) for detecting a directed structure (204) based on the score.

13. Computer program product with program code for carrying out the method according to one of claims 1 to 12, when the program is executed on an information system.