WO2017153372A1

WO2017153372A1 - Method for recognizing a deposit on an optical element of a camera by a feature space and a hyperplane as well as camera system and motor vehicle

Info

Publication number: WO2017153372A1
Application number: PCT/EP2017/055261
Authority: WO
Inventors: David Hurych; Pavel Krizek; Jiri Kula; Michael Starr
Original assignee: Connaught Electronics Ltd
Current assignee: Connaught Electronics Ltd
Priority date: 2016-03-07
Filing date: 2017-03-07
Publication date: 2017-09-14
Anticipated expiration: 2018-09-07
Also published as: DE102016104044A1

Abstract

The invention relates to a method for recognizing a deposit (17) on an outer side (14) of an optical element (13) of a camera (3, 4, 5, 6) disposed in the optical path of the camera (3, 4, 5, 6) of a motor vehicle (1), in which at least one image (15) of an environmental region (11) of the motor vehicle (1) is captured by the camera (3, 4, 5, 6), and the deposit (17) is recognized based on the image (15), wherein at least one image feature (25, f₁to f₆) is determined at least in an image area (18) of the image (15) and the image feature (25, f₁to f₆) is registered in a feature space (22) characterizing at least the image feature(25, f to f6), wherein a hyperplane (27) of a predetermined classifier (26, SVM_Ato SVM_D) is registered in the feature space (22), and the feature space (22) is divided into a first partial feature space (28) and a second partial feature space (29) by the hyperplane (27), wherein the deposit (17) is recognized in the image area (18) if the image feature (25, f₁to f₆) is registered in the first partial feature space (28), which is associated with a deposit type (30) characterizing the deposit (17) by the classifier (26, SVM_Ato SVM_D).

Description

Method for recognizing a deposit on an optical element of a camera by a feature space and a hyperplane as well as camera system and motor vehicle

The invention relates to a method for recognizing a deposit on an outer side of an optical element of a camera disposed in the optical path of the camera of a motor vehicle, in which at least one image of an environmental region of the motor vehicle is captured by the camera, and the deposit is recognized based on the image. The invention also relates to a camera system for a motor vehicle as well as to a motor vehicle with a corresponding camera system.

Methods for recognizing a deposit on an optical element of a camera are known from the prior art. DE 10 2012 015 282 A1 describes a method for recognizing a covered state of an image capturing device of a motor vehicle.

It is the object of the invention to provide a method, a camera system as well as a motor vehicle, in which or by which a deposit on an outer side of an optical element disposed in the optical path of a camera of the camera system can be more precisely recognized.

According to the invention, this object is solved by a method, by a camera system as well as by a motor vehicle having the features according to the respective independent claims.

In a method according to the invention, a deposit on an outer side of an optical element of a camera disposed in the optical path of the camera of a motor vehicle is recognized. An image of an environmental region of the motor vehicle is captured by the camera and the deposit is recognized based on the image. An essential idea of the invention is to be regarded in that at least one image feature is determined at least in an image area of the image and the image feature is registered in an image feature space characterizing at least the image feature. A hyperplane of a predetermined classifier is registered in the feature space and the feature space is divided into a first partial feature space and a second partial feature space by the hyperplane. The deposit is recognized in the image area if the image feature is registered in the first partial feature space. The first partial feature space is associated with a deposit type characterizing the deposit by the classifier.

The invention is based on the realization that the deposit on the outer side of the optical element can be more precisely recognized if the image feature is generated and registered in the feature space in order to be associated there with the first partial feature space or the second partial feature space by the hyperplane.

For example, the optical element of the camera can be formed as an objective. Thus, the outer side of the optical element can for example be an outer side of a lens of the objective facing the environmental region of the motor vehicle.

For example, the image feature can be determined from a pixel of the image or else based on multiple pixels of the image. The registration of the image feature in the feature space and the application of the hyperplane are in particular effected according to the principle of a so-called Support Vector Machine (SVM). The Support Vector Machine divides an amount of objects in classes such that an area as wide as possible remains free of objects around the class boundary. Thereby, the Support Vector Machine is a so- called Large Margin Classifier.

In particular, the classifier is trained based on training examples or training vectors. In generating the hyperplane, it is not required to consider all of the available training vectors in the feature space. Training vectors, which are farther away from the hyperplane and are virtually hidden behind a front of other vectors, do not influence the location and position of the hyperplane. The hyperplane is only dependent on the training vectors closest to it - and also only these are required to mathematically uniquely describe the hyperplane. These closest vectors are called support vectors according to their function.

The feature space can be present as a space with any dimension. Similarly, the hyperplane is a plane, which can be present in any dimension. The dimensions of the feature space and the hyperplane are preferably formed corresponding to each other. The Support Vector Machine is preferably executed as a linear Support Vector Machine since the run time is thereby shortened and the recognition of the deposit is faster effected. However, alternatively, it can also be possible that the Support Vector Machine is executed with a non-linear kernel.

Preferably, it is provided that the image area is recognized as free of deposits if the image feature is registered in the second partial feature space, which is associated as different from the deposit type characterizing the deposit by the classifier. Thereby, it can then not only be recognized if the deposit is present on the outer side of the optical element, but also if a deposit is not present on the outer side of the optical element. Thereby, a covered state of an image sensor of the camera, which is triggered by the deposit on the outer side of the optical element, can be more precisely recognized. Thus, the unrestricted operation of the camera can thereby be more reliably determined. By the first partial feature space and the second partial feature space, so-called classes are provided, which describe the deposit on the one hand and the state of the camera free of deposits on the other hand. By the association of the image feature with the respective class, the image feature and thus the image area of the image are then recognized as polluted or affected by the deposit or else as free of deposits or clean. The affected image area can then be associated with an area of the outer side of the optical element after recognizing the deposit. Thus, the local position of the deposit is known not only in the image, but also on the real object, thus the optical element.

Furthermore, it is preferably provided that a plurality of images of the environmental region of the motor vehicle is captured by the camera, and the image feature is determined based on the plurality of the images. By determining the image feature in the plurality of the images, the image feature can be more significantly determined. Thereby, a temporal component can be included in the image feature. Thus, a temporal progression of a state of the image area is thereby described. The association of the image feature with the first partial feature space or with the second partial feature space can thereby be more precisely and reliably effected. This in turn results in the fact that the deposit on the outer side of the optical element is more precisely and reliably recognized.

Furthermore, it is provided that an intensity value of the image, in particular an intensity value of a Y channel of the image and/or an intensity value of a U channel of the image and/or an intensity value of a V channel of the image, and/or a normalized color saturation value of the image and/or a normalized maximum color saturation value of the plurality of the images and/or a maximum gradient intensity value of the plurality of the images is determined as the image feature. Thus, the intensity value of the image is in particular determined from an image present in a YUV color space. However, the image can preferably also be present in an RGB color space, wherein the intensity value is then for example determined from an R channel of the image and/or a G channel of the image and/or a B channel of the image. A conversion of the image from an RGB color space into a YUV color space or vice versa can be effected any number of times. For the normalized color saturation value of the image, a color saturation of the entire image is determined and a maximum color saturation of the image area is reduced by the entire color saturation of the image. Subsequently, the result is again divided by the entire color saturation. The normalized maximum color saturation value is determined over the plurality of the images. Here, the maximum color saturation value of the image area is simply determined over the plurality of the images. The maximum gradient intensity value is also determined based on the plurality of the images. Thereto, the gradients are preferably determined in all of the channels of the image in line and column direction of the image. Subsequently, a sum of squared gradients in line and column direction of the image is for example determined for all of the image channels. Thereby, the gradient magnitudes of the image are in particular determined and the maximum of these gradient magnitudes can be selected over the plurality of the images. The deposit on the outer side of the optical element can be more precisely and reliably determined by the image features.

Furthermore, it is preferably provided that a plurality n>1 of hyperplanes of a

corresponding plurality m>1 of predetermined classifiers is registered in the feature space, and the feature space is divided into a plurality n>2 of partial feature spaces by the plurality of the hyperplanes, wherein the deposit is recognized in the image area if the image feature is registered in one of the partial feature spaces, which is associated with the deposit by the classifier. Therein, m and n are preset as integers. The plurality of the hyperplanes is then preferably provided by the plurality of the classifiers. In particular, a plurality of Support Vector Machines is present by the plurality of the classifiers. Therein, the classifiers can for example be trained based on different training data. Thus, the classifiers can for example each be classified to a certain type of deposit. By the plurality of the hyperplanes, the feature space can then for example be divided such that overlapping partial feature spaces are registered in the feature space. Now, it can for example occur that the image feature is arranged in multiple partial feature spaces associated with the deposit. However, it can also be that the image feature is recognized only by one classifier as image feature associated with the class of the deposit. In this case, it is preferably provided that upon recognizing the image feature by only one classifier as associated with the deposit, it is already assumed that the deposit is present on the outer side of the optical element. By the plurality of the hyperplanes, which are provided by the plurality of the classifiers, the deposit can be more reliably recognized on the one hand since the classifiers can be more precisely trained and thus the feature space can be more precisely divided by the hyperplanes provided by the more precisely trained classifiers. On the other hand, the type of deposit can thereby be simpler determined. Thus, the deposit itself can be associated with respect to its type. Thereby, further information about the deposit can thus be provided in reliable manner. Based on the information about the deposit, thus the type of deposit, a method for eliminating the deposit can then for example be selected. Thus, upon recognizing the deposit for example as ice, a heating of the camera can be activated, while upon recognizing the deposit as dirt, for example a water jet for cleaning the outer side of the optical element is employed.

Furthermore, it is preferably provided that by the classification of the plurality of the classifiers, a different type of deposit is respectively recognized, in particular a type of water and/or a type of mud and/or a type of sand and/or a type of grass. By recognizing the different type of the deposit, an adequate solution for eliminating the deposit can for example then be selected, as already described. Thus, the water can for example be eliminated by a wiper-like apparatus, while the elimination of mud or sand is better effected by a water jet in order that the optical element is for example prevented from a mechanical damage, for example scratching. The recognition of the type of the deposit is in particular possible in that multiple classifiers, in particular Support Vector Machines, are applied to recognize the deposit.

Furthermore, it is preferably provided that a plurality of classifier groups each including a plurality of classifiers is provided or assigned and the plurality of the classifiers is selected from the plurality of the classifier groups depending on a current speed of the motor vehicle and/or a current brightness state of the environmental region. Thus, it can thereby in particular be provided that classifiers are generated and provided offline, which are provided or have been trained for different speeds of the motor vehicle and/or different brightness states of the environmental region. Thus, classifiers can for example be trained based on training examples, which have been captured at night to a certain speed of the motor vehicle. The classifier thus generated is then for example also selected if the motor vehicle currently advances with this speed at night. Preferably, for this situation too, the plurality of the classifiers is again provided to be able to determine the different type of the deposit. The plurality of the classifier groups thus provides multiple pluralities of the classifiers, which can then be selected depending on the current speed of the motor vehicle and/or the current brightness state of the environmental region of the motor vehicle. It is advantageous that the classifiers can thereby be appropriately selected and the recognition of the deposit is more reliably and precisely effected. The closer the training examples of the classifiers are to the actually prevailing situation during the recognition of the deposit, the more reliably and precisely the deposit can be recognized.

Furthermore, it is preferably provided that a first classifier group is assigned to a first speed of the motor vehicle during capture of the image, and a second classifier group is assigned to a second speed of the motor vehicle different from the first speed during capture of the image. Thereby, the first classifier group can be selected and used for recognizing the deposit if the motor vehicle is currently moved with the first speed, and the second classifier group can be used for recognizing the deposit if the motor vehicle is currently moved with the second speed. Thereby, the plurality of the classifiers can be appropriately provided and the deposit can be more reliably and precisely recognized.

Furthermore, it is preferably provided that a first classifier group is assigned to a first brightness state of the environmental region and a second classifier group is assigned to a second brightness state of the environmental region different from the first brightness state. In particular, a first brightness state of the environmental region, which has a brightness value of the environmental region below a first brightness limit value, or a second brightness state of the environmental region, which has a brightness value of the environmental region above the first brightness limit value and below a second brightness limit value, or a third brightness state, which has a brightness value of the environmental region above the second brightness limit value. Thereby, a classifier group can then for example respectively be provided for the brightness states of bright, twilight and darkness. The first brightness state can then for example correspond to bright, while the second brightness state corresponds to twilight and the third brightness state then accordingly corresponds to darkness. However, more than only three exemplarily mentioned brightness states of the environmental region can also be used to select the respective classifier group. It is again advantageous that the deposit can thereby be more reliably and precisely recognized.

In a further embodiment, it is provided that a brightness feature of the image in particular different from the image feature is determined and the brightness feature is registered in a brightness feature space characterizing the brightness feature, wherein at least one brightness hyperplane of a predetermined brightness classifier is registered in the brightness feature space and the brightness feature space is divided into a first partial brightness feature space and a second partial brightness feature space by the brightness hyperplane, wherein the brightness state is determined during capture of the image in that the brightness feature is registered in the first partial brightness feature space, which is associated with the corresponding brightness state by the brightness classifier. The determination of the brightness state can therefore be determined with the brightness feature and the brightness feature space depending on the brightness classifier in analogous manner to the recognition of the deposit. Therein, the brightness classifier can in particular also be provided according to the principle of a Support Vector Machine. For example, the value of an automatic exposure control and/or a signal gain value of the camera and/or a gamma value from a gamma look-up table and/or an exposure time of the camera can be used as the brightness feature. The brightness feature, which is determined based on automatic exposure of the camera, can then for example be inserted into the image or more precisely the image file of the image as meta data. In particular, the brightness classifier is generated offline based on training data. Overall, up to eleven different parameters of the camera characterizing the exposure time of the camera and many more can be used for the brightness feature. Thereby, the brightness state in the current environmental region of the motor vehicle can be precisely determined before recognizing the deposit. Based on the precisely determined brightness state, the corresponding classifier group can then be selected. This in turn results in the fact that the deposit on the optical element is more precisely and reliably recognized.

Furthermore, it is preferably provided that a reliability value is determined for a result of recognition of the deposit and the result of recognition of the deposit is output in particular for all of the image areas of the image if the reliability value is greater than a reliability limit value. The reliability value is also referred to as so-called confidence value. By the reliability value, it is indicated how likely the deposit is present on the optical element or not. The reliability value can for example be determined by determining a distance of the image feature to the hyperplane. The larger the distance of the image feature to the hyperplane, the more likely the deposit is considered recognized. In contrast thereto, it can for example be assumed that the closer the image feature is to the hyperplane, the less likely the deposit can be considered recognized without error. Finally, the image feature if it is registered close to the hyperplane is also already close to the second partial feature space, which is associated with the class free of deposits. For associating the image feature with the class free of deposits, a distance to the hyperplane can also be taken into account. The result of the association can then also be output with a reliability value.

Furthermore, it can be provided that the result of recognition of the deposit is output if a yaw angle variation of the motor vehicle is greater than a yaw angle variation limit value, in particular 90° and/or if a traveled drive dista nee of the motor vehicle is greater than a drive distance limit value, in particular 100 m. Thus, the result of recognition of the deposit can for example also already be output if the reliability value is less than the reliability limit value, but the yaw angle variation is greater than the yaw angle variation limit value or the traveled drive distance is greater than the drive distance limit value. The yaw angle variation of the motor vehicle can for example be picked up from a CAN (Controller Area Network) bus or a FlexRay bus of the motor vehicle. Similarly, the traveled drive distance of the motor vehicle can be picked up from a CAN bus or a FlexRay bus of the motor vehicle. The result of recognition of the deposit can then for example be output to the display unit of the motor vehicle. Furthermore, the result of recognition of the deposit can also be acoustically or haptically output in the motor vehicle. Thus, a warning can for example be output to a driver of the motor vehicle if the deposit is recognized.

Advantageously, the result can also be output depending on the yaw angle variation or the traveled drive distance such that the result can also be output if the reliability value is less than the reliability limit value.

In particular, it is preferably provided that the predetermined classifier is determined based on a plurality of training images before registering the hyperplane in the feature space, wherein the training images are respectively associated with one of at least two classes, in particular a deposit class and a class free of deposits. In particular, each classifier is determined by a plurality of training images. Therein, the training images are for example manually selected and manually associated with a class. Thus, for example with the class deposit or the class free of deposits. However, the training images can also be in particular associated with the different type of the deposit. Thus, the training images can for example be associated with the type of water and/or the type of mud and/or the type of sand and/or the type of grass. The respective classifier can then be trained for this respective class. By training the classifier based on the plurality of training images, a so- called monitored classification method is provided, which is characterized by the training based on the plurality of the training images in contrast to the monitored classification method. Thereby, the deposit is more precisely and reliably recognized.

The invention also relates to a camera system for a motor vehicle with at least one camera and an evaluation unit, which is formed to perform a method according to the invention.

The camera has a motor vehicle fixing element for fixing to the motor vehicle.

Preferably, the camera system includes multiple cameras. A deposit can then be recognized on each individual one of the cameras.

Furthermore, the invention also relates to a motor vehicle with a camera system according to the invention. The preferred embodiments presented with respect to the method according to the invention and the advantages thereof correspondingly apply to the camera system according to the invention as well as to the motor vehicle according to the invention.

Further features of the invention are apparent from the claims, the figures and the description of figures. The features and feature combinations mentioned above in the description as well as the features and feature combinations mentioned below in the description of figures and/or shown in the figures alone are usable not only in the respectively specified combination, but also in other combinations without departing from the scope of the invention. Thus, implementations are also to be considered as encompassed and disclosed by the invention, which are not explicitly shown in the figures and explained, but arise from and can be generated by separated feature combinations from the explained implementations. Implementations and feature combinations are also to be considered as disclosed, which thus do not have all of the features of an originally formulated independent claim. Moreover, implementations and feature combinations are also to be considered as disclosed, in particular by the explanations set out above, which extend beyond or deviate from the feature combinations set out in the relations of the claims.

Below, the embodiments of the invention are explained in more detail based on schematic drawings.

There show:

Fig. 1 a schematic plan view of an embodiment of a motor vehicle according to the invention with a camera system;

Fig. 2 a schematic representation of an image of an environmental region of the motor vehicle captured by a camera of the camera system;

Fig. 3 a schematic representation analogously to Fig. 2 with multiple image areas of the image, in which a deposit is recognized;

Fig. 4 a schematic representation of a further image of the environmental region of the motor vehicle captured by the camera; Fig. 5 a schematic representation of a further image analogously to Fig. 4, however with image areas of the image, in which a deposit is recognized;

Fig. 6 a schematic representation of a feature space, in which an image feature and a hyperplane of a predetermined classifier are registered; and

Fig. 7 a flow diagram of an embodiment of a method according to the invention.

In the figures, identical or functionally identical elements are provided with the same reference characters.

In Fig. 1 , a plan view of a motor vehicle 1 with a camera system 2 is schematically illustrated. The camera system 2 includes a first camera 3, a second camera 4, a third camera 5 and a fourth camera 6. The first camera 3 is disposed at a front 7 of the motor vehicle 1 , above a front motor vehicle license plate of the motor vehicle 1 . The second camera 4 is disposed at a rear 8 of the motor vehicle 1 , above a rear motor vehicle license plate of the motor vehicle 1 . The third camera 5 is disposed at a left wing mirror 9 of the motor vehicle 1 and the fourth camera 6 is disposed at a right wing mirror 10 of the motor vehicle 1 . However, the arrangement of the respective camera 3, 4, 5, 6 is variously possible at the motor vehicle 1 , however, preferably such that an environmental region 1 1 of the motor vehicle 1 can be at least partially captured by the respective camera 3, 4, 5, 6.

Furthermore, the camera system 2 includes an evaluation unit 12. The evaluation unit 12 is electrically connected to the cameras 3, 4, 5, 6 and disposed below a front passenger's seat of the motor vehicle 1 according to the embodiment. The arrangement of the evaluation unit 13 is again variously possible, however, preferably such that the connection to the cameras 3, 4, 5, 6 of the camera system 2 can be provided. The evaluation unit 12 can either be formed separately from the cameras 3, 4, 5, 6 or else be integrated in one of the cameras 3, 4, 5, 6. Furthermore, the evaluation unit 12 can for example be formed of multiple partial evaluation units, which are for example each integrated in one of the cameras 3, 4, 5, 6 of the camera system 2.

The cameras 3, 4, 5, 6 can be formed as CMOS (complementary metal-oxide

semiconductor) camera or else as CCD (charge-coupled device) camera or else various image capturing device. The cameras 3, 4, 5, 6 are in particular formed as a video camera, which continuously provides an image sequence of frames.

The cameras 3, 4, 5, 6 each have an optical element 13. Therein, the optical element 13 is disposed in the optical path of the respective camera 3, 4, 5, 6. In the optical path means that light from the environmental region 1 1 , which is incident on an image sensor of the respective camera 3, 4, 5, 6, at least passes through the optical element 13.

Therein, the optical element can for example be formed as an objective of the respective camera 3, 4, 5, 6. However, the optical element 13 can for example also be formed only as a lens of the objective of the respective camera 3, 4, 5, 6 or else as a cover pane of the respective camera 3, 4, 5, 6. The optical element 13 has an outer side 14. According to the embodiment, the outer side 14 is a side of the optical element 13 facing the environmental region 1 1 . In particular, the outer side 14 is in contact with air from the environmental region 1 1 . Thereby, the outer side 14 of the optical element 13 can for example also become polluted by dirt from the environmental region 1 1 .

Fig. 2 shows an image 15, which is captured by one of the cameras 3, 4, 5, 6. The image 15 shows a deposit 17 of water in a central area 16. The deposit 17 of water is present on the optical element 13 and is thereby depicted in the image 15.

Fig. 3 shows the image 15 with multiple image areas 18. The image areas 18 are examined with respect to the deposit 17 and the water is recognized as a deposit type 30 of the deposit 17.

Fig. 4 shows a further image 19, which is captured by one of the cameras 3, 4, 5, 6. The deposit 17 in the form of mud is depicted in the further image 19. The deposit 17 is substantially depicted in the right lateral part 20 of the further image 19 and in a central area 21 of the further image 19.

Fig. 5 shows the further image 19 with the image areas 18, in which the mud is

recognized as the deposit type 30 of the deposit 17.

Fig. 6 shows a feature space 22. The feature space 22 has a first coordinate axis 23 and a second coordinate axis 24. According to Fig. 6, the feature space 22 is only two- dimensionally illustrated, however, the feature space 22 can assume any dimensional order. The dimension of the feature space 22 depends on image features of the image 15, 19. Thus, the first coordinate axis 23 can for example have a range of values of an intensity value of an image channel of a pixel of the respective image 15, 19. The second coordinate axis 24 can for example have a range of values of an intensity value of a further image channel of a pixel of the respective image 15, 19. The intensity value of the image channel and the intensity value of the further image channel can then for example be registered in the feature space 22 as an image feature 25 of the respective image 15, 19.

In order to recognize the deposit 17 in the image area 18, the image feature 25 is generated from the information of the image area 18. However, the image feature 25 can also be generated based on multiple image areas 18 following consecutive in time. A hyperplane 27 is provided by a predetermined classifier 26. The hyperplane 27 is registered in the feature space 22. The feature space 22 is divided into a first partial feature space 28 and a second partial feature space 29 by the hyperplane 27. The first partial feature space 28 is associated with the deposit type 30 characterizing the deposit 17 by the classifier 26. The deposit type 30 can for example be preset as water, mud, sand or grass. The second partial feature space 29 is associated as different from the deposit type 30 characterizing the deposit 17 by the classifier 26. According to the embodiment of Fig. 6, the image feature 25 is registered in the first partial feature space 28. Thereby, the image feature 25 originates from one of the image areas 18, in which the deposit 17 is recognized. The classifier 26 is determined based on a plurality of training images before registering the hyperplane 27 in the feature space 22. The determination of the classifier 26 is referred to as so-called training. In the training, the classifier 26 is for example iteratively generated with training images associated with the deposit class or the class free of deposits. In the training of the classifier 26, a cross-validation technique is applied to render the classifier 26 more robust with respect to unknown data. The result of the cross-validation technique is for example used for the selection of the image features 25, thus which image features 25 are to be used at all and then are to be determined during the operative operation, thus online, based on the image area 18.

Fig. 7 shows a flow diagram of an embodiment of the method. In a step S1 , the image 15 is captured by one of the cameras 3, 4, 5, 6, transmitted to the evaluation unit 12 and read in there. According to the embodiment, the image 15 is present in a YUV color space. Thus, the image 15 has a Y channel 31 , a U channel 32 and a V channel 33. In a step S2, the image 15 is reduced with respect to its size. Thus, the image 15 is for example present with a size of 1280 x 800 pixels in the step S1 , while it is then for example reduced to a size of 640 x 400 pixels in the step S2. The respective image area 18 of the image 15 is thereby for example 32 x 32 pixels large in the step S1 , and in step S2 for example 16 x 16 pixels large after reducing the size. Subsequently, the image features 25 for the individual pixels of the image 15 or the image areas 18 of the image 15 are determined. The image area 18 can be composed of one pixel or else include multiple pixels. Thus, in a step S3, an intensity value of the Y channel 31 of the image area 18 is determined. In a step S4, an intensity value of the U channel 32 of the image area 18 is determined. In a step S5, an intensity value of the V channel 33 of the image area 18 is determined. In particular, calculation is not required for the steps S3, S4 and S5 if the image 15 is provided in the YUV color space. If the image is provided in an RGB color space, the intensity values of the color channels of the YUV color space can be calculated as follows, wherein x and y stand for the coordinates of the respective pixel of the image 15.

Y describes an intensity value of the Y channel 31 , U describes an intensity value of the U channel 32 and V describes an intensity value of the V channel 33. R describes an intensity value of an R channel of the RGB color space, G describes an intensity value of a G channel of the RGB color space and B describes an intensity value of a B channel of the RGB color space.

In a step S6, the color saturation value S of the individual pixels of the image 15 or of the image area 18 of the image 15 is determined. The color saturation value 34 of a pixel of the image 15 is determined as follows.

Furthermore, a maximum color saturation value S_f is determined in the respective image area 18 and an overall maximum color saturation S₀ is determined for the entire image 15. The maximum color saturation value S_f can be mathematically described as follows.

In a step S7, a maximum color saturation value S_m of a plurality of the images 15 is determined. Thus, in particular images, which are captured before the image 15 in time, are brought in for determining the maximum color saturation value S_m. For determining the maximum color saturation value S_m, the maximum of the maximum color saturation value S_f Over the plurality of the images is simply selected. This can be mathematically described as follows, wherein C is the number of the current images 15.

In a step S8, the gradients of the Y channel 31 are determined. The gradients are compared to gradients of the Y channel 31 of the plurality of the images 15, and a maximum is determined. This image feature 25 is described as a maximum gradient intensity value M of the plurality of the images 15. This is effected in a step S9. The maximum gradient intensity value M can be determined for the image 15 from the RGB color space as follows. An index parameter k characterizes the respective image of the plurality of the images 15.

If the image 15 is present in the YUV color space, thus, the maximum gradient intensity value M is determined as follows.

Thus, the image 15 can be present in the RGB format or else in the YUV format. For the YUV format, for example, concrete formats like the YUV-420 format, the YUV-444 format, the YUV-41 1 format, the YUV-422 format or further variants of the YUV format are possible.

Before the image features 25 are forwarded, they are normalized. There arises a first normalized image feature f₁ , a second normalized image feature f₂, a third normalized image feature f_3, a fourth normalized image feature f₄, a fifth normalized image feature f₅ and a sixth normalized image feature f₆, respectively as the image feature 25. For calculating the normalized image features f₁ to f₆, an average value μ, is determined in a later described offline training step. Furthermore, a standard deviation∑, is determined over the training data. By a further index parameter i, the number of the respective

normalized image feature f, is described. The normalized image features f₁ to f₆ are mathematically described as follows.

After calculating the normalized image features \_Λ to f₆, they are each forwarded to a first classifier SVM_A, a second classifier SVM_B, a third classifier SVM_C and a fourth classifier SVM_D. The first classifier SVM_A is provided in a step S10 and recognizes the deposit 17 if it is associated with a type of water. The second classifier SVM_B is provided in a step S1 1 and recognizes the deposit 17 if it is associated with a type of mud. The third classifier SVMc is provided in a step S12 and recognizes the deposit 17 if it is associated with a type of sand. The fourth classifier SVM_D is provided in a step S13 and recognizes the deposit 17 if it is associated with a type of grass. The first classifier SVM_A classifies the image feature 25, thus the normalized image features \_Λ to f₆, in a step S14. Analogously thereto, the second classifier SVM_B classifies the image feature 25 in a step S15. The third classifier SVM_C classifies the image feature 25 in a step S16 and the fourth classifier SVM_D classifies the image feature 25 in a step S17. The classification of the steps S14 to S17 can be mathematically represented as follows.

Each classifier SVM_A to SVM_D has weights w_{1 ;} w₂ and w₃ as well as a deviation coefficient b. Preferably, as many weights w_{1 ;} w₂, w₃ are used as normalized image features f₁ to f₆ are used for the respective classifier SVM_A to SVM_D. The deviation coefficient preferably occurs only once per classifier SVM_A to SVM_D. In particular, each of the classifiers SVM_A to SVM_D operates on three normalized image features f₁ to f₆. In equation 16, a limit value c is introduced, which is determined during the training of the respective classifier SVM_A to SVM_D and allows statement if the result exceeds a desired reliability value of the decision of the respective classifier SVM_A to SVM_D.

Each of the classifiers SVM_A to SVM_D provides a binary result in the form of true or false. These results are combined in a step S18. There, the deposit 17 on the optical element 13 is then recognized if at least one of the classifiers SVM_A to SVM_D outputs true as the result and thus the image feature 25 is arranged in the first partial feature space 28. This can be mathematically described as follows.

In step S18, a decision D is made. Based on the decision D, it is then determined for each of the image areas 18 whether or not it is affected by the deposit 17. If the image area 18 has multiple pixels, thus, the image area 18 is marked as affected by the deposit if a plurality of the pixels has been marked as affected by the deposit 17 by the decision D. In contrast, it also applies if a plurality of the pixels of the image area 18 is marked as free of deposits by the decision D, thus, the entire image area 18 is characterized as free of deposits. In a step S19, a first image area 34 is marked as unpolluted based on the result. In a step S20, a second image area 35 is marked as polluted based on the result D. In a step S21 , a third image area 36 is marked as unpolluted based on the result D.

The provision of the respective classifier SVM_A to SVM_D in the steps S10 to S13 is effected depending on a current speed 37 of the motor vehicle 1 and depending on a current brightness state 38 of the environmental region 1 1 . The current speed 37 is provided in a step S22. Thus, the current speed 37 can for example be provided based on CAN bus and/or Flexray bus data of the motor vehicle 1 . The current brightness state 38 is determined in a step S23. In the step S23, brightness features 39 are determined. The brightness features 39 in particular provide information about an exposure adjustment of the respective camera 3, 4, 5, 6 during the capture of the image 15. Therein, the automatic exposure adjustment can for example be evaluated for the various color channels of the image 15. Therein, an applied gamma value can for example also be used as the brightness feature 39. Overall, eleven brightness features 39 describing the current brightness state 38 are preferably used, which are assigned to a first brightness state, a second brightness state or a third brightness state by a brightness classifier 40 in a step S24. Therein, the first brightness state can for example be described as bright, the second brightness state as twilight and the third brightness state as darkness. Based on the current brightness state 38, thus, either the first brightness state, the second brightness state or the third brightness state, and the current speed 37, the classifiers SVM_A to SVM_D are selected from a plurality of classifier groups 41 in a step S25.

By the plurality of the classifier groups 41 , a respective classifier group is provided for the current brightness state 38 and/or the current speed 37. Thus, each of the classifier groups in particular includes the plurality of the classifiers, which is described by the classifiers SVM_A to SVM_D. Thus, each of the classifiers SVM_A to SVM_D is in particular present in the various classifier groups 41 for different current speeds 37 and/or different brightness states 38.

If the current brightness state 38 is recognized as the second brightness state or the third brightness state, thus twilight or darkness, thus, the image 15 is preferably treated with a noise suppression method before step S1 .

Preferably, the classifiers SVM_A to SVM_D of the plurality of the classifier groups 41 are trained offline before recognizing the deposit 17. This is effected based on so-called training data. Thereto, the training data is in particular manually associated with individual classes or deposit types of the deposit 17. The training data or training examples can for example include several 1 ,000 example images.

The selection of the classifiers SVM_A to SVM_D from the plurality of the classifier groups 41 based on the current speed 37 can for example be effected depending on three speed intervals of the motor vehicle 1 . Thus, a first speed interval of the motor vehicle 1 can for example be determined from 0 km/h to 30 km/h, a second speed interval of the motor vehicle 1 can for example be determined from 30 km/h to 60 km/h and a third speed interval of the motor vehicle 1 can for example be determined from 60 km/h to infinite. However, the respective speed intervals can also be appropriately adapted.

Claims

1 . Method for recognizing a deposit (17) on an outer side (14) of an optical element

(13) of a camera (3, 4, 5, 6) disposed in the optical path of the camera (3, 4, 5, 6) of a motor vehicle (1 ), in which at least one image (15) of an environmental region (1 1 ) of the motor vehicle (1 ) is captured by the camera (3, 4, 5, 6), and the deposit (17) is recognized based on the image (15),

characterized in that

at least one image feature (25, f₁ to f₆) is determined in at least one image area (18) of the image (15) and the image feature (25, f₁ to f₆) is registered in a feature space (22) characterizing at least the image feature (25, f₁ to f₆), wherein a hyperplane (27) of a predetermined classifier (26, SVM_A to SVM_D) is registered in the feature space (22), and the feature space (22) is divided into a first partial feature space (28) and a second partial feature space (29) by the hyperplane (27), wherein the deposit (17) is recognized in the image area (18) if the image feature (25, f₁ to f₆) is registered in the first partial feature space (28), which is associated with a deposit type (30) characterizing the deposit (17) by the classifier (26, SVM_A to SVM_D).

2. Method according to claim 1 ,

characterized in that

the image area (18) is recognized as free of deposits if the image feature (25, f₁ to f₆) is registered in the second partial feature space (29), which is associated as different from the deposit type (30) characterizing the deposit (17) by the classifier (26, SVM_A to SVM_D).

3. Method according to claim 1 or 2,

characterized in that

a plurality of images (15) of the environmental region (1 1 ) of the motor vehicle (1 ) is captured by the camera (3, 4, 5, 6), and the image feature (25, f₁ to f₆) is determined based on the plurality of the images (15).

4. Method according to claim 3,

characterized in that

an intensity value ( f₁ to f₃) of the image (15), in particular an intensity value (f^ of a Y channel (31 ) of the image (15) and/or an intensity value (f₂) of a U channel (32) of the image (15) and/or an intensity value (f₃) of a V channel (33) of the image (15), and/or a normalized color saturation value (f₄) of the image (15) and/or a normalized maximum color saturation value (f₅) of the plurality of the images (15) and/or a maximum gradient intensity value (f₆) of the plurality of the images (15) is determined as the image feature (25).

5. Method according to any one of the preceding claims,

characterized in that

a plurality m>1 of hyperplanes (27) of a corresponding plurality m>1 of

predetermined classifiers (26, SVM_A to SVM_D) is registered in the feature space (22), and the feature space is divided into a plurality n>2 of partial feature spaces (28, 29) by the plurality of the hyperplanes (27), wherein the deposit (17) is recognized in the image area (18) if the image feature (25, f₁ to f₆) is registered in one of the partial feature spaces (28, 29), which is associated with the deposit by the classifier (26, SVM_A to SVM_D).

6. Method according to claim 5,

characterized in that

a different type (30) of the deposit (17) is respectively recognized by the classifiers (26, SVM_A to SVM_D) of the plurality of the classifiers, in particular a type of water and/or a type of mud and/or a type of sand and/or a type of grass.

7. Method according to claim 5 or 6,

characterized in that

a plurality of classifier groups (41 ) each including a plurality of classifiers is provided and the plurality of the classifiers is selected from the plurality of the classifier groups (41 ) depending on a current speed (37) of the motor vehicle (1 ) and/or a current brightness state (38) of the environmental region (1 1 ).

8. Method according to claim 7,

characterized in that

a first classifier group is assigned to a first speed of the motor vehicle (1 ) during the capture of the image (15), and a second classifier group is assigned to a second speed of the motor vehicle (1 ) different from the first speed during the capture of the image (15).

9. Method according to claim 7 or 8,

characterized in that

a first classifier group is assigned to a first brightness state of the environmental region (1 1 ) and a second classifier group is assigned to a second brightness state of the environmental region (1 1 ) different from the first brightness state.

10. Method according to claim 9,

characterized in that

at least one brightness feature (39) of the image (15) is determined and the brightness feature (39) is registered in a brightness feature space characterizing the brightness feature (39), wherein at least one brightness hyperplane of a

predetermined brightness classifier (40) is registered in the brightness feature space, and the brightness feature space is divided into a first partial brightness feature space and a second partial brightness feature space by the brightness hyperplane, wherein the brightness state (38) during the capture of the image (15) is determined in that the brightness feature (39) is registered in the first partial brightness feature space, which is associated with the corresponding brightness state (38) by the brightness classifier (40).

1 1 . Method according to any one of the preceding claims,

characterized in that

a reliability value for a result of the recognition of the deposit (17) is determined, and the result of the recognition of the deposit (17) is output in particular for all of the image areas (18) of the image (15) if the reliability value is greater than a reliability limit value.

12. Method according to claim 1 1 ,

characterized in that

the result of the recognition of the deposit (17) is output if a yaw angle variation of the motor vehicle (1 ) is greater than a yaw angle variation limit value, in particular 90° and/or if a traveled drive distance of the mot or vehicle (1 ) is greater than a drive distance limit value, in particular 100 m.

13. Method according to any one of the preceding claims,

characterized in that

the predetermined classifier (26, SVM_A to SVM_D) is determined based on a plurality of training images before registering the hyperplane (27) in the feature space (22), wherein the training images are each associated with one of at least two classes (28, 29), in particular a deposit class (28) or a class (29) free of deposits.

14. Camera system (2) for a motor vehicle (1 ) with at least one camera (3, 4, 5, 6) and an evaluation unit (12), which is formed to execute a method according to any one of the preceding claims.

15. Motor vehicle (1 ) with a camera system (2) according to claim 14.