JP2019099008A - Raindrop detection device and wiper control device - Google Patents

Abstract

To provide a raindrop detection device capable of efficiently determining a rainy weather.SOLUTION: A raindrop detection device includes: a camera 11 for imaging a windshield 2 to acquire information of the outside surface 2a of the windshield 2; and a control circuit 21 including a raindrop determination part 22 for estimating the existence of foreign matters on the windshield 2 from the information of the outside surface 2a of the windshield 2, and a storage part 23 storing data that states of the windshield 2 are a non-rainy weather state stored by machine learning. In the case that the raindrop determination part 22 determines that the existence of raindrops (foreign matters) on the windshield 2 is positive, the control circuit 21 determines whether to be a wet weather state by comparing the state of the windshield 2 with the data stored in the storage part 23.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a raindrop detection device and a wiper control device.

  In recent years, as a vehicle wiper device, one that detects raindrops attached to a window with a raindrop detection device and automatically drives the wiper by a wiper control device based on the detection result has been developed (for example, Patent Document 1) reference).

  As a raindrop detection device (adhesion object sensor in Patent Document 1) used in such a vehicle wiper device, the amount of foreign matter (raindrops) adhering to the vehicle window is determined, and the wiper motor is controlled according to the amount of foreign matter. A signal is output to the control unit.

JP-A-9-189533

  By the way, in the above-described vehicle wiper device, the operation of the wiper motor is determined according to the amount of foreign matter adhering to the vehicle window. For this reason, for example, if a threshold is set in order to avoid a malfunction of the wiper, the initial movement at the time of rainy weather tends to be delayed. Even if it is rainy, there are various variations such as light rain, heavy rain, heavy rain, etc. If the situation is divided according to the situation, the threshold to be set becomes a huge amount and it can not be judged efficiently as rainy weather.

  The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to provide a raindrop detection device and a wiper control device capable of efficiently determining that it is raining.

  The raindrop detection device which solves the above-mentioned subject detects the existence of the foreign substance on a window surface from the window surface detection sensor for imaging a vehicle window and acquiring the information on the surface of the vehicle window, and the information on the window surface of the vehicle window. A control unit comprising: a foreign matter estimation unit to be estimated; and a storage unit storing a wind surface condition discriminant that is a non-weathering condition derived by machine learning from the wind surface condition at the time of non-weathering trained in advance. And the control unit compares the state of the window surface with the window surface state discriminant stored in the storage unit when the foreign matter estimation unit affirms the presence of the foreign matter on the window surface. It is determined whether or not it is rainy.

  According to the above aspect, it is determined whether or not the raining state is obtained by comparing the window surface state discriminant that is in the non-raining state stored in the storage unit with the state of the window surface. That is, since it becomes possible to determine that it is raining when it deviates from the wind surface state discriminant (threshold value) that is not raining, it is not necessary to obtain data on fine raining, and it is judged efficiently as raining. Is possible.

  In the raindrop detection device, preferably, the foreign matter estimation unit detects the position and temporal continuity of the foreign matter on the window surface within the imaging range of the window surface detection sensor.

  According to the above aspect, the position and temporal continuity of the foreign matter on the window surface within the imaging range are detected. Here, when a raindrop adheres on the window surface, it remains in place or slightly moves by wind or the like. That is, the raindrops on the window surface usually have little positional change with the passage of time. Therefore, false detection of raindrops can be suppressed by detecting the position of the foreign matter on the window surface and the temporal continuity as described above.

  In the raindrop detection device, the control unit calculates an adhesion rate of foreign matter on the window surface, and controls a wiper motor for driving a wiper, the adhesion rate of the foreign matter and information indicating that the raining state is present. It is preferable to output to the wiper control unit.

  According to the above aspect, the wiper control unit can determine the operation mode of the wiper by outputting the adhesion rate of the foreign matter and the information indicating the rain state to the wiper control unit that controls the wiper motor.

  Further, a wiper control device for solving the above problems has a wiper control unit for controlling the rotation of a wiper motor, and the wiper control unit is an adhesion rate of the foreign matter output from the control unit of the raindrop detection device and a rainy state. The operation of the wiper is determined based on the information indicating that

According to the above aspect, it is possible to determine the operation mode of the wiper by the wiper control unit that controls the wiper motor from the adhesion rate of the foreign matter and the information indicating that it is wet.
In the wiper control device, an angle detection unit that detects an angle of an output shaft of the wiper motor, and angle information that outputs information of an angle of the output shaft detected by the angle detection unit to the control unit of the raindrop detection device Preferably, an output unit is provided.

  According to the above aspect, by outputting information on the angle of the output shaft of the wiper motor to the control unit of the raindrop detection device, for example, suppressing raindrop detection in a state where the wiper (wiper blade) is missing within the imaging range. Is possible.

  According to the raindrop detection device and the wiper control device of the present invention, it can be determined efficiently as rain.

BRIEF DESCRIPTION OF THE DRAWINGS The schematic cross section of the vehicle in one Embodiment. Explanatory drawing for demonstrating a learning method. (A)-(c) is an explanatory view for explaining the difference by the difference in the weather. (A)-(c) is an explanatory view for explaining a raindrop detection method. The flowchart which shows the control example of a raindrop detection apparatus. The control block diagram in a modification. Explanatory drawing for demonstrating the relationship between the imaging range and the wiping range in the modification.

  Hereinafter, an embodiment of a vehicle wiper device provided with a raindrop detection device will be described. In the drawings, for convenience of explanation, part of the configuration may be shown exaggerated or simplified. Also, the dimensional ratio of each part may be different from the actual one.

  As shown in FIG. 1, the vehicle C is provided with a windshield 2 as a vehicle window so as to extend obliquely downward from the vehicle front side end of the roof panel 1 toward the vehicle front side. Further, on the lower surface of the roof panel 1 in the vehicle C, a rearview mirror 4 for confirming the rear from the driver's seat 3 is supported.

  In addition, the vehicle C is provided with a camera 11 as a window surface detection sensor for capturing an image of the windshield 2 from inside the vehicle C to obtain information on the surface of the windshield 2. The camera 11 has a housing 12, a CMOS (image) sensor 13 as an imaging device housed and held in the housing 12, and a lens 14 held so as to be exposed to the outside from the housing 12. The camera 11 is fixed to the roof panel 1 in the present embodiment so as to be disposed on the vehicle front side of the rearview mirror 4. Specifically, the camera 11 has a support 12a extending upward to support the housing 12, and the support 12a is fixed to the roof panel 1 such that the camera 11 is disposed on the vehicle front side of the rearview mirror 4 It is done. The rearview mirror 4 is disposed at the center of the vehicle C in the vehicle width direction (width direction of the windshield 2), and the camera 11 is also disposed at the center of the vehicle C in the vehicle width direction (width direction of the windshield 2).

  The imaging surface 13a of the CMOS sensor 13 and the principal surface 14a of the lens 14 are provided non-parallel to each other so that the focal plane Pm is inclined toward the vehicle interior with reference to the plane including the outer surface 2a of the windshield 2 ing. More specifically, the imaging surface 13a and the main surface 14a are provided in an inclined manner so that the lower end side in the imaging range is on the vehicle interior side. The imaging surface 13a and the main surface 14a are provided so as to be inclined such that the plane including the focusing surface Pm and the imaging surface 13a intersects with the main surface 14a on the straight line L. Although the outer surface 2a of the windshield 2 is actually curved and not a flat surface, the inclination between the imaging surface 13a of the CMOS sensor 13 and the main surface 14a of the lens 14 is determined using a plane closest to the curved surface. It is set. FIG. 1 is a schematic cross-sectional view of the vehicle C as viewed from the side (vehicle width direction), showing a plane including the imaging surface 13a as a straight line S1, and illustrating a plane including the main surface 14a as a straight line S2. It shows. Furthermore, in FIG. 1, a plane including the outer surface 2a of the windshield 2 is illustrated as a straight line S3, and illustrated as a straight line S4 including a focus plane Pm including a line S3 as a reference and inclined toward the vehicle interior. Because it extends in the direction, it is illustrated by a point.

  By setting it as the structure mentioned above, in the camera 11 of this embodiment, the depth of field D will be set on the basis of the said focus plane Pm. Note that the depth of field D is a range that can be considered to be in focus (in-focus), and the front depth of field D1 located on the front side (rear side in the vehicle C) with respect to the focus plane Pm This is a range obtained by adding the rear depth of field D2 located on the rear side (the front side in the vehicle C) with reference to the focal plane Pm. Further, the depth of field D of this example is shallower toward the upper end side of the windshield 2 within the imaging range of the camera 11 and deeper toward the lower end side.

  Therefore, as described above, the imaging surface 13a and the main surface 14a are inclined so that the lower end side in the imaging range of the focus plane Pm is on the vehicle interior side, the front depth of field D2 in the imaging range is front glass It will be set near the outer surface 2a of 2. Further, the depth of field D in this example is a position including the outer surface 2a in the imaging range.

  In addition, in the camera 11, the position P at which the imaging central axis J intersects the outer surface 2a of the windshield 2 so that the imaging central axis J faces the vertical direction side (in other words, obliquely downward) than the horizontal plane It is provided to be within the range H1 of the upper half of the glass 2. In addition, the range H1 of the upper half of the windshield 2 is a range between the upper end of the windshield 2 and the vertical center Z of the windshield 2 in detail.

  In addition, the vehicle C is provided with a control circuit 21 connected to the camera 11. The control circuit 21 performs machine learning from the raindrop determination unit 22 that determines (estimates) the presence or absence of raindrops as foreign objects based on the image data, and the state of the outer surface 2a of the windshield 2 in non-rainy weather trained in advance. And a communication unit 24 that stores the wind surface state discriminant equation that the wind surface state is derived.

  The raindrop determination unit 22 estimates raindrops from the image data captured by the camera 11. The raindrop determination unit 22 is configured to detect the continuity of raindrops as foreign matter in the raindrop determination. That is, the positional and temporal continuity of raindrops in each of a plurality of frames is detected in the image data captured by the camera 11. A Kalman filter can be used as an example of such a detection method.

The control circuit 21 is connected to the wiper control circuit 31 via the communication unit 24.
The wiper control circuit 31 has a wiper control unit 32 that controls the drive of the wiper motor M of the vehicle wiper device, and a communication unit 33 that communicates with the communication unit 24 of the control circuit 21. For example, an in-vehicle network such as CAN, LIN, or FlexRay (registered trademark) can be used as a communication method (connection method) with each of the communication units 24 and 33.

  Next, the machine learning method used when deriving the wind surface state discriminant that it is a non-rainy state will be described. In addition, the wind surface state discriminant obtained by the learning method shown below is stored beforehand by the memory | storage part 23, and it demonstrates as what does not update the information at the time of vehicle travel.

  In this example, machine learning is performed from an image in a state of "not raining" using 1-class SVM (Support Vector Machine) which is one of pattern recognition methods. At this time, the "non-rainy" state (non-rainy state) is learned as a normal range (normal value). Here, the information in the "not wet" image includes, for example, information on weather such as fine weather and cloudy weather, information on scratches on the windshield 2 and lights such as headlights of other vehicles.

  FIG. 2 shows a normal distribution ND1 in the case other than rain and a normal distribution ND2 in the case true. In addition, "the likeness of rain" is shown as a common horizontal axis of the normal distributions ND1 and ND2. "Raininess" indicates the degree to which the foreign matter in the acquired image is similar to the characteristics of raindrops. Further, in FIG. 2, the case of true rain is plotted as “o”, and the case of true other than rain (as an example, true scratch) is plotted as “x”.

  As can be seen from FIG. 2, the normal distribution ND1 and the normal distribution ND2 overlap in a part of the region A. The area A in the normal distribution ND1 is an area in which the “likeliness of rain” is high in the normal distribution ND1. The region A in the normal distribution ND2 is a region where the "likeliness of rain" is low in the normal distribution ND2.

  Further, in this example, for example, “Raininess” in the range X of 4σ in the normal distribution ND1 is scored in 10 stages from the reference value O between the average value μ1 of the normal distribution ND1 and the average value μ2 of the normal distribution ND2. , Histogram them for every point.

  Then, as shown in FIG. 3A, in the case of sunny, the percentage of low scores is high, and the percentage of high scores is low. As shown in FIG. 3 (b), in the case of light rain, there is a feature that the rate of middle to high scores tends to be higher than that in the case of fine weather. As shown in FIG. 3 (c), in the case of rain, there is a feature that the rate of high scores tends to be higher than in the case of light rain or sunny. The histograms shown in FIGS. 3A to 3C are examples, and not all of the histograms apply to this pattern.

  As described above, histograms are prepared, and they are prepared, for example, for a plurality of frames (10 frames of t-9 to t in this example), and classified into multiple dimensions (100 dimensions in this example) It is possible to derive a wind surface state discriminant (threshold) which is a boundary area of the normal range) or the "rainy" state (abnormal range). That is, in the present example, since the "rain" state is not directly learned, it is possible to reduce the amount of data necessary for learning.

Next, processing performed by the control circuit 21 will be described.
As shown in FIG. 5, the raindrop determination unit 22 of the control circuit 21 acquires image data (step S101). The image data referred to here are, for example, the current frame which is the current image data captured by the camera 11, the first frame which is the image data immediately before the current image data, and the current image data It is image data of a plurality of frames with a third frame which is image data of a front (one before the first frame). It should be noted that various methods can be considered such as acquiring each frame from memory after being temporarily stored in the memory in the camera 11, or acquiring data captured by the camera 11 and stored in the external storage unit 23.

Next, an image feature amount is extracted from the acquired image data (step S102). For example, information such as luminance in the image is extracted as the image feature amount.
Next, the raindrop determination unit 22 detects raindrops from the image feature amount extracted from the acquired image data (step S103). At this time, raindrops are individually detected in each frame.

  Next, the raindrop determination unit 22 detects the continuity of raindrops in each frame (step S104). Here, raindrops, once attached, remain in place or move slightly by wind or the like. That is, by detecting positional continuity and temporal continuity of raindrops in each frame in the acquired image, correct raindrops can be detected, and erroneous detection of raindrops is suppressed.

  In FIG. 4A, the raindrops A1, A2 and A3 for every three frames are schematically shown superimposed. The raindrops A1 have been detected as raindrops in the step S103 in the current frame. The raindrops A2 are detected as raindrops in the step S103 in the immediately preceding frame. The raindrops A3 are detected as raindrops in the step S103 in a frame two frames before the current frame.

  In FIG. 4 (b), the raindrops A1, A2 and A3 of FIG. 4 (a) are added (combined). That is, positional continuity and temporal continuity can be detected by adding the raindrops A1, A2 and A3 for each frame. In FIG. 4B, it is estimated that the dark part of the dot is likely to be a part with high positional continuity, that is, a real raindrop.

  By using the detection method described above, the portion indicated by the solid line in FIG. 4C is detected as a raindrop. In FIG. 4C, the case where raindrops are detected only in the current frame is indicated by a two-dot chain line, and raindrops should be detected focusing on positional continuity and temporal continuity as described above. False positives can be suppressed. In addition, a Kalman filter is mentioned as an example of what paid its attention to positional continuity and temporal continuity in this way. If the presence of raindrops (foreign matter) is not recognized in step S104 or step S103, the process ends. With such a configuration, it is not necessary to perform unnecessary calculations and the like.

  Next, as shown in FIG. 5, the raindrop determination unit 22 compares the detected raindrops with the window surface state discriminant stored in the storage unit 23 to determine whether it is within the normal range (step S105). When the raindrop determination unit 22 determines that the detected raindrops are within the normal range by comparing the detected raindrops with the window surface state discriminant stored in the storage unit 23 (step S105: YES), the process ends. Thereafter, for example, the process is repeated from step S101.

  In addition, when the raindrop determination unit 22 determines that the detected raindrop has deviated from the normal range by comparing the detected raindrop with the window surface state discriminant stored in the storage unit 23 (step S105: NO), the acquired image data The deposition rate of raindrops in the inside is calculated (step S106).

  Next, the raindrop determination unit 22 outputs the adhesion rate of raindrops and information indicating that it is raining to the wiper control unit 32 via the communication units 24 and 33 (S107), and the process ends. Thereafter, for example, the process is repeated from step S101.

  Then, the drive control of the wiper motor M is performed based on the determination of the presence or absence of raindrops by the raindrop determination unit 22 and the information of the deposition rate of the raindrops. Specifically, for example, the wiper control unit 32 of the present embodiment drives the wiper motor M in a mode (a low speed mode, a high speed mode, an intermittent mode, or the like) according to the amount of raindrops determined by the raindrop determination unit 22. .

The operation of this embodiment will be described.
For example, when the lever switch SW provided in the vehicle is operated to the automatic position, the windshield 11 is periodically imaged by the camera 11 from the inside of the vehicle C. In FIG. 1, the imaging range (angle) by the camera 11 is illustrated by a broken line. Then, if it is determined by the raindrop determination unit 22 that raindrops or the like are attached on the windshield 2 based on the captured image data, the wiper control unit 32 drives the wiper motor M and the windshield is not shown. 2 is wiped off and raindrops are automatically removed.

The effects of this embodiment will be described.
(1) Whether the wind surface state discriminant in the non-rainy state stored in the storage unit 23 and the detected raindrops in the state of the outer surface 2 a of the windshield 2 are compared and whether it is the raining state or not judge. That is, since it becomes possible to determine that it is raining when it deviates from the wind surface state discriminant (threshold value) that is not raining, it is not necessary to obtain data on fine raining, and it is judged efficiently as raining. Is possible.

  (2) The position of the raindrops on the windshield 2 within the imaging range and the temporal continuity are detected. Here, when raindrops adhere to the windshield 2, the raindrops remain in place or slightly move by wind or the like. That is, the raindrops on the windshield 2 usually have little positional change with the passage of time. Therefore, false detection of raindrops can be suppressed by detecting the position and temporal continuity of the foreign matter on the windshield 2 as described above.

  (3) The wiper control unit 32 determines the operation mode of the wiper according to the adhesion rate of the foreign matter by outputting the adhesion rate of foreign matter and information indicating that it is raining to the wiper control unit 32 that controls the wiper motor M. be able to.

The above embodiment may be modified as follows.
In the above embodiment, although not particularly mentioned, for example, the position of the wiper (wiper blade) on the windshield 2 is detected, and the camera under the situation where the detected position of the wiper does not fall within the imaging range SH A configuration for performing imaging by 11 may be adopted. As shown in FIG. 7, the imaging range SH is such that all or part of the wiping range WH of the wiper overlaps with the wiping range WH. The position of the wiper can be specified (estimated) by, for example, detecting the angle of the output shaft of the wiper motor M for driving the wiper. That is, as shown in FIG. 6, an angle detection unit 40 for detecting the angle of the output shaft is provided, and the information is output to the control circuit 21 of the raindrop detection apparatus via the communication unit 33 as an angle information output unit Raindrop detection timing can be controlled. As a result, raindrops can be appropriately detected using image data under a situation where the wiper can not be found within the imaging range SH.

  In the above embodiment, the Kalman filter is used as a method for detecting the position and temporal continuity of the foreign material, but another method may be used as long as the position and temporal continuity of the foreign material can be detected. In the above embodiment, although the position and temporal continuity of the foreign substance are detected from the video of three frames in the captured video, the present invention is not limited to this, for example, the foreign substance from the video of four or more frames It is also possible to detect the position as well as the temporal continuity of In addition, it is not necessary to use consecutive frames, and for example, the position and temporal continuity of the foreign matter may be detected using the current frame, the second previous frame, and the fourth previous frame.

  In the above embodiment, learning is performed using the 1-class SVM method, which is one learning method of machine learning. However, other learning methods such as the MT (Mahalanobis-Taguchi) method may be used.

  In the above embodiment, the camera 11 is fixed to the roof panel 1 so as to be disposed on the vehicle front side of the rearview mirror 4, but may be fixed to another position. For example, it may be fixed to the inner surface of the windshield 2. Even in such a configuration, the camera 11 is disposed on the vehicle front side of the rearview mirror 4. Further, with such a configuration, for example, the working distance (the distance from the lens 14 to the outer surface 2a of the windshield 2) is shorter than that fixed to the roof panel 1 (see the above embodiment). The depth of field can be reduced (shallow). Therefore, it is possible to suppress that the background is in focus (in focus), which in turn can suppress erroneous detection.

Moreover, you may employ | adopt the structure which provides the camera 11 not only at the upper end side of the windshield 2 but at the lower end side of the windshield 2, such as the upper surface of a dashboard.
In the embodiment described above, the focusing surface Pm is arranged closer to the lower end side of the vehicle interior, but the invention is not limited thereto. For example, the focusing surface Pm may be a straight line S3, that is, a configuration along the outer surface 2a of the windshield 2 It may be

In the above embodiment, the camera 11 picks up an image of the windshield 2 from the inside of the vehicle C, but may pick up another vehicle window such as a rear glass.
In the above embodiment, the image sensor is the CMOS sensor 13. However, the present invention is not limited to this, and may be changed to another image sensor such as a CCD (image) sensor.

In the above embodiment, the wiper motor M is driven and controlled based on the determination of the presence or absence of raindrops by the raindrop determination unit 22, but the result of the determination may be used for other applications.
In the above embodiment, the camera 11 is described as having only the lens 14 exposed to the outside, but the present invention is not limited to this, and the camera may be a camera having a lens also inside the housing 12, The lens (principal surface) and the imaging surface 13a may be inclined to use the principle of shine proof.

  -The above-mentioned embodiment and each modification may be combined suitably.

  Reference Signs List 2 windshield (vehicle window) 2a outer surface (window surface) 11 camera (window surface detection sensor) 22 raindrop determination unit (foreign substance estimation unit) 23 storage unit 32 wiper control unit 40 ... angle detection unit, C ... vehicle, M ... wiper motor, SH ... imaging range.

Claims (5)

A window surface detection sensor for imaging a vehicle window to obtain information on the window surface of the vehicle window, and a foreign matter estimation unit for estimating the presence of foreign matter on the window surface from the information on the surface of the vehicle window A control unit including a storage unit storing a wind surface state discriminant that the wind surface state is non-rainy state derived by machine learning from the wind surface surface state at the time of non-rainy weather;
The control unit compares the state of the window surface with the window surface state discriminant stored in the storage unit when the foreign matter estimation unit affirms the presence of the foreign matter on the window surface, and the control unit determines a rainy weather state It is determined whether it is or not.   The raindrop detection device according to claim 1, wherein the foreign matter estimation unit detects the position and temporal continuity of the foreign matter on the window surface within the imaging range of the window surface detection sensor.   The control unit calculates an adhesion rate of the foreign matter on the window surface, and outputs the adhesion rate of the foreign matter and information indicating that the raining state is to a wiper control unit that controls a wiper motor for driving the wiper. The raindrop detection device according to claim 1 or 2, characterized in that: A wiper control unit that controls the wiper motor;
The wiper control unit determines the operation of the wiper based on the adhesion rate of the foreign matter output from the control unit of the raindrop detection device according to claim 3 and the information to the effect of being in a rainy condition. Wiper control device. An angle detection unit that detects an angle of an output shaft of the wiper motor;
5. The wiper control device according to claim 4, further comprising: an angle information output unit that outputs the information on the angle of the output shaft detected by the angle detection unit to the control unit of the raindrop detection device. .