JP2021050935A - Attached matter detector and attached matter detecting method - Google Patents

Abstract

To improve the accuracy of detecting attached matter.SOLUTION: An attached matter detector pertaining to an embodiment comprises a calculation unit, a determination unit and a setting unit. The calculation unit calculates, for each unit area composed of a prescribed number of pixels included in an imaged image, an area feature quantity based on the edge vector of each pixel. The determination unit temporarily determines the buried state of a camera due to attached matter on the basis of the area feature quantity, and also confirms the buried state as confirmed result when a prescribed confirmation condition is established in the past history of a prescribed number of temporary determinations including the latest one. When the ignition of a vehicle to which the camera is mounted is turned on, the setting unit sets a prescribed initial value to the temporary determination history immediately after ignition-on so that the establishment of the confirmation condition is expedited.SELECTED DRAWING: Figure 2

Description

The disclosed embodiments relate to a deposit detection device and a deposit detection method.

Conventionally, there is known a deposit detection device that detects deposits adhering to a camera lens based on an image captured by a camera mounted on a vehicle or the like. Some deposit detection devices detect deposits based on, for example, differences in captured images over time (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2012-038048

However, there is room for further improvement in the above-mentioned prior art in improving the detection performance of deposits.

One aspect of the embodiment is made in view of the above, and an object of the present invention is to provide a deposit detection device and a deposit detection method capable of improving the deposit detection performance.

The deposit detection device according to one aspect of the embodiment includes a calculation unit, a determination unit, and a setting unit. The calculation unit calculates a region feature amount based on the edge vector of each pixel for each unit region composed of a predetermined number of pixels included in the captured image. The determination unit provisionally determines the state of the camera being buried by deposits based on the area feature amount, and corresponds to the case where a predetermined determination condition is satisfied in a predetermined number of past provisional determination histories including the latest portion. The buried state is determined as a confirmation result. When the vehicle equipped with the camera is ignited, the setting unit sets a predetermined initial value in the provisional determination history so that the determination condition is quickly satisfied immediately after the ignition is turned on.

According to one aspect of the embodiment, the performance of detecting deposits can be improved.

FIG. 1A is a schematic explanatory view (No. 1) of the deposit detection method according to the embodiment. FIG. 1B is a schematic explanatory view (No. 2) of the deposit detection method according to the embodiment. FIG. 1C is a schematic explanatory view (No. 3) of the deposit detection method according to the embodiment. FIG. 1D is a schematic explanatory view (No. 4) of the deposit detection method according to the embodiment. FIG. 1E is a schematic explanatory view (No. 5) of the deposit detection method according to the embodiment. FIG. 2 is a block diagram of the deposit detection device according to the embodiment. FIG. 3 is a diagram (No. 1) showing the processing contents of the calculation unit. FIG. 4 is a diagram (No. 2) showing the processing contents of the calculation unit. FIG. 5 is a diagram (No. 3) showing the processing contents of the calculation unit. FIG. 6 is a diagram (No. 4) showing the processing contents of the calculation unit. FIG. 7 is a diagram (No. 5) showing the processing contents of the calculation unit. FIG. 8 is a diagram (No. 6) showing the processing content of the calculation unit. FIG. 9 is a diagram (No. 7) showing the processing content of the calculation unit. FIG. 10 is a diagram (No. 8) showing the processing contents of the calculation unit. FIG. 11 is a diagram (No. 9) showing the processing content of the calculation unit. FIG. 12 is a diagram (No. 10) showing the processing contents of the calculation unit. FIG. 13 is a diagram showing the processing contents of the setting unit. FIG. 14 is a flowchart showing a processing procedure executed by the deposit detection device according to the embodiment.

Hereinafter, embodiments of the deposit detection device and the deposit detection method disclosed in the present application will be described in detail with reference to the accompanying drawings. The present invention is not limited to the embodiments shown below.

First, the outline of the deposit detection method according to the embodiment will be described with reference to FIGS. 1A to 1E. 1A to 1E are schematic explanatory views (No. 1) to (No. 5) of the deposit detection method according to the embodiment.

As shown in FIG. 1A, for example, it is assumed that there is a captured image I captured with snow attached to the lens surface of an in-vehicle camera. In the following, when the deposit detection device 1 (see FIG. 2) to which the deposit detection method according to the embodiment is applied has a feature amount related to the brightness gradient of each pixel of the captured image I (hereinafter referred to as “edge feature amount”). Based on (there is), the case where most of the lens of the in-vehicle camera is buried by snow (hereinafter, may be referred to as “buried state”) is detected as an example.

Specifically, as shown in FIG. 1A, the deposit detection device 1 is based on the edge feature amount of each pixel PX calculated from the ROI (Region Of Interest) which is a predetermined region of interest of the captured image I. Detects the state of snow adhesion. The edge features include angular features and intensity features. The angular feature amount is the direction of the edge vector (luminance gradient) of each pixel PX (hereinafter, may be referred to as “edge direction”). The intensity feature amount is the magnitude of the edge vector of each pixel PX (hereinafter, may be referred to as “edge intensity”).

The deposit detection device 1 handles the edge feature amount in units of 100 cells composed of a predetermined number of pixels PX in order to reduce the processing load in image processing. This can contribute to reducing the processing load in image processing. Further, the unit area UA constituting the ROI is a collection of such cells 100.

Subsequently, the deposit detection device 1 calculates the area feature amount, which is the feature amount for each unit area UA, based on the edge feature amount calculated for each of the cells 100. The region feature quantity is, so to speak, a statistical feature quantity of the edge feature quantity in each unit region UA, and includes, for example, the number of pair regions and the sum of the edge strengths of the pair regions. Here, the pair area is a combination of cells 100 that are adjacent to each other and whose edge orientations are opposite to each other.

Then, the deposit detecting device 1 determines the adhered state of the deposit for each unit region UA based on the region feature amount. Then, the deposit detection device 1 determines the buried state of the lens based on the determination result for each unit region UA.

More specifically, as shown in FIG. 1A, the deposit detection device 1 is a unit region UA in which a predetermined number of cells 100 are arranged in the vertical direction and the horizontal direction in the captured image I. First, the edge feature amount is calculated in units of 100 cells (step S1). The edge features are edge orientation and edge strength as described above.

As shown in FIG. 1A, the edge orientation of the cell 100 is a representative value of the vector orientation of each pixel PX classified by an angle in a predetermined angle range, and in the example of FIG. 1A, it is divided into 90 ° angle ranges. It is determined by either up, down, left or right. The edge strength of the cell 100 is a representative value of the vector strength of each pixel PX. The calculation process of the edge feature amount in 100 units of the cell will be described later with reference to FIGS. 3 and 4.

Subsequently, the deposit detection device 1 calculates the area feature amount for each unit area UA based on the edge feature amount for each cell 100 calculated in step S1 (step S2). Then, the deposit detection device 1 determines the adhesion state (“adhesion” or “non-adhesion”) for each unit region UA based on the calculated region feature amount (step S3).

Then, for example, when the adhesion rate in the ROI exceeds a certain level, the deposit detection device 1 tentatively determines that most of the lenses of the in-vehicle camera are buried (step S4). Here, the adhesion rate is, for example, the area ratio of the adhesion portion which is the unit region UA determined to be "adhesion" in the ROI.

Further, the “provisional determination” means that the deposit detection device 1 determines the determination result not only by the processing result for one frame but also by the comprehensive determination considering the contents of the past determination history, so that the determination result is determined in step S4. At the stage of, it is expressed as "provisional judgment".

Here, the content of the provisional determination is indicated by the provisional determination flag as shown in FIG. 1B. The tentative determination flag may take, for example, "1", "-1", or "0" as a value.

The meaning of the value "1" is "buried". Such "1" is set when, for example, the adhesion rate is 40% or more. Moreover, the meaning of the value "-1" is "not buried". Such "-1" is set when, for example, the adhesion rate is less than 30%. Further, the meaning of the value "0" is "keep". Such "0" is set, for example, when the adhesion rate is other than the above, or when it is difficult to determine.

Then, the foreign substance detecting device 1, as shown in FIG. 1C, includes a captured image I _n in process as the most recent content, a predetermined number (here, for example, 9) based on the history of the provisional determination flag in the past frames To determine the buried state. The history of such a provisional determination flag is referred to as "determination history information".

As shown in FIG. 1D, the deposit detection device 1 determines the buried state as "buried" if 5 or more of the 9 histories of the judgment history information are "1", and the book for the provisional judgment flag. Turn on the "buried flag" which is the judgment flag.

Further, the deposit detection device 1 determines that the buried state is "not buried" and turns off the "buried flag" if 5 or more of the 9 histories are "-1". In the following, 5 or more of the 9 histories may be referred to as "5/9 conditions".

For this reason, the buried state is usually an operation of turning on the buried flag after waiting for the state to stabilize for at least 5 frames from the provisional determination of "buried". As a result, it is possible to prevent the determination result of the buried state from becoming unstable.

However, when the IG (ignition) of the vehicle is turned on, for example, the influence of snow on the parked vehicle may be considered, so it is preferable to be notified of the confirmed result of the buried state with a better response than usual.

Therefore, as shown in FIG. 1E, in the deposit detection method according to the embodiment, when the vehicle's IG is turned on, a predetermined number (here, two) of the determination history information is set as an initial value from the latest portion to the past portion. It was decided to set "1".

Then, as shown in FIG. 1E, this makes it possible to confirm "buried" in the shortest third frame after the IG is turned on. That is, in the deposit detection method according to the embodiment, when the IG of the vehicle is turned on, a predetermined initial value is set in the determination history information so that the determination condition of the determination result of the buried state is quickly satisfied.

As a result, according to the deposit detection method according to the embodiment, the deposit detection performance can be improved.

In the example shown in FIG. 1E, it was decided to accelerate the establishment of the confirmation condition of the judgment result of the buried state by setting the initial value for the judgment history information, but the confirmation condition itself (for example, the above-mentioned 5/9 condition). May be relaxed. Such an example will be described later with reference to FIG.

Hereinafter, a configuration example of the deposit detection device 1 to which the deposit detection method according to the above-described embodiment is applied will be described in more detail.

FIG. 2 is a block diagram of the deposit detection device 1 according to the embodiment. Note that, in FIG. 2, only the components necessary for explaining the features of the present embodiment are represented by functional blocks, and the description of general components is omitted.

In other words, each component shown in FIG. 2 is a functional concept and does not necessarily have to be physically configured as shown in the figure. For example, the specific form of distribution / integration of each functional block is not limited to the one shown in the figure, and all or part of it is functionally or physically distributed in arbitrary units according to various loads and usage conditions. -It is possible to integrate and configure.

As shown in FIG. 2, the deposit detection device 1 according to the embodiment includes a storage unit 2 and a control unit 3. Further, the deposit detection device 1 is connected to the camera 10, the IG switch 20, the vehicle speed sensor 30, and various devices 50.

Note that FIG. 2 shows a case where the deposit detection device 1 is configured separately from the camera 10, the IG switch 20, the vehicle speed sensor 30, and various devices 50, but the present invention is not limited to this, and the camera 10 and the IG are not limited to this. It may be integrally configured with at least one of the switch 20, the vehicle speed sensor 30, and various devices 50.

The camera 10 is, for example, an in-vehicle camera including a lens such as a fisheye lens and an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is provided, for example, at a position where the front and rear of the vehicle and the side of the vehicle can be imaged, and the captured image I is output to the deposit detection device 1.

The IG switch 20 is an ignition switch. The vehicle speed sensor 30 is a sensor that detects the vehicle speed of the vehicle.

The various devices 50 are devices that acquire the detection results of the deposit detection device 1 and perform various controls on the vehicle. The various devices 50, for example, have a display device that notifies the user that deposits are attached to the lens of the camera 10 and an instruction to wipe off the deposits, and jets fluid, gas, or the like toward the lens to remove the deposits. It includes a removal device for removing, a vehicle control device for controlling automatic driving, and the like.

The storage unit 2 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk. , Template information 22 and threshold information 23 are stored.

The determination history information 21 corresponds to the determination history information shown in FIGS. 1D and 1E. The template information 22 is information about a template used in the matching process executed by the calculation unit 33, which will be described later. The threshold value information 23 is information regarding the threshold value used in the determination process executed by the determination unit 34, which will be described later.

The control unit 3 is a controller, and for example, various programs stored in the storage device inside the deposit detection device 1 work on the RAM by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like. It is realized by executing as an area. Further, the control unit 3 can be realized by, for example, an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

The control unit 3 has an acquisition unit 31, a setting unit 32, a calculation unit 33, and a determination unit 34, and realizes or executes an information processing function or operation described below.

The acquisition unit 31 acquires the captured image I captured by the camera 10. The acquisition unit 31 performs grayscale processing for converting each pixel in the acquired captured image I into each gradation from white to black according to the brightness, and also performs smoothing processing for each pixel to the calculation unit 33. Output. For the smoothing process, for example, an averaging filter, an arbitrary smoothing filter such as a Gaussian filter, or the like can be used. Further, the grayscale processing and the smoothing process may be omitted.

Further, the acquisition unit 31 acquires the vehicle status based on the input signal from the IG switch 20 or the vehicle speed sensor 30. In addition, the acquisition unit 31 notifies the setting unit 32 of the status of the acquired vehicle.

The setting unit 32 sets the determination history information 21 based on the vehicle status notified from the acquisition unit 31. When the IG on is notified via, for example, the IG switch 20 as the vehicle status, the setting unit 32 has a predetermined number of determination history information 21 from the latest to the past (two in the example shown in FIG. 1E). ), Set "1" as the initial value.

The setting unit 32 relaxes the above-mentioned 5/9 condition so that the condition for determining the buried state is quickly satisfied when the vehicle is notified that the vehicle is stopped, for example, via the vehicle speed sensor 30. You can also do it. The determination unit 34, which will be described later, determines the buried state while responding to the change of the condition by the setting unit 32. Such a point will be described later with reference to FIG.

The calculation unit 33 calculates the edge feature amount for each cell 100 of the captured image I acquired from the acquisition unit 31. Here, the calculation process of the edge feature amount by the calculation unit 33 will be specifically described with reference to FIGS. 3 and 4.

3 and 4 are diagrams (No. 1) and (No. 2) showing the processing contents of the calculation unit 33. As shown in FIG. 3, the calculation unit 33 first performs edge detection processing for each pixel PX to obtain the strength of the edge ex in the X-axis direction (horizontal direction of the captured image I) and the Y-axis direction (captured image I). The strength of the edge eye (in the vertical direction) is detected. For the edge detection process, for example, an arbitrary edge detection filter such as a Sobel filter or a Prewitt filter can be used.

Subsequently, the calculation unit 33 calculates the edge vector V by using a trigonometric function based on the detected intensity of the edge ex in the X-axis direction and the intensity of the edge ey in the Y-axis direction. The edge direction, which is the angle θ formed by the X-axis, and the edge strength, which is the length L of the edge vector V, are calculated.

Subsequently, the calculation unit 33 extracts a representative value for the edge in the cell 100 based on the calculated edge vector V of each pixel PX. Specifically, as shown in the upper part of FIG. 4, the calculation unit 33 sets the edge directions −180 ° to 180 ° of the edge vector V of each pixel PX in the cell 100 in four directions of up / down / left / right at 90 ° intervals. It is classified into angle classifications (0) to (3) (hereinafter, may be referred to as "upper, lower, left, and right 4 classifications").

More specifically, when the edge orientation of the pixel PX is in the angle range of −45 ° or more and less than 45 °, the calculation unit 33 classifies it into the angle classification (0), and the angle of 45 ° or more and less than 135 °. If it is a range, it is classified into angle classification (1), and if it is 135 ° or more and less than 180 °, or if it is an angle range of -180 ° or more and less than -135 °, it is classified into angle classification (2) and -135. If the angle range is greater than or equal to ° and less than −45 °, it is classified into angle classification (3).

Then, as shown in the lower part of FIG. 4, the calculation unit 33 generates a histogram for each cell 100 with the angle classifications (0) to (3) as each class. Then, when the frequency of the class having the highest frequency is equal to or higher than the predetermined threshold value THa in the generated histogram, the calculation unit 33 indicates the angle classification corresponding to the class (in the example of FIG. 4, the angle classification (1)). Is extracted as a representative value for the edge orientation in the cell 100.

The frequency of the above-mentioned histogram is calculated by adding the edge intensities of the pixel PXs classified in the same angle range among the pixel PXs in the cell 100. Specifically, consider the power of the histogram in the class of angle classification (0). For example, it is assumed that there are three pixel PXs classified in the angle classification (0), and the edge intensities in each pixel PX are 10, 20, and 30. In this case, the frequency of the histogram in the class of angle classification (0) is calculated as 10 + 20 + 30 = 60.

Based on the histogram calculated in this way, the calculation unit 33 calculates a representative value of the edge strength in the cell 100. Specifically, as a representative value of such an edge strength, when the frequency of the class having the highest frequency in the histogram is equal to or higher than a predetermined threshold value THa, the frequency corresponding to the class is defined as the edge strength of the cell 100. That is, the process of calculating the representative value of the edge strength in the calculation unit 33 can be said to be the process of calculating the feature related to the edge strength in the cell 100 corresponding to the representative value for the edge.

On the other hand, when the frequency of the highest frequency class is less than the predetermined threshold value THa, the calculation unit 33 is "invalid" for the edge orientation of the cell 100, in other words, "no representative value for the edge orientation". Treat as. This makes it possible to prevent the calculation of a specific edge orientation as a representative value when the variation in the edge orientation of each pixel PX is large.

The processing content of the calculation unit 33 shown in FIGS. 3 and 4 is only an example, and the processing content may be arbitrary as long as the representative value for the edge can be calculated. For example, the average value of each pixel PX in the cell 100 in the edge direction may be calculated, and the angle classifications (0) to (3) corresponding to the average value may be used as the representative value in the edge direction.

Further, in FIG. 4, a case where a total of 16 pixel PXs of 4 × 4 is regarded as one cell 100, but the number of pixel PXs in the cell 100 may be arbitrarily set, and 3 × 5 Etc., the number of pixels PX in the vertical direction and the horizontal direction may be different.

Returning to the description of FIG. Further, the calculation unit 33 calculates the area feature amount for each unit area UA based on the calculated edge feature amount for each cell 100.

First, the calculation unit 33 calculates the brightness average, the average edge strength of the cell 100, and the variance in each unit area UA as the area feature amount. Further, the calculation unit 33 calculates the total number of pair regions 200 and the edge strength as the region features.

Here, a case of calculating the total number of pair regions 200 and the edge strength will be described with reference to FIGS. 5 and 6. 5 and 6 are diagrams (No. 3) and (No. 4) showing the processing contents of the calculation unit 33.

Note that FIG. 5 shows a case where the two pair areas 200 do not share the cell 100, and FIG. 6 shows a case where the two pair areas 200 share the cell 100.

As shown in FIG. 5, the calculation unit 33 scans a plurality of cells 100 arranged in the left-right direction and the up-down direction of the unit area UA in the left-right direction and the up-down direction to search for the pair area 200. That is, the calculation unit 33 extracts the cells 100 that are adjacent to each other and whose edge directions are opposite to each other as the pair area 200 among the cells 100 in the unit area UA.

Then, the calculation unit 33 calculates the number of the extracted pair regions 200 and the total edge strength in the pair regions 200. As shown in FIG. 5, when the extracted two pair areas 200 do not share the cell 100, the calculation unit 33 calculates the number of the pair areas 200 as two and sums the edge strengths. It is calculated as the total value of the edge intensities of the four cells 100 included in the two pair areas 200.

Further, as shown in FIG. 6, when the extracted two pair areas 200 share the cell 100, the calculation unit 33 calculates the number of the pair areas 200 as two and sums the edge strengths. It is calculated as the total value of the edge intensities of the three cells 100 included in the two pair areas 200.

In addition, the calculation unit 33 calculates representative values for two or more types of edges for one cell 100 based on, for example, not only the angle classification of "up / down / left / right 4 classifications" but also the angle classification of "diagonal 4 classifications". Allocation and area feature amount may be calculated. This point will be described with reference to FIGS. 7 and 8. 7 and 8 are diagrams (No. 5) and (No. 6) showing the processing contents of the calculation unit 33.

If the calculation unit 33 uses "up, down, left, and right 4 classifications" as the first angle classification and the representative value for the edge direction based on this as the first representative value, as shown in FIG. 7, the "diagonal 4 classification" is the first. The angle classification of 2 can be used, and the representative value for the edge direction based on this can be calculated as the second representative value.

In such a case, the calculation unit 33 classifies the edge direction −180 ° to 180 ° of the edge vector V of each pixel PX in the cell 100 by the second angle classification into four diagonal directions of 90 ° (4). Classify into ~ (7).

More specifically, when the edge orientation of the pixel PX is in the angle range of 0 ° or more and less than 90 °, the calculation unit 33 classifies it into the angle classification (4), and the angle range of 90 ° or more and less than 180 °. If it is, it is classified into angle classification (5), if it is in the angle range of -180 ° or more and less than -90 °, it is classified in angle classification (6), and in the angle range of -90 ° or more and less than 0 °. If there is, it is classified into angle classification (7).

Then, as shown in the lower part of FIG. 4, the calculation unit 33 generates a histogram for each cell 100 with the angle classifications (4) to (7) as each class. Then, when the frequency of the class having the highest frequency is equal to or higher than the predetermined threshold value THa in the generated histogram, the calculation unit 33 uses the angle classification corresponding to the class as the second representative value for the edge in the cell 100. calculate.

As a result, as shown in FIG. 8, two edge-oriented representative values can be assigned to each cell 100. Then, as shown in FIG. 8, in the adjacent cell 100, when at least one of the first representative value and the second representative value in the edge direction are opposite to each other, the adjacent cell 100 Is extracted as the pair area 200.

That is, the calculation unit 33 can extract the pair area 200 that could not be extracted only by one type of edge direction by calculating the first representative value and the second representative value for the edge direction in each cell 100. It becomes.

For example, a pixel PX having an edge orientation of 140 ° and a pixel PX having an edge orientation of −40 ° are not opposite in the first angle range, but are opposite in the second angle range, so that the cell 100 It becomes possible to detect the change in the edge orientation in more accurately.

Further, the calculation unit 33 calculates the number of intersections at the time of pattern matching as the area feature amount. Here, the case of calculating the number of intersections at the time of pattern matching will be described with reference to FIGS. 9 to 12.

9 to 12 are diagrams (No. 7) to (No. 10) showing the processing contents of the calculation unit 33. The amorphous hatched portions shown in FIGS. 7, 9 and 10 are assumed to be pattern portions having a predetermined edge feature amount in the captured image I.

The calculation unit 33 searches for a predetermined search pattern that matches a predetermined template by using the edge orientation of the calculated edge feature amounts of the cell 100. As shown in FIG. 9, the search directions are the left-right direction and the up-down direction.

For example, the calculation unit 33 searches for a search pattern on the condition that "opposite angle classifications do not appear on both sides of the angle classification of interest". Specifically, when the angle classification of interest is searched in the left-right direction as the angle classification (1), as shown in FIG. 10, for example, the start position is the cell of the angle classification (2) in which the angle classification is not in the opposite direction. The starting position is cell 100-2 of the angle classification (1) adjacent to 100-1.

Then, when the array of the angle classification (1) continues and the cell 100-4 of the angle classification (0) in which "the angle classification is not in the opposite direction" appears, the cell of the angle classification (1) adjacent to the cell 100-4 appears. 100-3 is the end position. In such a case, the match length is "8" in the example of FIG. When there is a match with the search pattern in this way, the calculation unit 33 holds the position and the match length.

Further, when there is a match in the search pattern shown in FIG. 10, the brightness changes between the adjacent classifications among the four types of angle classifications can be seen at the start position and the end position.

Then, as shown in FIG. 11, when the search patterns match in the left-right direction and the up-down direction, the calculation unit 33 horizontally matches each angle classification as storage information corresponding to the unit area UA corresponding to the intersection. Cumulatively add the product of length and top and bottom match lengths.

Specifically, according to the examples of FIGS. 10 and 11, as shown in FIG. 12, the calculation unit 33 cumulatively adds 5 × 8 in association with the angle classification (1) of the unit area. Further, although not shown, the calculation unit 33 also cumulatively adds the number of intersections in association with the angle classification (1) of the unit area.

By repeating such a matching process, the determination unit 34, which will be described later, determines, for example, three or more of the cumulative addition results associated with the angle classifications (0) to (3) of the unit region UA based on the predetermined detection conditions. If it is equal to or more than the threshold value of, the adhered state of the unit region UA is determined to be “adhered”. Further, if the determination condition is not satisfied, it is determined to be "non-adherent". The processing contents of the matching processing shown in FIGS. 9 to 12 are merely examples, and the processing contents are not limited.

Returning to the description of FIG. Then, the calculation unit 33 outputs the calculated area feature amount for each unit area UA to the determination unit 34.

The determination unit 34 determines the adhesion state of the deposits for each unit region UA based on the region feature amount calculated by the calculation unit 33. Further, the determination unit 34 calculates the adhesion rate based on the adhesion state for each determined unit region UA, and determines the buried state of the lens of the camera 10 based on the adhesion rate.

Here, the determination unit 34 determines whether the adhered state of the unit region UA is “adhered” or “non-adhered”, respectively. Then, the determination unit 34 calculates the area ratio of the unit region UA that is "adhered" in the predetermined region of interest of the captured image I, that is, the adhesion ratio, based on the determination result.

Then, when the adhesion rate is a certain value or more (for example, 40% or more), the determination unit 34 considers that it is "buried" and sets "1" in the above-mentioned provisional determination flag. Further, when the adhesion rate is less than a certain value (for example, less than 30%), the determination unit 34 sets the provisional determination flag to "-1" as "not buried". Further, the determination unit 34 sets the provisional determination flag to "0" as "keep" when the adhesion rate is other than the above or when the determination is difficult.

Then, the determination unit 34 determines the buried state based on the determination history information 21. For example, when the above-mentioned 5/9 condition is satisfied, the determination unit 34 reflects the corresponding flag value as a determination result in the buried flag and notifies the various devices 50. The determination unit 34 turns on the buried flag, for example, if the flag value "1" satisfies the 5/9 condition. Further, the determination unit 34 turns off the buried flag if, for example, the 5/9 condition is satisfied by the flag value “-1”.

By the way, the establishment of the confirmation condition of the buried state can be accelerated by setting the initial value in the determination history information 21 at the time of IG on, but it can also be accelerated by relaxing the confirmation condition itself. Such an example will be described with reference to FIG. FIG. 13 is a diagram showing the processing contents of the setting unit 32.

As shown in FIG. 13, it is assumed that the vehicle in which the determination history information 21 is followed by "1" is stopped. In such a case, it is assumed that the user who has been continuously notified of "1 = buried" wipes off the deposits.

If the wiping is performed, it is preferable that the "not buried" is confirmed and notified at an early stage, and the determination history information 21 that is mostly filled with "1" is cleared.

Therefore, in such a case, if the setting unit 32 notifies that the vehicle is stopped, for example, via the vehicle speed sensor 30, the setting unit 32 assumes that wiping will occur, and sets the above-mentioned 5/9 condition to 3/5 condition (3/5 condition). Change to 3 or more in 5 history).

Then, after the change, the determination unit 34 can determine that "it is not buried" in the shortest third frame. That is, in the deposit detection method according to the embodiment, when the vehicle is stopped, the confirmation condition is relaxed so that the determination condition of the determination result of the buried state is quickly established.

As a result, according to the deposit detection method according to the embodiment, the deposit detection performance can be improved.

Further, the setting unit 32 clears the determination history information 21 with "0" if it is determined that "it is not buried" after the determination condition is relaxed. After such clearing, the setting unit 32 may restore the relaxed definite condition.

Even if the vehicle whose determination history information 21 is followed by "1" is stopped, it is not always wiped off. However, even if the wiping is not performed, the "buried" is confirmed at an early stage by relaxing the confirmation condition, so that there is no problem in the buried state determination process itself.

Next, the processing procedure executed by the deposit detection device 1 according to the embodiment will be described with reference to FIG. FIG. 14 is a flowchart showing a processing procedure executed by the deposit detection device 1 according to the embodiment. Note that FIG. 14 shows a processing procedure for the captured image I for one frame.

As shown in FIG. 14, first, the acquisition unit 31 acquires the status of the vehicle (step S101). Here, when the IG is turned on (step S102, Yes), the setting unit 32 sets an initial value in the determination history information 21 so that the determination condition is satisfied earlier (step S103), and proceeds to step S106.

Further, when the vehicle is not stopped immediately after the IG is turned on (step S102, No) and the vehicle is stopped (step S104, Yes), the setting unit 32 relaxes the confirmation condition so that the confirmation condition is satisfied earlier (step S105). ).

If the vehicle is not stopped (steps S104, No), the process proceeds to step S106 as it is.

Then, the acquisition unit 31 acquires the captured image I (step S106). At the same time, the acquisition unit 31 performs grayscale processing and smoothing processing on the captured image I.

Subsequently, the calculation unit 33 calculates the edge feature amount for each cell 100 of the captured image I (step S107). Further, the calculation unit 33 calculates the area feature amount for each unit area UA based on the calculated edge feature amount (step S108).

Then, the determination unit 34 determines the adhesion state for each unit area UA based on the area feature amount calculated by the calculation unit 33 (step S109), and determines the buried state based on the determined adhesion state and the determination history information 21. Confirm (step S110).

Then, the determination unit 34 notifies the various devices 50 of the confirmation result (step S111), and ends the process. Although not shown in FIG. 14, when the buried state is determined based on the definite condition relaxed while the vehicle is stopped, the setting unit 32 clears the determination history information 21 with "0". It becomes.

As described above, the deposit detection device 1 according to the embodiment includes a calculation unit 33, a determination unit 34, and a setting unit 32. The calculation unit 33 calculates a region feature amount based on the edge vector of each pixel PX for each unit region UA composed of a predetermined number of pixel PX included in the captured image I. The determination unit 34 provisionally determines the buried state of the camera 10 due to the deposits based on the region feature amount, and also determines a predetermined determination condition in the determination history information 21 which is a provisional determination history of a predetermined number of past minutes including the latest portion. When is satisfied, the corresponding buried state is confirmed as a confirmation result. When the vehicle on which the camera 10 is mounted is IG-on, the setting unit 32 sets a predetermined initial value in the determination history information 21 so that the above-mentioned determination condition is quickly satisfied immediately after the IG is turned on.

Therefore, according to the deposit detection device 1 according to the embodiment, the deposit detection performance can be improved.

Further, when the setting unit 32 sets the initial value immediately after the IG is turned on, the setting unit 32 sets the initial value by a predetermined number from the latest portion to the past portion of the determination history information 21.

Therefore, according to the deposit detection device 1 according to the embodiment, it is possible to improve the response performance of the buried state determination immediately after the IG is turned on.

Further, when the setting unit 32 sets the initial value immediately after the IG is turned on, the setting unit 32 sets the flag value “1” indicating that the camera 10 is buried as the initial value.

Therefore, according to the deposit detection device 1 according to the embodiment, it is possible to improve the response performance of "buried" detection immediately after the IG is turned on.

Further, the setting unit 32 relaxes the definite condition so that the definite condition is satisfied earlier when the vehicle is stopped.

Therefore, according to the deposit detection device 1 according to the embodiment, when it is assumed that the user has wiped off the deposit, it is possible to confirm and notify the “not buried” at an early stage. Further, along with this, it becomes possible to clear the determination history information 21 at an early stage.

In the above-described embodiment, the case where the initial value is set in the determination history information 21 when the IG is turned on is set as an example, but it is a numerical value that can be adjusted as a parameter and is 1. It may be 3 or more.

Further, in the above-described embodiment, the determination history information 21 is for the past 9 frames including the latest information, but it may be a plurality of frames, and the number of frames is not limited. Further, the above-mentioned 5/9 condition and 3/5 condition are merely examples, and do not limit the contents of the definite condition.

Further, in the above-described embodiment, the case where −180 ° to 180 ° is divided into four directions divided by an angle range of 90 ° is shown, but the angle range is not limited to 90 °, for example, every 60 °. The angle may be classified into 6 directions divided by the angle range of.

Further, the width of each angle range may be different between the first angle classification and the second angle classification. For example, in the first angle classification, the angle may be classified by 90 °, and in the second angle classification, the angle may be classified by 60 °. Further, in the first angle classification and the second angle classification, the boundary of the angle in the angle range is shifted by 45 °, but the shifting angle may be more than 45 ° or less than 45 °.

Further effects and variations can be easily derived by those skilled in the art. For this reason, the broader aspects of the invention are not limited to the particular details and representative embodiments expressed and described as described above. Therefore, various modifications can be made without departing from the spirit or scope of the general concept of the invention as defined by the appended claims and their equivalents.

1 Adhesion detection device 2 Storage unit 3 Control unit 10 Camera 21 Judgment history information 22 Template information 23 Threshold information 31 Acquisition unit 32 Setting unit 33 Calculation unit 34 Judgment unit 50 Various devices 100 cells 200 Pair area I Captured image PX pixel UA unit region

Claims (5)

A calculation unit that calculates a region feature amount based on the edge vector of each pixel for each unit region consisting of a predetermined number of pixels included in the captured image.
The buried state of the camera due to the deposits is tentatively determined based on the region feature amount, and when a predetermined definite condition is satisfied in a predetermined number of tentative determination histories including the latest portion, the corresponding buried state is determined. Judgment unit to confirm as a confirmation result and
When the vehicle equipped with the camera is ignited, immediately after the ignition is turned on, a setting unit for setting a predetermined initial value in the provisional determination history is provided so that the determination condition is satisfied earlier. A featured deposit detection device. The setting unit
The deposit according to claim 1, wherein when the initial value is set immediately after the ignition is turned on, the initial value is set by a predetermined number from the latest portion to the past portion of the provisional determination history. Detection device. The setting unit
The deposit detection device according to claim 1 or 2, wherein when the initial value is set immediately after the ignition is turned on, a flag value indicating that the camera is buried is set as the initial value. .. The setting unit
The deposit detection device according to claim 1, 2 or 3, wherein when the vehicle is stopped, the definite condition is relaxed so that the definite condition is satisfied earlier. A calculation process for calculating a region feature amount based on the edge vector of each pixel for each unit region consisting of a predetermined number of pixels included in the captured image, and a calculation process.
The buried state of the camera due to the deposits is tentatively determined based on the region feature amount, and when a predetermined definite condition is satisfied in a predetermined number of tentative determination histories including the latest portion, the corresponding buried state is determined. Judgment process to confirm as a confirmation result and
Immediately after the ignition is turned on, when the vehicle equipped with the camera is ignited, a setting step of setting a predetermined initial value in the provisional determination history so that the determination condition is quickly satisfied is included. A characteristic deposit detection method.

Images