JP7172932B2 - Attached matter detection device and attached matter detection method - Google Patents

Description

The disclosed embodiments relate to a deposit detection device and a deposit detection method.

2. Description of the Related Art Conventionally, there has been known an adhering matter detection device that detects an adhering matter adhering to a camera lens based on an image captured by a camera mounted on a vehicle or the like. 2. Description of the Related Art For example, there is an attached matter detection device that detects an attached matter based on a difference between time-series captured images (see, for example, Patent Document 1).

JP 2012-038048 A

However, the conventional technology described above has room for further improvement in terms of improving the performance of detecting adhering matter.

One aspect of the embodiments has been made in view of the above, and an object thereof is to provide an adhering matter detection device and an adhering matter detection method capable of improving the performance of detecting an adhering matter.

An attached matter detection device according to an aspect of an embodiment includes a calculation unit, a determination unit, and a setting unit. The calculation unit calculates an area feature amount based on an edge vector of each pixel for each unit area made up of a predetermined number of pixels included in the captured image. The determination unit provisionally determines whether or not the camera is obscured by the adhering matter based on the region feature amount, and when a predetermined confirmation condition is satisfied in a predetermined number of past provisional determination histories including the most recent history, The burial state is determined as a determination result. The setting unit sets a predetermined initial value in the temporary determination history immediately after the ignition is turned on when the vehicle equipped with the camera is turned on so that the determination condition is quickly established. The setting unit, when the vehicle is stopped, relaxes the confirmation condition so that the confirmation condition is established more quickly.

According to one aspect of the embodiment, it is possible to improve the detection performance of adhering matter.

FIG. 1A is a schematic explanatory diagram (part 1) of an attached matter detection method according to an embodiment; FIG. 1B is a schematic explanatory diagram (part 2) of the attached matter detection method according to the embodiment; FIG. 1C is a schematic explanatory diagram (part 3) of the attached matter detection method according to the embodiment; FIG. 1D is a schematic explanatory diagram (part 4) of the attached matter detection method according to the embodiment; FIG. 1E is a schematic explanatory diagram (No. 5) of the attached matter detection method according to the embodiment; FIG. 2 is a block diagram of the attached matter detection device according to the embodiment. FIG. 3 is a diagram (part 1) illustrating the processing contents of the calculation unit; FIG. 4 is a diagram (part 2) illustrating the processing contents of the calculation unit; FIG. 5 is a diagram (part 3) showing the processing contents of the calculation unit; FIG. 6 is a diagram (part 4) showing the processing contents of the calculation unit. FIG. 7 is a diagram (No. 5) showing the processing contents of the calculation unit. FIG. 8 is a diagram (No. 6) showing the processing contents of the calculation unit. FIG. 9 is a diagram (No. 7) showing the processing contents of the calculation unit. FIG. 10 is a diagram (No. 8) showing the processing contents of the calculation unit. FIG. 11 is a diagram (No. 9) showing the processing contents of the calculation unit. FIG. 12 is a diagram (No. 10) illustrating the processing contents of the calculation unit; FIG. 13 is a diagram showing the processing contents of the setting unit. FIG. 14 is a flowchart illustrating a processing procedure executed by the adhering matter detection device according to the embodiment;

DETAILED DESCRIPTION OF THE INVENTION Embodiments of the attached matter detection device and the attached matter detection method disclosed in the present application will be described in detail below with reference to the accompanying drawings. In addition, this invention is not limited by embodiment shown below.

First, with reference to FIGS. 1A to 1E, an overview of the attached matter detection method according to the embodiment will be described. 1A to 1E are schematic explanatory diagrams (part 1) to (part 5) of an attached matter detection method according to an embodiment.

As shown in FIG. 1A, for example, there is a captured image I captured with snow adhering to the lens surface of an in-vehicle camera. In the following, the attached matter detection apparatus 1 (see FIG. 2) to which the attached matter detection method according to the embodiment is applied detects a feature amount (hereinafter referred to as an "edge feature amount") related to the luminance gradient of each pixel of the captured image I. An example of detecting a state in which most of the lens of an in-vehicle camera is buried in snow (hereinafter sometimes referred to as a “buried state”) will be taken as an example.

Specifically, as shown in FIG. 1A, the adhering matter detection device 1 is based on the edge feature amount of each pixel PX calculated from an ROI (Region Of Interest), which is a predetermined region of interest in the captured image I, Detects the state of snow adhesion. The edge features include angle features and strength features. The angle feature amount is the orientation of the edge vector (brightness gradient) of each pixel PX (hereinafter sometimes referred to as "edge orientation"). The intensity feature amount is the magnitude of the edge vector of each pixel PX (hereinafter sometimes referred to as "edge intensity").

In order to reduce the processing load in image processing, the adhering matter detection apparatus 1 handles such edge feature amounts in units of cells 100 each having a predetermined number of pixels PX. This can help reduce the processing load in image processing. A unit area UA that constitutes the ROI is a collection of such cells 100 .

Subsequently, the adhering matter detection apparatus 1 calculates an area feature amount, which is a feature amount for each unit area UA, based on the edge feature amount calculated for each cell 100 . The area feature amount is, so to speak, a statistical feature amount of the edge feature amount for each unit area UA, and includes, for example, the number of pair areas and the sum of edge strengths of the pair areas. Here, a pair region is a combination of adjacent cells 100 whose edge directions are opposite to each other.

Then, the adhering matter detection device 1 determines the adhering state of the adhering matter for each unit area UA based on the area feature amount. Then, the adhering matter detection device 1 determines the buried state of the lens based on the determination result for each unit area UA.

More specifically, as shown in FIG. 1A, the adhering matter detection device 1 detects a predetermined number of cells 100 in a captured image I for each unit area UA, which is a predetermined area in which cells 100 are arranged vertically and horizontally. First, an edge feature amount is calculated in units of 100 cells (step S1). The edge feature amounts are the edge direction and the edge strength as described above.

The edge orientation of the cell 100 is, as shown in FIG. 1A, a representative value of the vector orientation of each pixel PX that is classified into a predetermined angular range. It is determined by either up, down, left, or right. Also, the edge intensity of the cell 100 is the representative value of the vector intensity of each pixel PX. Note that the processing for calculating the edge feature amount for each cell 100 will be described later with reference to FIGS. 3 and 4. FIG.

Subsequently, the adhering matter detection device 1 calculates an area feature amount for each unit area UA based on the edge feature amount for each cell 100 calculated in step S1 (step S2). Then, the adhering matter detection apparatus 1 determines the adhering state (“attached” or “non-adhered”) for each unit area UA based on the calculated area feature amount (step S3).

Then, for example, when the adherence rate in the ROI reaches a certain level or more, the adhering matter detection device 1 tentatively determines that most of the lens of the vehicle-mounted camera is "buried" (step S4). Here, the adhesion rate is, for example, the area ratio of the adhesion portion, which is the unit area UA determined to be "adhered" within the ROI.

Further, the "temporary determination" means that the adhering matter detection apparatus 1 finalizes the determination result not only based on the processing result for one frame but also based on comprehensive determination taking into consideration the contents of the past determination history. At the stage of , it is expressed as "provisional judgment".

Here, the content of such provisional determination is indicated by a provisional determination flag, as shown in FIG. 1B. The temporary determination flag can take values such as "1", "-1", and "0".

The meaning of the value "1" is "buried". Such "1" is set, for example, when the adhesion rate is 40% or more. Also, the meaning of the value "-1" is "not buried". Such "-1" is set, for example, when the adhesion rate is less than 30%. Also, the meaning of the value "0" is "keep". Such "0" is set, for example, when the adhesion rate is other than the above or when determination is difficult.

Then, as shown in FIG. 1C, the adhering matter detection apparatus 1 is based on the history of the provisional determination flags for a predetermined number (here, for example, 9) of past _frames including the captured image In being processed as the most recent one. to confirm the buried state. The history of such provisional determination flags is called "determination history information".

As shown in FIG. 1D, when five or more of the nine determination history information are "1", the adhering matter detection apparatus 1 confirms the buried state as "buried", Turn on the "buried flag" which is a judgment flag.

Further, if five or more of the nine histories are "-1", the attached matter detection device 1 determines the buried state as "not buried" and turns off the "buried flag". In the following description, a condition that satisfies 5 or more of the 9 histories may be referred to as a "5/9 condition".

Therefore, in the buried state, the buried flag is turned on after waiting for at least five frames to stabilize from the provisional determination of "buried". As a result, it is possible to prevent the buried state determination result from becoming unstable.

However, when the IG (ignition) of the vehicle is turned on, for example, the parked vehicle may be affected by accumulated snow or the like, so it is preferable to be notified of the confirmation result of the buried state with better response than usual.

Therefore, as shown in FIG. 1E, in the attached object detection method according to the embodiment, when the ignition of the vehicle is turned on, a predetermined number (here, two) of the determination history information is set as an initial value from the latest to the past. It was decided to set "1".

Then, as shown in FIG. 1E, it is possible to determine "buried" at the shortest third frame after the IG is turned on. That is, in the attached object detection method according to the embodiment, when the ignition of the vehicle is turned on, a predetermined initial value is set in the determination history information so that the determination condition for the determination result of the buried state is quickly established.

As a result, the adhering matter detection method according to the embodiment can improve the adhering matter detection performance.

In the example shown in FIG. 1E, by setting the initial value for the determination history information, the confirmation condition for the determination result of the burial state is set earlier. may be relaxed. Such an example will be described later with reference to FIG.

Hereinafter, a configuration example of the adhering matter detection device 1 to which the adhering matter detection method according to the above-described embodiment is applied will be described more specifically.

FIG. 2 is a block diagram of the adhering matter detection device 1 according to the embodiment. In addition, in FIG. 2, only the components necessary for explaining the features of the present embodiment are represented by functional blocks, and the description of general components is omitted.

In other words, each component illustrated in FIG. 2 is functionally conceptual and does not necessarily need to be physically configured as illustrated. For example, the specific forms of distribution and integration of each functional block are not limited to those shown in the figure, and all or part of them can be functionally or physically distributed in arbitrary units according to various loads and usage conditions.・It is possible to integrate and configure.

As shown in FIG. 2 , the attached matter detection device 1 according to the embodiment includes a storage unit 2 and a control unit 3 . The attached matter detection device 1 is also connected to the camera 10 , the IG switch 20 , the vehicle speed sensor 30 and various devices 50 .

Although FIG. 2 shows the case where the adhering matter detection device 1 is configured separately from the camera 10, the IG switch 20, the vehicle speed sensor 30, and the various devices 50, the present invention is not limited to this. At least one of the switch 20 , the vehicle speed sensor 30 and the various devices 50 may be configured integrally.

The camera 10 is, for example, an in-vehicle camera that includes a lens such as a fisheye lens and an imaging device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is provided at a position capable of capturing images of the front, rear, and sides of the vehicle, for example, and outputs the captured image I to the adhering matter detection device 1 .

The IG switch 20 is an ignition switch. The vehicle speed sensor 30 is a sensor that detects the vehicle speed of the vehicle.

The various devices 50 are devices that acquire detection results of the adhering matter detection device 1 and perform various vehicle controls. The various devices 50 include, for example, a display device that notifies the user that there is a deposit on the lens of the camera 10 and instructs the user to wipe off the deposit; It includes a removal device that removes, and a vehicle control device that controls automatic driving and the like.

The storage unit 2 is realized by, for example, a semiconductor memory device such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. , template information 22 and threshold information 23 are stored.

The determination history information 21 corresponds to the determination history information shown in FIGS. 1D and 1E. The template information 22 is information about a template used in matching processing executed by the calculation unit 33, which will be described later. The threshold information 23 is information about a threshold used in determination processing executed by the determination unit 34, which will be described later.

The control unit 3 is a controller, and various programs stored in a storage device inside the adhering matter detection apparatus 1 operate in the RAM by means of, for example, a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). It is realized by being executed as a region. Also, the control unit 3 can be implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

The control unit 3 includes an acquisition unit 31, a setting unit 32, a calculation unit 33, and a determination unit 34, and implements or executes information processing functions and actions described below.

The acquisition unit 31 acquires the captured image I captured by the camera 10 . The acquisition unit 31 performs grayscaling processing for converting each pixel in the acquired captured image I into each gradation from white to black according to the luminance, and performs smoothing processing for each pixel, and outputs the data to the calculation unit 33. Output. Any smoothing filter such as an averaging filter or a Gaussian filter can be used for the smoothing process. Also, the grayscaling process and the smoothing process may be omitted.

Acquisition unit 31 also acquires the status of the vehicle based on an input signal from IG switch 20 or vehicle speed sensor 30 . In addition, the obtaining unit 31 notifies the setting unit 32 of the obtained vehicle status.

The setting unit 32 sets the determination history information 21 based on the vehicle condition notified from the acquisition unit 31 . For example, when IG ON is notified via IG switch 20 as the state of the vehicle, setting unit 32 sets a predetermined number (two in the example shown in FIG. 1E) of determination history information 21 from the latest to the past. ), and set "1" as the initial value.

When the vehicle is notified, for example, that the vehicle is stopped via the vehicle speed sensor 30, the setting unit 32 relaxes the above-mentioned 5/9 condition so as to hasten the establishment of the buried state determination condition. can also The determination unit 34, which will be described later, determines the buried state while responding to the condition change by the setting unit 32. FIG. This point will be described later in the description using FIG. 13 .

The calculation unit 33 calculates an edge feature amount for each cell 100 of the captured image I acquired from the acquisition unit 31 . Here, the calculation processing of the edge feature amount by the calculation unit 33 will be specifically described with reference to FIGS. 3 and 4. FIG.

3 and 4 are diagrams (part 1) and (part 2) showing the processing contents of the calculation unit 33. FIG. As shown in FIG. 3, the calculation unit 33 first performs edge detection processing for each pixel PX, and determines the intensity of the edge ex in the X-axis direction (horizontal direction of the captured image I) and the intensity of the edge ex in the Y-axis direction (the captured image I and the strength of the edge ey in the vertical direction of ). Any edge detection filter such as a Sobel filter or a Prewitt filter can be used for edge detection processing.

Subsequently, the calculation unit 33 calculates an edge vector V using a trigonometric function based on the detected intensity of the edge ex in the X-axis direction and the intensity of the detected edge ey in the Y-axis direction. The edge orientation, which is the angle θ formed with the X axis, and the edge strength, which is the length L of the edge vector V, are calculated.

Subsequently, the calculation unit 33 extracts a representative value of the edge direction in the cell 100 based on the calculated edge vector V of each pixel PX. Specifically, as shown in the upper part of FIG. 4, the calculation unit 33 calculates the edge direction of the edge vector V of each pixel PX in the cell 100 from −180° to 180° in four directions of up, down, left, and right at every 90°. It is classified into angle classifications (0) to (3) (hereinafter sometimes referred to as "up, down, left, and right four classifications").

More specifically, the calculation unit 33 classifies the edge direction of the pixel PX into the angle category (0) when the angle range is −45° or more and less than 45°, If it is in the range, it is classified into angle classification (1), and if it is in the angle range of 135° or more and less than 180°, or -180° or more and less than -135°, it is classified into angle classification (2), and -135 If the angle range is greater than or equal to degrees and less than -45 degrees, it is classified as angle classification (3).

Then, as shown in the lower part of FIG. 4, the calculation unit 33 generates a histogram for each cell 100 with angle classifications (0) to (3) as each class. Then, when the frequency of the class with the highest frequency in the generated histogram is equal to or greater than a predetermined threshold value THa, the calculation unit 33 performs an angle classification corresponding to the class (angle classification (1) in the example of FIG. 4). is extracted as the representative value of the edge orientation in the cell 100 .

The frequency of the histogram described above is calculated by adding together the edge intensities of the pixels PX classified into the same angular range among the pixels PX in the cell 100 . Specifically, consider the histogram frequency in the class of angle classification (0). For example, assume that there are three pixels PX classified into angle classification (0), and the edge strengths of the pixels PX are 10, 20, and 30, respectively. In this case, the frequency of the histogram in the class of angle classification (0) is calculated as 10+20+30=60.

Based on the histogram calculated in this manner, the calculator 33 calculates the representative value of the edge strength in the cell 100 . Specifically, when the frequency of the class with the highest frequency in the histogram is equal to or greater than a predetermined threshold THa, the representative value of the edge strength is the edge strength of the cell 100 corresponding to the class. That is, the calculation processing of the representative value of the edge strength in the calculation unit 33 can also be said to be processing of calculating the feature related to the strength of the edge in the cell 100 corresponding to the representative value of the edge direction.

On the other hand, when the frequency of the class with the highest frequency is less than the predetermined threshold value THa, the calculation unit 33 determines that the edge direction of the cell 100 is “invalid”, in other words, “there is no edge direction representative value”. treated as As a result, it is possible to prevent a specific edge orientation from being calculated as a representative value when there is a large variation in the edge orientation of each pixel PX.

Note that the processing contents of the calculation unit 33 shown in FIGS. 3 and 4 are merely examples, and the processing contents may be arbitrary as long as the representative value of the edge direction can be calculated. For example, the average value of the edge orientation of each pixel PX in the cell 100 may be calculated, and the angle classifications (0) to (3) corresponding to the average value may be used as the representative values of the edge orientation.

In addition, although FIG. 4 shows a case where a total of 16 pixels PX (4×4) are defined as one cell 100, the number of pixels PX in the cell 100 may be set arbitrarily. For example, the number of pixels PX in the vertical direction and the horizontal direction may be different.

Returning to the description of FIG. Further, the calculation unit 33 calculates an area feature amount for each unit area UA based on the calculated edge feature amount for each cell 100 .

First, the calculation unit 33 calculates the average brightness for each unit area UA, and the average and variance of the edge strength of the cell 100 as area feature amounts. The calculation unit 33 also calculates the sum of the number of pair regions 200 and the edge strength as the region feature amount.

Here, a case of calculating the number of pair regions 200 and the sum of edge strengths will be described with reference to FIGS. 5 and 6. FIG. 5 and 6 are diagrams (part 3) and (part 4) showing the processing contents of the calculation unit 33. FIG.

5 shows the case where the two pair regions 200 do not share the cell 100, and FIG. 6 shows the case where the two pair regions 200 share the cell 100. As shown in FIG.

As shown in FIG. 5 , the calculation unit 33 scans the plurality of cells 100 arranged in the horizontal direction and the vertical direction of the unit area UA in the horizontal direction and the vertical direction to search for the pair area 200 . That is, the calculation unit 33 extracts, as the pair area 200, the cells 100 that are adjacent to each other and whose edge directions are opposite to each other among the cells 100 in the unit area UA.

Calculation unit 33 then calculates the number of extracted pair regions 200 and the sum of edge strengths in pair regions 200 . Note that, as shown in FIG. 5, the calculation unit 33 calculates the number of the pair regions 200 as two when, for example, the two pair regions 200 extracted do not share the cell 100, and the sum of the edge strengths is calculated as follows. It is calculated as a sum of the edge strengths of the four cells 100 included in the two pair regions 200 .

Further, as shown in FIG. 6 , if, for example, two paired regions 200 that are extracted share a cell 100, the calculation unit 33 calculates the number of paired regions 200 as two, and calculates the sum of the edge strengths as follows: It is calculated as a sum of the edge strengths of the three cells 100 included in the two pair regions 200 .

Note that the calculation unit 33 calculates representative values of two or more types of edge orientations for one cell 100 based on not only the above-described "upper, lower, left, and right 4 categories" but also, for example, the "diagonal 4 categories" angle classification. Assignment and area feature amount may be calculated. This point will be described with reference to FIGS. 7 and 8. FIG. 7 and 8 are diagrams (part 5) and (part 6) showing the processing contents of the calculation unit 33. FIG.

If the calculation unit 33 sets the "top, bottom, left, and right four categories" as the first angle classification and sets the representative value of the edge orientation based on this as the first representative value, then as shown in FIG. 2, and the representative value of the edge orientation based on this can be calculated as the second representative value.

In this case, the calculation unit 33 classifies the edge direction of the edge vector V of each pixel PX in the cell 100 from −180° to 180° into angle classification (4), which is four oblique directions at every 90°, by the second angle classification. to (7).

More specifically, the calculation unit 33 classifies the edge direction of the pixel PX into the angle category (4) when it is in the angle range of 0° or more and less than 90°, If it is, it is classified into angle classification (5), if it is in the angle range of -180 ° or more and less than -90 °, it is classified into angle classification (6), and in the angle range of -90 ° or more and less than 0 ° In some cases, it is classified into angle classification (7).

Then, in the same manner as shown in the lower part of FIG. 4, the calculation unit 33 generates a histogram with angle classifications (4) to (7) for each class for each cell 100. FIG. Then, when the frequency of the class with the highest frequency in the generated histogram is equal to or greater than a predetermined threshold value THa, the calculation unit 33 sets the angle classification corresponding to this class as the second representative value of the edge direction in the cell 100. calculate.

As a result, as shown in FIG. 8, two edge orientation representative values can be assigned to each cell 100 . Then, as shown in FIG. 8, when at least one of the edge-oriented first representative value and the edge-oriented second representative value is opposite to each other in adjacent cells 100, the calculation unit 33 are extracted as the pair region 200 .

That is, the calculation unit 33 calculates the first representative value and the second representative value of the edge orientation in each cell 100, thereby extracting the pair region 200 that could not be extracted with only one type of edge orientation. becomes.

For example, a pixel PX with an edge orientation of 140° and a pixel PX with an edge orientation of −40° are not reversed in the first angle range, but are reversed in the second angle range. It becomes possible to detect the change in the edge orientation at the higher accuracy.

Further, the calculation unit 33 calculates the number of intersections during pattern matching as the region feature amount. Here, a case of calculating the number of intersections during pattern matching will be described with reference to FIGS. 9 to 12. FIG.

9 to 12 are diagrams (No. 7) to (No. 10) showing the processing contents of the calculation unit 33. FIG. 7, 9, and 10 are assumed to be pattern portions having a predetermined edge feature amount in the captured image I. In FIGS.

The calculation unit 33 searches for a predetermined search pattern that matches a predetermined template using the edge direction of the calculated edge feature amount of the cell 100 . As shown in FIG. 9, the search directions are the horizontal direction and the vertical direction.

For example, the calculation unit 33 searches for a search pattern under the condition that "an opposite angle classification does not appear on both sides of the angle classification of interest". Specifically, when the target angle classification is angle classification (1) and the search is performed in the horizontal direction, as shown in FIG. Cell 100-2 of angle classification (1) adjacent to 100-1 is the starting position.

Then, when the array of angle classification (1) continues and a cell 100-4 of angle classification (0) "angle classification is not reversed" appears, a cell of angle classification (1) adjacent to the cell 100-4 appears. 100-3 is the end position. In this case, the match length is "8" in the example of FIG. Note that, when there is a match with the search pattern in this way, the calculation unit 33 holds the position and the match length.

Also, if there is a match in the search pattern shown in FIG. 10, a change in brightness between adjacent categories among the four angle categories will be seen at the start position and the end position.

Then, as shown in FIG. 11, when the matching search patterns intersect in the left-right direction and the up-down direction, the calculation unit 33 stores a horizontal match for each angle classification as stored information corresponding to the unit area UA corresponding to the intersection. Accumulate the product of the length and the upper and lower match lengths.

10 and 11, the calculation unit 33 cumulatively adds 5×8 in association with the angle classification (1) of the unit area, as shown in FIG. Also, although not shown, the calculation unit 33 also cumulatively adds the number of intersections in association with the angle classification (1) of the unit area.

Such matching processing is repeated, and the determination unit 34, which will be described later, determines, based on predetermined detection conditions, that, for example, three or more of the cumulative addition results associated with each angle classification (0) to (3) of the unit area UA are predetermined. is equal to or greater than the threshold value, the adhesion state of the unit area UA is determined to be "adhesion". Also, if the determination condition is not satisfied, it is determined as "non-adhesion". Note that the processing contents of the matching processing shown in FIGS. 9 to 12 are merely examples, and the processing contents are not limited.

Returning to the description of FIG. Then, the calculation unit 33 outputs the calculated area feature amount for each unit area UA to the determination unit 34 .

Based on the area feature amount calculated by the calculation unit 33, the determination unit 34 determines the adherence state of the attached matter for each unit area UA. The determination unit 34 also calculates the adhesion rate based on the determined adhesion state for each unit area UA, and determines the buried state of the lens of the camera 10 based on the adhesion rate.

Here, the determination unit 34 determines whether the adhesion state of each unit area UA is "adhesion" or "non-adhesion". Then, based on the determination result, the determination unit 34 calculates the area ratio of the unit area UA determined as "attached" in the predetermined target area of the captured image I, that is, the attachment rate.

Then, when the adhesion rate is equal to or higher than a certain value (for example, 40% or higher), the determination unit 34 sets the temporary determination flag to "1" as "buried". Further, when the adhesion rate is less than a certain value (for example, less than 30%), the determination unit 34 sets the provisional determination flag to "-1" as "not buried". Further, when the adhesion rate is other than the above or when the determination is difficult, the determining unit 34 sets the provisional determination flag to "0" as "keep".

Based on the determination history information 21, the determination unit 34 then determines the buried state. For example, when the above-mentioned 5/9 condition is established, the determination unit 34 reflects the corresponding flag value in the buried flag as the determination result, and notifies the various devices 50 of it. For example, if the flag value "1" satisfies the 5/9 condition, the determination unit 34 turns on the buried flag. Further, the determination unit 34 turns off the buried flag if the flag value "-1" satisfies the 5/9 condition, for example.

By the way, establishment of the definite condition for the buried state can be hastened by setting the initial value to the determination history information 21 at the time of IG ON as described above, but it can also be hastened by relaxing the definite condition itself. Such an example will be described with reference to FIG. 13 . FIG. 13 is a diagram showing the processing contents of the setting unit 32. As shown in FIG.

As shown in FIG. 13, it is assumed that the vehicle whose determination history information 21 continues with "1" is stopped. In such a case, it is assumed that the user who has been notified of "1=buried" has wiped off the adhering matter.

If the wiping has been performed, it is preferable that "not buried" is confirmed and notified early, and the determination history information 21 filled with "1" is cleared.

Therefore, in such a case, the setting unit 32 assumes that wiping will occur if the vehicle is notified, for example, that the vehicle is stopped via the vehicle speed sensor 30, and changes the 5/9 condition described above to the 3/5 condition ( 5 history, 3 or more).

As a result, after the change, the determining unit 34 can determine that "not buried" at the shortest third frame. That is, in the attached matter detection method according to the embodiment, when the vehicle is stopped, the confirmation condition is relaxed so that the establishment of the confirmation condition for the buried state determination result is hastened.

As a result, the adhering matter detection method according to the embodiment can improve the adhering matter detection performance.

Further, the setting unit 32 clears the judgment history information 21 to "0" if it is confirmed that the item is "not hidden" after the confirmation condition is relaxed. In addition, after such clearing, the setting unit 32 may restore the relaxed confirmation condition.

Note that even if the vehicle whose judgment history information 21 continues to be "1" is stopped, it is not always the case that the wiping is actually performed. However, even if the wiping is not performed, since the "buried" state is confirmed early due to relaxation of the confirmation condition, there is no problem in the process of determining the buried state.

Next, a processing procedure executed by the adhering matter detection device 1 according to the embodiment will be described with reference to FIG. 14 . FIG. 14 is a flow chart showing a processing procedure executed by the adhering matter detection device 1 according to the embodiment. Note that FIG. 14 shows the processing procedure for the captured image I for one frame.

As shown in FIG. 14, first, the acquisition unit 31 acquires the vehicle status (step S101). Here, if it is immediately after the IG is turned on (step S102, Yes), the setting unit 32 sets an initial value to the determination history information 21 so as to hasten the establishment of the determination condition (step S103), and the process proceeds to step S106.

Further, if it is not immediately after the IG is turned on (step S102, No) and the vehicle is stopped (step S104, Yes), the setting unit 32 relaxes the confirmation condition so that the establishment of the confirmation condition is hastened (step S105). ).

If the vehicle is not stopped (step S104, No), the process proceeds directly to step S106.

Then, the acquiring unit 31 acquires the captured image I (step S106). In addition, the acquisition unit 31 performs grayscaling processing and smoothing processing on the captured image I. FIG.

Subsequently, the calculation unit 33 calculates the edge feature amount for each cell 100 of the captured image I (step S107). Further, the calculation unit 33 calculates the area feature amount for each unit area UA based on the calculated edge feature amount (step S108).

Then, the determination unit 34 determines the adhesion state for each unit area UA based on the area feature amount calculated by the calculation unit 33 (step S109), and determines the buried state based on the determined adhesion state and the determination history information 21. Confirm (step S110).

Then, the determination unit 34 notifies the various devices 50 of the determination result (step S111), and ends the process. Although not shown in FIG. 14, when the buried state is determined based on the relaxed determination condition while the vehicle is stopped, the setting unit 32 clears the determination history information 21 to "0". becomes.

As described above, the adhering matter detection device 1 according to the embodiment includes the calculator 33 , the determiner 34 , and the setter 32 . The calculation unit 33 calculates, for each unit area UA made up of a predetermined number of pixels PX included in the captured image I, an area feature amount based on the edge vector of each pixel PX. The determination unit 34 provisionally determines whether or not the camera 10 is covered by the adhering matter based on the area feature amount, and also determines whether or not the determination history information 21, which is a predetermined number of past provisional determination histories including the most recent history, has a predetermined confirmation condition. is established, the above buried state is determined as a determination result. The setting unit 32 sets a predetermined initial value in the determination history information 21 so that the determination condition is quickly established immediately after the IG is turned on when the vehicle equipped with the camera 10 is turned on.

Therefore, according to the adhering matter detection device 1 according to the embodiment, it is possible to improve the adhering matter detection performance.

Further, when setting the initial value immediately after the IG is turned on, the setting unit 32 sets the initial value by a predetermined number from the latest to the past in the determination history information 21 .

Therefore, according to the adhering matter detection device 1 according to the embodiment, it is possible to improve the response performance of buried state determination immediately after the IG is turned on.

Further, when setting the initial value immediately after the IG is turned on, the setting unit 32 sets the flag value "1" indicating that the camera 10 is buried as the initial value.

Therefore, according to the attached matter detection device 1 according to the embodiment, it is possible to improve the response performance of the "buried" detection immediately after the IG is turned on.

In addition, the setting unit 32 relaxes the determination conditions so that the determination conditions are established more quickly when the vehicle is stopped.

Therefore, according to the adhering matter detection device 1 according to the embodiment, when it is assumed that the user has wiped off, it is possible to quickly confirm that the object is "not buried" and notify the user. Further, along with this, it becomes possible to clear the determination history information 21 at an early stage.

In the above-described embodiment, the case where two initial values are set in the determination history information 21 when the ignition is turned on is taken as an example. , or 3 or more.

Also, in the above-described embodiment, the determination history information 21 is the past nine frames including the most recent frame, but the number of frames is not limited as long as it is plural. Also, the 5/9 condition and the 3/5 condition described above are only examples, and do not limit the content of the determination condition.

Further, in the above-described embodiment, the case of classifying the angles into four directions by dividing −180° to 180° into angular ranges of 90° is shown, but the angular range is not limited to 90°, for example, every 60°. The angles may be classified into six directions divided by the angle range of .

Also, the width of each angle range may be different between the first angle classification and the second angle classification. For example, in the first angle classification, angles may be classified by 90°, and in the second angle classification, angles may be classified by 60°. Also, in the first angle classification and the second angle classification, the angle boundary of the angle range is shifted by 45°, but the shifted angle may be more than 45° or less than 45°.

Further effects and modifications can be easily derived by those skilled in the art. Therefore, the broader aspects of the invention are not limited to the specific details and representative embodiments so shown and described. Accordingly, various changes may be made without departing from the spirit or scope of the general inventive concept defined by the appended claims and equivalents thereof.

Reference Signs List 1 adhering matter detection device 2 storage unit 3 control unit 10 camera 21 determination history information 22 template information 23 threshold information 31 acquisition unit 32 setting unit 33 calculation unit 34 determination unit 50 various devices 100 cell 200 pair region I captured image PX pixel UA unit region

Claims (4)

a calculation unit that calculates an area feature amount based on an edge vector of each pixel for each unit area made up of a predetermined number of pixels included in a captured image;
Based on the area feature amount, the camera is tentatively determined to be buried due to adhering matter, and when a predetermined confirmation condition is satisfied in a predetermined number of past temporary determination histories including the most recent history, the corresponding buried state is determined. a determination unit for determining as a determination result;
a setting unit that sets a predetermined initial value to the temporary determination history immediately after the ignition is turned on when the vehicle equipped with the camera is turned on, so that the confirmation condition is quickly established ,
The setting unit
An attached matter detection device characterized in that, when the vehicle is stopped, the confirmation condition is relaxed so that the establishment of the confirmation condition is hastened . The setting unit
The deposit according to claim 1, wherein when the initial value is set immediately after the ignition is turned on, a predetermined number of the initial values are set from the latest to the past in the temporary determination history. detection device. The setting unit
3. The adhering matter detection device according to claim 1, wherein when the initial value is set immediately after the ignition is turned on, a flag value indicating that the camera is buried is set as the initial value. . a calculation step of calculating an area feature value based on an edge vector of each pixel for each unit area made up of a predetermined number of pixels included in the captured image;
Based on the area feature amount, the camera is tentatively determined to be buried due to adhering matter, and when a predetermined confirmation condition is satisfied in a predetermined number of past temporary determination histories including the most recent history, the corresponding buried state is determined. A judgment step of confirming as a confirmation result;
When the ignition of the vehicle equipped with the camera is turned on, immediately after the ignition is turned on, a predetermined initial value is set in the temporary determination history so that the establishment of the confirmation condition is hastened, and the vehicle is stopped. a setting step of relaxing the definite condition so that the establishment of the definite condition is hastened in some cases;
A deposit detection method , comprising :

Images