JP7463159B2 - Droplet Detector - Google Patents

Description

The present invention relates to a droplet detection device that detects droplets adhering to a lens.

Images captured by the onboard camera are displayed on a monitor inside the vehicle. In onboard cameras, dirt can adhere to the lens. Patent Document 1 discloses a lens cleaning device that removes cloudiness that has adhered to the lens of an onboard camera. This lens cleaning device calculates the degree of cloudiness on the lens surface based on the luminance gradient of the image generated by the onboard camera. The cloudiness that adheres to the lens is, for example, dust stirred up when the vehicle travels on an unpaved road. This lens cleaning device changes the spray form of cleaning liquid or compressed air based on the calculated degree of cloudiness.

JP 2014-27539 A

The lens cleaning device described in Patent Document 1 may not be able to remove water droplets that have adhered to the lens. This is because if the water droplets contain very few impurities, they do not become cloudy.

In addition, if water droplets or other liquid droplets adhere to the lens, the environment around the vehicle may be distorted in the image captured by the camera. Therefore, it is desirable to remove the liquid droplets from the lens as soon as they adhere to the lens.

In view of the above problems, the present invention aims to provide a droplet detection device that can detect droplets adhering to a lens.

In order to solve the above problems, a first invention is a droplet detection device comprising an image acquisition unit, a candidate region identification unit, and a droplet determination unit. The image acquisition unit acquires a plurality of captured images generated by a camera at different times . The candidate region identification unit identifies a time change in the level of a video signal in a predetermined direction in the captured images acquired by the image acquisition unit for each predetermined unit region, and identifies a droplet candidate region by comparing the identified time change in the level with a preset droplet candidate threshold. The droplet determination unit determines whether or not a droplet is attached to the camera lens based on the shape of the droplet candidate region identified by the candidate region identification unit.

According to the first invention, the droplet detection device can detect droplets adhering to a lens.

A second invention is the first invention, in which the candidate region identification unit includes a statistical value identification unit, a difference calculation unit, and a region determination unit. The statistical value identification unit identifies a maximum value and a minimum value of a level of a video signal in a predetermined direction in one unit region. The difference calculation unit calculates an absolute difference value between the maximum value and the minimum value identified by the statistical value identification unit as a time change in the level in one unit region. The region determination unit determines whether or not the one unit region is included in the droplet candidate region based on a result of comparing the absolute difference value calculated by the difference calculation unit with a droplet candidate threshold.

According to the second invention, it is possible to easily identify the level change in one unit area.

The third invention is the first or second invention, and the droplet determination unit includes a contour identification unit, a distance calculation unit, and a distance determination unit. The contour identification unit creates an array of contour pixels that form the contour of the droplet candidate region. When the number of contour pixels is N, the distance calculation unit calculates the distance between the kth pixel (k is a natural number between 1 and N/2) and the k+N/2th pixel in the array of contour pixels created by the contour identification unit. The distance determination unit determines whether or not a droplet is attached to the camera lens based on the variation in the distance calculated by the distance calculation unit.

According to the third invention, whether or not a droplet is attached to the camera lens is determined by taking into account the shape of the droplet attached to the lens, thereby improving the accuracy of droplet detection.

A fourth invention is any one of the first to third inventions, in which the image acquisition unit includes a pixel line acquisition unit, a pixel line determination unit, and a synthesis unit. The pixel line acquisition unit acquires a first pixel line in the horizontal direction from a first captured image generated by the camera at a first time, and acquires a second pixel line in the horizontal direction from a second captured image generated by the camera at a second time. When the pixel line acquisition unit acquires the first pixel line from the first captured image, the pixel line determination unit determines a second pixel line based on a position of the first pixel line, and notifies the pixel line acquisition unit of the determined second pixel line. The synthesis unit generates a synthesis image including the first pixel line and the second pixel line acquired by the pixel line acquisition unit. The candidate region identification unit identifies a droplet candidate region from the synthesis image generated by the synthesis unit.

According to the fourth invention, a composite image generated from the first captured image and the second captured image is used to determine whether or not droplets have adhered to the lens, so the amount of information acquired from one captured image can be reduced. As a result, droplets can be detected while preventing the processing load of the droplet detection device from temporarily increasing each time a captured image is acquired.

A fifth invention is a droplet detection method for detecting droplets adhering to a lens, comprising steps a), b), and c). In step a), a plurality of captured images generated by a camera at different times are acquired. In step b), a change over time in the level of a video signal in a predetermined direction in the acquired captured images is identified for each predetermined unit area, and a droplet candidate area is identified by comparing the change over time in the identified level with a preset droplet candidate threshold . In step c), it is determined whether or not a droplet is adhering to the camera lens based on the shape of the identified droplet candidate area.

The fifth invention is used in the first invention.

The present invention makes it possible to detect droplets adhering to a lens.

1 is a functional block diagram showing a configuration of a droplet removal system according to an embodiment of the present invention. FIG. 2 is a functional block diagram showing a configuration of the droplet detection device shown in FIG. 1 . 3 is a functional block diagram showing a configuration of a candidate region identifying unit shown in FIG. 2 . 3 is a functional block diagram showing the configuration of a droplet determination unit shown in FIG. 2 . 3 is a functional block diagram showing a configuration of an image acquisition unit shown in FIG. 2 . 2 is a flowchart showing the operation of the liquid droplet detection device shown in FIG. 1; FIG. 3 is a diagram showing the configuration of pixel lines of a captured image acquired by the image acquisition unit shown in FIG. 1; FIG. 7 is a diagram for explaining the composite image generating process (step S1) shown in FIG. 6; 8 is a graph showing an example of a change over time in the level of a video signal included in the composite image shown in FIG. 7 . 3 is a diagram showing an example of a droplet candidate region included in a composite image generated by the image acquisition unit shown in FIG. 2 . 3 is a flowchart showing the operation of a candidate region identifying unit 5 shown in FIG. 2 . FIG. 10 is an enlarged view of a droplet candidate region 50a shown in FIG. 9. 10 is a diagram showing an array generated from contour pixels of the droplet candidate region shown in FIG. 9 . FIG. 10 is an enlarged view of a droplet candidate region 50b shown in FIG. 9. 7 is a flowchart showing the droplet determination unit process (step S3) shown in FIG. 6. 10 is a flowchart showing an operation of a liquid droplet detection device according to a modified example of an embodiment of the present invention. FIG. 2 is a diagram showing a CPU bus configuration.

The following describes in detail an embodiment of the present invention with reference to the drawings. The same or corresponding parts in the drawings are designated by the same reference numerals and their description will not be repeated.

[1. Configuration of droplet removal system 100]
1 is a functional block diagram showing the configuration of a droplet removal system 100 according to an embodiment of the present invention. The droplet removal system 100 includes a camera C, a droplet detection device 2, and a droplet removal device 3. The droplet removal system 100 removes droplets such as water droplets adhering to the lens of the camera C.

Camera C is attached, for example, around the vehicle. The vehicle is, for example, an automobile. Camera C captures the surroundings of the vehicle to generate captured image 1. Captured image 1 is an analog video signal, for example a composite video signal. Camera C outputs the generated captured image 1 to a display unit (not shown) installed inside the vehicle cabin. The display unit is, for example, a monitor of a car navigation device. The driver and passengers of the vehicle can check the surroundings of the vehicle from captured image 1 displayed on the display unit.

The droplet detection device 2 receives the captured image 1 generated by the camera C, and determines whether or not droplets are attached to the lens of the camera C based on the received captured image 1. If the droplet detection device 2 determines that droplets are attached to the lens of the camera C, it outputs a drive signal 2A for driving the droplet removal device 3.

When the droplet removal device 3 receives the drive signal 2A from the droplet detection device 2, it sprays, for example, compressed air toward the lens of the camera C. By spraying the compressed air toward the lens of the camera C, droplets adhering to the lens are removed.

[2.1. Configuration of droplet detection device 2]
Fig. 2 is a functional block diagram showing a configuration of the liquid droplet detection device 2 shown in Fig. 1. Referring to Fig. 2, the liquid droplet detection device 2 includes an image acquisition unit 4, a candidate region specification unit 5, and a liquid droplet determination unit 6.

The image acquisition unit 4 acquires the captured image 1 generated by the camera C. Specifically, the image acquisition unit 4 generates a composite image 4A from a plurality of captured images 1 generated at different times, and outputs the generated composite image 4A to the candidate area identification unit 5. The generation of the composite image 4A will be described later.

The candidate area identification unit 5 receives the composite image 4A from the image acquisition unit 4, and identifies the level change of the video signal in a predetermined direction in the received composite image 4A for each predetermined unit area. In this embodiment, the predetermined direction is the horizontal scanning direction of the composite image 4A. The predetermined unit area is a pixel. The candidate area identification unit 5 identifies a droplet candidate area 50 in the composite image 4A based on the identified level change. Details of the video signal level change and the droplet candidate area 50 will be described later.

The droplet determination unit 6 determines whether or not a droplet is attached to the lens of the camera C based on the shape of the droplet candidate region 50 identified by the candidate region identification unit 5. If the droplet determination unit 6 determines that a droplet is attached to the lens of the camera C, it outputs a drive signal 2A to the droplet removal device 3.

[2.2. Configuration of image acquisition unit 4]
3, image acquisition unit 4 includes a pixel line acquisition unit 7, a pixel line determination unit 8, and a synthesis unit 9. In the following description, a line of pixels aligned in the horizontal scanning direction will be referred to as a "pixel line".

The pixel line acquisition unit 7 acquires a first horizontal pixel line L1 from a captured image 1A generated by the camera C at a first time, and acquires a second horizontal pixel line L2 from a captured image 1B generated by the camera C at a second time. The second time is a time later than the first time.

In this embodiment, the first pixel line L1 is the odd-numbered pixel line counting from the top of the pixel lines in the captured image 1A. The second pixel line L2 is the even-numbered pixel line counting from the top of the pixel lines in the captured image 1B.

When the pixel line acquisition unit 7 acquires a first pixel line, it outputs line position information 71 indicating the position of the acquired first pixel line L1 to the pixel line determination unit 8. When the pixel line acquisition unit 7 acquires a second pixel line L2, it outputs line position information 71 indicating the position of the acquired second pixel line L2 to the pixel line determination unit 8.

When the pixel line acquisition unit 7 receives line designation information 72 from the pixel line determination unit 8, which designates the next pixel line to be acquired, it acquires the pixel line designated by the received line designation information 72 from the newly acquired captured image 1. Each time the pixel line acquisition unit 7 acquires each of the first pixel line L1 and the second pixel line L2, it outputs the acquired first pixel line L1 and second pixel line L2 to the synthesis unit 9.

When the pixel line determination unit 8 receives line position information 71 from the pixel line acquisition unit 7, it determines the next pixel line to be acquired based on the position of the pixel line indicated by the received line position information 71. Specifically, when the line position information 71 indicates the first pixel line L1, the pixel line determination unit 8 determines that the pixel line L2 is to be acquired next. When the line position information 71 indicates the second pixel line L2, the pixel line determination unit 8 determines that the pixel line L1 is to be acquired next. The pixel line determination unit 8 outputs line designation information 72 indicating the determined pixel line to the pixel line acquisition unit 7.

The synthesis unit 9 receives the first pixel line L1 and the second pixel line L2 from the pixel line acquisition unit 7, and generates a synthetic image 4A including the received first pixel line L1 and second pixel line L2. The synthesis unit 9 outputs the generated synthetic image 4A to the candidate area identification unit 5.

[2.3. Configuration of candidate region identification unit 5]
With reference to FIG. 4, the candidate region identifying unit 5 includes a statistical value identifying unit 10 , a difference calculation unit 11 , and a region determination unit 12 .

The statistical value determination unit 10 receives the composite image 4A from the image acquisition unit 4, and determines the statistical value of the pixel signal in one unit area contained in the received composite image 4A. The unit area is a pixel. The statistical value is the maximum and minimum values of the image signal level in the horizontal scanning direction.

Specifically, the statistical value determination unit 10 divides a signal corresponding to one pixel line included in the composite image 4A into pixel units, and determines the maximum and minimum values of the signal level divided into pixel units. The maximum and minimum values of the signal level are determined for all pixels included in the composite image 4A. The statistical value determination unit 10 outputs statistical value information 101 including the maximum and minimum values of each pixel included in the composite image 4A to the difference calculation unit 11.

The difference calculation unit 11 calculates the absolute difference between the maximum and minimum values in one unit area identified by the statistical value identification unit 10 as the maximum level change amount in one unit area. Specifically, the difference calculation unit 11 receives statistical value information 101 from the statistical value identification unit 10, and uses the received statistical value information 101 to calculate the maximum level change amount of each pixel included in the composite image 4A. The maximum level change amount is obtained by subtracting the minimum signal level value from the maximum signal level value in one pixel. The difference calculation unit 11 outputs level change information 111 recording the maximum level change amount of each pixel included in the composite image 4A to the area determination unit 12.

The area determination unit 12 compares the absolute difference value of a unit area identified by the difference calculation unit 11 with a predetermined droplet candidate threshold, and determines whether or not the unit area is included in the droplet candidate area based on the comparison result. The predetermined droplet candidate threshold is determined based on the maximum level change amount obtained when the unit area corresponds to a droplet.

Specifically, the area determination unit 12 receives the level change information 111 from the difference calculation unit 11. The area determination unit 12 compares the maximum level change amount of each pixel recorded in the received level change information 111 with the droplet candidate threshold. Based on the comparison result, the area determination unit 12 determines whether each pixel included in the composite image 4A is included in the droplet candidate region. The area determination unit 12 identifies the droplet candidate region 50 in the composite image 4A based on the determination result of whether each pixel in the composite image 4A is included in the droplet candidate region.

[2.4. Configuration of droplet determination unit 6]
With reference to FIG. 5, droplet determination unit 6 includes an outline specifying unit 13 , a distance calculation unit 14 , and a distance determination unit 15 .

The contour identification unit 13 receives the droplet candidate region 50 from the candidate region identification unit 5, and identifies the contour pixels that form the contour of the received droplet candidate region 50. The contour identification unit 13 creates an array 131 of the identified contour pixels. The contour identification unit 13 outputs the created array 131 of contour pixels to the distance calculation unit 14.

The distance calculation unit 14 receives the array 131 of contour pixels from the contour identification unit 13, and calculates the distance between two pixels that satisfy a predetermined condition among the contour pixels included in the received array 131 as the provisional opposing distance 141. Specifically, when the number of contour pixels is N, the distance calculation unit 14 identifies the kth pixel and the (k+N/2)th pixel in the array 131. The distance calculation unit 14 calculates the distance between the two identified pixels as the provisional opposing distance 141. The kth pixel and the (k+N/2)th pixel are pixels that are assumed to be opposing when it is assumed that the contour pixels have a circular shape. k is a natural number between 1 and N/2. Since there are N/2 possible pixel combinations, the distance calculation unit 14 repeats the calculation of the provisional opposing distance 141 N/2 times.

The distance determination unit 15 receives the provisional facing distance 141 from the distance calculation unit 14, and determines whether or not droplets are attached to the lens of the camera C based on the statistical variation of the received provisional facing distance 141. If droplets are attached to the lens of the camera C, the distance determination unit 15 outputs a drive signal 2A to the droplet removal device 3 to cause the droplet removal device 3 to remove the droplets attached to the lens of the camera C.

[3. motion]
[3.1. Operation of Droplet Detection Device 2]
Fig. 6 is a flow chart showing the operation of the liquid droplet detection device 2 shown in Fig. 1. When the camera C starts generating the photographed image 1, the liquid droplet detection device 2 starts the process shown in Fig. 6.

Referring to FIG. 6, the image acquisition unit 4 generates a composite image 4A using multiple captured images 1 generated by the camera C (step S1). The candidate area identification unit 5 identifies a droplet candidate area 50 in the composite image 4A generated in step S1 (step S2). The droplet determination unit 6 determines whether or not a droplet is attached to the lens of the camera C based on the shape of the droplet candidate area 50 identified in step S2 (step S3). If the droplet determination unit 6 determines that a droplet is attached to the lens of the camera C, it drives the droplet removal device 3.

The droplet detection device 2 determines whether the camera C has finished generating the captured image 1 (step S4). If the camera C is continuing to generate the captured image 1 (Yes in step S4), the droplet detection device 2 returns to step S1. If the camera C has finished generating the captured image 1 (No in step S4), the droplet detection device 2 ends the process shown in FIG. 6.

[3.2. Generation of composite image 4A (step S1)]
Fig. 7 is a diagram for explaining the composite image generating process (step S1) shown in Fig. 6. In the example shown in Fig. 7, a composite image 4A is generated from photographed images 1A and 1B. The size of the composite image 4A is the same as the size of the photographed images 1A and 1B. The photographed image 1A is generated by the camera C at time t1. The photographed image 1B is generated by the camera C at time t2, which is after time t1. The photographed image 1B is the next frame of the photographed image 1A.

The captured image 1A includes pixel lines La1 to La10. Pixel line La1 is located at the top of the captured image 1A. Pixel line La2 is adjacent to and below pixel line La1. Pixel line La10 is located at the bottom of the captured image 1A.

Captured image 1B includes pixel lines Lb1 to Lb10. The positions of pixel lines Lb1 to Lb10 correspond to the positions of pixel lines La1 to La10.

When the pixel line acquisition unit 7 receives the captured image 1A from the camera C, it acquires the first pixel line from the captured image 1A. The first pixel line is the odd-numbered pixel line counting from the top of the captured image 1. Specifically, the first pixel line in the captured image 1A is pixel line La1, La3, ..., La9. The pixel line acquisition unit 7 outputs line position information 71 indicating the positions of the acquired pixel lines La1, La3, ..., La9 to the pixel line determination unit 8. The pixel line acquisition unit 7 waits until it receives the captured image 1B from the camera C.

The pixel line determination unit 8 determines the pixel line to be acquired from the captured image 1B based on the line position information 71 indicating the first pixel line. Specifically, because the line position information 71 indicates the first pixel line, the pixel line determination unit 8 determines to acquire the second pixel line from the captured image 1B. The second pixel line is an even-numbered pixel line in the captured image 1B. In the captured image 1B, the second pixel line is pixel line Lb2, Lb4, ..., Lb10. The pixel line determination unit 8 outputs line designation information 72 indicating pixel lines Lb2, Lb4, ..., Lb10 to the pixel line acquisition unit 7.

When the pixel line acquisition unit 7 receives the captured image 1B from the camera C, it acquires pixel lines Lb2, Lb4, ..., L10 from the captured image 1B based on line designation information 72 that designates the second pixel line.

The synthesis unit 9 receives the first pixel line and the second pixel line from the pixel line acquisition unit 7, and generates a synthetic image 4A including the received first pixel line L1 and second pixel line L2. Specifically, the synthesis unit 9 arranges the pixel lines La1, La3, ..., La9 acquired from the photographed image 1A on odd-numbered pixel lines in the synthetic image 4A. The synthesis unit 9 arranges the pixel lines Lb2, Lb4, ..., Lb10 acquired from the photographed image 1B on even-numbered pixel lines in the synthetic image 4A.

In other words, the synthesis unit 9 places the first pixel line and the second pixel line at the same positions in the synthetic image 4A as the positions of the pixel lines in the captured images 1A and 1B. As shown in FIG. 7, the pixel lines of the captured image 1A and the pixel lines of the captured image 1B are alternately arranged in the synthetic image 4A.

After generating a composite image 4A from the captured images 1A and 1B, the image acquisition unit 4 repeats the above process. The frame interval of the composite image 4A is twice the frame interval of the captured image 1.

The image acquisition unit 4 may generate a composite image using three or more captured images 1. Even in this case, the position of the pixel line acquired by the synthesis unit 9 from the captured images 1 is the same as the position of the pixel line arranged in the composite image 4A. When the composite image 4A is generated from six captured images 1A, the frame interval of the composite image 4A is six times the frame interval of the captured images 1.

[3.3. Identification of droplet candidate regions]
The candidate region identifying unit 5 determines whether or not each pixel included in the composite image 4A is included in a droplet candidate region. In this way, it is determined whether or not each pixel included in the composite image 4A is included in a droplet candidate region.

[3.3.1. Acquisition of signals from each pixel]
The statistical value specification unit 10 acquires the composite image 4A from the composition unit 9, and acquires the statistical value of each pixel included in the acquired composite image 4A. Specifically, the statistical value is a maximum value and a minimum value.

When identifying the maximum and minimum levels of each pixel included in the composite image 4A, the statistical value identification unit 10 acquires the signal of each pixel included in the composite image 4A. The signal of each pixel included in the composite image 4A is an analog video signal. This is because the composite image 4A is generated from the captured image 1, which is an analog video signal. Below, the operation of the statistical value identification unit 10 that acquires the signal of each pixel will be described using pixel lines La1 and Lb2 shown in Figure 7 as an example.

Note that pixel line La1 includes pixels P1 to P10. Pixel P1 is located at the left end of pixel line La1. Pixel Pm is adjacent to pixel Pm+1 on the left. m is an integer between 1 and R-1. R is the number of pixels included in pixel line La1, which is 10 in the example shown in FIG. 7. Pixel P10 is located at the right end of pixel line La1.

When the statistical value identification unit 10 acquires the vertical synchronization signal of the composite image 4A, it starts acquiring the signal of pixel line La1 and starts counting the horizontal synchronization signal of the received composite image 4A. Pixel line La1 corresponds to the signal in the video signal of the composite image 4A during the period from the time when the vertical synchronization signal is acquired to the time of the first horizontal synchronization signal. Therefore, the statistical value identification unit 10 stops acquiring the signal of pixel line La1 at the time when the first horizontal synchronization signal is acquired. The statistical value identification unit 10 acquires signals corresponding to each of the pixels P1 to P10 included in pixel line La1 by dividing the video signal of pixel line La1 by pixel.

Figure 8 is a graph showing an example of the change over time in the level of the video signal contained in the composite image 4A. With reference to Figure 8, time t0 is the time of the vertical synchronization signal of the composite image 4A. Time t3 is the time of the first horizontal synchronization signal in the composite image 4A. Therefore, the period TL from time t0 to time t3 corresponds to the video signal of pixel line La1.

The statistical value identification unit 10 acquires the signals of pixels P1 to P10 by dividing the signal of pixel line La1 by the time per pixel. The time per pixel is obtained by dividing the period TL, which is one horizontal scanning period, by the number of pixels in the horizontal scanning direction. The statistical value identification unit 10 acquires the signal for period Ta, from time t0 to time t1, as the signal of pixel P1. The statistical value identification unit 10 acquires the signal for period Tb, from time t1 to time t2, as the signal of pixel P2. In Figure 8, the signals of pixels P3 to P10 are not shown.

The statistical value determination unit 10 acquires the signal from the time of the first horizontal synchronization signal to the time of the second horizontal synchronization signal in the composite image 4A as pixel line Lb2. The signals of the other pixel lines included in the composite image 4A are also acquired in the same manner as pixel line Lb2.

[3.3.2. Obtaining Statistics]
The statistical value determination unit 10 selects one pixel from the pixels included in the composite image 4A, and obtains the maximum and minimum values of the signal level of the selected pixel. In the following description of the operation of the droplet detection device 2, the operation of the statistical value determination unit 10 will be described with reference to FIG.

The statistical value determination unit 10 determines the maximum value A1 and the minimum value A2 as the statistical values of pixel P1. This is because the period Ta corresponds to pixel P1. The statistical value determination unit 10 outputs statistical value information 101 that records the maximum and minimum values of pixel P1 to the difference calculation unit 11.

When pixel P2 is selected, the statistical value identification unit 10 identifies a maximum value A3 and a minimum value A4 as statistical values of the signal level corresponding to pixel P2. This is because period Tb corresponds to pixel P2. The statistical value identification unit 10 identifies statistical values of other pixels included in the composite image 4A, similar to pixels P1 and P2.

[3.3.3. Calculation of maximum level change amount]
The difference calculation unit 11 calculates the maximum amount of change in level of the pixel P1 based on the statistical value information 101 of the pixel P1 received from the statistical value specification unit 10. Hereinafter, the operation of the difference calculation unit 11 will be described using pixels P1 and P2 as an example.

The difference calculation unit 11 acquires the maximum value A1 and minimum value A2 of pixel P1 from the statistical value information 101. The difference calculation unit 11 calculates the absolute difference between the acquired maximum value A1 and minimum value A2 as the maximum level change amount of pixel P1. The difference calculation unit 11 generates level change information 111 that links the maximum level change amount of pixel P1 with the position of pixel P1. Note that the maximum level change amount of pixel P2 is the absolute difference value between the maximum value A3 and minimum value A4. The statistical value identification unit 10 calculates the maximum level change amounts of other pixels included in the composite image 4A in the same way as pixels P1 and P2.

3.3.4. Comparison with Droplet Candidate Threshold
The region determination unit 12 determines whether or not each pixel included in the composite image 4A is included in a droplet candidate region, based on the maximum level change amount recorded in the level change information 111. The operation of the region determination unit 12 will be described below using pixel P1 as an example.

The area determination unit 12 obtains the maximum level change amount of pixel P1 from the level change information 111 received from the difference calculation unit 11, and compares the obtained maximum level change amount with a preset droplet candidate threshold. If the maximum level change amount of pixel P1 is smaller than the droplet candidate threshold, the area determination unit 12 determines that pixel P1 is included in the droplet candidate area. This is because when a droplet adheres to the lens of camera C, the maximum level change amount of the signal of the pixel corresponding to the position where the droplet adheres tends to be small. If the maximum level change amount of pixel P1 exceeds the droplet candidate threshold, the area determination unit 12 determines that pixel P1 is included in the background area. Note that the background area is an area in the composite image where no droplets are attached.

After determining whether pixel P1 is included in the droplet candidate region, the statistical value acquisition unit 10 selects a new pixel (e.g., pixel P2). As with pixel P1, it is determined whether the selected pixel is included in the droplet candidate region based on the maximum level change amount of the selected pixel.

[3.3.5. Determination of droplet candidate regions]
The region determination unit 12 identifies a droplet candidate region in the composite image 4A based on the result of determining whether each pixel included in the composite image 4A is included in the droplet candidate region. In the following description, a pixel determined to be included in a droplet candidate region will be referred to as a "candidate pixel."

Figure 9 is a diagram showing an example of a droplet candidate region included in a composite image 4A generated by the image acquisition unit 4. Using Figure 9 as an example, the operation of the region determination unit 12 that determines the droplet candidate region will be explained.

The region determination unit 12 identifies a region where candidate pixels are consecutive by a labeling process, and determines the identified region as a droplet candidate region. In FIG. 9, the region where candidate pixels are consecutive is region 50a surrounded by pixels Pa1 to Pa18, and region 50b consisting of pixels Pb1 to Pb12. The region determination unit 12 determines each of regions 50a and 50b as a droplet candidate region. Region 50a corresponds to a raindrop adhering to the lens of camera C. Region 50b corresponds to a white line drawn on the road. In region 50a shown in FIG. 9, pixels other than pixels Pa1 to Pa18 are omitted from the display. The same applies to FIG. 11 and FIG. 13 described below.

In the following description, region 50a will be referred to as "droplet candidate region 50a" and region 50b will be referred to as "droplet candidate region 50b." Details will be described later, but if a group of pixels contained in one droplet candidate region meets a predetermined condition, the one droplet candidate region is determined to be a raindrop on the windshield.

[3.3.6. Operation of candidate region identification unit 5]
Fig. 10 is a flowchart showing the operation of the candidate region identifying section 5 shown in Fig. 2. When the candidate region identifying section 5 receives the composite image 4A from the image acquisition section 4, it starts the process shown in Fig. 10.

Referring to FIG. 10, the statistical value determination unit 10 acquires the signal of each pixel included in the composite image 4A (step S201).

The statistical value determination unit 10 selects one of the pixels contained in the composite image 4A (step S202).

The statistical value determination unit 10 determines the maximum and minimum signal levels of the pixels selected in step S202 (step S203).

The difference calculation unit 11 calculates the maximum change in the signal level of the pixel selected in step S202 based on the maximum and minimum values identified in step S203 (step S204).

The region determination unit 12 compares the maximum level change amount calculated in step S204 with the droplet candidate threshold (step S205). If the maximum level change amount is smaller than the droplet candidate threshold (Yes in step S205), the region determination unit 12 determines that the pixel selected in step 202 is included in the droplet candidate region (step S206). If the maximum level change amount exceeds the droplet candidate threshold (No in step S205), the region determination unit 12 determines that the pixel selected in step 202 is included in the background region (step S207).

When all pixels in the composite image 4A have been selected (Yes in step S208), the region determination unit 12 determines the region of consecutive candidate pixels as the droplet candidate region 50 (step S209). This causes the candidate region identification unit 5 to end the process shown in FIG. 10. When there are pixels that have not been selected in step S202 (No in step S208), the candidate region identification unit 5 returns to step S202.

When the candidate region identification unit 5 is newly executed in step S202, it selects pixels for which it has not yet been determined whether they are included in the droplet candidate region 50. The order in which the pixels are selected may be random or may be set in advance.

[3.4. Droplet Judgment]
The droplet determination unit 6 determines whether or not a droplet is attached to the lens of the camera C based on the shape of the droplet candidate region 50 identified by the candidate region identification unit 5. When it is determined that droplets are attached to the lens of the camera C, the droplet removing device 3 is driven.

[3.4.1. Identifying Contour Pixels]
The contour identifying section 13 receives the droplet candidate regions 50 from the region determining section 12, selects one droplet candidate region from the received droplet candidate regions 50, and identifies contour pixels that form the contour of the selected droplet candidate region.

Below, the operation of the contour identification unit 13 will be explained using an example in which the selected droplet candidate region is the droplet candidate region 50a shown in Figure 9.

Figure 11 is an enlarged view of the droplet candidate region 50a shown in Figure 9. With reference to Figure 11, the contour identification unit 13 identifies pixels Pa1 to Pa18 from among the pixels included in the droplet candidate region 50a as contour pixels. Specifically, the contour identification unit 13 selects one candidate pixel included in the droplet candidate region 50a. If at least one vertex out of all vertices of the selected candidate pixel matches a vertex of a pixel that is not a candidate pixel, the contour identification unit 13 determines that the selected candidate pixel is a contour pixel. Alternatively, if the selected candidate pixel has at least one side that is not in contact with other candidate pixels included in the droplet candidate region 50a, the contour identification unit 13 may determine that the selected candidate pixel is a contour pixel.

3.4.2. Comparison with Exclusion Threshold
The contour specifying unit 13 compares the number of pixels of the specified contour pixels Pa1 to Pa18 with a preset exclusion threshold. The exclusion threshold is, for example, 8. As a result of the comparison, since the number of pixels of the contour pixels Pa1 to Pa18 is equal to or greater than the exclusion threshold, the contour specifying unit 13 continues to judge the droplet candidate region 50a including the contour pixels Pa1 to Pa18. If the number of contour pixels included in the selected droplet candidate region 50 is smaller than the exclusion threshold, the contour specifying unit 13 ends the judgment of the selected droplet candidate region 50. This makes it possible to exclude droplets that adhere to the lens of the camera C that are large enough not to affect the user's visibility. In addition, it is possible to improve the accuracy of judging whether or not the droplet candidate region is substantially circular.

[3.4.3. Creating an array of contour pixels]
11, contour identification unit 13 selects contour pixel Pa1 as the first pixel of the array, and determines the element number of the contour pixel array to be "1". Contour identification unit 13 identifies contour pixels in counterclockwise order starting from the selected contour pixel Pa1, and assigns array element numbers to the identified contour pixels. The array element numbers are assigned to each contour pixel. Contour identification unit 13 generates array 131 according to the element numbers assigned to each contour pixel. The number of elements in array 131 corresponds to the number of contour pixels.

Figure 12 shows an array 131 generated from the contour pixels of the droplet candidate region 50a shown in Figure 9. The numbers placed under each pixel in Figure 12 are element numbers of the array 131.

[3.4.4. Shape Judgment]
The droplet that adheres to the lens of the camera C becomes approximately circular due to the surface tension of the droplet. Therefore, when the droplet candidate region 50a is a droplet, the following opposing pixels located on the outline of the droplet are The line segment connecting two pixels that satisfy the condition corresponds to the diameter of a circle. The opposing condition is that the pixels located on the contour of the droplet face each other across the center of gravity of the droplet candidate region 50a. Even if the pair of pixels that satisfies the facing condition is changed, the line segment connecting the two pixels included in the changed pair still corresponds to the diameter of the circle. In the case of droplets, the variance in the distance between two opposing pixels is expected to be small.

The distance calculation unit 14 identifies two pixels that are expected to face each other if the droplet candidate region 50a is a droplet, among the pixels included in the droplet candidate region 50a. The distance calculation unit 14 calculates the distance between the two identified pixels as the provisional facing distance 141. This is explained in detail below.

Referring to Figures 11 and 12, when the droplet candidate region 50a is a droplet, the contour pixels Pa1 to Pa18 are evenly arranged along the contour of the droplet. The pixel furthest from the kth pixel is the (k+2/N)th pixel. N is the number of contour pixels, and k is a natural number between 1 and (2/N). The distance calculation unit 14 identifies a pixel pair including the kth pixel and the (k+2/N)th pixel. The distance calculation unit 14 calculates the distance between the two pixels included in the identified pixel pair.

In FIG. 12, N is 18. Therefore, when k=1, the contour pixels identified by the distance calculation unit 14 are pixels Pa1 and Pa10. The distance calculation unit 14 calculates the distance Mk1 from pixel Pa1 to pixel Pa10 as the provisional opposing distance 141. When k=2, the contour pixels identified are pixels Pa2 and Pa11. The distance calculation unit 14 calculates the distance Mk2 from pixel Pa2 to pixel Pa11 as the provisional opposing distance 141. After that, the distance calculation unit 14 increments k. The distance calculation unit 14 repeats the process of calculating the provisional opposing distance 141 until k reaches 9.

The distance determination unit 15 calculates the variance of all the provisional facing distances 141 calculated by the distance calculation unit 14. Note that the distance determination unit 15 does not have to use the variance as long as it calculates the statistical variation of all the calculated provisional facing distances 141. The distance determination unit 15 compares the calculated variance with a preset variance threshold, and determines whether the droplet candidate region 50a is a droplet or not based on the comparison result. The variance threshold is calculated in advance using droplets attached to the camera lens.

As shown in FIG. 12, the droplet candidate region 50a corresponds to a raindrop, so the variance calculated from the droplet candidate region 50a is lower than the variance threshold. The distance determination unit 15 determines that the droplet candidate region 50a is a droplet because it has a shape close to a circle. In other words, the droplet candidate region 50a is determined to be a droplet region.

The following describes how to determine the shape of the droplet candidate region 50b. Figure 13 is an enlarged view of the droplet candidate region 50b shown in Figure 9.

Referring to FIG. 13, the distance calculation unit 14 identifies two pixels that are assumed to face each other if the droplet candidate region 50b is a droplet, from among the contour pixels Pb1 to Pb12 contained in the droplet candidate region 50b. For example, the distance calculation unit 14 identifies contour pixels Pb1 and Pb7, and calculates the distance Nk1 from contour pixel Pb1 to contour pixel Pb7. The distance calculation unit 14 identifies contour pixels Pb3 and Pb9, and calculates the distance Nk2 from contour pixel Pb3 to contour pixel Pb9.

As described above, droplet candidate region 50b corresponds to a white line painted on the road and is not circular. Because distance Nk2 is several times greater than distance Nk1, the variance calculated from droplet candidate region 50b is greater than or equal to the variance threshold. Because droplet candidate region 50b is not a droplet, distance determination unit 15 determines that droplet candidate region 50b is a background region.

[3.4.5. Operation of droplet determination unit 6]
Fig. 14 is a flow chart showing the droplet determination unit process (step S3) shown in Fig. 6. When the droplet determination unit 6 receives a droplet candidate region included in the composite image 4A from the candidate region identification unit 5, it starts the process shown in Fig. 14.

Referring to FIG. 14, the contour identification unit 13 selects one droplet candidate region from the droplet candidate regions received from the candidate region identification unit 5 (step S104).

The contour identification unit 13 identifies the contour pixels of the droplet candidate region selected in step S104 (step S105).

The contour identification unit 13 compares the number of contour pixels identified in step S105 with the exclusion threshold (step S106). If the number of contour pixels is equal to or greater than the exclusion threshold (Yes in step S106), the contour identification unit 13 creates an array of contour pixels (step S107). If the number of contour pixels is less than the exclusion threshold (No in step S106), the contour identification unit 13 determines that the droplet candidate region selected in step S104 is a background region (step S114). The droplet determination unit 6 then proceeds to step S115, which will be described later.

The distance calculation unit 14 uses the array created in step S107 to identify pairs of contour pixels that are assumed to be opposing if the selected droplet candidate region is a droplet (step S108).

The distance calculation unit 14 calculates the distance between the two contour pixels determined in step S108 as the provisional opposing distance 141 (step S109).

If there are any contour pixel pairs that have not been identified in step S108 (No in step S110), the distance calculation unit 14 repeats the processes of steps S108 and S109. If all contour pixel pairs have been identified in step S108 (Yes in step S110), the distance calculation unit 14 calculates the variance of all distances calculated in step S109 (step S111).

If the number of contour pixels contained in the selected droplet candidate region is odd, the distance calculation unit 14 determines in step S110 whether the number of contour pixels not identified in step S108 is 1. If the number of contour pixels not identified is 1, the distance calculation unit 14 proceeds to step S111.

The distance determination unit 15 compares the variance calculated in step S111 with the variance threshold (step S112). If the variance calculated in step S111 is equal to or less than the variance threshold (Yes in step S112), the distance determination unit 15 determines that the droplet candidate region selected in step S104 is a droplet region (step S113).

If the variance of the distances calculated in step S111 exceeds the variance threshold (No in step S112), the droplet candidate region selected in step S104 is determined to be a background region (step S114).

After step S113 or S115, the droplet determination unit 6 determines whether there are any droplet candidate regions that have not yet been selected in step S104 (step S115). If there are any droplet candidate regions (No in step S115), the droplet determination unit 6 returns to step S104 to select a new droplet candidate region. If all droplet candidate regions have been selected (Yes in step S115), the droplet determination unit 6 ends the droplet determination unit process (step S3).

After determining all droplet candidate regions included in the composite image 4A, the droplet determination unit 6 determines whether or not to drive the droplet removal device 3. For example, if the number of droplet regions included in the composite image 4A exceeds a reference value, the droplet determination unit 6 determines to drive the droplet removal device 3.

Alternatively, the droplet determination unit 6 determines to drive the droplet removal device 3 when the area of the droplet region included in the composite image 4A exceeds a preset area reference value. For example, the droplet determination unit 6 may use the integrated value of the number of pixels included in the droplet candidate region determined to be a droplet region as the area of the droplet region.

As described above, the droplet detection device 2 identifies a droplet candidate region 50 based on the change in the level of the video signal for each pixel in the captured image 1. The droplet detection device 2 can determine whether or not a droplet is attached to the lens of the camera C based on the shape of the identified droplet candidate region 50, and can detect droplets attached to the lens.

The droplet detection device 2 identifies the maximum and minimum values of the video signal level at a single pixel. The droplet detection device 2 calculates the absolute difference between the identified maximum and minimum values as the level change at a single pixel, so that the level change at a single pixel can be easily identified.

The droplet detection device 2 uses the contour pixels that form the contour of the identified droplet candidate region 50 to determine whether or not a droplet is attached to the lens of the camera C. In other words, the droplet detection device 2 determines whether or not a droplet is attached to the lens of the camera C by taking into account the shape of the droplet attached to the lens of the camera C, thereby improving the accuracy of droplet detection.

The droplet detection device 2 uses a composite image 4A generated from the captured image 1A and the captured image 1B to determine whether or not a droplet has adhered to the lens, so the amount of information acquired from one captured image can be reduced. As a result, droplets can be detected while preventing the processing load of the droplet detection device 2 from temporarily increasing each time a captured image is acquired.

[Modification]
In the above embodiment, an example has been described in which the liquid droplet detection device 2 acquires the maximum amount of change in level of pixels included in the composite image 4A after generating the composite image 4A from the captured image 1. However, the liquid droplet detection device 2 may acquire the maximum amount of change in level of each pixel included in the pixel line in parallel with acquiring the pixel line.

Figure 15 is a flowchart showing the operation of a droplet detection device 2 according to a modified embodiment of the present invention. When executing the process shown in Figure 6, the droplet detection device 2 according to this modified embodiment executes the process shown in Figure 15 instead of steps S1 and S2. As a result of the process shown in Figure 15, a droplet candidate image of the same size as the captured image 1 is generated. Each pixel included in the droplet candidate image indicates whether it is included in the droplet candidate region.

The image acquisition unit 4 sets pixel lines to be acquired from the captured image 1 (step S501). For example, the pixel line determination unit 8 generates line designation information 72 indicating odd-numbered pixel lines in the captured image 1A, similar to the above embodiment.

Although not shown in FIG. 15, when executing step S501, the image acquisition unit 4 initializes the count value of the horizontal synchronization signal to 1. This is because the pixels contained in the captured image 1 are input in order from the top.

The image acquisition unit 4 starts inputting the video signal. The image acquisition unit 4 determines whether the count value of the horizontal synchronization signal corresponds to the pixel line specified by the line specification information 72 (step S502).

If the count value does not correspond to the pixel line specified by the line specification information 72 (No in step S502), the image acquisition unit 4 waits until a time equivalent to one pixel line has elapsed (step S508). For example, if the count value is 2 and the line specification information 72 specifies an odd-numbered pixel line from the top, the count value does not correspond to the line specification information 72. In this case, the image acquisition unit 4 waits until input of the third pixel line from the top begins.

If the count value corresponds to the pixel line specified by the line specification information 72 (Yes in step S502), the image acquisition unit 4 starts acquiring the pixel line (step S503). The image acquisition unit 4 acquires a signal for one pixel (step S504).

The candidate area identification unit 5 acquires statistical values of the pixel signals acquired in step S504 (step S505). The statistical values are maximum and minimum values. Step S505 corresponds to step S203 shown in FIG. 10. The candidate area identification unit 5 calculates the maximum level change amount of the pixel from which the statistical values were acquired based on the statistical values acquired in step S505 (step S506). Step S506 corresponds to step S204 shown in FIG. 10. The candidate area identification unit 5 determines whether the pixel from which the statistical values were identified is included in the droplet candidate area based on the maximum level change amount of the pixel from which the statistical values were identified (step S507). Step S507 corresponds to steps S205 to S207 shown in FIG. 10.

If neither the horizontal synchronization signal nor the vertical synchronization signal is input (No in step S509), the image acquisition unit 4 returns to step S504 to determine whether the next pixel is included in the droplet candidate region.

If the horizontal synchronization signal is input (Yes in step S509, Yes in step S510), the image acquisition unit 4 increments the count value of the horizontal synchronization signal (step S511). After that, the image acquisition unit 4 returns to step S502 to acquire a new pixel line.

When a vertical synchronization signal is input (Yes in step S509, No in step S510), the image acquisition unit 4 determines that the input of one frame of captured image 1 has been completed. In this case, the image acquisition unit 4 determines whether or not it has acquired a pixel line that can generate a droplet candidate image (step S512).

In the above embodiment, when the odd-numbered pixel lines from the top are acquired from the captured image 1A, the composite image 4A is not generated. Similarly, when the candidate area identification unit 5 acquires the odd-numbered pixel lines from the top from the captured image 1A, it is unable to generate a droplet candidate image. In this case, one frame's worth of pixel lines have not been acquired (No in step S512), so the image acquisition unit 4 returns to step S501. This is to acquire the even-numbered pixel lines from the top from the captured image 1B.

When pixel lines for one frame have been acquired from multiple captured images 1 (Yes in step S512), the candidate area identification unit 5 generates a droplet candidate image based on the results of step S507 for each pixel (step S513). The candidate area identification unit 5 then performs a labeling process on the droplet candidate image to identify the droplet candidate area.

Note that step S504 shown in FIG. 8 is executed in parallel with steps S505 to S507.

The image acquisition unit 4 starts acquiring pixel line La1 from time t0 when the vertical synchronization signal is input. Since the period from time t0 to time t1 corresponds to the time for one pixel, the image acquisition unit 4 acquires the video signal from time t0 to time t1 as the signal of pixel P1 (step S504). The candidate area identification unit 5 determines whether pixel P1 is included in the droplet candidate area based on the video signal from time t0 to time t1 (steps S505 to S507).

The input of the video signal of the captured image 1A continues even after time t1. In parallel with the above-mentioned determination of pixel P1 by the candidate area identification unit 5, the image acquisition unit 4 acquires the video signal from time t1 to time t2 as the signal of pixel P2 (step S504). Based on the signal of pixel P2, the candidate area identification unit 5 determines whether pixel P2 is included in the droplet candidate area. Thereafter, acquisition of the signals of pixels P3 to P10 and determination of whether pixels P3 to P10 are included in the droplet candidate area are repeated. As a result, acquisition of the pixel video signal and determination of whether the pixel is included in the droplet candidate area are performed in parallel.

[Other Modifications]
In the above embodiment, an example in which the unit region is one pixel has been described, but this is not limiting. The unit region may be a plurality of pixels arranged in the horizontal direction, or a plurality of pixels arranged in the vertical direction. Alternatively, the unit region may be a rectangle including a plurality of pixels arranged in the horizontal direction and a plurality of pixels arranged in the vertical direction.

In the above embodiment, an example has been described in which the candidate area identification unit 5 uses the maximum level change amount to determine whether or not each pixel is included in the droplet candidate area, but this is not limiting. The candidate area identification unit 5 only needs to be able to obtain the level change of each pixel. For example, the candidate area identification unit 5 may identify the extreme values of the pixel signal and identify the variance of the identified extreme values as the level change amount.

In the above embodiment, an example has been described in which the droplet determination unit 6 determines whether or not a droplet candidate region is a droplet based on the statistical variation of the provisional facing distance 141 calculated from the contour pixels, but this is not limited to this. If the droplet determination unit 6 determines whether or not a droplet candidate region is a droplet based on the shape of the droplet candidate region, the algorithm used by the droplet determination unit 6 is not particularly limited. For example, the droplet determination unit 6 may use a Hough transform.

In the above embodiment, an example has been described in which the captured image 1 is an image signal in analog format, but this is not limiting. The captured image 1 may be an image signal in digital format. In this case, the image acquisition unit 4 may generate the captured image 1 in analog format by D/A (Digital to Analog) converting the captured image 1 received from the camera.

In the above embodiment, each functional block of the droplet detection device 2 may be individually integrated into a single chip using a semiconductor device such as an LSI, or may be integrated into a single chip to include some or all of the functional blocks. Here, we refer to it as an LSI, but depending on the level of integration, it may also be called an IC, system LSI, super LSI, or ultra LSI.

In addition, the method of integration is not limited to LSI, but may be realized by a dedicated circuit or a general-purpose processor. It is also possible to use a field programmable gate array (FPGA) that can be programmed after LSI manufacturing, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells inside the LSI.

In addition, some or all of the processing performed by each functional block of the droplet detection device 2 may be realized by a program. Some or all of the processing of each functional block in each of the above embodiments is performed by a central processing unit (CPU) in a computer. The programs for performing each process are stored in a storage device such as a hard disk or ROM, and are read out and executed in the ROM or RAM.

In addition, each process in the above embodiments may be realized by hardware, or by software (including cases where it is realized together with an OS (operating system), middleware, or a specified library). Furthermore, it may be realized by a mixed process of software and hardware.

In addition, the execution order of the processing methods in the above embodiments is not necessarily limited to that described in the above embodiments, and the execution order may be changed without departing from the spirit of the invention.

The scope of the present invention includes a computer program that causes a computer to execute the above-mentioned method and a computer-readable recording medium on which the program is recorded. Here, examples of computer-readable recording media include flexible disks, hard disks, CD-ROMs, MOs, DVDs, DVD-ROMs, DVD-RAMs, large-capacity DVDs, next-generation DVDs, and semiconductor memories.

The computer program is not limited to one recorded on the recording medium, but may be one transmitted via a telecommunications line, a wireless or wired communication line, a network such as the Internet, etc.

Although the embodiments of the present invention have been described above, the above-mentioned embodiments are merely examples for implementing the present invention. Therefore, the present invention is not limited to the above-mentioned embodiments, and it is possible to implement the above-mentioned embodiments by appropriately modifying them within the scope of the spirit of the present invention.

Reference Signs List 2 Droplet detection device 3 Droplet removal device 4 Image acquisition unit 5 Candidate area identification unit 6 Droplet judgment unit 7 Pixel line acquisition unit 8 Pixel line determination unit 9 Synthesis unit 10 Statistical value identification unit 11 Difference calculation unit 12 Area judgment unit 13 Contour identification unit 14 Distance calculation unit 15 Distance judgment unit

Claims (5)

an image acquisition unit that acquires a plurality of captured images generated by a camera at different times ;
a candidate region identifying unit that identifies a time change in a level of a video signal in a predetermined direction for each predetermined unit region in a plurality of captured images acquired by the image acquiring unit, and identifies a droplet candidate region by comparing the time change in the identified level with a preset droplet candidate threshold value;
a droplet determination unit that determines whether or not a droplet is attached to the camera lens based on the shape of the droplet candidate region identified by the candidate region identification unit. 2. The droplet detection device of claim 1,
The candidate region identification unit
a statistical value determination unit that determines a maximum value and a minimum value of the level of the video signal in the predetermined direction in one unit area;
a difference calculation unit that calculates an absolute difference between the maximum value and the minimum value specified by the statistical value specification unit as a time change in level in the one unit area;
A droplet detection device including: a region determination unit that determines whether the one unit region is included in the droplet candidate region based on the result of comparing the absolute difference value calculated by the difference calculation unit with the droplet candidate threshold. 3. The droplet detection device according to claim 1,
The droplet determination unit is
a contour identifying unit that creates an array of contour pixels that form the contour of the droplet candidate region;
a distance calculation unit that calculates a distance between a k-th pixel (k is a natural number between 1 and N/2) and a k+N/2-th pixel in the array of contour pixels created by the contour identification unit when the number of contour pixels is N;
and a distance determination unit that determines whether or not a droplet is attached to the camera lens based on the variation in distance calculated by the distance calculation unit. A droplet detection device according to any one of claims 1 to 3,
The image acquisition unit includes:
a pixel line acquisition unit that acquires a first pixel line in a horizontal direction from a first captured image generated by the camera at a first time and acquires a second pixel line in the horizontal direction from a second captured image generated by the camera at a second time;
a pixel line determination unit that, when the pixel line acquisition unit acquires the first pixel line from the first captured image, determines the second pixel line based on a position of the first pixel line and notifies the pixel line acquisition unit of the determined second pixel line;
a synthesis unit that generates a synthetic captured image including the first pixel line and the second pixel line acquired by the pixel line acquisition unit,
The candidate region identifying unit identifies the droplet candidate region from the composite captured image generated by the combining unit. A method for detecting droplets on a lens, comprising the steps of:
acquiring a plurality of captured images generated by a camera at different times ;
identifying a time change in a level of a video signal in a predetermined direction in the plurality of captured images for each predetermined unit area, and identifying a droplet candidate area by comparing the time change in the identified level with a preset droplet candidate threshold value ;
and determining whether or not a droplet is attached to the camera lens based on the shape of the identified droplet candidate region.