JP2021158524A - Droplet detection device - Google Patents

Abstract

To provide a droplet detection device that is able to detect droplets adhering to a lens.SOLUTION: A droplet detection device 2 receives a photographic image 1 generated by a camera C and determines whether a droplet is adhering to a lens of the camera C, based on the received photographic image 1. The droplet detection device 2 includes an image acquisition unit 4, a candidate area specifying unit 5, and a droplet determination unit 6. The image acquisition unit 4 acquires the photographic image 1 generated by the camera C. The candidate area specifying unit 5 specifies, for each predetermined unit area, a level change of an image signal in a predetermined direction in the photographic image 1 acquired by the image acquisition unit 4 and, based on the specified level change, specifies a droplet candidate area 50. The droplet determination unit 6 determines whether a droplet is adhering to the lens of the camera C, based on a shape of the droplet candidate area 50 specified by the candidate area specifying unit 5.SELECTED DRAWING: Figure 2

Description

The present invention relates to a droplet detection device that detects droplets adhering to a lens.

The image taken by the in-vehicle camera is displayed on the monitor in the vehicle interior. In an in-vehicle camera, dirt may adhere to the lens. Patent Document 1 discloses a lens cleaning device that removes white turbidity adhering to the lens of an in-vehicle camera. This lens cleaning device calculates the degree of white turbidity on the surface of the lens based on the brightness gradient of the image generated by the in-vehicle camera. The cloudiness that adheres to the lens is, for example, dust that is rolled up when the vehicle travels on an unpaved road. This lens cleaning device changes the injection form of the cleaning liquid or compressed air based on the calculated degree of white turbidity.

Japanese Unexamined Patent Publication No. 2014-27539

The lens cleaning device according to Patent Document 1 may not be able to remove water droplets adhering to the lens. This is because when the impurities contained in the water droplets are very small, the water droplets do not become cloudy.

Further, when droplets such as water droplets adhere to the lens, the environment around the vehicle may be distorted by the droplets in the image taken by the camera. Therefore, it is desirable to remove the droplets attached to the lens when the droplets adhere to the lens.

Therefore, in view of the above problems, an object of the present invention is to provide a droplet detection device capable of detecting droplets adhering to a lens.

In order to solve the above problems, the first invention is a droplet detection device, which includes an image acquisition unit, a candidate region identification unit, and a droplet determination unit. The image acquisition unit acquires a captured image generated by the camera. The candidate area identification unit specifies the level change of the video signal in the predetermined direction in the captured image acquired by the image acquisition unit for each predetermined unit area, and specifies the droplet candidate area based on the specified level change. The droplet determination unit determines whether or not the droplet is attached to the lens of the camera based on the shape of the droplet candidate region specified by the candidate region identification unit.

According to the first invention, the droplet detection device can detect droplets adhering to the lens.

The second invention is the first invention, and the candidate area specifying unit includes a statistical value specifying unit, a difference calculation unit, and an area determining unit. The statistical value specifying unit specifies the maximum value and the minimum value of the level of the video signal in a predetermined direction in one unit area. The difference calculation unit calculates the absolute difference between the maximum value and the minimum value specified by the statistical value identification unit as a level change in one unit area. The region determination unit determines whether or not one unit region is included in the droplet candidate region based on the result of comparing the absolute difference value calculated by the difference calculation unit with a predetermined threshold value.

According to the second invention, the level change in one unit domain can be easily specified.

The third invention is the first invention or the second invention, and the droplet determination unit includes a contour specifying unit, a distance calculation unit, and a distance determination unit. The contour specifying unit creates an array of contour pixels that form the contour of the droplet candidate region. When the number of contour pixels is N, the distance calculation unit has the k-th pixel (k is a natural number of 1 or more and N / 2 or less) and the k + N / 2nd pixel in the array of contour pixels created by the contour identification unit. Calculate the distance to the pixel of. The distance determination unit determines whether or not the droplets are attached to the lens of the camera based on the variation in the distance calculated by the distance calculation unit.

According to the third invention, it is determined whether or not the droplet is attached to the lens of the camera in consideration of the shape of the droplet attached to the lens, so that the detection accuracy of the droplet can be improved.

The fourth invention is any one of the first to third inventions, and the image acquisition unit includes a pixel line acquisition unit, a pixel line determination unit, and a composition unit. The pixel line acquisition unit acquires the first pixel line in the horizontal direction from the first captured image generated by the camera at the first time, and horizontally from the second captured image generated by the camera at the second time. Acquire the second pixel line. When the pixel line acquisition unit acquires the first pixel line from the first captured image, the pixel line determination unit determines the second pixel line based on the position of the first pixel line, and determines the determined second pixel line as a pixel. Notify the line acquisition department. The compositing unit generates a composite image including the first pixel line and the second pixel line acquired by the pixel line acquisition unit. The candidate area identification unit identifies the candidate area from the composite image generated by the composite unit.

According to the fourth invention, information acquired from one captured image is used to determine whether or not droplets have adhered to the lens by using the first captured image and the composite image generated from the second captured image. The amount can be suppressed. As a result, it is possible to detect droplets while preventing the processing load of the droplet detection device from temporarily increasing each time a captured image is acquired.

A fifth invention is a droplet detection method for detecting droplets adhering to a lens, which includes a) step, b) step, and c) step. a) The step acquires a captured image generated by the camera. b) In the step, the level change of the video signal in the predetermined direction in the acquired captured image is specified for each predetermined unit area, and the droplet candidate area is specified based on the specified level change. c) The step determines whether or not the droplet has adhered to the lens of the camera based on the shape of the identified droplet candidate region.

The fifth invention is used in the first invention.

According to the present invention, droplets adhering to the lens can be detected.

It is a functional block diagram which shows the structure of the droplet removal system which concerns on embodiment of this invention. It is a functional block diagram which shows the structure of the droplet detection apparatus shown in FIG. It is a functional block diagram which shows the structure of the candidate area identification part shown in FIG. It is a functional block diagram which shows the structure of the droplet determination part shown in FIG. It is a functional block diagram which shows the structure of the image acquisition part shown in FIG. Flow chart showing the operation of the droplet detection device shown in FIG. 1 is a diagram showing a configuration of pixel lines of a captured image acquired by the image acquisition unit shown in FIG. 1. It is a figure explaining the composite image generation process (step S1) shown in FIG. It is a graph which shows an example of the time change of the level of the video signal included in the composite image shown in FIG. 7. It is a figure which shows an example of the droplet candidate region included in the composite image generated by the image acquisition part shown in FIG. It is a flowchart which shows the operation of the candidate area identification part 5 shown in FIG. It is an enlarged view of the droplet candidate region 50a shown in FIG. It is a figure which shows the arrangement generated from the contour pixel of the droplet candidate region shown in FIG. It is an enlarged view of the droplet candidate region 50b shown in FIG. It is a flowchart which shows the droplet determination part processing (step S3) shown in FIG. It is a flowchart which shows the operation of the droplet detection apparatus which concerns on the modification of embodiment of this invention. It is a figure which shows the CPU bus configuration.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The same or corresponding parts in the drawings are designated by the same reference numerals, and the description thereof will not be repeated.

[1. Configuration of Droplet Removal System 100]
FIG. 1 is a functional block diagram showing a configuration of a droplet removal system 100 according to an embodiment of the present invention. The droplet removal system 100 includes a camera C, a droplet detection device 2, and a droplet removal device 3. The droplet removal system 100 removes droplets such as water droplets adhering to the lens of the camera C.

The camera C is mounted around the vehicle, for example. The vehicle is, for example, an automobile. The camera C photographs the surroundings of the vehicle to generate a captured image 1. The captured image 1 is an analog-format video signal, for example, a composite video signal. The camera C outputs the generated captured image 1 to a display unit (not shown) installed in the vehicle interior. The display unit is, for example, a monitor of a car navigation device. The driver and passengers of the vehicle can confirm the surroundings of the vehicle by the photographed image 1 displayed on the display unit.

The droplet detection device 2 receives the captured image 1 generated by the camera C, and determines whether or not the droplet is attached to the lens of the camera C based on the captured image 1 received. When the droplet detection device 2 determines that the droplets are attached to the lens of the camera C, the droplet detection device 2 outputs a drive signal 2A for driving the droplet removal device 3.

When the droplet removing device 3 receives the drive signal 2A from the droplet detecting device 2, for example, the droplet removing device 3 injects compressed air toward the lens of the camera C. By injecting the compressed air onto the lens of the camera C, the droplets adhering to the lens are removed.

[2.1. Configuration of Droplet Detection Device 2]
FIG. 2 is a functional block diagram showing the configuration of the droplet detection device 2 shown in FIG. With reference to FIG. 2, the droplet detection device 2 includes an image acquisition unit 4, a candidate region identification unit 5, and a droplet determination unit 6.

The image acquisition unit 4 acquires the captured image 1 generated by the camera C. Specifically, the image acquisition unit 4 generates a composite image 4A from a plurality of captured images 1 generated at different times, and outputs the generated composite image 4A to the candidate region identification unit 5. The generation of the composite image 4A will be described later.

The candidate area specifying unit 5 receives the composite image 4A from the image acquisition unit 4, and specifies the level change of the video signal in the predetermined direction in the received composite image 4A for each predetermined unit area. In the present embodiment, the predetermined direction is the horizontal scanning direction of the composite image 4A. The predetermined unit area is a pixel. The candidate region specifying unit 5 identifies the droplet candidate region 50 in the composite image 4A based on the specified level change. Details of the level change of the video signal and the droplet candidate region 50 will be described later.

The droplet determination unit 6 determines whether or not the droplet is attached to the lens of the camera C based on the shape of the droplet candidate region 50 specified by the candidate region identification unit 5. When the droplet determination unit 6 determines that the droplet is attached to the lens of the camera C, the droplet determination unit 6 outputs a drive signal 2A to the droplet removal device 3.

[2.2. Configuration of image acquisition unit 4]
With reference to FIG. 3, the image acquisition unit 4 includes a pixel line acquisition unit 7, a pixel line determination unit 8, and a composition unit 9. In the following description, a line of pixels arranged in a row in the horizontal scanning direction will be referred to as a “pixel line”.

The pixel line acquisition unit 7 acquires the first pixel line L1 in the horizontal direction from the captured image 1A generated at the first time by the camera C, and is horizontal from the captured image 1B generated at the second time by the camera C. The second pixel line L2 in the direction is acquired. The second time is a time after the first time.

In the present embodiment, the first pixel line L1 is an odd-numbered pixel line counted from the top among the pixel lines of the captured image 1A. The second pixel line L2 is an even-numbered pixel line counted from the top among the pixel lines of the captured image 1B.

When the first pixel line is acquired, the pixel line acquisition unit 7 outputs line position information 71 indicating the position of the acquired first pixel line L1 to the pixel line determination unit 8. When the second pixel line L2 is acquired, the pixel line acquisition unit 7 outputs line position information 71 indicating the acquired position of the second pixel line L2 to the pixel line determination unit 8.

When the pixel line acquisition unit 7 receives the line designation information 72 that specifies the pixel line to be acquired next from the pixel line determination unit 8, the pixel line acquisition unit 7 newly acquires the pixel line specified by the received line designation information 72. Obtained from captured image 1. Each time the pixel line acquisition unit 7 acquires each of the first pixel line L1 and the second pixel line L2, the pixel line acquisition unit 7 outputs the acquired first pixel line L1 and the second pixel line L2 to the synthesis unit 9.

When the line position information 71 is received from the pixel line acquisition unit 7, the pixel line determination unit 8 determines the pixel line to be acquired next based on the position of the pixel line indicated by the received line position information 71. .. Specifically, when the line position information 71 indicates the first pixel line L1, the pixel line determination unit 8 determines to acquire the pixel line L2 next. When the line position information 71 indicates the second pixel line L2, the pixel line determination unit 8 determines to acquire the pixel line L1 next. The pixel line determination unit 8 outputs line designation information 72 indicating the determined pixel line to the pixel line acquisition unit 7.

The compositing unit 9 receives the first pixel line L1 and the second pixel line L2 from the pixel line acquisition unit 7, and generates a composite image 4A including the received first pixel line L1 and the second pixel line L2. .. The synthesis unit 9 outputs the generated composite image 4A to the candidate region identification unit 5.

[2.3. Configuration of Candidate Area Identification Unit 5]
With reference to FIG. 4, the candidate region identification unit 5 includes a statistical value identification unit 10, a difference calculation unit 11, and an area determination unit 12.

The statistical value specifying unit 10 receives the composite image 4A from the image acquisition unit 4 and specifies the statistical value of the pixel signal in one unit region included in the received composite image 4A. The unit area is a pixel. The statistical values are the maximum and minimum values of the level of the image signal in the horizontal scanning direction.

Specifically, the statistical value specifying unit 10 classifies the signal corresponding to one pixel line included in the composite image 4A for each pixel, and specifies the maximum value and the minimum value of the signal level divided for each pixel. do. The maximum and minimum values of the signal level are specified in all the pixels included in the composite image 4A. The statistical value specifying unit 10 outputs statistical value information 101 including the maximum value and the minimum value of each pixel included in the composite image 4A to the difference calculation unit 11.

The difference calculation unit 11 calculates the absolute difference between the maximum value and the minimum value in one unit area specified by the statistical value specifying unit 10 as the maximum level change amount in one unit area. Specifically, the difference calculation unit 11 receives the statistical value information 101 from the statistical value specifying unit 10, and uses the received statistical value information 101 to calculate the maximum level change amount of each pixel included in the composite image 4A. do. The maximum level change amount is obtained by subtracting the minimum value of the signal level from the maximum value of the signal level in one pixel. The difference calculation unit 11 outputs the level change information 111 recording the maximum level change amount of each pixel included in the composite image 4A to the area determination unit 12.

The region determination unit 12 compares the absolute difference value of one unit region specified by the difference calculation unit 11 with a predetermined droplet candidate threshold value, and based on the comparison result, one unit region becomes the droplet candidate region. Determine if it is included. A predetermined droplet candidate threshold is determined based on the maximum level change obtained when one unit region corresponds to a droplet.

Specifically, the area determination unit 12 receives the level change information 111 from the difference calculation unit 11. The area determination unit 12 compares the maximum level change amount of each pixel recorded in the received level change information 111 with the droplet candidate threshold value. The region determination unit 12 determines whether or not each pixel included in the composite image 4A is included in the droplet candidate region based on the comparison result. The region determination unit 12 identifies the droplet candidate region 50 in the composite image 4A based on the determination result of whether or not each pixel of the composite image 4A is included in the droplet candidate region.

[2.4. Configuration of droplet determination unit 6]
With reference to FIG. 5, the droplet determination unit 6 includes a contour specifying unit 13, a distance calculation unit 14, and a distance determination unit 15.

The contour specifying unit 13 receives the droplet candidate region 50 from the candidate region specifying unit 5, and specifies the contour pixels that form the contour of the received droplet candidate region 50. The contour specifying unit 13 creates an array 131 of the specified contour pixels. The contour specifying unit 13 outputs the created array of contour pixels 131 to the distance calculation unit 14.

The distance calculation unit 14 receives the array 131 of the contour pixels from the contour identification unit 13, and calculates the distance between two of the contour pixels included in the received array 131 as the provisional facing distance 141. .. Specifically, when the number of contour pixels is N, the distance calculation unit 14 identifies the k-th pixel and the (k + N / 2) -th pixel in the array 131. The distance calculation unit 14 calculates the distance between the specified two pixels as the provisional facing distance 141. The k-th pixel and the (k + N / 2) -th pixel are pixels that are assumed to face each other when the shape of the contour pixel is assumed to be circular. k is a natural number of 1 or more and N / 2 or less. Since N / 2 combinations of pixels are conceivable, the distance calculation unit 14 repeats the calculation of the provisional facing distance 141 N / 2 times.

The distance determination unit 15 receives the provisional facing distance 141 from the distance calculation unit 14, and determines whether or not the droplets are attached to the lens of the camera C based on the statistical variation of the received provisional facing distance 141. .. When the droplets are attached to the lens of the camera C, the distance determination unit 15 outputs the drive signal 2A to the droplet removing device 3 and outputs the droplets attached to the lens of the camera C to the droplet removing device 3. Let it be removed.

[3. motion]
[3.1. Operation of Droplet Detection Device 2]
FIG. 6 is a flowchart showing the operation of the droplet detection device 2 shown in FIG. When the camera C starts generating the captured image 1, the droplet detection device 2 starts the process shown in FIG.

With reference to FIG. 6, the image acquisition unit 4 generates a composite image 4A using a plurality of captured images 1 generated by the camera C (step S1). The candidate region specifying unit 5 identifies the droplet candidate region 50 in the composite image 4A generated in step S1 (step S2). The droplet determination unit 6 determines whether or not the droplet is attached to the lens of the camera C based on the shape of the droplet candidate region 50 identified in step S2 (step S3). When the droplet determination unit 6 determines that the droplets are attached to the lens of the camera C, the droplet determination unit 6 drives the droplet removal device 3.

The droplet detection device 2 determines whether or not the camera C has finished generating the captured image 1 (step S4). When the camera C continues to generate the captured image 1 (Yes in step S4), the droplet detection device 2 returns to step S1. When the camera C has finished generating the captured image 1 (No in step S4), the droplet detection device 2 finishes the process shown in FIG.

[3.2. Generation of composite image 4A (step S1)]
FIG. 7 is a diagram illustrating the composite image generation process (step S1) shown in FIG. In the example shown in FIG. 7, the composite image 4A is generated from the captured image 1A and the captured image 1B. The size of the composite image 4A is the same as the size of the captured images 1A and 1B. The captured image 1A is generated by the camera C at time t1. The captured image 1B is generated by the camera C at a time t2 after the time t1. The captured image 1B is the next frame of the captured image 1A.

The captured image 1A includes pixel lines La1 to La10. The pixel line La1 is located at the upper end of the captured image 1A. The pixel line La2 is adjacent below the pixel line La1. The pixel line La10 is located at the lower end of the captured image 1A.

The captured image 1B includes pixel lines Lb1 to Lb10. The positions of the pixel lines Lb1 to Lb10 correspond to the positions of the pixel lines La1 to La10.

When the captured image 1A is received from the camera C, the pixel line acquisition unit 7 acquires the first pixel line from the captured image 1A. The first pixel line is an odd-numbered pixel line counted from the top of the captured image 1. Specifically, the first pixel line in the captured image 1A is the pixel lines La1, La3, ..., La9. The pixel line acquisition unit 7 outputs line position information 71 indicating the positions of the acquired pixel lines La1, La3, ..., La9 to the pixel line determination unit 8. The pixel line acquisition unit 7 waits until the captured image 1B is received from the camera C.

The pixel line determination unit 8 determines the pixel line to be acquired from the captured image 1B based on the line position information 71 indicating the first pixel line. Specifically, since the line position information 71 indicates the first pixel line, the pixel line determination unit 8 determines to acquire the second pixel line from the captured image 1B. The second pixel line is an even-numbered pixel line in the captured image 1B. In the captured image 1B, the second pixel line is the pixel lines Lb2, Lb4, ..., Lb10. The pixel line determination unit 8 outputs line designation information 72 indicating pixel lines Lb2, Lb4, ..., Lb10 to the pixel line acquisition unit 7.

When the captured image 1B is received from the camera C, the pixel line acquisition unit 7 acquires the pixel lines Lb2, Lb4, ... L10 from the captured image 1B based on the line designation information 72 that specifies the second pixel line. ..

The compositing unit 9 receives the first pixel line and the second pixel line from the pixel line acquisition unit 7, and generates a composite image 4A including the received first pixel line L1 and the second pixel line L2. Specifically, the compositing unit 9 arranges the pixel lines La1, La3, ..., La9 acquired from the captured image 1A on the odd-numbered pixel lines in the composite image 4A. The compositing unit 9 arranges the pixel lines Lb2, Lb4, ..., Lb10 acquired from the captured image 1B on the even-numbered pixel lines in the composite image 4A.

That is, the compositing unit 9 arranges each of the first pixel line and the second pixel line at the same position of the composite image 4A as the position of the pixel line in the captured images 1A and 1B. As shown in FIG. 7, the pixel lines of the captured image 1A and the pixel lines of the captured image 1B are alternately arranged in the composite image 4A.

The image acquisition unit 4 repeats the above process after generating the composite image 4A from the captured images 1A and 1B. The frame interval of the composite image 4A is twice the frame interval of the captured image 1.

The image acquisition unit 4 may generate a composite image using three or more captured images 1. Even in this case, the compositing unit 9 has the same position of the pixel line acquired from the captured image 1 and the position of the pixel line arranged in the composite image 4A. When the composite image 4A is generated from the six captured images 1A, the frame interval of the composite image 4A is 6 times the frame interval of the captured image 1.

[3.3. Identification of droplet candidate area]
The candidate region specifying unit 5 determines whether or not each pixel included in the composite image 4A is included in the droplet candidate region. As a result, it is determined whether or not each pixel included in the composite image 4A is included in the droplet candidate region.

[3.3.1. Acquisition of signal for each pixel]
The statistical value specifying unit 10 acquires the composite image 4A from the composite unit 9, and acquires the statistical value of each pixel included in the acquired composite image 4A. The statistical values are specifically the maximum value and the minimum value.

The statistical value specifying unit 10 acquires the signal of each pixel included in the composite image 4A in specifying the maximum value and the minimum value of the level of each pixel included in the composite image 4A. The signal of each pixel included in the composite image 4A is an analog-format video signal. This is because the composite image 4A is generated from the captured image 1 which is an analog-format video signal. Hereinafter, the operation of the statistical value specifying unit 10 for acquiring the signal of each pixel will be described by taking the pixel lines La1 and Lb2 shown in FIG. 7 as an example.

The pixel line La1 includes pixels P1 to P10. The pixel P1 is located at the left end of the pixel line La1. Pixel Pm is adjacent to the left of pixel Pm + 1. m is an integer of 1 or more and R-1 or less. R is the number of pixels included in the pixel line La1 and is 10 in the example shown in FIG. The pixel P10 is located at the right end of the pixel line La1.

When the statistical value specifying unit 10 acquires the vertical synchronization signal of the composite image 4A, it starts acquiring the signal of the pixel line La1 and starts counting the horizontal synchronization signal of the received composite image 4A. The pixel line La1 corresponds to a signal in the period from the time when the vertical synchronization signal is acquired to the time of the first horizontal synchronization signal among the video signals of the composite image 4A. Therefore, the statistical value specifying unit 10 ends the acquisition of the signal of the pixel line La1 at the time when the first horizontal synchronization signal is acquired. The statistical value specifying unit 10 acquires signals corresponding to each of the pixels P1 to P10 included in the pixel line La1 by classifying the video signal of the pixel line La1 for each pixel.

FIG. 8 is a graph showing an example of time-dependent changes in the level of the video signal included in the composite image 4A. With reference to FIG. 8, the time t0 is the time of the vertical synchronization signal of the composite image 4A. Time t3 is the time of the first horizontal synchronization signal in the composite image 4A. Therefore, the period TL from the time t0 to the time t3 corresponds to the video signal of the pixel line La1.

The statistical value specifying unit 10 acquires the signals of the pixels P1 to P10 by dividing the signal of the pixel line La1 by the time per pixel. The time per pixel is obtained by dividing the period TL, which is one horizontal scanning period, by the number of pixels in the horizontal scanning direction. The statistical value specifying unit 10 acquires the signal of the period Ta from the time t0 to the time t1 as the signal of the pixel P1. The statistical value specifying unit 10 acquires the signal of the period Tb from the time t1 to the time t2 as the signal of the pixel P2. In FIG. 8, the display of the signals of the pixels P3 to P10 is omitted.

The statistical value specifying unit 10 acquires the signal from the time of the first horizontal synchronization signal to the time of the second horizontal synchronization signal in the composite image 4A as the pixel line Lb2. Signals of other pixel lines included in the composite image 4A are also acquired in the same manner as the pixel line Lb2.

[3.3.2. Get statistics]
The statistical value specifying unit 10 selects one pixel from the pixels included in the composite image 4A, and acquires the maximum value and the minimum value of the signal level of the selected pixel. Hereinafter, in the description of the operation of the droplet detection device 2, the operation of the statistical value specifying unit 10 will be described with reference to FIG.

The statistical value specifying unit 10 specifies the maximum value A1 and the minimum value A2 as the statistical values of the pixel P1. This is because the period Ta corresponds to the pixel P1. The statistical value specifying unit 10 outputs the statistical value information 101 recording the maximum value and the minimum value of the pixel P1 to the difference calculation unit 11.

When the pixel P2 is selected, the statistical value specifying unit 10 specifies the maximum value A3 and the minimum value A4 as statistical values of the signal level corresponding to the pixel P2. This is because the period Tb corresponds to the pixel P2. The statistical value specifying unit 10 specifies the statistical values of other pixels included in the composite image 4A, similarly to the pixels P1 and P2.

[3.3.3. Calculation of maximum level change]
The difference calculation unit 11 calculates the maximum level change amount of the pixel P1 based on the statistical value information 101 of the pixel P1 received from the statistical value specifying unit 10. Hereinafter, the operation of the difference calculation unit 11 will be described using pixels P1 and P2 as examples.

The difference calculation unit 11 acquires the maximum value A1 and the minimum value A2 of the pixel P1 from the statistical value information 101. The difference calculation unit 11 calculates the absolute difference between the acquired maximum value A1 and the minimum value A2 as the maximum level change amount of the pixel P1. The difference calculation unit 11 generates level change information 111 in which the maximum level change amount of the pixel P1 and the position of the pixel P1 are linked. The maximum level change amount of the pixel P2 is the absolute difference between the maximum value A3 and the minimum value A4. The statistical value specifying unit 10 calculates the maximum level change amount of other pixels included in the composite image 4A, similarly to the pixels P1 and P2.

[3.3.4.4. Comparison with Droplet Candidate Threshold]
The area determination unit 12 determines whether or not each pixel included in the composite image 4A is included in the droplet candidate region based on the maximum level change amount recorded in the level change information 111. Hereinafter, the operation of the area determination unit 12 will be described using the pixel P1 as an example.

The area determination unit 12 acquires the maximum level change amount of the pixel P1 from the level change information 111 received from the difference calculation unit 11, and compares the acquired maximum level change amount with the preset droplet candidate threshold value. When the maximum level change amount of the pixel P1 is smaller than the droplet candidate threshold value, the region determination unit 12 determines that the pixel P1 is included in the droplet candidate region. This is because when the droplet adheres to the lens of the camera C, the maximum level change amount of the signal of the pixel corresponding to the position where the droplet adheres tends to be small. When the maximum level change amount of the pixel P1 exceeds the droplet candidate threshold value, the region determination unit 12 determines that the pixel P1 is included in the background region. The background region is a region in which no droplets are attached in the composite image.

After determining whether or not the pixel P1 is included in the droplet candidate region, the statistical value acquisition unit 10 selects a new pixel (for example, the pixel P2). Similar to the pixel P1, it is determined whether or not the selected pixel is included in the droplet candidate region based on the maximum level change amount of the selected pixel.

[3.3.5. Determining Droplet Candidate Area]
The region determination unit 12 identifies the droplet candidate region in the composite image 4A based on the result of determining whether or not each pixel included in the composite image 4A is included in the droplet candidate region. In the following description, pixels determined to be included in the droplet candidate region will be referred to as “candidate pixels”.

FIG. 9 is a diagram showing an example of a droplet candidate region included in the composite image 4A generated by the image acquisition unit 4. The operation of the region determination unit 12 for determining the droplet candidate region will be described with reference to FIG. 9.

The area determination unit 12 identifies a region in which candidate pixels are continuous by a labeling process, and determines the specified region as a droplet candidate region. In FIG. 9, the region where the candidate pixels are continuous is a region 50a surrounded by the pixels Pa1 to Pa18 and a region 50b composed of the pixels Pb1 to Pb12. The region determination unit 12 determines each of the regions 50a and 50b as a droplet candidate region. The region 50a corresponds to raindrops adhering to the lens of the camera C. Region 50b corresponds to a white line drawn on the road. In the region 50a shown in FIG. 9, the display of pixels other than the pixels Pa1 to Pa18 is omitted. The same applies to FIGS. 11 and 13 described later.

In the following description, the region 50a will be referred to as a “droplet candidate region 50a” and the region 50b will be referred to as a “droplet candidate region 50b”. Although the details will be described later, if the pixel group included in the one droplet candidate region satisfies a predetermined condition, the one droplet candidate region is determined to be raindrops adhering to the windshield.

[3.3.6.6. Operation of candidate area identification unit 5]
FIG. 10 is a flowchart showing the operation of the candidate area specifying unit 5 shown in FIG. When the composite image 4A is received from the image acquisition unit 4, the candidate area identification unit 5 starts the process shown in FIG.

With reference to FIG. 10, the statistical value specifying unit 10 acquires the signal of each pixel included in the composite image 4A (step S201).

The statistical value specifying unit 10 selects one of the pixels included in the composite image 4A (step S202).

The statistical value specifying unit 10 specifies the maximum value and the minimum value of the signal level of the pixel selected in step S202 (step S203).

The difference calculation unit 11 calculates the maximum level change amount of the signal of the pixel selected in step S202 based on the maximum value and the minimum value specified in step S203 (step S204).

The region determination unit 12 compares the maximum level change amount calculated in step S204 with the droplet candidate threshold value (step S205). When the maximum level change amount is smaller than the droplet candidate threshold value (Yes in step S205), the region determination unit 12 determines that the pixel selected in step 202 is included in the droplet candidate region (step S206). When the maximum amount of change in the level exceeds the droplet candidate threshold value (No in step S205), the region determination unit 12 determines that the pixel selected in step 202 is included in the background region (step S207).

When all the pixels in the composite image 4A are selected (Yes in step S208), the region determination unit 12 determines the region of continuous candidate pixels as the droplet candidate region 50 (step S209). As a result, the candidate area identification unit 5 ends the process shown in FIG. If there is a pixel not selected in step S202 (No in step S208), the candidate region specifying unit 5 returns to step S202.

When newly executed in step S202, the candidate region specifying unit 5 selects a pixel for which it has not been determined whether or not it is included in the droplet candidate region 50. The order of selecting pixels may be random or may be preset.

[3.4. Droplet judgment]
The droplet determination unit 6 determines whether or not the droplet is attached to the lens of the camera C based on the shape of the droplet candidate region 50 specified by the candidate region identification unit 5. When the droplet determination unit 6 determines that the droplets are attached to the lens of the camera C, the droplet determination unit 6 drives the droplet removal device 3.

[3.4.1. Identification of contour pixels]
The contour specifying unit 13 receives the droplet candidate region 50 from the region determination unit 12, selects one droplet candidate region from the received droplet candidate region 50, and forms the contour of the selected droplet candidate region. To identify.

Hereinafter, the operation of the contour specifying unit 13 will be described by taking the case where one selected droplet candidate region is the droplet candidate region 50a shown in FIG. 9 as an example.

FIG. 11 is an enlarged view of the droplet candidate region 50a shown in FIG. With reference to FIG. 11, the contour specifying unit 13 specifies pixels Pa1 to Pa18 as contour pixels among the pixels included in the droplet candidate region 50a. Specifically, the contour specifying unit 13 selects one candidate pixel included in the droplet candidate region 50a. When at least one of the vertices of all the selected candidate pixels coincides with the vertices of a pixel that is not a candidate pixel, the contour specifying unit 13 determines that the selected candidate pixel is a contour pixel. Alternatively, when the selected candidate pixel has at least one side that is not in contact with another candidate pixel included in the droplet candidate region 50a, the contour specifying unit 13 determines that the selected candidate pixel is a contour pixel. You may.

[3.4.2. Comparison with exclusion threshold]
The contour specifying unit 13 compares the number of pixels of the specified contour pixels Pa1 to Pa18 with a preset exclusion threshold value. The exclusion threshold is, for example, 8. As a result of the comparison, since the number of pixels of the contour pixels Pa1 to Pa18 is equal to or larger than the exclusion threshold value, the contour specifying unit 13 continues to determine the droplet candidate region 50a including the contour pixels Pa1 to Pa18. When the number of contour pixels included in the selected droplet candidate region 50 is smaller than the exclusion threshold value, the contour specifying unit 13 ends the determination of the selected droplet candidate region 50. As a result, it is possible to exclude droplets having a size such that the droplets adhering to the lens of the camera C do not affect the user's visual recognition. In addition, it is possible to improve the accuracy of determining whether or not the droplet candidate region is substantially circular.

[3.4.3. Creating an array of contour pixels]
With reference to FIG. 11, the contour specifying unit 13 selects the contour pixel Pa1 as the first pixel of the array, and determines the element number of the array of the contour pixels to be “1”. The contour specifying unit 13 specifies contour pixels in the counterclockwise order from the selected contour pixel Pa1, and assigns an array element number to the specified contour pixel. The element number of the array is given to each of the contour pixels. The contour specifying unit 13 generates the array 131 according to the element numbers assigned to each of the contour pixels. The number of elements in the array 131 is located at the number of contour pixels.

FIG. 12 is a diagram showing an array 131 generated from the contour pixels of the droplet candidate region 50a shown in FIG. The numerical value arranged below each pixel shown in FIG. 12 is the element number of the array 131.

[3.4.4. Shape judgment]
The droplet attached to the lens of the camera C becomes substantially circular due to the surface tension of the droplet. Therefore, when the droplet candidate region 50a is a droplet, the line segment connecting two pixels that satisfy the following facing conditions among the pixels located on the contour of the droplet corresponds to the diameter of the circle. The facing condition is that the pixels located on the contour of the droplet face each other with the center of gravity of the droplet candidate region 50a in between. Even when the pair of pixels satisfying the opposite conditions is changed, the line segment connecting the two pixels included in the changed pair still corresponds to the diameter of the circle. When the droplet candidate region 50a is a droplet, the variance of the distance between the two opposing pixels is considered to be small.

The distance calculation unit 14 identifies two pixels included in the droplet candidate region 50a that are assumed to face each other if the droplet candidate region 50a is a droplet. The distance calculation unit 14 calculates the distance between the specified two pixels as the provisional facing distance 141. The details will be described below.

With reference to FIGS. 11 and 12, when the droplet candidate region 50a is a droplet, the contour pixels Pa1 to Pa18 are evenly arranged along the contour of the droplet. The pixel farthest from the k-th pixel is the (k + 2 / N) -th pixel. N is the number of contour pixels, and k is a natural number of 1 or more (2 / N) or less. The distance calculation unit 14 identifies a pixel pair including the k-th pixel and the (k + 2 / N) -th pixel. The distance calculation unit 14 calculates the distance between two pixels included in the specified pixel pair.

In FIG. 12, N is 18. Therefore, in the distance calculation unit 14, when k = 1, the contour pixels specified by the distance calculation unit 14 are the pixels Pa1 and Pa10. The distance calculation unit 14 calculates the distance Mk1 from the pixel Pa1 to the pixel Pa10 as the provisional facing distance 141. When k = 2, the specified contour pixels are pixels Pa2 and Pa11. The distance calculation unit 14 calculates the distance Mk2 from the pixel Pa2 to the pixel Pa11 as the provisional facing distance 141. After that, the distance calculation unit 14 increments k. The distance calculation unit 14 repeats the process of calculating the provisional facing distance 141 until k reaches 9.

The distance determination unit 15 calculates the variance of all the provisional facing distances 141 calculated by the distance calculation unit 14. The distance determination unit 15 does not have to be a variance as long as it calculates the statistical variation of all the calculated provisional facing distances 141. The distance determination unit 15 compares the calculated variance with a preset variance threshold value, and determines whether or not the droplet candidate region 50a is a droplet based on the comparison result. The dispersion threshold is pre-calculated using the droplets attached to the camera lens.

As shown in FIG. 12, since the droplet candidate region 50a corresponds to raindrops, the variance calculated from the droplet candidate region 50a is lower than the dispersion threshold. The distance determination unit 15 determines that the droplet candidate region 50a is a droplet because the droplet candidate region 50a has a shape close to a circle. That is, the droplet candidate region 50a is determined to be the droplet region.

Hereinafter, the shape determination of the droplet candidate region 50b will be described. FIG. 13 is an enlarged view of the droplet candidate region 50b shown in FIG.

With reference to FIG. 13, the distance calculation unit 14 selects two pixels, which are assumed to face each other if the droplet candidate region 50b is a droplet, from the contour pixels Pb1 to Pb12 included in the droplet candidate region 50b. Identify. For example, the distance calculation unit 14 identifies the contour pixels Pb1 and Pb7, and calculates the distance Nk1 from the contour pixel Pb1 to the contour pixel Pb7. The distance calculation unit 14 identifies the contour pixels Pb3 and Pb9, and calculates the distance Nk2 from the contour pixel Pb3 to the contour pixel Pb9.

The droplet candidate region 50b corresponds to the white line drawn on the road as described above, and is not circular. Since the distance Nk2 is larger than several times the distance Nk1, the variance calculated from the droplet candidate region 50b is equal to or greater than the variance threshold. Since the droplet candidate region 50b is not a droplet, the distance determination unit 15 determines that the droplet candidate region 50b is a background region.

[3.4.5. Operation of droplet determination unit 6]
FIG. 14 is a flowchart showing the droplet determination unit processing (step S3) shown in FIG. When the droplet candidate region included in the composite image 4A is received from the candidate region identification unit 5, the droplet determination unit 6 starts the process shown in FIG.

With reference to FIG. 14, the contour specifying unit 13 selects one droplet candidate region from the droplet candidate regions received from the candidate region specifying unit 5 (step S104).

The contour specifying unit 13 specifies the contour pixels of the droplet candidate candidate region selected in step S104 (step S105).

The contour specifying unit 13 compares the number of the contour pixels specified in step S105 with the exclusion threshold value (step S106). When the number of contour pixels is equal to or greater than the exclusion threshold (Yes in step S106), the contour specifying unit 13 creates an array of contour pixels (step S107). When the number of pixels of the contour pixel is smaller than the exclusion threshold value (No in step S106), the contour specifying unit 13 determines that the droplet candidate candidate region selected in step S104 is the background region (step S114). After that, the droplet determination unit 6 proceeds to step S115, which will be described later.

The distance calculation unit 14 uses the array created in step S107 to identify a pair of contour pixels that are assumed to face each other when the selected droplet candidate region is a droplet (step S108).

The distance calculation unit 14 calculates the distance between the two contour pixels determined in step S108 as the provisional facing distance 141 (step S109).

When there is a pair of contour pixels not specified in step S108 (No in step S110), the distance calculation unit 14 repeats the processes of steps S108 and S109. When all the contour pixel pairs are specified in step S108 (Yes in step S110), the distance calculation unit 14 calculates the variance of all the distances calculated in step S109 (step S111).

When the number of contour pixels included in the selected droplet candidate region is an odd number, the distance calculation unit 14 determines whether or not the number of contour pixels not specified in step S108 is 1 in step S110. To judge. If the number of unspecified contour pixels is 1, the distance calculation unit 14 proceeds to step S111.

The distance determination unit 15 compares the variance calculated in step S111 with the variance threshold (step S112). When the variance calculated in step S111 is equal to or less than the variance threshold value (Yes in step S112), the distance determination unit 15 determines that the droplet candidate region selected in step S104 is the droplet region (step S113).

When the variance of the distance calculated in step S111 exceeds the variance threshold value (No in step S112), it is determined that the droplet candidate candidate region selected in step S104 is the background region (step S114).

After step S113 or S115, the droplet determination unit 6 determines whether or not there is a droplet candidate region that has not yet been selected in step S104 (step S115). When there is a droplet candidate region (No in step S115), the droplet determination unit 6 returns to step S104 in order to select a new droplet candidate region. When all the droplet candidate regions are selected (Yes in step S115), the droplet determination unit 6 ends the droplet determination unit process (step S3).

The droplet determination unit 6 determines whether or not to drive the droplet removal device 3 after determining all the droplet candidate regions included in the composite image 4A. For example, the droplet determination unit 6 determines to drive the droplet removal device 3 when the number of droplet regions included in the composite image 4A exceeds the reference value.

Alternatively, the droplet determination unit 6 determines to drive the droplet removal device 3 when the area of the droplet region included in the composite image 4A exceeds a preset area reference value. For example, the droplet determination unit 6 may use the integrated value of the number of pixels included in the droplet candidate region determined to be the droplet region as the area of the droplet region.

As described above, the droplet detection device 2 identifies the droplet candidate region 50 based on the level change of the video signal for each pixel in the captured image 1. The droplet detection device 2 can determine whether or not the droplet is attached to the lens of the camera C based on the shape of the specified droplet candidate region 50, and detects the droplet attached to the lens. be able to.

The droplet detection device 2 specifies the maximum value and the minimum value of the level of the video signal in one pixel. Since the droplet detection device 2 calculates the absolute difference between the specified maximum value and the minimum value as the level change in one pixel, the level change in one pixel can be easily specified.

The droplet detection device 2 determines whether or not the droplet is attached to the lens of the camera C by using the contour pixels that form the contour of the specified droplet candidate region 50. That is, the droplet detection device 2 determines whether or not the droplet is attached to the lens of the camera in consideration of the shape of the droplet attached to the lens of the camera C, so that the detection accuracy of the droplet is improved. be able to.

The droplet detection device 2 uses the composite image 4A generated from the captured image 1A and the captured image 1B to determine the amount of information acquired from one captured image in order to determine whether or not the droplet has adhered to the lens. It can be suppressed. As a result, it is possible to detect droplets while preventing the processing load of the droplet detection device 2 from temporarily increasing each time a captured image is acquired.

[Modification example]
In the above embodiment, an example has been described in which the droplet detection device 2 acquires the maximum level change amount of the pixels included in the composite image 4A after generating the composite image 4A from the captured image 1. However, the droplet detection device 2 may acquire the maximum level change amount of each pixel included in the pixel line in parallel with the acquisition of the pixel line.

FIG. 15 is a flowchart showing the operation of the droplet detection device 2 according to the modified example of the embodiment of the present invention. When executing the process shown in FIG. 6, the droplet detection device 2 according to the present modification executes the process shown in FIG. 15 instead of steps S1 and S2. As a result of the processing shown in FIG. 15, a droplet candidate image having the same size as the captured image 1 is generated. Each pixel included in the droplet candidate image indicates whether or not it is included in the droplet candidate region.

The image acquisition unit 4 sets a pixel line to be acquired from the captured image 1 (step S501). For example, the pixel line determination unit 8 generates line designation information 72 indicating an odd-numbered pixel line in the captured image 1A, as in the above embodiment.

Although not shown in FIG. 15, the image acquisition unit 4 initializes the count value of the horizontal synchronization signal to 1 when the step S501 is executed. This is because the pixels included in the captured image 1 are input in order from the top.

The image acquisition unit 4 starts inputting a video signal of the video signal. The image acquisition unit 4 determines whether or not the count value of the horizontal synchronization signal corresponds to the pixel line specified in the line designation information 72 (step S502).

When the count value does not correspond to the pixel line specified in the line designation information 72 (No in step S502), the image acquisition unit 4 waits until the time corresponding to one pixel line elapses (step S508). For example, when the count value is 2 and the line designation information 72 specifies the odd-numbered pixel line from the top, the count value does not correspond to the line designation information 72. In this case, the image acquisition unit 4 waits until the input of the third pixel line from the top is started.

When the count value corresponds to the pixel line specified in the line designation information 72 (Yes in step S502), the image acquisition unit 4 starts acquiring the pixel line (step S503). The image acquisition unit 4 acquires a signal of one pixel (step S504).

The candidate area identification unit 5 acquires the statistical value of the pixel signal acquired in step S504 (step S505). The statistical values are the maximum value and the minimum value. Step S505 corresponds to step S203 shown in FIG. The candidate area identification unit 5 calculates the maximum level change amount of the pixel for which the statistical value is acquired based on the statistical value acquired in step S505 (step S506). Step S506 corresponds to step S204 shown in FIG. The candidate region specifying unit 5 determines whether or not the pixel for which the statistical value is specified is included in the droplet candidate region based on the maximum level change amount of the pixel for which the statistical value is specified (step S507). Step S507 corresponds to steps S205 to S207 shown in FIG.

When both the horizontal synchronization signal and the vertical synchronization signal are not input (No in step S509), the image acquisition unit 4 returns to step S504 to determine whether or not the next pixel is included in the droplet candidate region.

When the horizontal synchronization signal is input (Yes in step S509, Yes in step S510), the image acquisition unit 4 increments the count value of the horizontal synchronization signal (step S511). After that, the image acquisition unit 4 returns to step S502 in order to acquire a new pixel line.

When the vertical synchronization signal is input (Yes in step S509, No in step S510), the image acquisition unit 4 determines that the input of the captured image 1 for one frame is completed. In this case, the image acquisition unit 4 determines whether or not a pixel line capable of generating a droplet candidate image has been acquired (step S512).

In the above embodiment, the composite image 4A is not generated when the odd-numbered pixel lines from the top are acquired from the captured image 1A. Similarly, the candidate region identification unit 5 cannot generate a droplet candidate image at the time when the odd-numbered pixel line from the top is acquired from the captured image 1A. In this case, since the pixel lines for one frame have not been acquired (No in step S512), the image acquisition unit 4 returns to step S501. This is because the even-numbered pixel lines from the top are acquired from the captured image 1B.

When the pixel lines for one frame are acquired from a plurality of captured images 1 (Yes in step S512), the candidate region identification unit 5 generates a droplet candidate image based on the result of step S507 for each pixel (Yes). Step S513). After that, the candidate region specifying unit 5 identifies the droplet candidate region by labeling the droplet candidate image.

Note that step S504 shown in FIG. 8 is executed in parallel with steps S505 to S507.

The image acquisition unit 4 starts acquiring the pixel line La1 from the time t0 when the vertical synchronization signal is input. Since the period from time t0 to time t1 corresponds to the time for one pixel, the image acquisition unit 4 acquires the video signal from time t0 to time t1 as the signal of pixel P1 (step S504). The candidate region specifying unit 5 determines whether or not the pixel P1 is included in the droplet candidate region based on the video signals from the time t0 to the time t1 (steps S505 to S507).

The input of the video signal of the captured image 1A continues even after the time t1. The image acquisition unit 4 acquires the video signal from the time t1 to the time t2 as the signal of the pixel P2 in parallel with the determination of the pixel P1 in the candidate area identification unit 5 (step S504). The candidate region specifying unit 5 determines whether or not the pixel P2 is included in the droplet candidate region based on the signal of the pixel P2. Hereinafter, the acquisition of the signals of the pixels P3 to P10 and the determination that the pixels P3 to P10 are included in the droplet candidate region are repeated. As a result, the acquisition of the video signal of the pixel and the determination of whether or not the pixel is included in the droplet candidate region are executed in parallel.

[Other variants]
In the above embodiment, an example in which the unit area is one pixel has been described, but the present invention is not limited to this. The unit area may be a plurality of pixels arranged in the horizontal direction, or may be a plurality of pixels arranged in the vertical direction. Alternatively, the unit area may be a rectangle including a plurality of pixels arranged in the horizontal direction and a plurality of pixels arranged in the vertical direction.

In the above embodiment, the candidate region specifying unit 5 has described an example of determining whether or not each pixel is included in the droplet candidate region by using the maximum level change amount, but the present invention is not limited to this. The candidate area specifying unit 5 may be able to acquire the level change of each pixel. For example, the candidate region specifying unit 5 may specify the extreme value of the signal of the pixel and specify the variance of the specified extreme value as the level change amount.

In the above embodiment, an example has been described in which the droplet determination unit 6 determines whether or not the droplet candidate region is a droplet based on the statistical variation of the provisional facing distance 141 calculated from the contour pixels. However, it is not limited to this. The algorithm used by the droplet determination unit 6 is not particularly limited as long as the droplet determination unit 6 determines whether or not the droplet candidate region is a droplet based on the shape of the droplet candidate region. For example, the droplet determination unit 6 may use the Hough transform.

In the above embodiment, an example in which the captured image 1 is an analog format image signal has been described, but the present invention is not limited to this. The captured image 1 may be an image signal in a digital format. In this case, the image acquisition unit 4 may generate the captured image 1 in the analog format by performing D / A (Digital to Analog) conversion of the captured image 1 received from the camera.

In the above embodiment, each functional block of the droplet detection device 2 may be individually integrated into one chip by a semiconductor device such as an LSI, or may be integrated into one chip so as to include a part or all of the functional blocks. Although it is referred to as LSI here, it may be referred to as IC, system LSI, super LSI, or ultra LSI depending on the degree of integration.

Further, the method of making an integrated circuit is not limited to the LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connection and settings of the circuit cells inside the LSI may be used.

Further, a part or all of the processing executed by each functional block of the droplet detection device 2 may be realized by a program. A part or all of the processing of each functional block of each of the above embodiments is performed by a central processing unit (CPU) in a computer. A program for performing each process is stored in a storage device such as a hard disk or a ROM, and is read and executed in the ROM or in the RAM.

Further, each process of the above embodiment may be realized by hardware, or may be realized by software (including a case where it is realized together with an OS (operating system), middleware, or a predetermined library). .. Further, it may be realized by mixed processing of software and hardware.

Further, the execution order of the processing methods in the above-described embodiment is not necessarily limited to the description of the above-described embodiment, and the execution order may be changed as long as the gist of the invention is not deviated.

A computer program that causes a computer to perform the above-mentioned method and a computer-readable recording medium on which the program is recorded are included in the scope of the present invention. Here, examples of computer-readable recording media include flexible disks, hard disks, CD-ROMs, MOs, DVDs, DVD-ROMs, DVD-RAMs, large-capacity DVDs, next-generation DVDs, and semiconductor memories. ..

The computer program is not limited to the one recorded on the recording medium, and may be transmitted via a telecommunication line, a wireless or wired communication line, a network represented by the Internet, or the like.

Although the embodiments of the present invention have been described above, the above-described embodiments are merely examples for carrying out the present invention. Therefore, the present invention is not limited to the above-described embodiment, and the above-described embodiment can be appropriately modified and implemented within a range that does not deviate from the gist thereof.

2 Droplet detection device 3 Droplet removal device 4 Image acquisition unit 5 Candidate area identification unit 6 Droplet determination unit 7 Pixel line acquisition unit 8 Pixel line determination unit 9 Synthesis unit 10 Statistical value specification unit 11 Difference calculation unit 12 Area judgment unit 13 Contour identification unit 14 Distance calculation unit 15 Distance judgment unit

Claims (5)

An image acquisition unit that acquires the captured image generated by the camera,
A candidate region specifying unit that specifies a level change of a video signal in a predetermined direction in a captured image acquired by the image acquisition unit for each predetermined unit region and specifies a droplet candidate region based on the specified level change.
A droplet detection device including a droplet determination unit that determines whether or not a droplet is attached to the lens of the camera based on the shape of the droplet candidate region specified by the candidate region identification unit. The droplet detection device according to claim 1.
The candidate area identification unit is
A statistical value specifying unit that specifies the maximum and minimum values of the level of the video signal in the predetermined direction in one unit area, and
A difference calculation unit that calculates the absolute value of the difference between the maximum value and the minimum value specified by the statistical value specification unit as a level change in the one unit area, and a difference calculation unit.
A region determination unit for determining whether or not the one unit region is included in the droplet candidate region based on the result of comparing the difference absolute value calculated by the difference calculation unit with a predetermined threshold value is included. Droplet detector. The droplet detection device according to claim 1 or 2.
The droplet determination unit
A contour identification unit that creates an array of contour pixels that form the contour of the droplet candidate region, and
When the number of the contour pixels is N, the k-th pixel (k is a natural number of 1 or more and N / 2 or less) and the k + N / second pixel in the array of contour pixels created by the contour identification unit. Distance calculation unit that calculates the distance of
A droplet detection device including a distance determination unit that determines whether or not a droplet is attached to the lens of the camera based on the variation in distance calculated by the distance calculation unit. The droplet detection device according to any one of claims 1 to 3.
The image acquisition unit
The first pixel line in the horizontal direction is acquired from the first captured image generated at the first time by the camera, and the second pixel line in the horizontal direction is obtained from the second captured image generated at the second time by the camera. Pixel line acquisition unit to acquire
When the pixel line acquisition unit acquires the first pixel line from the first captured image, the second pixel line is determined based on the position of the first pixel line, and the determined second pixel line is used as the second pixel line. A pixel line determination unit that notifies the pixel line acquisition unit,
A composite unit that generates a composite captured image including a first pixel line and a second pixel line acquired by the pixel line acquisition unit is included.
The candidate region identification unit is a droplet detection device that identifies the candidate region from a composite captured image generated by the composite unit. A droplet detection method that detects droplets adhering to a lens.
Steps to get the captured image generated by the camera,
A step of specifying a level change of a video signal in a predetermined direction in the acquired captured image for each predetermined unit region and specifying a droplet candidate region based on the specified level change.
A droplet detection method comprising a step of determining whether or not a droplet is attached to the lens of the camera based on the shape of the identified droplet candidate region.

Images