JP2024047879A - Dirt Detection System - Google Patents

Abstract

To provide technology with which it is possible to efficiently detect dirt adhered to an object.SOLUTION: A dirt detection system comprises: an image acquisition unit that acquires a captured image in which an object is imaged; a first image processing unit that compares each luminance value of the captured image with a predetermined binary threshold so as to generate a binary image; a second image processing unit that removes noise included in the binary image so as to generate a post-processing image; an examination unit that scans each pixel of the post-processing image at predetermined intervals so as to acquire the luminance value of pixels on the scan line; and a detection unit that compares the consecutive number of pixels in the scanned post-processing image that possess one of two luminance values with a predetermined determination threshold so as to detect dirt adhered to the object.SELECTED DRAWING: Figure 1

Description

This disclosure relates to a dirt detection system.

Conventionally, a technique is known for detecting dirt on an inspection object based on the property that the luminance value of dirty parts is lower than the luminance value of normal parts in an image of the inspection object (Patent Document 1). In this technique, first, the luminance value of each pixel in the image of the inspection object is compared to a reference luminance value, and pixels with a luminance value lower than the reference luminance value are set as dirt candidate data. Next, if the area value of adjacent dirt candidate data is larger than a preset reference area, it is determined that dirt is attached to the inspection object.

JP 2000-38256 A

In conventional technology, whether or not dirt is attached to an object to be inspected is determined by calculating the area value of adjacent dirt candidate data. Therefore, the calculation cost can increase as the size of the image of the object to be inspected increases. Therefore, there is room for improvement in the method of detecting dirt attached to an object to be inspected.

This disclosure can be realized in the following forms:

According to one embodiment of the present disclosure, a dirt detection system is provided. The dirt detection system includes an image acquisition unit that acquires a photographed image of an object, a first image processing unit that generates a binary image by comparing each luminance value of the photographed image with a predetermined binarization threshold, a second image processing unit that generates a processed image by removing noise contained in the binary image, an investigation unit that acquires the luminance value of the pixel on a scanning line by scanning each pixel of the processed image at a predetermined interval, and a detection unit that detects dirt attached to the object by comparing the number of consecutive pixels, which is the number of consecutive pixels having one of the two luminance values, in the scanned processed image with a predetermined judgment threshold. According to this embodiment, the dirt detection system can detect dirt attached to the object by utilizing the fact that the luminance value of each pixel of the photographed image differs depending on the presence or absence of dirt. At this time, the dirt detection system can acquire the number of consecutive pixels, which is the number of consecutive pixels having one of the binarized luminance values, to acquire the degree of collection of pixels having one of the luminance values. Then, the dirt detection system can detect dirt attached to the object by comparing the number of consecutive pixels with the judgment threshold. In this way, dirt on the object can be detected without calculating the area value of pixels having one of the luminance values for the processed image. This makes it possible to suppress an increase in the calculation cost for detecting dirt in accordance with an increase in the size of the captured image. Therefore, dirt on the object can be efficiently detected.
The present disclosure can be realized in various forms other than the above dirt detection system. For example, the present disclosure can be realized in the form of a manufacturing method for a dirt detection system, a control method for a dirt detection system, a computer program for implementing the control method, a non-transitory recording medium on which the computer program is recorded, etc. Also, for example, the first image processing unit may input a captured image to a machine learning model previously stored in a storage unit to obtain a binary image.

FIG. 4 is a diagram for explaining a dirt detection method. FIG. 11 is a diagram for explaining an example of a method for selecting a binarization threshold value. FIG. 4 is a diagram for explaining the processing content of a detection unit.

A. Embodiments:
A dirt detection system is a system that detects dirt such as dust particles and sand grains adhering to the surface of an object to be inspected by using a captured image of the object. Specifically, the dirt detection system detects dirt adhering to the surface of the object by utilizing the fact that the brightness value of each pixel in the captured image differs depending on the presence or absence of dirt.

The dirt detection system includes a camera that photographs an object, and a detection device that detects dirt adhering to the object by performing various image processing on the image captured by the camera. The detection device includes a communication unit, a storage unit, and a CPU. The detection device is communicatively connected to the camera via the communication unit. The storage unit stores various programs that control the operation of the dirt detection system, and various data including image data such as the captured image. The CPU functions as an image acquisition unit, a first image processing unit, a second image processing unit, an investigation unit, and a detection unit by deploying the various programs stored in the storage unit. Details of each function will be described later. The dirt detection system may include a display for displaying the dirt detection results, etc.

Figure 1 is a diagram for explaining a dirt detection method using a dirt detection system. The x-axis and y-axis shown in Figure 1 are an image coordinate system, and the same applies to the following figures and explanations. In this embodiment, the object is a radiator with a mirror-finished surface having projections and recesses. Therefore, in the captured image PF, the luminance value of a pixel corresponding to a portion D with dirt (hereinafter, the dirty portion D) is lower than the luminance value of a pixel corresponding to a portion N without dirt (hereinafter, the normal portion N). Note that in this embodiment, the distance between the object and the camera is constant. In other words, the description will be made assuming that the focal length during shooting is constant.

In step S1, the image acquisition unit acquires the captured image PF. In this embodiment, the captured image PF is a color image having 256 gradations from 0 (black) to 255 (white). Note that the captured image PF may be, for example, a grayscale image.

Next, in step S2, the first image processing unit generates a black and white binary image PB by binarizing the luminance value of each pixel of the captured image PF by comparing the luminance value of each pixel of the captured image PF with a predetermined threshold value (hereinafter, the binarization threshold value). In this embodiment, the first image processing unit converts the luminance value of pixels of the captured image PF that are less than the binarization threshold value to 0 (black) and converts the luminance value of pixels that are equal to or greater than the binarization threshold value to 255 (white) to generate the binary image PB.

FIG. 2 is a diagram for explaining an example of a method for selecting a binarization threshold. When the first image processing unit binarizes the captured image PF by setting an arbitrary luminance value as the binarization threshold, the luminance value at which the rate of change R1 to R5 of the ratio of the number of pixels with a luminance value of 255 to the total number of pixels of the binary image PB (hereinafter, the occupancy rate) is maximum is set as the binarization threshold. In other words, the first image processing unit adopts the luminance value at which the differential coefficient is maximum in the graph GB with the luminance value set as the binarization threshold as the horizontal axis and the occupancy rate as the vertical axis as the binarization threshold. FIG. 2 shows the change in the occupancy rate when the first image processing unit binarizes the captured image PF by setting each luminance value from 0 to 255 as the binarization threshold. When the shape of the graph GB showing the change in the occupancy rate is the shape shown in FIG. 2, the change rate R1 at the first change point P1 (luminance value 100) is greater than the change rate R5 at the fifth change point P5 (luminance value 250). In fact, the binary image PB1, which is obtained by binarizing the captured image PF by setting the luminance value 100 as the binarization threshold, is able to more accurately binarize the dirty part D and the normal part N than the binary image PB5, which is obtained by binarizing the captured image PF by setting the luminance value 250 as the binarization threshold. In addition, in the example shown in FIG. 2, the change rate R1 at the first change point P1 is greater than the change rates R2 to R4 at the other change points P2 to P4. Therefore, in this case, the first image processing unit selects the luminance value 100 as the optimal binarization threshold. In this way, even if there is a local difference in luminance value in the image regardless of the presence or absence of dirt, as in the captured image PF shown in FIG. 1, it is possible to generate a binary image PB that minimizes the influence of the part B1 where the difference in luminance value occurs.

Furthermore, as shown in FIG. 1, when the surface of the object is uneven, the shadow B2 caused by the unevenness may be reflected in the normal portion N. In this case, in the photographed image PF, the luminance value of the pixel corresponding to the shadow B2 caused by the unevenness is greater than the luminance value of the pixel corresponding to the dirty portion D. In other words, the difference between the luminance value of the pixel corresponding to the shadow B2 caused by the unevenness and the luminance value of the pixel corresponding to the normal portion N is smaller than the difference between the luminance value of the pixel corresponding to the dirty portion D and the luminance value of the pixel corresponding to the normal portion N. Therefore, as described above, by setting the luminance value at which the change rate of the occupancy ratio R1 to R5 is maximum as the binarization threshold, even if the shadow B2 occurs in the normal portion N due to the unevenness of the surface of the object, the binary image PB can be generated without converting the luminance value of the pixel corresponding to the shadow B2 to 0 (black). Therefore, it is possible to generate a binary image PB with the influence of disturbances minimized. This makes it possible to more accurately classify the pixels corresponding to the dirty portion D and the pixels corresponding to the normal portion N in the photographed image PF.

Note that the method of selecting the binarization threshold is not limited to this. The binarization threshold may be common to all pixels of the captured image PF. In this case, the binarization threshold may be selected, for example, by Otsu's binarization (discriminant analysis method) in which the luminance value that maximizes the inter-class separation of a histogram representing the distribution of luminance values of each pixel constituting the captured image PF is set as the binarization threshold. The binarization threshold may also be set for each pixel constituting the captured image PF. In this case, the binarization threshold may be selected, for example, by adaptive binarization in which an arbitrary filter size is set for each pixel of the captured image PF and the median or average value of each luminance value in the filter is set as the binarization threshold.

Next, in step S3 shown in FIG. 1, the second image processing unit generates a processed image PA by removing noise contained in the binary image PB. The second image processing unit removes noise contained in the binary image PB, for example, by performing a smoothing process using a median filter on the binary image PB. Note that the method for removing noise contained in the binary image PB is not limited to this. The noise contained in the binary image PB may be removed, for example, by performing a smoothing process using a moving average filter on the binary image PB.

Next, in step S4, the investigation unit acquires the luminance value of each pixel on the scanning line L by scanning each pixel of the processed image PA across a predetermined interval K (for example, 50 pixels). In this embodiment, the investigation unit acquires the luminance value of each pixel on the scanning line L while moving the scanning line L extending along the y direction of the image coordinate system in the x direction as the scanning direction SD with respect to the processed image PA. Note that the direction in which the scanning line L extends, the scanning direction SD which is the direction in which the scanning line L moves, and the interval K when scanning the scanning line L are not limited to the above. For example, the investigation unit may acquire the luminance value of each pixel on the scanning line L while moving the scanning line L extending along the x direction of the image coordinate system in the y direction as the scanning direction SD with respect to the processed image PA.

Next, in step S5, the detection unit obtains the number of consecutive pixels having one of the two luminance values in the scanned processed image PA (hereinafter, the number of consecutive pixels). FIG. 3 is a diagram for explaining the processing content of the detection unit. In this embodiment, the luminance value of the pixel corresponding to the dirty part D in the photographed image PF is lower than the luminance value of the pixel corresponding to the normal part N. Therefore, it is highly likely that the luminance value of the pixel corresponding to the dirty part D in the processed image PA is 0 (black). Therefore, the detection unit obtains the number of pixels on the scanning line L at each of the positions C1 to C7 obtained in step S4 where the luminance value is zero consecutively along the y direction as the number of consecutive pixels A2 and A5. For example, FIG. 3 illustrates a graph G2 representing the distribution of luminance values on the scanning line L at the second position C2 and a graph G5 representing the distribution of luminance values on the scanning line L at the fifth position C5 in the processed image PA shown in FIG. 1. In other embodiments, the numbers of consecutive pixels A2 and A5 may be the number of consecutive pixels in either of the two-dimensional directions defined by the x direction and the y direction.

Next, in step S6 shown in FIG. 1, the detection unit detects dirt on the object by comparing the consecutive pixel counts A2 and A5 with a predetermined threshold value AT (hereinafter, judgment threshold value AT). Specifically, as shown in FIG. 3, when the consecutive pixel count A5 is equal to or greater than the judgment threshold value AT, the detection unit determines that dirt is present at the position of the object (here, area C50 on the fifth position C5). On the other hand, when the consecutive pixel count A2 is less than the judgment threshold value AT, the detection unit determines that dirt is not present at the position of the object (here, area C20 on the second position C2). In this way, the dirt detection system detects dirt on the object by executing each process from step S1 to step S6 shown in FIG. 1. The detection unit may detect dirt on the object by comparing the maximum value of the acquired consecutive pixel counts A2 and A5 with the judgment threshold value AT.

According to the above embodiment, as shown in FIG. 3, the dirt detection system can count the number of adjacent pixels having one brightness value in the processed image PA, thereby obtaining the degree of collection (range) of pixels having one brightness value as the number of consecutive pixels A2, A5. The dirt detection system can detect dirt adhering to the object by comparing the number of consecutive pixels A2, A5 with the judgment threshold AT. In this way, the dirt detection system can detect dirt adhering to the object without calculating the area value of pixels having a brightness value lower than the judgment threshold AT. This makes it possible to suppress an increase in the calculation cost for detecting dirt in accordance with an increase in the size (number of pixels) of the captured image PF. Therefore, since dirt adhering to the object can be detected efficiently, the time required from the time the object is photographed by the camera to the time dirt adhering to the object is detected can be suppressed.

Furthermore, according to the above embodiment, the dirt detection system can identify the location of dirt in the captured image PF when the number of consecutive pixels A2, A5 is equal to or greater than the judgment threshold value AT.

B. Other embodiments:
B-1. Other embodiment 1:
In the above embodiment, the luminance value of the pixel corresponding to the dirty portion D in the photographed image PF is lower than the luminance value of the pixel corresponding to the normal portion N, but this is not limited to this. The luminance value of the pixel corresponding to the dirty portion D in the photographed image PF may be higher than the luminance value of the pixel corresponding to the normal portion N. In this case, the detection unit determines, for example, the number of consecutive pixels with high luminance values in the scanned post-processing image PA as the number of consecutive pixels. Even in this form, it is possible to detect dirt attached to the object by utilizing the fact that the luminance value of each pixel in the photographed image PF differs depending on the presence or absence of dirt.

B-2. Other embodiment 2:
In the above embodiment, the distance between the object and the camera is constant, but this is not limited to this. When the distance between the object and the camera is variable, for example, the binarization threshold is a value obtained by multiplying the binarization threshold (reference value) at a predetermined reference distance between the object and the camera by a coefficient corresponding to the actual distance between the object and the camera. The same applies to the judgment threshold AT. In this form, dirt attached to the object can be detected without fixing the distance between the object and the camera.

The present disclosure is not limited to the above-described embodiments, and can be realized in various configurations without departing from the spirit of the present disclosure. For example, the technical features of the embodiments corresponding to the technical features in each form described in the Summary of the Invention column can be replaced or combined as appropriate to solve some or all of the above-described problems or to achieve some or all of the above-described effects. Furthermore, if a technical feature is not described as essential in this specification, it can be deleted as appropriate.

A2, A5...number of consecutive pixels, AT...judgment threshold, B1...area where difference in brightness occurs, B2...shadow, C1-C7...position of scanning line, C20, C50...area, D...dirty area, G2, G5, GB...graph, K...interval, L...scanning line, N...normal area, P1-P5...change point, PA...processed image, PB, PB1, PB5...binary image, PF...captured image, R1-R5...change rate, SD...scanning direction

Claims (1)

1. A soil detection system comprising:
an image acquisition unit that acquires a captured image of an object;
a first image processing unit that generates a binary image by comparing each luminance value of the captured image with a predetermined binary threshold;
a second image processing unit that generates a processed image by removing noise contained in the binary image;
an investigation unit that scans each pixel of the processed image at a predetermined interval to obtain a luminance value of the pixel on a scanning line;
A dirt detection system comprising: a detection unit that detects dirt attached to the object by comparing the number of consecutive pixels, which is the number of consecutive pixels having one of the two brightness values in the scanned processed image, with a predetermined judgment threshold.