WO2022088620A1

WO2022088620A1 - State detection method and apparatus for camera lens, device and storage medium

Info

Publication number: WO2022088620A1
Application number: PCT/CN2021/088211
Authority: WO
Inventors: 姚兴华; 曾星宇
Original assignee: 北京市商汤科技开发有限公司
Priority date: 2020-10-28
Filing date: 2021-04-19
Publication date: 2022-05-05
Also published as: CN112348784A; KR20220058843A; JP2023503749A

Abstract

A state detection method and apparatus for a camera lens, a device and a storage medium. The method comprises: performing abnormality detection on a target image captured by a camera so as to obtain a final abnormal region in the target image (S11); and analyzing the final abnormal region to determine whether the camera lens is in an abnormal state (S12). According to the method, the detection difficulty of the camera lens can be reduced, and the detection costs are reduced.

Description

Method, device, device and storage medium for state detection of camera lens

CROSS-REFERENCE TO RELATED APPLICATIONS

This disclosure claims the priority of a Chinese patent application with the application number 202011172425.2 and the application title "Method, Apparatus, Equipment and Storage Medium for Condition Detection of Camera Lenses" filed with the China Patent Office on October 28, 2020. This Chinese patent application The entire contents of is hereby incorporated by reference into this disclosure in its entirety.

technical field

The present disclosure relates to the field of detection technology, and in particular, to a state detection method, device, device and storage medium of a camera lens.

Background technique

The camera has an image acquisition function, and the captured image data can be widely used in various computer vision applications such as smart cities, smart transportation, smart factories, smart campuses, smart communities, autonomous driving, and robot perception. Generally, a camera collects the external environment through its lens to obtain image data. Therefore, the lens of the camera serves as the source of data in the entire application and plays an extremely important role.

In real applications, the lens of the camera is exposed to the outdoor environment, and it usually suffers from wind and rain, dust pollution, abnormal occlusion and other abnormal conditions that affect its normal operation. Therefore, it is very important to monitor the status of the lens of the camera and find out the abnormality of the lens of the camera in time.

SUMMARY OF THE INVENTION

Embodiments of the present disclosure provide at least a method, apparatus, device, and storage medium for detecting a state of a camera lens.

A first aspect of the embodiments of the present disclosure provides a method for detecting the state of a camera lens. The method includes: performing abnormality detection on a target image captured by a camera to obtain a final abnormal area in the target image; analyzing the final abnormal area to determine Whether the lens of the camera is in an abnormal state.

Therefore, by performing abnormal detection on the target image captured by the camera, the abnormal state detection of the camera lens is directly realized by using the abnormal area of the image, without relying on other sensors, thus reducing the difficulty of detecting the camera lens and reducing the detection cost.

Wherein, the above-mentioned abnormal detection of the target image captured by the camera to obtain the final abnormal area in the target image includes: performing blur detection on the target image captured by the camera to obtain several candidate abnormal areas in the target image; Select at least one candidate abnormal area in the area as the final abnormal area.

Therefore, by performing blur detection on the target image, the abnormal area existing in the target image can be determined according to the fuzzy condition of the target image, and the abnormal area identification of the target image is realized, so that the camera can be judged according to the final abnormal area obtained later. Anomalies of the lens. Moreover, various types of abnormal conditions of the lens, such as water mist, smudges, occlusion, etc., often cause blurring of some areas in the target image. Therefore, the blur detection of the image can be used to detect all the above abnormal conditions. out to improve detection breadth.

Wherein, performing blur detection on the target image captured by the camera to obtain several candidate abnormal regions in the target image, including: performing preset transformation on the target image captured by the camera to obtain a transformed image, wherein the pixels in the transformed image are The pixel value can reflect the blur information of the transformed image; based on the pixel value of the transformed image, the candidate abnormal area is determined.

Therefore, by directly performing preset transformation on the target image, it is possible to detect the blurriness of the target image, and then obtain candidate abnormal regions.

Wherein, performing preset transformation on the target image captured by the camera to obtain the transformed image includes: preprocessing the target image to obtain the preprocessed image; performing Laplacian transformation on the preprocessed image to obtain the transformed image . The above-mentioned determination of the candidate abnormal region based on the pixel value of the transformed image includes: performing binarization processing based on the transformed image to obtain a binarized image; finding out the pixel points whose pixel value satisfies a preset pixel condition from the binarized image, to form candidate abnormal regions.

Therefore, the value obtained after Laplacian transformation can reflect the blur information of the changed image, making the edge of the blurred area more obvious. At the same time, the blurred area in the transformed image can be preliminarily obtained by the size of the obtained pixel value. In addition, by performing image binarization, the amount of data can be greatly reduced, which can improve the operation speed and speed up the detection of candidate abnormal areas; Candidate abnormal regions are highlighted.

Wherein, the above-mentioned preprocessing of the target image to obtain the preprocessed image includes: performing grayscale processing on the target image to obtain the preprocessed image. The above-mentioned binarization processing based on the transformed image to obtain a binarized image includes: filtering the transformed image to obtain a filtered image; the filtering processing includes morphological closing operation; and binarizing the filtered image to obtain a binarized image deal with. The above-mentioned finding out the pixel points whose pixel value meets the preset pixel condition from the binarized image to form the candidate abnormal area includes: inverting the pixel value of the binarized image to obtain an inverse binarized image; Pixels whose pixel values satisfy preset pixel conditions are found in the valued image to form candidate abnormal regions.

Therefore, by performing grayscale processing on the target image, the amount of data that needs to be calculated can be reduced, and the calculation speed can be improved. Through filtering processing such as morphological closing operation, the interference information inside the unblurred area can be eliminated, the accuracy of subsequent detection of the blurred area can be improved, and the accuracy of judging whether the camera lens is in an abnormal state can be improved. In addition, through the inversion operation, the subsequent processing of the blurred area can be facilitated, the operation speed is improved, and the detection difficulty is reduced.

Wherein, selecting at least one candidate abnormal area from the above several candidate abnormal areas as the final abnormal area includes: obtaining a first area of each candidate abnormal area; selecting a candidate whose first area satisfies a preset area condition from the several candidate abnormal areas The abnormal area is regarded as the pending abnormal area; at least one pending abnormal area is determined as the final abnormal area.

Therefore, by further analyzing the candidate abnormal area to determine the final abnormal area, the fuzzy area can be more accurately identified.

Wherein, the above-mentioned preset area condition is that the first area is greater than the first preset area threshold. The above-mentioned determination of at least one undetermined abnormal area as the final abnormal area includes: determining at least one statistical value of the undetermined abnormal area in the target image; wherein, the at least one statistical value of the undetermined abnormal area includes the saturation of the undetermined abnormal area obtained by statistics. At least one first statistical value, and/or, at least one second statistical value obtained by counting the pixel values of the abnormal area to be determined; if it is determined that at least one statistical value of the abnormal area to be determined satisfies the preset statistical conditions, the abnormal area to be determined is determined is the final abnormal area.

Therefore, by determining the statistical value of the abnormal area to be determined to further determine the blurring of the abnormal area to be determined, the accuracy of the blur detection can be improved, thereby improving the accuracy of the state detection of the camera lens.

The first statistical value and the second statistical value respectively include at least one of a mean value and a variance, and the above-mentioned preset statistical conditions include: each statistical value of the undetermined abnormal area is greater than a preset threshold value corresponding to the statistical value. The above-mentioned determining at least one statistical value of the undetermined abnormal area in the target image includes: generating at least one mask corresponding to the undetermined abnormal area based on the pixel position information of the undetermined abnormal area, wherein the at least one mask includes an area mask and/or Boundary mask; in the case where at least one mask includes a region mask, obtain a saturation image corresponding to the target image, and use the region mask corresponding to the undetermined abnormal region to obtain a saturation image corresponding to the undetermined abnormal region the first area to be counted; perform statistics on the saturation of the first area to be counted to obtain at least one first statistic value of the unusual area to be determined; and/or, in the case that at least one mask includes a boundary mask, use and The boundary mask corresponding to the undetermined abnormal area is obtained in the target image and the second to-be-statistical area corresponding to the undetermined abnormal area is obtained; the pixel values of the second to-be-statistical area are counted to obtain at least one second statistical value of the undetermined abnormal area .

Therefore, the mask is used to extract the area related to the abnormal area to be determined in the target image, so as to realize the accurate statistics of the statistical value of the abnormal area to be determined, and improve the accuracy of subsequent blur detection.

The above analysis of the final abnormal area to determine whether the lens of the camera is in an abnormal state includes: obtaining the second area of the final abnormal area; judging whether the second area of the final abnormal area satisfies the preset area condition; if so, determining The target image is in an abnormal state; when it is detected that there are at least a second preset number of frames in an abnormal state in the target image of the continuous first preset number of frames, it is determined that the camera lens is in an abnormal state; wherein the first preset number and the first preset number of frames are in an abnormal state; 2. The preset number is a positive integer.

Therefore, the abnormal state of the target image is determined by using the area of the final abnormal area in the target image, and whether the camera lens is in an abnormal state is determined based on the state of the target image of a preset number of frames, thereby realizing the abnormality detection of the camera lens, and, When the preset number is greater than 1, the state of the target images of consecutive multiple frames is used to comprehensively determine whether the camera lens is in an abnormal state, so the detection accuracy of the state of the camera lens can be improved.

Wherein, judging whether the second area of the final abnormal area satisfies the preset area condition includes: judging whether the second area of the final abnormal area satisfies the first preset area condition or the second preset area condition. If the above-mentioned conditions are satisfied, then determining that the target image is in an abnormal state includes: if the first preset area condition is satisfied, determining that the target image is in the first abnormal state; if the second preset area condition is satisfied, determining that the target image is in the second abnormal state abnormal state. The above-mentioned determining that the camera lens is in an abnormal state when it is detected that there are at least a second preset number of frames in an abnormal state in the target image of the continuous first preset number of frames includes: detecting that the continuous first preset number of frames of the target image is in an abnormal state. In the case where the images are all in the first abnormal state, it is determined that the camera lens is in the first abnormal state. When it is detected that the target images of the first preset number of consecutive frames are not uniformly in the first abnormal state, and there are at least the second preset number of frames in the first abnormal state. In the case of an abnormal state or a second abnormal state, it is determined that the camera lens is in the second abnormal state.

Therefore, by further dividing the abnormal state of the camera lens, a more accurate judgment of the abnormal state of the camera lens can be realized.

Wherein, the above-mentioned first preset area condition is that the second area is greater than the second preset area threshold; the second preset area condition is that the second area is greater than the second preset area threshold and less than the third preset area threshold, wherein, The second preset area threshold is greater than the third preset area threshold.

A second aspect of the embodiments of the present disclosure provides a device for detecting a state of a camera lens, the device including: a region detecting part and a state analyzing part.

The area detection part is configured to perform abnormality detection on the target image captured by the camera to obtain the final abnormal area in the target image. The state analysis part is configured to analyze the final abnormal area to determine whether the lens of the camera is in an abnormal state.

A third aspect of the embodiments of the present disclosure provides an electronic device, including a mutually coupled memory and a processor, where the processor is configured to execute a computer program stored in the memory to implement the method for detecting the state of a camera lens in the first aspect.

A fourth aspect of the embodiments of the present disclosure provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, implements the method for detecting the state of a camera lens in the first aspect.

A fifth aspect of the embodiments of the present disclosure provides a computer program, including computer-readable codes. When the computer-readable codes are executed in an electronic device, a processor in the electronic device executes a process for implementing the camera lens in the first aspect. Status detection method.

The above solution, by performing abnormal detection on the target image captured by the camera, directly uses the abnormal area of the image to detect the abnormal state of the camera lens without relying on other sensors, thus reducing the difficulty of detecting the camera lens and reducing the detection cost. .

It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

Description of drawings

The accompanying drawings, which are incorporated into and constitute a part of this specification, illustrate embodiments consistent with the present disclosure, and together with the description, serve to explain the technical solutions of the present disclosure.

FIG. 1A is a schematic diagram 1 of an application scenario of an electronic device provided by an embodiment of the present disclosure;

FIG. 1B is a second schematic diagram of an application scenario of an electronic device method provided by an embodiment of the present disclosure;

2 is a schematic flowchart of a first embodiment of a method for detecting a state of a camera lens provided by an embodiment of the present disclosure;

3 is a first schematic flowchart of a second embodiment of a method for detecting a state of a camera lens provided by an embodiment of the present disclosure;

4 is a second schematic flowchart of a second embodiment of a method for detecting a state of a camera lens provided by an embodiment of the present disclosure;

FIG. 5 is a third schematic flowchart of the second embodiment of the state detection method for a camera lens provided by an embodiment of the present disclosure;

FIG. 6 is a fourth schematic flowchart of the second embodiment of the state detection method of the camera lens provided by the embodiment of the present disclosure;

7 is a fifth schematic flowchart of the second embodiment of the camera lens state detection method provided by the embodiment of the present disclosure;

8 is a schematic diagram of an application scenario of a method for detecting a state of a camera lens provided by an embodiment of the present disclosure;

9 is a schematic diagram of a processing flow of an abnormal area detection module provided by an embodiment of the present disclosure;

10 is a schematic diagram of a grayscale image in each scene provided by an embodiment of the present disclosure;

11 is a schematic diagram of a transformed image in each scene provided by an embodiment of the present disclosure;

12 is a schematic diagram of filtered images in various scenarios provided by an embodiment of the present disclosure;

13 is a schematic diagram of an inverse binarized image in each scene provided by an embodiment of the present disclosure;

14 is a schematic diagram of candidate abnormal regions in various scenarios provided by an embodiment of the present disclosure;

FIG. 15 is a schematic frame diagram of an embodiment of a state detection device for a camera lens provided by an embodiment of the present disclosure;

16 is a schematic diagram of a framework of an embodiment of an electronic device provided by an embodiment of the present disclosure;

FIG. 17 is a schematic framework diagram of an embodiment of a computer-readable storage medium provided by an embodiment of the present disclosure.

Detailed ways

The solutions of the embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

In the following description, for purposes of explanation and not limitation, specific details are set forth, such as specific system structures, interfaces, techniques, etc., in order to provide a thorough understanding of the present disclosure.

The term "and/or" in this article is only an association relationship to describe the associated objects, indicating that there can be three kinds of relationships, for example, A and/or B, it can mean that A exists alone, A and B exist at the same time, and A and B exist independently B these three cases. In addition, the character "/" in this document generally indicates that the related objects are an "or" relationship. Also, "multiple" herein means two or more than two. In addition, the term "at least one" herein refers to any combination of any one of the plurality or at least two of the plurality, for example, including at least one of A, B, and C, and may mean including from A, B, and C. Any one or more elements selected from the set of B and C.

The camera described in the embodiments of the present disclosure can be any device that can realize image acquisition, such as a video camera, a camera, a mobile phone and other terminal devices, and the lens of the camera is the optical component of the device for forming an image. It can be understood that as long as it is a device capable of capturing images or videos, the method for detecting the state of a camera lens described in the embodiments of the present disclosure can be applied. In some possible implementations, the method for detecting the state of a camera lens described in the embodiments of the present disclosure may be implemented by a processor calling a computer program stored in a memory.

The execution subject of the method for detecting the state of the camera lens provided by the embodiment of the present disclosure may be an electronic device.

In a possible implementation manner, referring to the schematic diagram 1 of the application scenario of the electronic device shown in FIG. 1A , the electronic device may include a processor 11 and a camera 12 . In this way, the electronic device 11 can collect the target image through the camera 12 , and the processor 12 can detect the target image. The target image is analyzed to determine whether the lens of the camera 12 is in an abnormal state. For example, the electronic device may be implemented as a cell phone.

In another possible implementation manner, referring to the second schematic diagram of an electronic device application scenario shown in FIG. 1B , the electronic device 10 can receive the real-time captured target image transmitted by other devices 13 through the network 14 . In this way, the electronic device 10 can analyze and process the received target image, so as to determine whether the camera lens of the other device 13 is in an abnormal state. For example, the electronic device can be implemented as a computer, and the computer can receive the target image collected by the camera through the network.

Please refer to FIG. 2 , which is a schematic flowchart of a first embodiment of a method for detecting a state of a camera lens according to an embodiment of the present disclosure. Specifically, the following steps can be included:

Step S11: Perform abnormality detection on the target image captured by the camera to obtain the final abnormal area in the target image.

In this embodiment of the present disclosure, an image captured by a camera is defined as a target image. The target image may be a certain frame image in the video captured by the camera lens, or may be a single image captured directly. In some disclosed embodiments, the target image may be a color image or a grayscale image.

After the target image is obtained, abnormality detection can be performed directly on the target image to obtain the final abnormal area in the target image. Anomaly detection is to detect the abnormal area in the target image. The abnormal area of the target image can be understood as the area without normal imaging, such as the fuzzy area in the target image, or the whole target image is black, etc. . Among them, the situations that cause the target image to have an abnormal area are, for example, that the lens is contaminated with water mist, stains, the lens is blocked, the lens is out of focus, and so on.

After the abnormality detection is performed on the target image, the obtained area with abnormality in the target image is defined as the final abnormal area. The final abnormal area may be a part of the target image, and the number may be one or more, and the final abnormal area may also be that the entire target image is an abnormal area.

Step S12: Analyze the final abnormal area to determine whether the lens of the camera is in an abnormal state.

After the abnormality detection is performed, the obtained final abnormal area can be analyzed to more accurately determine whether the lens is in an abnormal state. For example, the position, size, shape, pixel information, etc. of the final abnormal area can be analyzed. It can be understood that all the information about the final abnormal area can be used as basic data for analysis. In some disclosed embodiments, a plurality of basic data for analysis can be combined, and whether the lens of the camera is in an abnormal state can be more accurately determined through the analysis of the plurality of data.

During the analysis, the analysis can be performed in a corresponding manner according to the type of the basic data of the final abnormal area to be analyzed. For example, the saturation analysis method can be used to calculate the saturation of the final abnormal area, and then the saturation of the final abnormal area can be used to determine whether the camera lens is in an abnormal state, or the pixel point statistics method can be used to calculate the final abnormal area. The included pixels are used to determine the size of the final area.

In the embodiment of the present disclosure, by performing abnormal detection on the target image captured by the camera, the abnormal state detection of the camera lens is directly realized by using the abnormal area of the image, without relying on other sensors, thus reducing the difficulty of detecting the camera lens, and Reduced inspection costs.

Please refer to FIG. 3 . FIG. 3 is a first schematic flowchart of a second embodiment of a state detection method for a camera lens according to an embodiment of the present disclosure. This embodiment is further described on the basis of the above-mentioned first embodiment. Specifically, the following steps may be included:

Step S21: Perform blur detection on the target image captured by the camera to obtain several candidate abnormal regions in the target image.

Generally speaking, when the lens of the camera is in an abnormal state, the target image captured by the camera will have a blurred area, that is, there is an unclear area in the target image. Therefore, in this embodiment, blur detection may be performed on the target image first to preliminarily determine the abnormality of the target image. After fuzzy detection, fuzzy areas in the target image can be obtained, which are defined as candidate abnormal areas. When the candidate abnormal region is a part of the target image, the number of candidate abnormal regions may be one or more, and the specific number is determined according to the blur detection result.

Wherein, the blur situation can be determined according to the transformation result by performing relevant transformation on the target image. Specifically, an example may be described with reference to FIG. 4 . FIG. 4 is a schematic flowchart of a second embodiment of a method for detecting a state of a camera lens according to an embodiment of the present disclosure. In a disclosed embodiment, blur detection is performed on a target image captured by a camera to obtain several candidate abnormal regions in the target image, which can be specifically implemented by the following steps:

Step S211: Perform preset transformation on the target image captured by the camera to obtain a transformed image, wherein the pixel value of each pixel in the transformed image can reflect the blur information of the transformed image.

A preset transformation is performed on the target image, and the transformed image is defined as a transformed image. After the preset transformation, the pixel value of each pixel in the obtained transformed image can reflect the blur information of the transformed image. Since the preset transformation is performed on each pixel point, the pixel value corresponding to each pixel point can be obtained accordingly. The pixel value of the pixel can reflect the blur information of the changing image, which can be reflected by the change trend of the pixel value of the pixel. For example, the boundary between the blurred area and the non-blurred area can be made more obvious by the change trend of the pixel value. , that is, to make the candidate anomaly regions more obvious. In addition, the size of the pixel value of the pixel point can also reflect whether the area where the pixel point is located is blurred. That is, the fuzzy area and the non-blurred area can be preliminarily determined by judging the size of the pixel value of the pixel point.

The preset transformation may be to process the information contained in each pixel in the target image, and the value obtained after processing is the pixel value. For example, when the target image is a color image, it can be processed according to the values of the red (R), green (G), and blue (B) three color channels of each pixel of the target image, and the final value is the pixel value. The preset transformation is performed by using, for example, a Brenner gradient function, a Tenengrad gradient function, a Laplacian (Laplace) gradient function, or an SMD (grayscale variance) function. It can be understood that any method that can be used to detect the blurring of an image can be applied to the embodiments of the present disclosure.

In a disclosed embodiment, a preset transformation is performed on a target image captured by a camera to obtain a transformed image, which specifically includes the following steps:

Step S2111: Preprocess the target image to obtain a preprocessed image.

When performing preset transformation on the target image, the target image may be preprocessed first, and the obtained image is the preprocessed image. The preprocessing may be to process the information contained in each pixel in the image, for example, to process the RGB three-channel information of the target image.

In a disclosed embodiment, the preprocessing may be to perform grayscale processing on the target image, and the obtained grayscale image is the preprocessed image. For example, the target image can be grayscaled according to the following formula:

Gray=R×0.299+G×0.587+B×0.114 (1)

In the above formula (1), R, G, and B respectively represent the values of the three color channels of red (R), green (G), and blue (B) of a pixel, and the obtained value Gray is the value of the pixel. grayscale value. It can be understood that the specific calculation formula of the grayscale processing can be adjusted accordingly according to the specific situation.

Through grayscale processing, the information contained in each pixel can be converted from RGB three-channel to single-channel information of gray value, which can simplify the amount of data in subsequent processing of the information contained in the pixel. Improve operation speed.

Step S2112: Laplace transform is performed on the preprocessed image to obtain a transformed image.

The preprocessed image obtained after preprocessing can be processed by using the Laplacian gradient function transformation, and the obtained image is the transformed image. By using the Laplacian transform, the real blur area in the preprocessed image can be detected to obtain the blurry area in the preprocessed image. For example, the specific process of Laplacian is to first use the Sobel operator to calculate the second-order x and y differences, and then sum them up. The formula is as follows:

In the above formula (2), x and y are the coordinates of each pixel in the pixel coordinates, respectively, src represents the input image, and dst represents the output image.

In this way, after Laplacian transformation, the obtained value is the pixel value of each pixel point, and the pixel value can reflect the blur information of the changed image, for example, to make the edge of the blurred area more obvious, and at the same time, the obtained pixel value can be obtained by Judging the size of the image, the blurred area in the transformed image is initially obtained.

Please continue to refer to Figure 4.

Step S212: Determine candidate abnormal regions based on pixel values of the transformed image.

Because the pixel value of the transformed image can reflect the blur information of the transformed image, the candidate abnormal region can be determined based on the pixel value of the transformed image. Based on the pixel value of the transformed image, it means that the candidate abnormal area can be determined according to the change trend of the pixel value of the transformed image, the overall distribution of the pixel value, or the size of the pixel value and other information related to the pixel value. That is, the candidate abnormal region may be determined according to the pixel values of a part of the transformed image, or may be determined according to the pixel points of the entire transformed image. For example, a region enclosed by a range in which the change trend of the pixel value of the pixel point exceeds a certain level can be determined as a candidate abnormal region.

Wherein, the candidate abnormal region can be determined from the transformed image by binarizing the transformed image. Specifically, it can be illustrated with reference to FIG. 5 , which is a third schematic flowchart of the second embodiment of the state detection method of the camera lens according to the embodiment of the present disclosure. In a disclosed embodiment, “determining candidate abnormal regions based on pixel values of the transformed image” can be specifically implemented through the following steps:

Step S2121: Perform binarization processing based on the transformed image to obtain a binarized image.

In order to further make the difference between the pixel values of each pixel more obvious, and to promote the boundary between the blurred area and the non-blurred area in the transformed image to be more obvious, so as to accurately determine the candidate abnormal area, the transformed image can be binarized. For example, the transformed image can be thresholded first. Threshold segmentation is to classify the pixel value of the pixel, for example, a feature threshold is set, the pixels whose pixel value is less than or equal to the feature threshold are classified into one category, and the pixels whose pixel value is greater than the feature threshold are classified into one category. The threshold is, for example, any number from 50 to 80, such as 50.

After threshold segmentation, the pixel value of each pixel in the transformed image is divided into two categories. At this time, a binarization operation can be performed to make the difference between the pixel values of the two types of pixel points more obvious. For example, the pixel degree of a pixel whose pixel value is greater than the feature threshold may be set as the first preset pixel value, and the pixel value of the pixel whose pixel value is less than or equal to the feature threshold may be set as the second preset pixel value; The pixel value may be 0, and the second preset pixel value may be 255; or the first preset pixel value may be 255, and the second preset pixel value may be 0. Of course, the above-mentioned first preset pixel value and second preset pixel value may also be set to other pixel values, which are not limited herein.

Through the above image binarization, the amount of data contained in the image can be greatly reduced, thereby improving the operation speed and speeding up the detection of candidate abnormal regions. At the same time, the boundary of the blurred area in the binarized image can be made more obvious, and the candidate abnormal area can be highlighted.

In one disclosed embodiment, "performing threshold segmentation and image binarization based on the transformed image to obtain a binarized image" may specifically include the following steps 1 and 2:

Step 1: Filter the transformed image to obtain a filtered image.

In the transformed image obtained after the preset transformation, within the unblurred area, there may be a large difference between the pixel values of a small part of the pixels and the pixel values of other pixels. The reason for these differences may be that the color information or gray level of the target image in the non-blurred area changes greatly, while the color information or gray level change in the blurred area is very small, even close to 0. Compared with other pixel points, the pixel values of some pixel points after the preset transformation are quite different. Reflected in the transformed image, that is, in the interior of the non-blurred area, there may be holes (point-shaped collection of larger difference pixels) or small cracks (stripe-shaped collection of larger difference pixels), due to these Holes or small cracks do not belong to the fuzzy area. In order to avoid these holes and small cracks from affecting the subsequent fuzzy detection, filtering can be used to eliminate these holes or small cracks and connect the unambiguous areas together, and Keep the overall position and shape of the unblurred area unchanged.

Specifically, the filtering process may include, but is not limited to, a morphological closing operation. Morphological closing operation is to perform dilation operation first, and then perform erosion operation.

The dilation operation can be used to fill in the holes of the unblurred areas and to remove the small grain noise (interference information in the unblurred areas) contained in the unblurred areas. The formula is as follows:

Among them, A can be a non-blurred area, and B can be a structural element. The unblurred area A is expanded by the structural element B.

The erosion operation can cause the boundary of the unblurred area to shrink, which can be used to eliminate some small holes or small cracks. The formula is as follows:

After the transformed image is subjected to morphological closing operation, the obtained image is defined as a filtered image. At this time, the interference information in the unblurred area has been eliminated by the morphological closing operation in the filtered image, which can improve the accuracy of subsequent detection of the blurred area and the accuracy of judging whether the camera lens is in an abnormal state.

Step 2: Binarize the filtered image to obtain a binarized image.

After removing the holes or small cracks in the non-blurred area by filtering, in order to make the boundary between the non-blurred area and the fuzzy area more obvious and simplify the calculation amount, the above binarization process can be used, and the image obtained after processing is a binary value. image.

Continue to refer to Figure 5.

Step S2122: Find out the pixel points whose pixel value meets the preset pixel condition from the binarized image to form a candidate abnormal area.

After the binarized image is obtained, the pixel values of the pixels in the binarized image at this time are only two. At this time, the pixel points whose pixel values satisfy the preset pixel condition can be found, and the area composed of these pixel points is the candidate abnormal area. For example, when the pixel value of the pixel point in the unblurred area is set to a smaller value, the point with the larger pixel value in the binarized image can be selected to form the blurred area. In a specific implementation scenario, the pixel value of the pixel point in the unblurred area can be set to 255, and the pixel point with the pixel value of 255 can be found by using the cv::findContours function in the OpenCV open source computer vision library.

In a disclosed embodiment, when performing an image binarization operation on the transformed image, the pixel value of the blurred area may be set to 0. Since the pixel value of the blurred area is 0, it is inconvenient for subsequent processing of the blurred area. The speed of operation reduces the difficulty of calculation. At this time, the following steps can be performed to realize "find out the pixel points whose pixel value meets the preset pixel condition from the binarized image to form a candidate abnormal area", which specifically includes the following steps 1 and 2:

Step 1: Invert the pixel values of the binarized image to obtain an inverse binarized image.

After the image binarization operation is performed, if the pixel value of the blurred area is set to 0 and the unblurred area is set to a value greater than 0, the inversion operation can be performed, that is, the pixel values of the unblurred area and the blurred area can be exchanged . For example, when the pixel value of the blurred area is 0 and the pixel value of the unblurred area is 255, after the inversion operation is performed, the pixel value of the blurred area is 255, and the pixel value of the unblurred area is 0. In this way, the pixel value of the blurred area can be made not to be 0, which can facilitate subsequent processing of the blurred area, improve the operation speed, and reduce the difficulty of detection.

In a disclosed embodiment, for threshold segmentation, the formulas for image binarization and inversion operations are as follows:

Among them, thresh is the pixel threshold of the preset value, and max Val is the maximum pixel value.

After inverting the pixel values of the binarized image, the obtained image is defined as an inverse binarized image.

Step 2: Find out the pixel points whose pixel value satisfies the preset pixel condition from the inverse binarization image to form a candidate abnormal area.

In the inverse binarization image, it is also possible to find out the pixel points whose pixel value satisfies the preset pixel condition, and the pixel point whose pixel value satisfies the preset pixel condition. Since the pixel value of the blurred area is greater than 0, the preset pixel condition may be that the pixel value of the pixel point is greater than 0. The way to find pixels can be to use the cv::findContours function in the OpenCV open source computer vision library to find them.

Please continue to refer to Figure 3.

Step S22: Select at least one candidate abnormal area from several candidate abnormal areas as the final abnormal area.

After obtaining the candidate abnormal regions, at least one of the candidate abnormal regions can be selected as the final abnormal region. For example, after performing certain processing on the candidate abnormal regions, at least one of them can be selected as the final abnormal region.

Please refer to FIG. 6 , FIG. 6 is a fourth schematic flowchart of the second embodiment of the state detection method of the camera lens according to the embodiment of the present disclosure. In one disclosed embodiment, in order to more accurately identify the ambiguous area, the candidate abnormal area may be further analyzed to determine the final abnormal area. For example, "select at least one candidate abnormal area from several candidate abnormal areas as the final abnormal area" may specifically include the following steps:

Step S221: Obtain the first area of each candidate abnormal region.

The area of the candidate abnormal region is defined as the first area. The area of the candidate abnormal area can be determined by counting the number of pixels contained in the candidate abnormal area. For example, if the number of pixels contained in a candidate abnormal area is 100, the first area size of the candidate abnormal area is 100 pixel size.

Step S222: From several candidate abnormal areas, select a candidate abnormal area whose first area satisfies a preset area condition as a pending abnormal area.

After the area of the candidate abnormal area is obtained, the candidate abnormal area can be screened by judging whether the area of the candidate abnormal area satisfies the preset area condition, and the candidate abnormal area that satisfies the preset area condition is defined as the pending abnormal area.

In a specific implementation scenario, the preset area condition may be that the first area is greater than the first preset area threshold. The first preset area threshold is, for example, any value of 2500-3000 pixels in size, such as 2500 pixels in size. Since the area of the fuzzy area is generally large, by setting the first area threshold, the candidate abnormal area with a small area can be excluded, and the accuracy of identifying the fuzzy area is improved.

Step S223: Determine at least one pending abnormal area as the final abnormal area.

After the candidate abnormal areas are screened, at least one abnormal area to be determined can be selected as the final abnormal area. In the specific selection, after a certain processing of the to-be-determined abnormal area, at least one of them can be selected as the final abnormal area.

Please refer to FIG. 7 , FIG. 7 is a fifth schematic flowchart of the second embodiment of the state detection method of the camera lens according to the embodiment of the present disclosure. In a disclosed embodiment, during further analysis of the pending abnormal area, "determining at least one pending abnormal area as the final abnormal area" may specifically include the following steps:

Step S2231: Determine at least one statistical value of the abnormal region to be determined in the target image.

After obtaining the pending abnormal area in the target image, processing may be performed on the pending abnormal area to further determine whether the pending abnormal area is the final abnormal area.

In the embodiment of the present disclosure, statistics may be performed on the pending abnormal area to obtain statistical values related to the pending abnormal area. For example, further analysis can be carried out by using the saturation information of the target image to determine the abnormal area. Specifically, at least one first statistical value may be obtained by performing statistics on the saturation of the abnormal region to be determined. Alternatively, the pixel values of the to-be-determined abnormal region can also be counted, so as to obtain at least one second statistical value. It can be understood that when implementing the state detection method for a camera lens described in this disclosure, the first statistical value and the second statistical value can be obtained as the statistical value of the pending abnormal area, and one of them can also be selected as the statistical value of the pending abnormal area. value.

In a disclosed embodiment, the calculation formula of saturation is as follows:

Among them, max(R, G, B) is the maximum value obtained after processing the RGB value of each pixel in the target image, and min(R, G, B) is the pixel value of each pixel in the target image. The minimum value obtained after processing the RGB values, S represents the saturation of the target image. The method for processing the RGB values of the pixels in the target image is, for example, the above-mentioned formula (1).

In a disclosed embodiment, the above-mentioned first statistical value and second statistical value respectively include at least one of a mean value and a variance. That is, the at least one first statistical value includes the mean value and/or the variance of the saturation of the abnormal area to be determined, and the at least one second statistical value includes the mean value and/or the variance of the pixel values of the abnormal area to be determined.

In a disclosed embodiment, step S2231 may specifically include the following steps 1 to 3.

Step 1: Based on the pixel position information of the abnormal region to be determined, generate at least one mask corresponding to the abnormal region to be determined, wherein the at least one mask includes an area mask and/or a boundary mask.

After obtaining the pixel points constituting the undetermined abnormal area, the position of the pixel points can be determined by analyzing the position information, such as coordinates, of the pixel points in the undetermined abnormal area. In addition, it is also possible to generate a mask corresponding to the undetermined abnormal area according to the pixel position information of the undetermined abnormal area. The mask corresponding to the undetermined abnormal area is consistent with the shape and size of the entire undetermined abnormal area, or only the shape and size of a part of the undetermined abnormal area. Masks can be used to block out pending anomaly regions. In a specific implementation scenario, the mask may be an image, a figure, etc., used to block the abnormal area to be determined.

When generating a mask, a mask corresponding to a complete undetermined abnormal region is a region mask; generating a mask corresponding to the boundary of a pending abnormal region is a boundary mask. It will be appreciated that the number and shape of the generated masks are not limited. For example, an area mask and a boundary mask corresponding to each pending anomaly area may be generated for all pending anomaly areas.

Step 2: In the case where at least one mask includes an area mask, obtain a saturation image corresponding to the target image, and use the area mask corresponding to the undetermined abnormal area to obtain a saturation image corresponding to the undetermined abnormal area. The first area to be counted; the saturation of the first area to be counted is counted to obtain at least one first statistical value of the to-be-determined abnormal area.

In step 1, if the generated mask includes a region mask, a saturation image corresponding to the target image can be obtained at this time. Specifically, saturation calculation can be performed on each pixel of the target image, so as to obtain a saturation image corresponding to the target image. The saturation calculation is, for example, converting an RGB image to a hue-saturation-lightness (HSV) image and extracting the value of the saturation S channel.

After obtaining the saturation image, the corresponding position of the undetermined abnormal area on the saturation image, that is, the first area to be counted, can be obtained through the position information, such as coordinates, of the pixels of the undetermined abnormal area in the target image. For example, the coordinates of a certain pixel in the undetermined abnormal area on the target image are (1,1), then in the saturation image, the coordinates of the pixel corresponding to the pixel are also (1,1). Therefore, the first to-be-statistical area corresponding to the to-be-determined abnormal area can be obtained in the saturation image.

In a disclosed embodiment, it is also possible to directly use the position information of the pixels in the abnormal area to be determined, and directly extract the corresponding pixels in the saturation image to form the first area to be counted, and then perform saturation statistics. .

At the same time, the saturation of the first region to be counted may also be counted, and a value obtained from the count is defined as the first statistic value. The statistics on the saturation of the first to-be-statistical area may be performed on the entire to-be-determined abnormal area, or may be performed on a certain part of the to-be-determined abnormal area. The statistical method is not limited, for example, the average saturation, variance, standard deviation, etc., of the undetermined abnormal area can be counted. When statistics are performed on different parts of the abnormal area to be determined, and various types of statistics are performed on the abnormal area to be determined, several different statistical values can be obtained.

In addition, in step 1, if the generated mask includes a boundary mask, step 3 can be performed at this time.

Step 3: Using the boundary mask corresponding to the abnormal area to be determined, obtain a second area to be counted corresponding to the abnormal area to be determined in the target image; Count the pixel values of the second area to be counted to obtain at least one area of the abnormal area to be determined. Second statistic.

After the boundary mask corresponding to the undetermined abnormal area is generated, the area covered by the boundary mask is the second to-be-statistical area corresponding to the undetermined abnormal area. At this time, the pixel values of the second area to be counted may be counted, for example, the pixel values are grayscale values, and the statistical value obtained by the statistics is the second statistical value.

It can be understood that, if the area mask and the mask exist at the same time, both steps 2 and 3 may be performed, or only one of the steps may be performed.

In the above, by using the mask to extract the region related to the abnormal region to be determined, accurate statistics of the first statistical value and the second statistical value of the abnormal region to be determined can be achieved, and the accuracy of the calculation can be improved.

Proceed to Figure 7.

Step S2232: If it is determined that at least one statistical value of the pending abnormal area satisfies the preset statistical condition, the pending abnormal area is determined as the final abnormal area.

After the first statistical value and the second statistical value are obtained, the statistical value can be analyzed to further determine whether the pending abnormal area is a fuzzy area, and obtain the final abnormal area. Specifically, if it is determined that at least one statistical value of the pending abnormal area satisfies the preset statistical condition, the pending abnormal area is determined as the final abnormal area. The preset statistical condition may be that the size of the statistical value is within a certain range. For example, when the statistical value is an arithmetic mean value, the preset condition may be that the arithmetic mean value is between 40-216.

When there are multiple statistical values, it can be considered that as long as one statistical value satisfies the preset statistical conditions, the pending abnormal area can be determined as the final abnormal area, or several statistical values meet the preset statistical conditions, or all statistical values can be determined as the final abnormal area. The values all meet the preset statistical conditions.

In one disclosed embodiment, the preset statistical condition may be that each statistical value of the abnormal region to be determined is greater than a preset threshold corresponding to the statistical value. For example, when the statistical value includes the arithmetic mean and variance of the whole and boundary of the undetermined abnormal area, it means that there will be 4 statistical values in an undetermined abnormal area, which are the arithmetic mean and variance of the entire undetermined abnormal area, to be determined Arithmetic mean and variance of anomaly region boundaries. Only when the four statistical values are all greater than the preset threshold corresponding to the statistical value, the pending abnormal area is determined as the final abnormal area.

In one disclosed embodiment, the preset threshold value of the arithmetic mean may be 40, and the preset threshold value of the variance may be 100.

By using the statistical value to further analyze the abnormal area to be determined, the blur situation of the abnormal area to be determined can be further judged, thereby improving the accuracy of blur detection and further improving the accuracy of the state detection of the camera lens.

In a disclosed embodiment, if the final abnormal area has been obtained, it can be directly considered that the target image of the frame is in an abnormal state, and subsequent operations are not necessary. In this way, the detection steps can be simplified and the detection speed can be accelerated.

After the final abnormal area is obtained, the final blurred area of the image is obtained. At this time, it can be determined whether the camera lens is in an abnormal state by analyzing the final abnormal area.

Please continue to refer to Figure 3. Step S23: obtaining the second area of the final abnormal area;

The fuzzy area can be analyzed by obtaining the area of the final abnormal area. The area of the final abnormal region is defined as the second area. The area of the final abnormal area can also be represented by the number of pixels it contains. For example, if a final abnormal area contains 3000 pixels, the area of the final abnormal area is 3000 pixels in size.

Step S24: Determine whether the second area of the final abnormal area satisfies the preset area condition. If satisfied, go to step S25, if not, go to step S27.

After the second area of the final abnormal area is obtained, the final abnormal area can be classified by judging whether the second area satisfies the preset area condition. The preset area condition is, for example, the size of the area, or the position of the final abnormal area in the target image, and so on.

In one disclosed embodiment, the preset area condition may include a first preset area condition or a second preset area condition. In this case, "judging whether the second area of the final abnormal area satisfies the preset area condition" specifically includes: judging whether the second area of the final abnormal area satisfies the first preset area condition or the second preset area condition.

In one implementation scenario, the first preset area condition is that the second area is greater than the second preset area threshold; the second preset area condition is that the second area is greater than the second preset area threshold and smaller than the third preset area threshold, Wherein, the second preset area threshold is greater than the third preset area threshold. The second preset area threshold may be a size of 5000 pixels, and the third preset area threshold may be a size of 3000 pixels.

Step S27: If not satisfied, determine that the target image is in a normal state.

In one disclosed embodiment, if the second area of the final abnormal area does not satisfy the preset area condition, it may be considered that the target image is in a normal state.

Step S25: If satisfied, determine that the target image is in an abnormal state.

After judging whether the second area of the final abnormal area satisfies the preset area condition, it can be determined whether the target image is in an abnormal state according to the obtained result.

Specifically, when the second area of the final abnormal area satisfies the preset area condition, it can be considered that the final abnormal area has a great influence on the target image obtained by the camera lens, and the target image can be considered to be in an abnormal state. It can be understood that when the second area of the final abnormal area does not meet the preset area condition, it can be considered that the target image of the frame is in a normal state.

When the preset area condition includes the first preset area condition or the second preset area condition, the final abnormal area can be further classified according to these two conditions. Specifically, when the second area of the final abnormal area satisfies the first preset area condition, it can be determined that the target image is in the first abnormal state; when the second area of the final abnormal area satisfies the second preset area condition, the target image is determined in the second abnormal state.

In a specific implementation scenario, because the area of the first preset area is larger than the area of the second preset area, it can be considered that when the area of the first preset area is satisfied, the abnormal state of the target image is more serious. The first abnormal state is, for example, an error state, and the second abnormal state is, for example, a warning state.

By further classifying the final abnormal area, the abnormal state of the target image can be further divided, so that a more accurate judgment of the abnormal state of the target image can be realized.

In the above, by performing blur detection on a frame of target image, the abnormal state of the frame of target image can be determined. It can be understood that when the lens of the camera is in an abnormal state, generally speaking, several consecutive frames of target images captured by the camera are in an abnormal state. Therefore, the state of the camera lens can be further judged by analyzing the number of target images in an abnormal state. Based on this, after step S25, the following steps can be continued:

Step S26: in the case of detecting that there are at least a second preset number of frames in an abnormal state in the target image of the continuous first preset number of frames, determine that the camera lens is in an abnormal state; wherein the first preset number and the second preset number of frames are in an abnormal state; Quantity is a positive integer.

If it is detected that in the target image of the first preset number of frames in a row, at least the second preset number of frames are in an abnormal state, it means that the camera lens is likely to be in an abnormal state, so that in the target image of the continuous frame input , there are more abnormal target images. It can be understood that because the unit of the target image is the number of frames, the first preset number and the second preset number are positive integers. The first preset number is, for example, 30, and the second preset number is, for example, 15.

In one disclosed embodiment, part of the target image may be periodically extracted from the target avatar shot by the camera lens within a period of time to determine whether the extracted target image is in an abnormal state. In this way, it can be determined whether the camera lens is in an abnormal state within a certain period of time, thereby improving the detection accuracy of the camera lens.

In a disclosed embodiment, corresponding to the above-mentioned preset area condition including the first preset area condition or the second preset area condition, it can be determined that the target image is in the first abnormal state or the second abnormal state, The following situations may occur:

The first situation: if it is detected that the target images of the first preset number of consecutive frames are all in the first abnormal state, it can be determined that the camera lens is in the first abnormal state. At this time, it can be considered that the abnormal state of the camera lens is relatively serious, and the first abnormal state is, for example, an error state.

The second situation: if it is detected that the target images of the first consecutive preset number of frames are in a normal state, then the camera lens can be considered to be in a normal state at this time.

The third situation: if it is detected that the target images of the first preset number of consecutive frames are not all in the first abnormal state, and there is a situation that at least the target image of the second preset number of frames is in the first abnormal state or the second abnormal state , to determine that the camera lens is in the second abnormal state. At this time, it is possible that the target images of the consecutive first preset number of frames are all in the second abnormal state, or at least the second preset number of frame images of the target images of the consecutive first preset number of frames (the second preset number of frames) The number is not equal to the first preset number) is in the first abnormal state or the second abnormal state. In this case, it can be considered that the camera lens may be in an abnormal state, so it is determined that the camera lens is in the second abnormal state. The second abnormal state is, for example, a warning state. The second preset number can be specifically set according to the actual situation, for example, it can be any positive integer such as 1, 5, and 15.

The abnormal state of the target image is determined by using the area of the final abnormal area in the target image, and whether the camera lens is in an abnormal state is determined based on the state of the target image in a preset number of frames, thereby realizing the abnormality detection of the camera lens. When the number is set to be greater than 1, that is, whether the camera lens is in an abnormal state is comprehensively determined by using the state of the target images of multiple consecutive frames, so the detection accuracy of the state of the camera lens can be improved.

In the above scheme, by performing blur detection on the target image, it is possible to determine the abnormal area in the target image according to the fuzzy condition of the target image, and realize the recognition of the abnormal area of the target image, so that the follow-up can be based on the obtained final abnormal area. Determine the abnormality of the camera lens. Moreover, various types of abnormal conditions of the lens, such as water mist, smudges, occlusion, etc., often cause blurring of some areas in the target image. Therefore, the blur detection of the image can be used to detect all the above abnormal conditions. out to improve detection breadth.

Those skilled in the art can understand that in the above method of the specific implementation, the writing order of each step does not mean a strict execution order but constitutes any limitation on the implementation process, and the specific execution order of each step should be based on its function and possible Internal logic is determined.

The method for detecting the state of a camera lens provided by the embodiments of the present disclosure will be described in detail below with reference to specific application scenarios.

In the application scenario provided by the embodiment, referring to FIG. 8 , the state detection method of the camera lens can be completed by the abnormal area detection module 81 , the abnormal area filtering module 82 , the state analysis processing module 83 and the control processing module 84 .

The abnormal area detection module 81 can detect the abnormal area of the target image. The types of abnormal areas include: the lens is stained with water mist; the lens is stained, and the lens is blocked. In addition, the abnormal area filtering module 82 can filter the results of the abnormal area detection module. Filtering is achieved through a series of human-designed rules and characteristics. The filtering rules here include statistics such as region size, region color, region boundary features, etc.

The state analysis and processing module 83 can detect whether the state of the camera lens is an abnormal state through the threshold value set manually. The processing results of the state analysis and processing module can be divided into three states: normal, abnormal, and error. Among them, the normal state, that is, the camera lens does not have any abnormality. Abnormal state, that is, the state of the camera lens is slightly abnormal, but not serious. Error status, that is, the camera lens status is abnormally serious and needs to be shut down immediately. The output result of this module can be handed over to the control module 84 for processing, so that the control module can make different responses to different states.

In this embodiment of the present disclosure, as shown in FIG. 9 , the abnormal area detection module 81 may perform the following steps:

Step 811: Acquire a target image. The target image is an image captured by a camera.

Step 812: Perform grayscale processing on the target image to obtain a grayscale image.

Here, the target image is a color RGB image. The target image can be subjected to grayscale processing according to the above formula (1) to obtain the grayscale value of each pixel, thereby obtaining a grayscale image.

Exemplarily, Figure 10 shows grayscale images in different scenarios. Among them, 1001 in Figure 10 is the grayscale image in normal conditions, 1002 in Figure 10 is the grayscale image of the camera lens contaminated with water mist, 1003 in Figure 10 is the grayscale image of the camera lens contaminated with contamination, and 1004 in Figure 10 Grayscale image with occlusions for the camera lens.

Step 813: Perform Laplacian transformation on the grayscale image to obtain a transformed image.

Among them, the specific process of Laplacian is, for example, firstly use the sobel operator to calculate the second-order x and y differences, and then sum them up. For the formula, please refer to the above formula (2).

Exemplarily, after performing Laplacian transformation on the grayscale image shown in FIG. 10 , the transformed image shown in FIG. 11 can be obtained. Among them, 1101 in Fig. 11 is the transformed image collected under normal conditions, 1102 in Fig. 11 is the transformed image of the camera lens contaminated with water mist, 1103 in Fig. 11 is the transformed image of the camera lens contaminated with stains, 1104 in Fig. 11 A rollover image with occlusions for the camera lens.

Step 814: Perform filtering processing on the transformed image to obtain a filtered image.

The filtering process may include, but is not limited to, morphological closing operations. Morphological closing operation is to perform dilation operation first, and then perform erosion operation. The specific processes of the expansion operation and the erosion operation are the same as those of the above-mentioned embodiment, and are not repeated here.

Exemplarily, after performing morphological closing operation processing on the transformed image shown in FIG. 11 , the filtered image shown in FIG. 12 can be obtained. Among them, 1201 in Fig. 12 is the filtered image collected under normal conditions, 1202 in Fig. 12 is the filtered image of the camera lens contaminated with water mist, 1203 in Fig. 12 is the filtered image of the camera lens contaminated with contamination, 1204 in Fig. 12 A filtered image with occlusions for the camera lens.

Step 815: Perform binarization processing on the filtered image to obtain a binarized image.

Step 816: Invert the pixel values of the binarized image to obtain an inverse binarized image.

It should be noted that the formula for performing the binarization processing and the inversion operation on the filtered image may refer to the formula (5) in the above embodiment, which will not be repeated here.

Exemplarily, after performing binarization processing and inversion processing on the filtered image shown in FIG. 12 , the inverse binarized image shown in FIG. 13 can be obtained. Among them, 1301 in Figure 13 is the inverse binarization image under normal conditions, 1302 in Figure 13 is the inverse binarization image of the camera lens contaminated with water mist, and 1303 in Figure 13 is the inverse binary image of the camera lens contaminated with stains 1304 in Fig. 13 is an inverse binarized image where the camera lens is occluded.

Step 817 : Find out the pixel points whose pixel value meets the preset pixel condition from the inverse binarized image, so as to form several candidate abnormal regions.

Here, the contour region search is performed on the inverse binary image to obtain independent individual abnormal regions. The specific implementation can use the cv::findContours function in the OpenCV open source computer vision library.

Exemplarily, as shown in FIG. 14, the area framed by the solid line frame is the candidate abnormal area in different scenarios.

In this embodiment of the present disclosure, the abnormal area filtering module 82 may perform the following steps:

Step 821: Obtain the first area of each candidate abnormal region.

Step 822 , from several candidate abnormal areas, select at least one candidate abnormal area whose first area satisfies the preset area condition as the pending abnormal area.

The preset area condition is that the first area is greater than the first preset area threshold. That is to say, the abnormal candidate contours can be filtered by area, and the area that is too small can be deleted to obtain the undetermined abnormal area.

Step 823 , based on the pixel position information of the abnormal region to be determined, generate at least one mask corresponding to the abnormal region to be determined, wherein the at least one mask includes a region mask and/or a boundary mask.

Step 824: In the case where at least one mask includes an area mask, obtain a saturation image corresponding to the target image, and use the area mask corresponding to the undetermined abnormal area to obtain a saturation image corresponding to the undetermined abnormal area. The first area to be counted; the saturation of the first area to be counted is counted to obtain at least one first statistical value of the to-be-determined abnormal area.

Specifically, the saturation information of the target image can be calculated according to formula (6) to obtain a saturation image.

Wherein, the first statistical value may be one of the mean value and the variance of the saturation of the undetermined abnormal area.

Step 825: Obtain a second area to be counted corresponding to the abnormal area to be determined in the target image by using the boundary mask corresponding to the abnormal area to be determined; perform statistics on the pixel values of the second area to be counted to obtain at least one abnormal area to be determined. Second statistic.

The second statistical value includes at least one of a mean and a variance.

Step 826: If it is determined that at least one statistical value of the pending abnormal area satisfies the preset statistical condition, determine the pending abnormal area as the final abnormal area.

In this embodiment of the present disclosure, the state analysis processing module 83 may perform the following steps:

Step 831: Obtain the second area of the final abnormal area.

Here, the cv::contourArea function in the OpenCV open source computer vision library can be used to calculate the second area of the most total abnormal area.

Step 832: Determine whether the second area of the final abnormal area satisfies the first preset area condition or the second preset area condition.

The first preset area condition is that the second area is greater than the second preset area threshold. The second preset area condition is that the second area is greater than the second preset area threshold and smaller than the third preset area threshold, wherein the second preset area threshold is greater than the third preset area threshold.

Step 833: If the first preset area condition is satisfied, it is determined that the target image is in the first abnormal state; if the second preset area condition is satisfied, it is determined that the target image is in the second abnormal state.

Step 834 , when it is detected that the target images of the first preset number of consecutive frames are in the first abnormal state, determine that the camera lens is in the first abnormal state.

When it is detected that the target images of the first preset number of consecutive frames are unevenly in the first abnormal state, and there is at least the second preset number of frames in the first abnormal state or the second abnormal state, it is determined that the camera lens is in the first abnormal state 2. Abnormal state.

It can be seen that, by performing blur detection on the target image, the abnormal area existing in the target image can be determined according to the fuzzy condition of the target image, and the abnormal area identification of the target image is realized, so that the follow-up can be based on the obtained final abnormal area. Determine the abnormality of the camera lens. Moreover, various types of abnormal conditions of the lens, such as water mist, smudges, occlusion, etc., often cause blurring of some areas in the target image. Therefore, the blur detection of the image can be used to detect all the above abnormal conditions. out to improve detection breadth.

Please refer to FIG. 15 . FIG. 15 is a schematic frame diagram of an embodiment of a state detection apparatus for a camera lens according to an embodiment of the present disclosure. The detection device 70 includes an area detection part 71 and a state analysis part 72 . The area detection section 71 is configured to perform abnormality detection on the target image captured by the camera to obtain the final abnormal area in the target image. The state analysis section 72 is configured to analyze the final abnormal area to determine whether the lens of the camera is in an abnormal state.

The area detection part 71 is also configured to perform blur detection on the target image captured by the camera to obtain several candidate abnormal areas in the target image; select at least one candidate abnormal area from several candidate abnormal areas as the final abnormal area .

The region detection part 71 is also configured to perform preset transformation on the target image captured by the camera to obtain a transformed image, wherein the pixel value of each pixel in the transformed image can reflect the blur information of the transformed image; Pixel values to identify candidate abnormal regions.

The region detection part 71 is further configured to preprocess the target image to obtain a preprocessed image, and to perform Laplace transform on the preprocessed image to obtain a transformed image. The region detection part 71 is further configured to perform binarization processing based on the transformed image to obtain a binarized image; to find out the pixel points whose pixel value meets the preset pixel condition from the binarized image to form a candidate abnormal region.

The region detection part 71 is further configured to perform grayscale processing on the target image to obtain a preprocessed image. The region detection part 71 is further configured to perform filtering processing on the transformed image to obtain a filtered image. The filtering process may be, but not limited to, a morphological closing operation. Perform binarization processing on the filtered image to obtain a binarized image. The region detection section 71 is further configured to perform inversion of pixel values of the binarized image to obtain an inverse binarized image. Execute to find out the pixel points whose pixel value satisfies the preset pixel condition from the de-binarized image, so as to form a candidate abnormal area.

Among them, the area detection part 71 is further configured to obtain the first area of each candidate abnormal area; select a candidate abnormal area whose first area satisfies the preset area condition from several candidate abnormal areas, as the pending abnormal area; At least one pending abnormal area is determined as the final abnormal area.

Wherein, the above-mentioned preset area condition is that the first area is greater than the first preset area threshold.

Wherein, the region detection part 71 is further configured to perform determining at least one statistical value of the abnormal region to be determined in the target image; wherein, the at least one statistical value of the abnormal region to be determined includes at least one first statistical value obtained from the saturation statistics of the abnormal region to be determined A statistical value, and/or, at least one second statistical value obtained by counting the pixel values of the abnormal area to be determined; if it is determined that at least one statistical value of the abnormal area to be determined satisfies the preset statistical conditions, the abnormal area to be determined is determined as the final abnormality area.

Wherein, the above-mentioned at least one statistical value includes at least one of mean and variance. The above-mentioned preset statistical conditions include: each statistical value of the undetermined abnormal area is greater than a preset threshold corresponding to the statistical value.

Wherein, the area detection part 71 is further configured to generate at least one mask corresponding to the undetermined abnormal area based on the pixel position information of the undetermined abnormal area, wherein the at least one mask includes an area mask and/or a boundary mask; In the case where a mask includes a region mask, a saturation image corresponding to the target image is obtained, and at least one first pending abnormal region corresponding to the pending abnormal region is obtained in the saturation image by using the region mask corresponding to the pending abnormal region. Statistical area, wherein at least one area to be counted includes an overall area and/or a boundary area; obtain a saturation image corresponding to the target image, and determine the saturation of the area corresponding to the overall area in the saturation image of the first area to be counted and/or, in the case where at least one mask includes a boundary mask, use the boundary mask corresponding to the abnormal area to be determined to obtain in the target image a The second to-be-statistical area corresponding to the undetermined abnormal area; the pixel values of the second to-be-statistical area are counted to obtain at least one second statistical value of the undetermined abnormal area.

Wherein, the state analysis part 72 is further configured to execute obtaining the second area of the final abnormal area; execute determining whether the second area of the final abnormal area satisfies the preset area condition; if so, execute determining that the target image is in an abnormal state; execute in When it is detected that there are at least a second preset number of frames in an abnormal state in the target image of the continuous first preset number of frames, it is determined that the camera lens is in an abnormal state; wherein the first preset number and the second preset number are positive integers .

The state analysis part 72 is further configured to judge whether the second area of the final abnormal area satisfies the first preset area condition or the second preset area condition. If so, the state analysis part 72 is further configured to determine that the target image is in the first abnormal state if the first preset area condition is met; and to execute the determination that the target image is in the second abnormal state if the second preset area condition is met. The state analysis part 72 is further configured to determine that the camera lens is in the first abnormal state when it is detected that the target images of the consecutive first preset number of frames are in the first abnormal state; The frame target images are not uniformly in the first abnormal state, and when at least a second preset number of frames are in the first abnormal state or the second abnormal state, it is determined that the camera lens is in the second abnormal state.

Please refer to FIG. 16 . FIG. 16 is a schematic frame diagram of an embodiment of an electronic device according to an embodiment of the present disclosure. The electronic device 80 includes a memory 81 and a processor 82 coupled to each other, and the processor 82 is configured to execute the computer program stored in the memory 81 to implement the steps of any of the above embodiments of the camera lens state detection method. In a specific implementation scenario, the electronic device 80 may include, but is not limited to, a microcomputer and a server. In addition, the electronic device 80 may also include mobile devices such as a notebook computer and a tablet computer, which are not limited herein.

Specifically, the processor 82 is configured to control itself and the memory 81 to implement the steps of any of the above embodiments of the camera lens state detection method. The processor 82 may also be referred to as a CPU (Central Processing Unit, central processing unit). The processor 82 may be an integrated circuit chip with signal processing capability. The processor 82 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field-Programmable Gate Array, FPGA) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. In addition, the processor 82 may be jointly implemented by an integrated circuit chip.

Please refer to FIG. 17 , which is a schematic diagram of a framework of an embodiment of a computer-readable storage medium according to an embodiment of the present disclosure. The computer-readable storage medium 90 stores a computer program 901 that can be executed by the processor, and the computer program 901 is used to implement the steps of any of the foregoing embodiments of the camera lens state detection method.

An embodiment of the present disclosure further provides a computer program, including computer-readable code, when the computer-readable code is executed in an electronic device, a processor in the electronic device executes a state detection method configured to implement the above-mentioned camera lens Example steps.

In some embodiments, the functions or included parts of the apparatus, device, or medium provided in the embodiments of the present disclosure may be used to execute the methods described in the above method embodiments, and the specific implementation may refer to the descriptions in the above method embodiments. For the sake of brevity, details are not repeated here.

The above descriptions of the various embodiments tend to emphasize the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, details are not repeated herein.

In the several embodiments provided in the present disclosure, it should be understood that the disclosed method and apparatus may be implemented in other manners. For example, the device implementations described above are only illustrative, for example, the division of parts or units is only a logical function division, and other divisions may be used in actual implementation, for example, units or components may be combined or integrated to another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, which may be in electrical, mechanical or other forms.

In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware, and can also be implemented in the form of software functional units.

The integrated unit, if implemented as a software functional unit and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present disclosure essentially or contribute to the prior art, or all or part of the technical solutions may be embodied in the form of software products, and the computer software products are stored in a storage The medium includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to execute all or part of the steps of the methods in the various implementation manners of the embodiments of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes .

Claims

A state detection method for a camera lens, comprising:

Perform abnormality detection on the target image captured by the camera to obtain the final abnormal area in the target image;

The final abnormal area is analyzed to determine whether the lens of the camera is in an abnormal state.
The method according to claim 1, wherein, performing abnormality detection on the target image captured by the camera to obtain the final abnormal area in the target image, comprising:

performing blur detection on the target image captured by the camera to obtain several candidate abnormal regions in the target image;

At least one of the candidate abnormal areas is selected from the several candidate abnormal areas as the final abnormal area.
The method according to claim 2, wherein the blur detection is performed on the target image captured by the camera to obtain several candidate abnormal regions in the target image, including:

performing preset transformation on the target image captured by the camera to obtain a transformed image, wherein the pixel value of each pixel in the transformed image can reflect the blur information of the transformed image;

The candidate abnormal region is determined based on the pixel values of the transformed image.
The method according to claim 3, wherein, performing a preset transformation on the target image captured by a camera to obtain a transformed image, comprising:

Preprocessing the target image to obtain a preprocessed image;

Laplace transform is performed on the preprocessed image to obtain the transformed image;

The determining the candidate abnormal region based on the pixel value of the transformed image includes:

Perform binarization processing based on the transformed image to obtain a binarized image;

The pixel points whose pixel values satisfy a preset pixel condition are found from the binarized image to form the candidate abnormal region.
The method according to claim 4, wherein the preprocessing of the target image to obtain the preprocessed image comprises:

Grayscale processing is performed on the target image to obtain a preprocessed image;

The performing binarization processing based on the transformed image to obtain a binarized image includes:

Perform filtering processing on the transformed image to obtain a filtered image; the filtering processing includes a morphological closing operation;

performing binarization processing on the filtered image to obtain the binarized image;

The pixel points whose pixel values satisfy the preset pixel condition are found from the binarized image to form the candidate abnormal area, including:

Invert the pixel values of the binarized image to obtain an inverse binarized image;

The pixel points whose pixel values satisfy the preset pixel condition are found from the de-binarized image to form the candidate abnormal region.
The method according to any one of claims 2 to 5, wherein the selecting at least one candidate abnormal region from the several candidate abnormal regions as the final abnormal region comprises:

obtaining the first area of each of the candidate abnormal regions;

From the several candidate abnormal areas, select at least one candidate abnormal area whose first area satisfies the preset area condition as the pending abnormal area;

At least one of the pending abnormal areas is determined as the final abnormal area.
The method according to claim 6, wherein the preset area condition is that the first area is greater than a first preset area threshold; and/or,

The determining at least one of the pending abnormal areas as the final abnormal area includes:

determining at least one statistical value of the pending abnormal region in the target image; wherein the at least one statistical value of the pending abnormal region includes at least one first statistical value obtained from the saturation statistics of the pending abnormal region, And/or, at least one second statistical value obtained by statistics on the pixel values of the pending abnormal area;

If it is determined that at least one statistical value of the pending abnormal area satisfies a preset statistical condition, the pending abnormal area is determined as the final abnormal area.
The method according to claim 7, wherein the first statistical value and the second statistical value respectively comprise at least one of a mean value and a variance, and the preset statistical condition comprises: each of the undetermined abnormal regions The statistical values are all greater than the preset threshold corresponding to the statistical value;

The determining at least one statistical value of the undetermined abnormal area in the target image includes:

generating at least one mask corresponding to the pending anomaly region based on the pixel position information of the pending anomaly region, wherein the at least one mask includes an area mask and/or a boundary mask;

In the case where the at least one mask includes an area mask, a saturation image corresponding to the target image is acquired, and using the area mask corresponding to the undetermined abnormal area, a saturation image corresponding to the undetermined abnormal area is obtained in the saturation image the first to-be-statistical area corresponding to the undetermined abnormal area; perform statistics on the saturation of the first to-be-statistical area to obtain the at least one first statistical value of the to-be-determined abnormal area; and/or,

In the case that the at least one mask includes a boundary mask, using the boundary mask corresponding to the abnormal region to be determined, a second region to be counted corresponding to the abnormal region to be determined is obtained in the target image; The pixel values of the second to-be-statistical area are counted to obtain the at least one second statistical value of the to-be-determined abnormal area.
The method according to any one of claims 1 to 8, wherein the analyzing the final abnormal area to determine whether the lens of the camera is in an abnormal state comprises:

obtaining the second area of the final abnormal area;

judging whether the second area of the final abnormal area satisfies the preset area condition;

If satisfied, it is determined that the target image is in an abnormal state;

In the case where it is detected that there are at least a second preset number of frames in the target image that are in an abnormal state in a continuous first preset number of frames, it is determined that the camera lens is in an abnormal state; wherein the first preset number and the number of 2. The preset number is a positive integer.
The method according to claim 9, wherein the judging whether the second area of the final abnormal area satisfies a preset area condition comprises:

judging whether the second area of the final abnormal area satisfies the first preset area condition or the second preset area condition;

If it is satisfied, then determining that the target image is in an abnormal state, including:

If the first preset area condition is satisfied, determining that the target image is in a first abnormal state;

If the second preset area condition is satisfied, determining that the target image is in a second abnormal state;

The determining that the camera lens is in an abnormal state when it is detected that there are at least a second preset number of frames in the target image that are in an abnormal state in the first consecutive predetermined number of frames includes:

determining that the camera lens is in the first abnormal state when it is detected that the target images of the consecutive first preset number of frames are all in the first abnormal state;

When it is detected that the target images are not uniformly in the first abnormal state for the consecutive first preset number of frames, and there are at least the second preset number of frames in the first abnormal state or the second abnormality In the case of the state, it is determined that the camera lens is in the second abnormal state.
The method according to claim 10, wherein the first preset area condition is that the second area is greater than a second preset area threshold; the second preset area condition is that the second area is greater than the second area The preset area threshold is smaller than a third preset area threshold, wherein the second preset area threshold is greater than the third preset area threshold.
A state detection device for a camera lens, comprising:

an area detection part, configured to perform abnormality detection on the target image captured by the camera, and obtain the final abnormal area in the target image;

The state analysis part is configured to analyze the final abnormal area to determine whether the lens of the camera is in an abnormal state.
The device according to claim 12, wherein the area detection part is further configured to perform blur detection on the target image captured by the camera to obtain several candidate abnormal areas in the target image; At least one of the candidate abnormal regions is selected as the final abnormal region.
The device according to claim 13, wherein the area detection part is further configured to perform a preset transformation on the target image captured by the camera to obtain a transformed image, wherein the pixel value of each pixel in the transformed image can be Reflect the blur information of the transformed image; and determine candidate abnormal regions based on pixel values of the transformed image.
The device according to claim 14, wherein the region detection part is further configured to preprocess the target image to obtain a preprocessed image; and perform Laplace transform on the preprocessed image to obtain the preprocessed image. transform image;

The area detection part is further configured to perform binarization processing based on the transformed image to obtain a binarized image; and to find out the pixel points whose pixel value meets the preset pixel condition from the binarized image, and form the candidate abnormal region.
The device according to claim 15, wherein the region detection part is further configured to perform grayscale processing on the target image to obtain the preprocessed image;

The region detection part is further configured to perform filtering processing on the transformed image to obtain a filtered image; the filtering processing includes a morphological closing operation; and performing binarization processing on the filtered image to obtain the binary image image;

The area detection part is further configured to invert the pixel values of the binarized image to obtain an inverse binarized image; and to find out the pixel value from the inverse binarized image that satisfies a preset pixel condition The pixel points constitute the candidate abnormal region.
The device according to any one of claims 13 to 16, wherein the area detection part is further configured to obtain a first area of each of the candidate abnormal areas; and from the several candidate abnormal areas, select the first area At least one candidate abnormal area whose area satisfies the preset area condition is regarded as a pending abnormal area; at least one of the pending abnormal areas is determined as the final abnormal area.
The device according to claim 17, wherein the preset area condition is that the first area is greater than a first preset area threshold;

The region detection part is further configured to determine at least one statistical value of the pending abnormal region in the target image; wherein the at least one statistical value of the pending abnormal region includes saturation of the pending abnormal region At least one first statistical value obtained by statistics, and/or, at least one second statistical value obtained by statistics on the pixel values of the pending abnormal area; if it is determined that at least one statistical value of the pending abnormal area satisfies a preset statistical condition , the pending abnormal area is determined as the final abnormal area.
The apparatus according to claim 18, wherein the first statistical value and the second statistical value respectively comprise at least one of a mean value and a variance, and the preset statistical condition comprises: each of the undetermined abnormal regions The statistical values are all greater than the preset threshold corresponding to the statistical value;

The area detection part is further configured to generate at least one mask corresponding to the pending abnormal area based on the pixel position information of the pending abnormal area, wherein the at least one mask includes an area mask and/or a boundary mask; in the case where the at least one mask includes a region mask, acquiring a saturation image corresponding to the target image, and using the region mask corresponding to the undetermined abnormal region, at the saturation Obtaining a first area to be counted corresponding to the undetermined abnormal area in the image; performing statistics on the saturation of the first area to be counted to obtain the at least one first statistical value of the undetermined abnormal area; and/or , in the case that the at least one mask includes a boundary mask, using the boundary mask corresponding to the abnormal region to be determined, obtain a second region to be counted corresponding to the abnormal region to be determined in the target image; Counting the pixel values of the second to-be-statistical area to obtain the at least one second statistical value of the to-be-determined abnormal area.
The apparatus according to claims 12 to 18, wherein the state analysis part is further configured to obtain a second area of the final abnormal area; and to perform judgment on whether the second area of the final abnormal area satisfies a preset area condition; if satisfied, execute to determine that the target image is in an abnormal state; execute to determine that the camera is in an abnormal state when it is detected that there are at least a second preset number of frames in the target image in a continuous first preset number of frames The lens is in an abnormal state; wherein, the first preset number and the second preset number are positive integers.
The apparatus according to claim 20, wherein the state analysis part is further configured to judge whether the second area of the final abnormal area satisfies the first preset area condition or the second preset area condition; If the first preset area condition is satisfied, then execute to determine that the target image is in a first abnormal state; if the second preset area condition is satisfied, execute to determine that the target image is in a second abnormal state;

The state analysis part is further configured to determine that the camera lens is in the first abnormal state when it is detected that the target images of the consecutive first preset number of frames are in the first abnormal state ; When it is detected that the target image is not uniformly in the first abnormal state of the continuous first preset number of frames, and there is at least the second preset number of frames in the first abnormal state or the second abnormal state In the case of an abnormal state, it is determined that the camera lens is in a second abnormal state.
The device of claim 21, wherein the first preset area condition is that the second area is greater than a second preset area threshold; the second preset area condition is that the second area is greater than the second area The preset area threshold is smaller than a third preset area threshold, wherein the second preset area threshold is greater than the third preset area threshold.
An electronic device, comprising: a memory and a processor coupled to each other, the processor is configured to execute a computer program stored in the memory, so as to implement the method for detecting the state of a camera lens according to any one of claims 1 to 11 .
A computer-readable storage medium storing a computer program, when the computer program is executed by a processor, the method for detecting a state of a camera lens according to any one of claims 1 to 11 is implemented.
A computer program, comprising computer readable code, when the computer readable code is executed in an electronic device, a processor in the electronic device executes a camera lens for realizing any one of claims 1 to 11 state detection method.