WO2023124385A1

WO2023124385A1 - Photographic apparatus shielding detection method and apparatus, and electronic device, storage medium and computer program product

Info

Publication number: WO2023124385A1
Application number: PCT/CN2022/124934
Authority: WO
Inventors: 李阳阳; 许亮; 毛宁元
Original assignee: 上海商汤智能科技有限公司
Priority date: 2021-12-31
Filing date: 2022-10-12
Publication date: 2023-07-06
Also published as: CN114332720A

Abstract

Provided in the embodiments of the present disclosure are a photographic apparatus shielding detection method and apparatus, and an electronic device, a storage medium and a computer program product. The photographic apparatus shielding detection method comprises: acquiring video data of a driving region of a vehicle by means of a photographic apparatus; performing facial detection on the current image frame in the video data, and when a face is not detected, coding the current image frame according to a pixel value in the current image frame, so as to obtain feature coding information of the current image frame; and on the basis of the feature coding information of the current image frame and preset feature coding information, determining whether the photographic apparatus is shielded, wherein the preset feature coding information comprises feature coding information of an image frame that includes a face in the video data. By means of the embodiments of the present disclosure, the detection precision of the shielding of a photographic apparatus can be improved.

Description

Method, device, electronic device, storage medium, and computer program product for occlusion detection of camera device

Cross References to Related Applications

The embodiment of the present disclosure is proposed based on the Chinese patent application with the application number 202111668662.2, the application date is December 31, 2021, and the application name is "Camera Blocking Detection Method, Device, Electronic Equipment, and Storage Medium", and requires the Chinese patent application The priority of the Chinese patent application, the entire content of the Chinese patent application is hereby incorporated into this disclosure by reference.

technical field

Embodiments of the present disclosure relate to the technical field of image processing, and in particular, to a camera device occlusion detection method, device, electronic equipment, storage medium, and computer program product.

Background technique

With the improvement of people's living standards, vehicles have become an indispensable means of transportation in people's lives. The camera device in the cabin can be used to restrain the driver's driving behavior, thereby reducing the probability of traffic accidents, thereby assisting in improving Driving safety.

However, in the process of actual use, the camera will be blocked, and if the camera is blocked, the driver's behavior cannot be accurately detected. Accuracy is particularly important.

Contents of the invention

Embodiments of the present disclosure at least provide a method, device, electronic device, storage medium, and computer program product for occlusion detection of an imaging device, which can not only implement occlusion detection for an imaging device, but also improve detection accuracy.

An embodiment of the present disclosure provides an occlusion detection method for a camera device, including:

Obtain video data of the driving area of the vehicle through the camera device;

Perform face detection on the current image frame in the video data, and if no face is detected, encode the current image frame according to the pixel values in the current image frame to obtain the current image frame The feature encoding information of

Determine whether the camera is blocked based on feature encoding information of the current image frame and preset feature encoding information, wherein the preset feature encoding information includes feature encoding of an image frame containing a human face in the video data information.

An embodiment of the present disclosure provides an occlusion detection device for a camera device, including:

The video acquisition module is configured to acquire the video data of the driving area of the vehicle through the camera device;

The face detection module is configured to perform face detection on the current image frame in the video data, and encode the current image frame according to the pixel value in the current image frame if no face is detected , to obtain the feature encoding information of the current image frame;

The occlusion determination module is configured to determine whether the camera is occluded based on the feature encoding information of the current image frame and preset feature encoding information, wherein the preset feature encoding information includes that the video data contains a human face The feature encoding information of the image frame.

An embodiment of the present disclosure provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the The memories communicate with each other through a bus, and when the machine-readable instructions are executed by the processor, the camera-device occlusion detection method as described in any of the foregoing implementation manners is executed.

An embodiment of the present disclosure provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processor, the camera-device occlusion detection method as described in any of the preceding implementation modes is executed. .

An embodiment of the present disclosure also provides a computer program product, including a computer-readable storage medium storing program codes, and when instructions included in the program codes are executed by a processor of a computer device, the steps of the above method are implemented.

In the method, device, electronic equipment, storage medium, and computer program product provided by the embodiments of the present disclosure, when a face is not recognized, the current image is processed according to the pixel values in the current image frame. frame to obtain the feature encoding information of the current image frame, and based on the feature encoding information of the current image frame and the preset feature encoding information, it is judged whether the camera is blocked. In the case of a face, further judgment is made on the image, which can improve the accuracy of judgment.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory in nature and are not restrictive of the disclosure.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present disclosure more clearly, the following will briefly introduce the accompanying drawings used in the embodiments. The accompanying drawings here are incorporated into the specification and constitute a part of the specification. The drawings show the embodiments consistent with the present disclosure, and are used together with the description to explain the technical solution of the present disclosure. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be regarded as limiting the scope. For those skilled in the art, they can also make From these drawings other related drawings are obtained.

FIG. 1 shows a schematic flowchart of a method for detecting an occlusion of a camera device provided by an embodiment of the present disclosure;

FIG. 2 shows a schematic flowchart of a method for determining feature coding information of a current image frame provided by an embodiment of the present disclosure;

FIG. 3 shows a schematic flow chart of another camera device occlusion detection method provided by an embodiment of the present disclosure;

FIG. 4 shows a schematic flowchart of a method for outputting prompt information provided by an embodiment of the present disclosure;

FIG. 5 shows a schematic flowchart of another method for occlusion detection of a camera device provided by an embodiment of the present disclosure;

FIG. 6 shows a schematic structural diagram of an occlusion detection device for a camera device provided by an embodiment of the present disclosure;

FIG. 7 shows a schematic structural diagram of another camera device occlusion detection device provided by an embodiment of the present disclosure;

Fig. 8 shows a schematic diagram of an electronic device provided by an embodiment of the present disclosure.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are The present disclosure discloses some embodiments, but not all embodiments. The components of the disclosed embodiments generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations. Accordingly, the following detailed description of the embodiments of the present disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative effort shall fall within the protection scope of the present disclosure.

It should be noted that like numerals and letters denote similar items in the following figures, therefore, once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

The term "and/or" in this article describes an association relationship, which means that there can be three kinds of relationships, for example, A and/or B, which can mean: A exists alone, A and B exist simultaneously, and B exists alone. . In addition, the term "at least one" herein means any one of a variety or any combination of at least two of the more, for example, including at least one of A, B, and C, which may mean including from A, Any one or more elements selected from the set formed by B and C.

With the improvement of people's living standards, vehicles have become an indispensable means of transportation in people's lives. The camera device in the cabin can be used to restrain the driver's driving behavior, thereby reducing the probability of traffic accidents, thereby assisting in improving Driving safety. However, during actual use, the camera may be blocked, and if the camera is blocked, the behavior of the driver cannot be accurately detected.

After research, it is found that although there is a method in the prior art that can detect whether the camera is blocked, such as determining whether the camera is blocked by face detection, this method does not work when the overall brightness of the image is high or low. , it is prone to misjudgment or omission of judgment.

Based on the above research, the present disclosure provides a camera occlusion detection method, which includes: acquiring video data of the driving area of the vehicle through the camera device; performing face detection on the current image frame in the video data, and In the case of a human face, encode the current image frame according to the pixel values in the current image frame to obtain the feature encoding information of the current image frame; based on the feature encoding information of the current image frame and preset The feature encoding information is used to determine whether the camera is blocked, wherein the preset feature encoding information includes feature encoding information of an image frame containing a human face in the video data.

In the embodiment of the present disclosure, if no human face is recognized, the current image frame is encoded according to the pixel values in the current image frame to obtain the feature encoding information of the current image frame, and based on the The feature coding information of the current image frame and the preset feature coding information determine whether the camera is blocked. In this way, the image can be further judged when no face is recognized, thereby improving the accuracy of judgment.

The occlusion detection method of the camera device provided in the embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Referring to FIG. 1 , it is a schematic flowchart of a method for detecting occlusion of a camera device provided by an embodiment of the present disclosure. The method for occlusion detection of a camera device includes the following steps S101 to S103:

S101. Acquiring video data of a driving area of a vehicle through a camera device.

Wherein, the driving area refers to the area in the vehicle where the driver controls the driving of the vehicle. During the driving process of the vehicle, the driver is usually located in the driving area of the vehicle, and the terminal device can acquire the video data of the driving area of the vehicle.

Video data refers to a continuous image sequence, which is essentially composed of a group of continuous images. Among them, an image frame is the smallest visual unit that makes up a video, and is a static image. A sequence of temporally continuous image frames is synthesized to form a dynamic video. In this embodiment, in order to facilitate subsequent detection and recognition, it is necessary to extract image frames in the video data.

Exemplarily, since the video data usually includes many frames of images per second (for example, 24 frames of images per second), therefore, in the process of extracting the image frames in the video data, frame extraction can be performed, wherein , frame extraction refers to frame extraction according to the preset interval frame number, for example, extracting a frame of image every 20 frames; it can also perform frame extraction according to the preset time interval, such as every interval of 10 milliseconds (ms) Fetch the image once.

It should be noted that the preset interval frame number and interval time can be set according to actual needs, and are not limited here.

Optionally, the video data of the driving area may be captured by a camera device installed inside the vehicle, and then the terminal device acquires the video data captured by the camera device. That is to say, in the embodiment of the present disclosure, the subject of execution of the camera occlusion detection method may be a terminal device, wherein the terminal device includes but not limited to a vehicle-mounted device, a wearable device, a user terminal, and a handheld device.

In other implementations, the subject of execution of the camera occlusion detection method may also be a server, wherein the server may be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or may provide Cloud servers for basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, big data and artificial intelligence platforms.

In some possible implementation manners, the camera occlusion detection method may also be implemented in a manner in which a processor invokes computer-readable instructions stored in a memory.

S102. Perform face detection on the current image frame in the video data, and if no face is detected, encode the current image frame according to pixel values in the current image frame to obtain the current The feature encoding information of the image frame.

It can be understood that after image frame extraction processing is performed on video data, multiple image frames will be obtained. Among them, the current image frame refers to the image frame that needs to be detected and recognized currently. The image frame in the video data whose timing is before the current image frame is called the previous image frame, and the image frame whose timing is after the current image frame is called the subsequent image. frame.

Exemplarily, after the aforementioned multiple frames of images are extracted, face detection may be performed on the extracted images to determine whether there is a face in the current image frame. In some embodiments, if a human face is detected in the current image frame, it means that the driver is in the driving area at this time, and the camera device is not blocked; Make further judgments to improve the accuracy of judgment.

In this implementation manner, if no human face is recognized in the current image frame, the current image frame is further encoded according to the pixel values in the current image frame to obtain feature encoding information of the current image frame. In some embodiments, as shown in FIG. 2, the step of encoding the current image frame according to the pixel values in the current image frame may include the following S1021 and S1022:

S1021. Determine a reference pixel threshold of the current image frame.

In this embodiment, the reference pixel threshold is the average pixel value of the current image frame. In this way, the feature encoding information obtained based on the reference pixel threshold can have a unique correspondence with the current image frame, which can further improve The accuracy of judgment. Of course, in other embodiments, the reference pixel threshold can also be set according to actual conditions, for example, the corresponding reference pixel threshold can be determined according to the overall brightness of the current image, rather than the average pixel value of the current image frame.

Exemplarily, in order to improve the determination accuracy of the reference pixel threshold, before determining the reference pixel threshold of the current image frame, noise reduction processing may be performed on the current image frame, thereby reducing the impact of imaging noise.

In some implementation manners, a Gaussian smoothing algorithm may be used to perform noise reduction processing on the current image frame. In other embodiments, the noise reduction processing may also use a median filtering algorithm or an average filtering algorithm to perform the noise reduction processing, which is not limited here.

S1022, sequentially compare the pixel value of each pixel point of the current image frame with the reference pixel threshold, encode the pixel points greater than the reference pixel threshold value as 1, encode the pixels not greater than the reference pixel threshold value Point encoding is 0, and the feature encoding information of the current image frame is obtained.

Exemplarily, after the reference pixel threshold is determined, the current image frame can be encoded according to the reference pixel threshold, and the encoding process can be to encode pixels greater than the reference pixel threshold as 1, and encode pixels not greater than the The pixel point of the reference pixel threshold is encoded as 0, and then the feature encoding information is obtained. The feature encoding information may be a two-dimensional array or a two-dimensional matrix having the same size as the current image frame to be encoded. For example, in the case where the average pixel of the current image frame is 100, if the pixel value of a certain pixel in the current image frame is 123, the pixel is coded as 1, and if a certain pixel in the current image frame If the pixel value of the point is 80, the pixel point is coded as 0.

Optionally, in order to reduce the dimensionality of the feature encoding information to reduce the amount of calculation, after encoding each pixel in the current image frame, the encoding of each pixel can be sequentially encoded according to a certain scanning order (for example, first row and second column). The coding of each pixel is connected in series to form a one-dimensional array or vector, which is used as the characteristic coding information of the image frame.

S103. Based on the feature encoding information of the current image frame and preset feature encoding information, determine whether the camera is blocked, wherein the preset feature encoding information includes the image frame of the video data that contains a face Feature encoding information.

In this embodiment, the preset feature encoding information includes the feature encoding information of the image frame containing the human face in the video data, that is, the preset feature encoding information is the feature of the preceding image frame of the current image frame. information is encoded, and the preceding image frame includes a human face. That is, an image including a human face captured after the camera is activated is used as a basis for judging whether the camera is blocked by the camera in subsequent image frames.

It can be understood that after the vehicle is started, the camera device can be powered on and start to shoot the driving area of the vehicle to obtain video data. The stored feature encoding information including the face is used as the preset feature encoding information; and if the first frame image contains a human face, the feature encoding information of the first frame image is used as the preset feature encoding information.

It should be noted that if it is determined through the video data that all consecutive frames of images contain human faces, the preset feature coding information can be updated, that is, the subsequent feature coding information containing faces can replace the features of the first frame of images. coded information. For example, it may be updated every preset time interval, and the preset interval time is not limited, for example, it may be 1 minute or 2 minutes.

Exemplarily, the feature encoding information of the current image frame may be compared with the preset feature encoding information to determine whether the camera is blocked. In some embodiments, the Hamming distance between the feature encoding information of the current image frame and the preset feature encoding information may be calculated, and the distance between the feature encoding information of the current image frame and the preset feature encoding information If the Hamming distance is greater than the preset threshold, it is determined that the camera is blocked.

In the embodiment of the present disclosure, if no human face is recognized, the current image frame is further encoded according to the pixel values in the current image frame to obtain the feature encoding information of the current image frame, and based on the Describe the feature coding information of the current image frame and the preset feature coding information to judge whether the camera is blocked, so that the image can be further judged when the face is not recognized, and the accuracy of the judgment can be improved. In particular, it is more accurate in judging whether the driver leaves the post.

Referring to FIG. 3 , it is a schematic flow chart of another camera device occlusion detection method provided by an embodiment of the present disclosure, including the following S201 to S205:

S201. Acquire video data of a driving area of a vehicle through a camera device.

Wherein, this step is similar to the above step S101.

S202. Perform face detection on the current image frame in the video data, and if no face is detected, encode the current image frame according to the pixel values in the current image frame to obtain the current The feature encoding information of the image frame.

Wherein, this step is similar to the above step S102.

S203, judging whether the Hamming distance between the feature encoding information of the current image frame and the preset feature encoding information is greater than a preset threshold; if yes, execute step S205; if not, execute step S204.

Wherein, this step is similar to the above step S103.

S204. Based on the pixel distribution histogram of the current image frame, determine whether the camera is blocked.

Exemplarily, when the Hamming distance between the feature encoding information of the current image frame and the preset feature encoding information is not greater than the preset threshold, it indicates that there may be a human face in the current image frame, which may be due to some specific The reason (for example, the ambient light is dark) that the face is not recognized during the detection process. Therefore, in order to further improve the accuracy of the judgment, it is necessary to judge the current image frame from other angles to further judge whether the camera is blocked.

In the embodiment of the present disclosure, when the Hamming distance between the feature encoding information of the current image frame and the preset feature encoding information is not greater than the preset threshold, determine the pixel distribution histogram of the current image frame, and If the pixel distribution ratio of the preset interval in the pixel distribution histogram is greater than the preset ratio threshold, it is determined that the camera is blocked.

In some embodiments, the distribution of pixel values of each pixel in the current image frame may be counted to obtain a pixel distribution histogram of the current image frame. For example, the distribution of the pixel value of each pixel point in the current image frame can be counted according to a plurality of pre-divided pixel intervals, and the plurality of pixel intervals can be [0-19], [20-80], [81 ~126] and [127~255], etc. It can be understood that, in this implementation manner, the multiple pixel intervals shown are schematic, and in other implementation implementation manners, the multiple pixel intervals may also be divided according to other requirements.

In some implementations, the preset interval can be obtained through a large number of tests based on the actual use environment of the camera device. For example, the histogram distribution diagram of the image of the scene in the driving area of the vehicle can be calculated, and it can be determined that the camera device is not blocked. There is a clear boundary between the ratio of the number of pixels in the pixel interval [20-80] in the normal image taken and the ratio of the number of pixels in the pixel interval [20-80] in the image taken when the camera is blocked. Therefore, you can Set the default interval to the pixel interval of [20~80].

In addition, a lower pixel value (such as less than 20) or a larger proportion of total pixels greater than the high brightness threshold means that the probability of valid content in the image is lower. Therefore, in other embodiments, the preset interval can also be It is an interval with a lower pixel value or an interval with a higher pixel value, such as a pixel interval of [0-20] or a pixel interval of [130-255], which is not limited here.

In this embodiment, the preset interval is one of the plurality of pixel intervals. For example, the preset interval is [20-80]. If it is greater than the preset proportion threshold, it can be determined that the camera device is blocked.

S205. Determine that the camera is blocked.

S206, output prompt information.

For example, after it is determined that the camera is blocked, prompt information may be output to remind the driver to deal with the blocked camera so that the camera can normally capture images of the driver, thereby helping to improve driving safety.

Wherein, the prompt information includes, but is not limited to, voice prompt information, graphic prompt information, light prompt information, and the like. For example, a voice prompt message "The camera is blocked, please confirm" can be output.

In some implementations, in order to reduce the impact of the frequent output of prompt information due to instantaneous occlusion on the occupants of the vehicle, as shown in FIG. 4 , the above step S206 of outputting prompt information can be performed in the following manners of S2061 and S2062:

S2061. Determine the continuous shielding time of the camera according to the detection result of the camera in each frame of image in the video data.

S2062. Output the prompt information when the continuous shielding time reaches a preset time.

In this embodiment, when the current image frame determines that the camera is blocked, it is also necessary to determine whether the judgment result of the subsequent frame image is also blocked. It just flashes, and at this time, no prompt information is output; if the judgment result of at least one frame after the frame image is also that the camera is blocked, that is, it is judged that the camera is blocked for multiple consecutive frames of images, it means that the camera is continuously blocked at this time , is intentional occlusion, not false occlusion, and when the continuous occlusion time reaches a preset time (for example, 5 seconds), a prompt message is output for prompting.

Referring to FIG. 5 , it is a schematic flowchart of another method for occlusion detection of a camera device provided by an embodiment of the present disclosure, including the following S301 to S310:

S301. Acquire video data of a driving area of a vehicle through a camera device.

Wherein, this step is similar to the above step S101.

S302. Perform face detection on the current image frame in the video data, and if no face is detected, encode the current image frame according to pixel values in the current image frame to obtain the current The feature encoding information of the image frame.

Wherein, this step is similar to the above step S102.

S303, judging whether the Hamming distance between the feature encoding information of the current image frame and the preset feature encoding information is greater than a preset threshold; if yes, execute step S308; if not, execute step S304.

For example, if the Hamming distance between the feature encoding information of the current image frame and the preset feature encoding information is greater than the preset threshold, it means that the current image frame does not contain a human face. At this time, step S308 is executed to determine The camera is blocked; if the Hamming distance between the feature encoding information of the current image frame and the preset feature encoding information is not greater than the preset threshold, it means that there may be a human face in the current image frame, and further judgment is required. Therefore, Execute step S304.

S304. Determine the pixel distribution histogram of the current image frame.

Wherein, this step is similar to the above step S204.

S305, judging whether the proportion of pixel distribution in the preset interval in the pixel distribution histogram is greater than the preset proportion threshold; if yes, execute step S308; if not, execute step S306.

Exemplarily, if the proportion of pixel distribution in the preset interval in the pixel distribution histogram is greater than the preset proportion threshold, it means that the proportion of invalid content in the current image frame is relatively large, and step S308 is executed to determine that the camera is blocked; if the pixel The proportion of pixel distribution in the preset interval in the distribution histogram is not greater than the preset proportion threshold, indicating that the proportion of effective content in the current image frame is normal, and step S306 needs to be executed for further judgment.

S306. Determine the maximum connected domain of the current image frame.

S307, judging whether the area of the largest connected domain is greater than a preset area threshold; if yes, execute step S308; if not, execute step S310.

Among them, the maximum connected domain refers to the image area composed of all pixel values with the same pixel value or within a certain error and adjacent to each other, and the maximum connected domain is a closed area. For example, the preset area threshold can be set to an area that accounts for 60% of the entire image. If the maximum connected domain is greater than the preset area threshold, step S308 is performed to determine that the camera is blocked; if the maximum connected domain is not larger than the preset area threshold If the area threshold is set, step S310 is executed to determine that the camera is not blocked.

It can be understood that in order to further improve the judgment accuracy and reduce the misjudgment caused by sudden changes, for example, the misjudgment caused by instantaneous occlusion, in some implementations, when determining the area of the largest connected domain of the current image frame In the case of greater than the preset area threshold, the following (1) to (3) can also be performed:

(1) determining the maximum connected domain of at least one frame of images in at least one of before and after the current image frame;

(2) determining the average value of the area of the largest connected domain of the current image frame and the at least one frame of image;

(3) If the average value of the areas of the largest connected domains of the current image frame and the at least one image frame is greater than the preset area threshold, determine that the camera is blocked.

In this way, when it is determined that the area of the largest connected domain of the current image frame is greater than the preset area threshold, the average value of the area of the largest connected domain of the multi-frame images before or after the current image frame is also determined, thereby reducing the risk of flickering. (For example, the probability of misjudgment caused by instantaneously passing in front of the camera device) further improves the accuracy of judgment.

S308. Determine that the camera is blocked.

S309, output prompt information.

Wherein, this step is similar to the above step S206.

S310. Determine that the camera is not blocked.

Those skilled in the art can understand that in the above method of specific implementation, the writing order of each step does not mean a strict execution order and constitutes any limitation on the implementation process. The specific execution order of each step should be based on its function and possible The inner logic is OK.

Based on the same technical idea, the embodiment of the present disclosure also provides a camera device occlusion detection device corresponding to the camera device occlusion detection method, because the principle of solving the problem of the device in the embodiment of the present disclosure is the same as the above-mentioned camera device occlusion detection method in the embodiment of the present disclosure Similarly, the implementation of the device can refer to the implementation of the method.

Referring to FIG. 6 , it is a schematic diagram of a camera occlusion detection device 500 provided by an embodiment of the present disclosure. The camera occlusion detection device 500 includes:

The video acquisition module 501 is configured to acquire video data of the driving area of the vehicle through the camera device;

The face detection module 502 is configured to perform face detection on the current image frame in the video data, and perform face detection on the current image frame according to pixel values in the current image frame if no face is detected. Encoding, to obtain the feature encoding information of the current image frame;

The occlusion determining module 503 is configured to determine whether the camera is occluded based on the feature encoding information of the current image frame and preset feature encoding information, wherein the preset feature encoding information includes The feature encoding information of the image frame of the face.

In a possible implementation manner, the face detection module 502 is further configured to:

determining a reference pixel threshold of the current image frame;

Sequentially comparing the pixel value of each pixel of the current image frame with the reference pixel threshold, encoding the pixel points greater than the reference pixel threshold value as 1, and encoding the pixel points not greater than the reference pixel threshold value is 0, the feature encoding information of the current image frame is obtained.

In a possible implementation manner, the reference pixel threshold is an average pixel value of the current image frame.

In a possible implementation manner, the occlusion determining module 503 is further configured to:

If the Hamming distance between the feature encoding information of the current image frame and the preset feature encoding information is greater than a preset threshold, it is determined that the camera is blocked.

If the Hamming distance is not greater than the preset threshold, determine the pixel distribution histogram of the current image frame;

Based on the pixel distribution histogram of the current image frame, it is determined whether the camera is blocked.

In a case where the proportion of pixel distribution in a preset interval in the pixel distribution histogram is greater than a preset proportion threshold, it is determined that the camera is blocked.

When the proportion of pixel distribution in the preset interval in the pixel distribution histogram is not greater than the preset proportion threshold, determine the maximum connected domain of the current image frame;

If the area of the largest connected domain is greater than a preset area threshold, it is determined that the camera is blocked.

Referring to Figure 7, in a possible implementation manner, the device further includes:

The information output module 504 is configured to output prompt information when it is determined that the camera is blocked.

In a possible implementation manner, the information output module 504 is further configured to:

According to the camera detection results of each frame image in the video data, determine the continuous occlusion time of the camera;

When the continuous shielding time reaches a preset time, the prompt information is output.

For the description of the processing flow of each module in the device and the interaction flow between the modules, reference may be made to the relevant description in the above method embodiment, and details will not be described here.

Based on the same technical idea, an embodiment of the present disclosure also provides an electronic device. Referring to FIG. 8 , it is a schematic structural diagram of an electronic device 700 provided by an embodiment of the present disclosure, including a processor 701 , a memory 702 , and a bus 703 . Among them, the memory 702 is used to store execution instructions, including a memory 7021 and an external memory 7022; the memory 7021 here is also called an internal memory, and is used to temporarily store calculation data in the processor 701 and exchange data with an external memory 7022 such as a hard disk. The processor 701 exchanges data with the external memory 7022 through the memory 7021 .

In the embodiment of the present disclosure, the memory 702 is used to store application program codes for executing the solutions of the present disclosure, and the execution is controlled by the processor 701 . That is, when the electronic device 700 is running, the processor 701 communicates with the memory 702 through the bus 703, so that the processor 701 executes the application program code stored in the memory 702, and then executes the method described in any of the foregoing embodiments.

Wherein, memory 702 can be, but not limited to, random access memory (Random Access Memory, RAM), read-only memory (Read Only Memory, ROM), programmable read-only memory (Programmable Read-Only Memory, PROM), can Erasable Programmable Read-Only Memory (EPROM), Electric Erasable Programmable Read-Only Memory (EEPROM), etc.

The processor 701 may be an integrated circuit chip with signal processing capability. The above-mentioned processor can be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; it can also be a digital signal processor (Digital Signal Processing, DSP), dedicated integrated Circuit (Application-Specific Integrated Circuit, ASIC), Field Programmable Gate Array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. Various methods, steps and logic block diagrams disclosed in the embodiments of the present invention may be implemented or executed. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, and the like.

It can be understood that, the structure illustrated in the embodiment of the present disclosure does not constitute a limitation to the electronic device 700 . In other embodiments of the present disclosure, the electronic device 700 may include more or fewer components than shown in the illustration, or combine certain components, or separate certain components, or arrange different components. The illustrated components can be realized in hardware, software or a combination of software and hardware.

An embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processor, the steps of the camera-device occlusion detection method in the foregoing method embodiments are executed. Wherein, the storage medium may be a volatile or non-volatile computer-readable storage medium.

An embodiment of the present disclosure also provides a computer program product, the computer program product carries a program code, and the instructions contained in the program code can be used to execute the steps of the camera device occlusion detection method in the above method embodiment, and refer to the above method implementation example.

Wherein, the above-mentioned computer program product may be implemented by means of hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) and the like.

Those skilled in the art can clearly understand that for the convenience and brevity of description, for the working process of the above-described system and device, reference may be made to the corresponding process in the foregoing method embodiments. In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods may be implemented in other ways. The device embodiments described above are illustrative. For example, the division of the units is a logical function division. In actual implementation, there may be another division method. For example, multiple units or components can be combined or integrated. to another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.

If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present disclosure. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disc and other media that can store program codes. .

If the disclosed technical solution involves personal information, the products applying the disclosed technical solution have clearly notified the personal information processing rules and obtained the individual's independent consent before processing personal information. If the disclosed technical solution involves sensitive personal information, the products applying the disclosed technical solution have obtained individual consent before processing sensitive personal information, and at the same time meet the requirement of "express consent". For example, at a personal information collection device such as a camera, a clear and prominent sign is set up to inform that it has entered the scope of personal information collection, and personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed to agree to the collection of his personal information; or On the personal information processing device, when the personal information processing rules are informed with obvious signs/information, personal authorization is obtained through pop-up information or by asking individuals to upload their personal information; among them, the personal information processing rules may include Information processor, purpose of personal information processing, processing method, type of personal information processed and other information.

Finally, it should be noted that the above-mentioned embodiments are specific implementations of the present disclosure, and are used to illustrate the technical solutions of the present disclosure, rather than to limit them, and the protection scope of the present disclosure is not limited thereto. This example describes the present disclosure in detail, and those skilled in the art should understand that any skilled person within the technical scope of the present disclosure can still modify or modify the technical solutions described in the foregoing embodiments. Changes can be easily imagined, or equivalent replacements can be made to some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the scope of the technical solutions of the embodiments of the present disclosure. within the scope of protection. Therefore, the protection scope of the present disclosure should be defined by the protection scope of the claims.

Claims

A camera occlusion detection method, comprising:

Obtain video data of the driving area of the vehicle through the camera device;

Perform face detection on the current image frame in the video data, and if no face is detected, encode the current image frame according to the pixel values in the current image frame to obtain the current image frame The feature encoding information of

Determine whether the camera is blocked based on feature encoding information of the current image frame and preset feature encoding information, wherein the preset feature encoding information includes feature encoding of an image frame containing a human face in the video data information.
The method according to claim 1, wherein said encoding the current image frame according to the pixel values in the current image frame comprises:

determining a reference pixel threshold of the current image frame;

Sequentially comparing the pixel value of each pixel of the current image frame with the reference pixel threshold, encoding the pixel points greater than the reference pixel threshold value as 1, and encoding the pixel points not greater than the reference pixel threshold value is 0, the feature encoding information of the current image frame is obtained.
The method according to claim 2, wherein the reference pixel threshold is an average pixel value of the current image frame.
The method according to any one of claims 1 to 3, wherein the determining whether the camera is blocked based on the feature encoding information of the current image frame and preset feature encoding information includes:

If the Hamming distance between the feature encoding information of the current image frame and the preset feature encoding information is greater than a preset threshold, it is determined that the camera is blocked.
The method according to claim 4, wherein the method further comprises:

If the Hamming distance is not greater than the preset threshold, determine the pixel distribution histogram of the current image frame;

Based on the pixel distribution histogram of the current image frame, it is determined whether the camera is blocked.
The method according to claim 5, wherein the determining whether the camera is blocked based on the pixel distribution histogram of the current image frame comprises:

In a case where the proportion of pixel distribution in a preset interval in the pixel distribution histogram is greater than a preset proportion threshold, it is determined that the camera is blocked.
The method according to claim 5 or 6, wherein the method further comprises:

When the proportion of pixel distribution in the preset interval in the pixel distribution histogram is not greater than the preset proportion threshold, determine the maximum connected domain of the current image frame;

If the area of the largest connected domain is greater than a preset area threshold, it is determined that the camera is blocked.
The method according to any one of claims 1 to 7, wherein the method further comprises:

If it is determined that the camera is blocked, prompt information is output.
The method according to claim 8, wherein when it is determined that the camera is blocked, outputting prompt information includes:

According to the camera detection results of each frame image in the video data, determine the continuous occlusion time of the camera;

When the continuous shielding time reaches a preset time, the prompt information is output.
A camera blocking detection device, comprising:

The video acquisition module is configured to acquire the video data of the driving area of the vehicle through the camera device;

The face detection module is configured to perform face detection on the current image frame in the video data, and encode the current image frame according to the pixel value in the current image frame if no face is detected , to obtain the feature encoding information of the current image frame;

The occlusion determination module is configured to determine whether the camera is occluded based on the feature encoding information of the current image frame and preset feature encoding information, wherein the preset feature encoding information includes that the video data contains a human face The feature encoding information of the image frame.
The device according to claim 10, wherein the face detection module is further configured to: determine a reference pixel threshold of the current image frame; sequentially compare the pixel value of each pixel of the current image frame with the The reference pixel threshold is compared, and the pixel points greater than the reference pixel threshold are encoded as 1, and the pixel points not greater than the reference pixel threshold are encoded as 0, so as to obtain the feature encoding information of the current image frame.
The apparatus according to claim 11, wherein the reference pixel threshold is an average pixel value of the current image frame.
The device according to any one of claims 10 to 12, wherein the occlusion determination module is further configured to: the Hamming distance between the feature encoding information of the current image frame and the preset feature encoding information is greater than In the case of a preset threshold, it is determined that the camera is blocked.
The device according to claim 13, wherein the occlusion determination module is further configured to:

If the Hamming distance is not greater than the preset threshold, determine the pixel distribution histogram of the current image frame; determine whether the camera is blocked based on the pixel distribution histogram of the current image frame.
The device according to claim 14, wherein the occlusion determination module is further configured to: determine the The camera is blocked.
The device according to claim 13 or 14, wherein the occlusion determination module is further configured to:

In the case that the pixel distribution proportion of the preset interval in the pixel distribution histogram is not greater than the preset proportion threshold, determine the maximum connected domain of the current image frame; when the area of the maximum connected domain is greater than In the case of a preset area threshold, it is determined that the camera is blocked.
The device according to any one of claims 10 to 16, wherein the device further comprises: an information output module configured to output prompt information when it is determined that the camera is blocked.
The device according to claim 17, wherein the information output module is further configured to: determine the continuous occlusion time of the camera device according to the detection results of the camera device in each frame of image in the video data; When the time reaches the preset time, output the prompt information.
An electronic device, comprising: a processor, a memory, and a bus, the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory through the bus , when the machine-readable instructions are executed by the processor, the camera-device occlusion detection method according to any one of claims 1 to 9 is executed.
A computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the camera-device occlusion detection method according to any one of claims 1 to 9 is executed.
A computer program product, comprising a computer-readable storage medium storing program code, when the instructions included in the program code are executed by the processor of the computer device, the steps in the method described in any one of claims 1 to 9 are realized .