WO2023124387A1

WO2023124387A1 - Photographing apparatus obstruction detection method and apparatus, electronic device, storage medium, and computer program product

Info

Publication number: WO2023124387A1
Application number: PCT/CN2022/124951
Authority: WO
Inventors: 李阳阳; 许亮; 毛宁元
Original assignee: 上海商汤智能科技有限公司
Priority date: 2021-12-31
Filing date: 2022-10-12
Publication date: 2023-07-06
Also published as: CN114332721A

Abstract

Embodiments of the present disclosure provide a photographing apparatus obstruction detection method and apparatus, an electronic device, a storage medium, and a computer program product. The method comprises: acquiring video data of a scene area by means of a photographing apparatus; performing human face detection on a current image frame in the video data, and when no human face is detected, determining an average pixel value of the current image frame; when the average pixel value of the current image frame is less than a preset reference threshold, inverting pixel values of pixels in the current image frame, to obtain an inverted image; and on the basis of the inverted image, determining whether the photographing apparatus is obstructed. Embodiments of the present disclosure can improve the obstruction detection accuracy of a photographing apparatus.

Description

Method, device, electronic device, storage medium, and computer program product for occlusion detection of camera device

Cross References to Related Applications

The embodiment of the present disclosure is based on the Chinese patent application with the application number 202111668682.X, the application date is December 31, 2021, and the application name is "Camera Blocking Detection Method, Device, Electronic Equipment, and Storage Medium", and requires the Chinese Priority of the patent application, the entire content of the Chinese patent application is hereby incorporated by reference into this disclosure.

technical field

Embodiments of the present disclosure relate to the technical field of image processing, and in particular, to a camera device occlusion detection method, device, electronic equipment, storage medium, and computer program product.

Background technique

With the development of image processing technology, scene analysis and behavior decision-making based on images have been applied to more and more scenes. In most scenarios, image processing relies on the camera to capture high-quality images. If the camera is blocked, it is difficult to obtain images containing effective information, which in turn increases the difficulty of image-based scene analysis and behavioral decision-making.

Taking the driver behavior detection scene in the car cabin as an example, the camera device in the car cabin can be used to restrict the driver's driving behavior, thereby reducing the probability of traffic accidents, thereby helping to improve driving safety. However, in the process of actual use, the camera will be blocked, and if the camera is blocked, the driver's behavior cannot be accurately detected. Therefore, it is necessary to detect whether the camera is blocked and how to improve the accuracy of camera block detection , appears to be particularly important.

Contents of the invention

Embodiments of the present disclosure at least provide a method, device, electronic device, storage medium, and computer program product for occlusion detection of an imaging device, which can not only implement occlusion detection for an imaging device, but also improve detection accuracy.

An embodiment of the present disclosure provides an occlusion detection method for a camera device, including:

Obtain video data of the scene area through the camera device;

Perform face detection on the current image frame in the video data, and determine the pixel average value of the current image frame if no face is detected;

In the case that the pixel average value of the current image frame is lower than the preset reference threshold, performing inversion processing on the pixel values of the pixels in the current image frame to obtain an image after inversion processing;

Based on the inverted image, it is determined whether the camera is blocked.

An embodiment of the present disclosure also provides an occlusion detection device for an imaging device, including:

The first acquisition module is configured to acquire the video data of the scene area through the camera device;

A detection module configured to perform face detection on the current image frame in the video data, and determine the pixel average value of the current image frame if no face is detected;

A processing module configured to perform inversion processing on the pixel values of the pixels in the current image frame when the average value of the pixels in the current image frame is lower than a preset reference threshold, to obtain an image after inversion processing ;

The judging module is configured to determine whether the camera is blocked based on the inverted image.

An embodiment of the present disclosure also provides an electronic device, including: a processor, a memory, and a bus, the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the The memories communicate through a bus, and when the machine-readable instructions are executed by the processor, the camera device occlusion detection method described in any one of the foregoing possible implementation manners is executed.

An embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processor, the camera device described in any one of the above-mentioned possible implementation manners is executed. Detection method.

An embodiment of the present disclosure also provides a computer program product, including a computer-readable storage medium storing program codes, and when instructions included in the program codes are executed by a processor of a computer device, the steps of the above method are implemented.

In the embodiment of the present disclosure, face detection is performed on the current image frame in the video data, and if no face is detected, the pixel average value of the current image frame is determined, and when the pixel average value of the current image frame is low In the case of a preset reference threshold, the pixel value of the pixel in the current image frame is reversed, and then based on the reversed image, it is determined whether the camera is blocked. In this way, no face can be detected In the case of the current image frame, further judgment is made on the current image frame, and in the case of low image brightness, the pixel points in the image are reversed, so that the pixel value can be mapped to an easier-to-observe pixel value range, which improves the image quality. The judgment accuracy of the occlusion state of the device.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory in nature and are not restrictive of the disclosure.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present disclosure more clearly, the following will briefly introduce the accompanying drawings used in the embodiments. The accompanying drawings here are incorporated into the specification and constitute a part of the specification. The drawings show the embodiments consistent with the present disclosure, and are used together with the description to explain the technical solutions of the present disclosure. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be regarded as limiting the scope. For those skilled in the art, they can also make From these drawings other related drawings are obtained.

FIG. 1 shows a schematic flowchart of a first method for occlusion detection of a camera device provided by an embodiment of the present disclosure;

FIG. 2 shows a schematic flowchart of a method for determining the average value of pixels of a current image frame provided by an embodiment of the present disclosure;

FIG. 3 shows a schematic flowchart of a method for inverting pixel values of pixels in a current image frame provided by an embodiment of the present disclosure;

FIG. 4 shows a schematic flowchart of a second method for occlusion detection of a camera provided by an embodiment of the present disclosure;

FIG. 5 shows a schematic flowchart of a third method for occlusion detection of a camera device provided by an embodiment of the present disclosure;

FIG. 6 shows a schematic flowchart of a fourth method for occlusion detection of a camera device provided by an embodiment of the present disclosure;

FIG. 7 shows a schematic flowchart of a method for outputting prompt information based on an image judgment result provided by an embodiment of the present disclosure;

FIG. 8 shows a schematic structural diagram of an occlusion detection device for a camera device provided by an embodiment of the present disclosure;

FIG. 9 shows a schematic structural diagram of another camera device occlusion detection device provided by an embodiment of the present disclosure;

Fig. 10 shows a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are The present disclosure discloses some embodiments, but not all embodiments. The components of the disclosed embodiments generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations. Accordingly, the following description of the embodiments of the present disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative effort shall fall within the protection scope of the present disclosure.

It should be noted that like numerals and letters denote similar items in the following figures, therefore, once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

The term "and/or" in this article describes an association relationship, which means that there can be three kinds of relationships, for example, A and/or B, which can mean: A exists alone, A and B exist simultaneously, and B exists independently. Condition. In addition, the term "at least one" herein means any one of a variety or any combination of at least two of the more, for example, including at least one of A, B, and C, which may mean including from A, Any one or more elements selected from the set formed by B and C.

Taking the driver behavior detection scene in the car cabin as an example, the camera device in the car cabin can be used to restrict the driver's driving behavior, thereby reducing the probability of traffic accidents, thereby helping to improve driving safety. However, in the process of actual use, the camera device may be blocked, and if the camera device is blocked, the driver's behavior cannot be accurately detected.

After research, it is found that although there are methods in the prior art that can detect whether the camera is blocked, such as using face recognition to determine whether the camera is blocked, this method is prone to problems when the overall brightness of the image is low. Misjudgments or omissions.

In view of the above problems, the embodiment of the present disclosure provides a method for occlusion detection of a camera device. The video data of the scene area is obtained by the camera device, and then the face detection is performed on the current image frame in the video data. In the case of , determine the average value of pixels in the current image frame, and in the case that the average value of pixels in the current image frame is lower than the preset reference threshold, reverse the pixel values of the pixels in the current image frame, and then based on Invert the processed image to determine whether the camera is blocked. In this way, the current image frame can be further judged when no face is detected, and the pixels in the image can be further judged when the image brightness is low. Points are inverted, so that the pixel value can be mapped to an easier-to-observe pixel value range, which improves the accuracy of judging the occlusion state of the camera device.

The execution subject of the method for detecting occlusion of a camera device provided by the embodiments of the present disclosure may be an electronic device or a server or other processing device with certain computing capabilities, and the electronic device may be a mobile device, a handheld device, a computing device, a vehicle device, a wearable equipment etc. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or it can provide basic cloud computing such as cloud services, cloud databases, cloud computing, cloud storage, big data and artificial intelligence platforms. Cloud server for the service. In some possible implementation manners, the camera occlusion detection method may be implemented in a manner in which a processor invokes computer-readable instructions stored in a memory.

The occlusion detection method of the camera device provided by the embodiment of the present disclosure will be described below.

Referring to FIG. 1 , it is a schematic flowchart of a first method for occlusion detection of a camera device provided by an embodiment of the present disclosure, and the method includes the following steps S101 to S104:

S101. Acquire video data of a scene area through a camera device.

Exemplarily, the camera device refers to a device capable of recording video of the current scene area in real time. Taking the car cabin scene as an example, the scene area includes the driving area, and the camera device can be installed inside the vehicle to obtain video data of the driver in the driving area of the vehicle.

Among them, the camera device is the necessary hardware of the driver monitoring system (Driver Monitoring System, DMS). The driver monitoring system uses the camera to obtain images, and uses technologies such as visual tracking, target detection, and action recognition to perform real-time intelligent detection and reminders of driver fatigue, driving distraction, and dangerous actions to reduce the probability of traffic accidents . By detecting the blocking state of the camera device, prompts can be given in time when the camera device is blocked so that the camera device can be adjusted to a normal non-blocking state, thereby assisting in improving driving safety.

It can be understood that in other implementation manners, the scene area may also be a panoramic area in the vehicle cabin or other areas in the vehicle cabin including the driving area of the vehicle, which is not limited herein.

The vehicle driving area refers to an area where a driver can perform a vehicle driving operation. Wherein, the vehicle driving operation includes but not limited to a steering wheel control operation, an accelerator pedal control operation, and the like. When the driver is driving the vehicle in the vehicle driving area, the camera device will collect video of the driver's driving behavior and physiological state in the cabin in real time.

It should be noted that the number of camera devices may be set according to actual needs. In some embodiments, it can be set according to the shooting angle and shooting range of the camera, or according to factors such as cost. For example, there can be one camera, two or three cameras, etc., which is not limited here.

Video data refers to a continuous image sequence, which is essentially composed of a group of continuous images. Among them, an image frame (Frame) is the smallest visual unit of video data, and is a static image. A sequence of temporally continuous image frames is synthesized to form a dynamic video. Therefore, in order to facilitate subsequent detection, it is necessary to extract multiple frames of images in the video data.

Exemplarily, since video data usually includes multiple frames of images per second (for example, 24 frames of images per second), therefore, during the process of extracting images in the real-time video data, frame extraction can be performed, wherein , frame extraction refers to frame extraction according to the preset interval frame number, for example, extracting a frame of image every 20 frames; it can also perform frame extraction according to the preset time interval, such as every interval of 10 milliseconds (ms) Fetch the image once.

It should be noted that the preset number of frame intervals and the time interval can be set according to actual needs, and are not limited here.

S102. Perform face detection on a current image frame in the video data, and determine an average value of pixels of the current image frame if no face is detected.

Among them, the current image frame refers to the image that needs to be detected and recognized currently. In the video data, the image whose time sequence is before the current image frame is called the previous image frame, and the image whose time sequence is after the current image frame is called the subsequent image frame.

Exemplarily, after the aforementioned multiple frames of images are extracted, face detection may be performed on the extracted images to determine whether there is a face in the current image frame. In some embodiments, if a human face is detected in the current image frame, it means that the area captured by the camera device contains a human face, and the camera device is not blocked; The image frame is further judged to distinguish that no human face is detected due to the blocking of the camera device or that there is no human face in the collection area of the camera device, so as to improve the accuracy of judgment.

In this embodiment, if no human face is detected in the current image frame, the pixel average value of the current image frame is further determined. Wherein, the average value of the pixels of the current image frame refers to the average value of the pixels of the pixel points in the current image frame, which is used to judge the overall brightness and darkness of the current image frame.

S103. In the case that the average value of pixels in the current image frame is lower than a preset reference threshold, perform inversion processing on the pixel values of the pixels in the current image frame to obtain an inversion-processed image.

After determining the average value of the pixels of the current image frame, if the average value of the pixels of the current image frame is lower than the preset reference threshold, it means that the current image frame is darker. In order to improve the judgment accuracy, the pixels of the pixels in the current image frame can be The value is negated.

It can be understood that the preset reference threshold can be determined according to the scene where the camera device is located. For example, if the background of the driving area is mostly bright, the preset reference threshold at this time can be set to a relatively high value, and if the driving area If the background is mostly dark, the preset reference threshold at this time can be set to a lower value accordingly, so that the accuracy of judgment can be further improved.

Among them, there will be many pixels in the current image frame (such as 320*320), and each pixel has a total of 256 gray levels, that is, the pixel value of each pixel is between 0 and 255, according to the current image The frame average pixel value can determine the brightness and darkness of the current image frame. In some embodiments, the preset reference threshold can be set to 80. If it is lower than the preset reference threshold, it proves that the current image frame is overall dark.

In some embodiments, the inversion process refers to the processing method of mapping the pixel value of the image to a range of pixel values that is easier to analyze by inversion, that is, the inversion process is to subtract the pixel value of the current pixel point from the pixel value 255, The inverted pixel value is obtained. For example, the pixel value of a pixel in the current image frame is 103, and the inverted pixel value is 152.

S104. Based on the inverted image, determine whether the camera is blocked.

After inverting the pixel values of the pixels in the current image frame, it is determined whether the camera is blocked based on the inverted image.

In the embodiment of the present disclosure, face detection is performed on the current image frame in the video data, and if no face is detected, the pixel average value of the current image frame is determined, and when the pixel average value of the current image frame is low In the case of a preset reference threshold, the pixel value of the pixel in the current image frame is reversed, and then based on the reversed image, it is determined whether the camera is blocked. In this way, no face can be detected In the case of the current image frame, further judgment is made on the current image frame, and in the case of low image brightness, the pixel points in the image are reversed, so that the pixel value can be mapped to the pixel value range that is easier to analyze, which improves the image quality. The judgment accuracy of the occlusion state of the device.

For the above S102, see FIG. 2 , which is a schematic flowchart of a method for determining the average value of pixels of a current image frame provided by an embodiment of the present disclosure, including the following S1021 and S1022:

S1021. If no human face is detected, perform noise reduction processing on the current image frame.

S1022. Determine an average value of pixels of the current image frame after the noise reduction processing.

After it is determined that no human face is detected, noise reduction processing is performed on the current image frame. Among them, noise is an important cause of image interference, and a frame of image may have various noises in practical applications, and these noises may be generated during transmission or quantization and other processing. Therefore, the process of reducing noise in the image is very important, and the determination accuracy of the pixel average value of the current image frame can be improved through noise reduction processing.

In the embodiment of the present disclosure, the Gaussian smoothing algorithm is used for noise reduction processing, and the Gaussian smoothing is applied to the blurred image to uniformly smooth the transition of the entire image, remove details, reduce imaging snowflake noise, and make the current image frame clearer, which is beneficial to Improve the accuracy of determination of pixel average value. In other embodiments, the noise reduction processing may also use a median filtering algorithm or an average filtering algorithm to perform the noise reduction processing, which is not limited here.

For the above S103, see FIG. 3 , which is a schematic flowchart of a method for inverting the pixel value of a pixel in the current image frame provided by the embodiment of the present disclosure, including the following S1031 and S1032:

S1031. Determine a target pixel point from pixels in the current image frame, where a pixel value of the target pixel point is greater than a first preset pixel value and smaller than a second preset pixel value.

It can be understood that although the average pixel value of the current image frame is lower than the preset reference threshold, it does not mean that the pixel value of each pixel in the current image frame is very low, and for pixels with higher pixel values in the current image frame , then there is no need to perform inversion processing. Therefore, before the inversion processing, it is necessary to further determine the target pixel points that need to be inversion processing. In this embodiment, a pixel point in the current image frame whose pixel value is greater than the first preset pixel value and smaller than the second preset pixel value is used as the target pixel point. For example, the first preset pixel value can be set to 90, and the second preset pixel value can be set to 150. If the pixel value of the pixel point is greater than the first preset pixel value and smaller than the second preset pixel value, it proves that the pixel The point is darker, and the pixel value of the pixel point needs to be reversed.

It should be noted that the first preset pixel value and the second preset pixel value are determined by the imaging parameters of the camera device, and the imaging parameters of the camera device determine the quality of the imaging effect of the camera device, that is, if the camera device If the imaging effect is better than other camera devices, the first preset pixel value and the second preset pixel value can be set to higher pixel values, so that the blocking state of the camera device can be judged more accurately.

In the embodiment of the present disclosure, a third preset pixel value is also set. If the pixel value of the pixel point is smaller than the first preset pixel value, it proves that the pixel point is a black pixel point; if the pixel value of the pixel point is greater than the first preset pixel value If the pixel value is less than the second preset pixel value, it proves that the pixel point is darker; if the pixel value of the pixel point is greater than the second preset pixel value and smaller than the third preset pixel value, it proves that the pixel point is of normal brightness; If the pixel value of the pixel point is greater than the third preset pixel value, it is proved that the pixel point is a white pixel point.

S1032. Perform inversion processing on the pixel value of the target pixel point.

Exemplarily, after determining that the average value of the pixels of the current image frame is lower than the preset reference threshold, determining the target pixel from the pixels in the current image frame, and performing inverse processing on the pixel value of the target pixel, In this way, based on the first preset pixel value and the second preset pixel value, a relatively dark pixel point in the current image frame can be determined, and the pixel value of the pixel point can be reversed, thereby improving the judgment accuracy of the occlusion state of the camera device .

Referring to FIG. 4 , it is a schematic flow chart of the second camera device occlusion detection method provided by the embodiment of the present disclosure, including the following S201 to S210:

S201. Acquire video data of a scene area through a camera device.

Wherein, the scene area may be, for example, a vehicle driving area. This step is similar to step S101 in FIG. 1 .

S202. Perform face detection on the current image frame in the video data, and determine whether there is a human face in the current image frame; if yes, execute step S210; if not, execute step S203.

This step is similar to step S102 in FIG. 1 .

S203. Determine the pixel average value of the current image frame.

This step is similar to step S102 in FIG. 1 .

S204. Determine whether the average value of pixels of the current image frame is lower than a preset reference threshold; if yes, perform step S206; if not, perform step S205.

For example, if the average value of the pixels of the current image frame is lower than the preset reference threshold, it means that the current image frame is relatively dark, and step S206 needs to be performed to invert the pixels of the current image frame to improve the judgment accuracy; if the current If the average pixel value of the image frame is not lower than the preset reference threshold, it means that the brightness of the current image frame is normal. At this time, step S205 is performed to determine the maximum connected area of the current image frame.

S205. Determine the maximum connected domain of the current image frame.

Among them, the maximum connected domain refers to the image area composed of all pixel values with the same pixel value or within a certain error and adjacent to each other. For example, the preset area threshold may be set to an area of 60% of the entire image, and if the largest connected domain exceeds the preset area threshold, it is determined that the camera is blocked. It should be noted that the maximum connected domain is a closed area.

In this embodiment, after the maximum connected domain of the current image frame is determined, step S208 may be directly executed.

S206. Perform inversion processing on the pixel values of the pixel points in the current image frame to obtain an inversion-processed image.

This step is similar to step S103 in FIG. 1 .

S207. Determine the maximum connected domain in the image after the inversion process.

This step is similar to step S205.

S208, judging whether the area of the largest connected domain is greater than a preset area threshold; if yes, execute step S209; if not, execute step S210.

Exemplarily, if the area of the largest connected domain is greater than the preset area threshold, then perform step S209 to determine that the camera is blocked; if the area of the largest connected domain is less than or equal to the preset area threshold, then perform step S210, Make sure the camera is not blocked.

It should be noted that the preset area threshold can be determined by the area size of the entire image frame, that is, if the image has a larger area, the preset area threshold can be set to a larger threshold, so that it can be more accurate to determine the blocking state of the camera.

S209. Determine that the occlusion detection result of the current image frame is that the camera is occluded.

S210. Determine that the occlusion detection result of the current image frame is that the camera is not occluded.

Referring to FIG. 5 , it is a schematic flowchart of a third method for occlusion detection of a camera device provided by an embodiment of the present disclosure. This method is different from the method in FIG. 4 in that it also includes the following S211 to S213 after step S208:

S211. Determine a maximum connected domain in an image of at least one frame of image before and after at least one of the current image frame in the video data after inversion processing.

It can be understood that after it is determined that the area of the largest connected domain of the image after inversion processing is greater than the preset area threshold, it is necessary to further determine at least one frame of image in at least one of before and after the current image frame in the video data The maximum connected domain in the image after inversion processing is taken to reduce the misjudgment caused by the sudden change, wherein the sudden change refers to suddenly blocking the camera device, or suddenly not blocking the camera device.

S212. Determine the average of the area of the largest connected domain in the image after the inversion processing of the current image frame, and at least one frame of image before and after the current image frame. value.

For at least one frame of image in at least one of before and after the current image frame in the video data, after determining the maximum connected domain in the image after its inversion processing, it is necessary to further determine the current image frame after inversion processing In the image, and at least one frame of image before and after the current image frame is taken as an average value of the area of the largest connected domain in the image after inverse processing.

Among them, the average refers to the sum of all data in a set of data divided by the number of this set of data, which is used to reflect the average level of a set of data.

S213, judging whether the average value is greater than the preset area threshold; if yes, execute step S209; if not, execute step S210.

After determining the average value of the area of the largest connected domain in the image after the inverse processing of the current image frame and at least one frame of image before and after the current image frame, if the If the average value is greater than the preset area threshold, it is determined that the camera is blocked. In this way, the sudden change can be ruled out by determining the average value of the area of the largest connected domain, so that the occlusion state of the camera is not affected by the sudden change, and the judgment can be improved. The accuracy of the occlusion state of the camera.

Referring to FIG. 6 , it is a schematic flowchart of a fourth method for occlusion detection of an imaging device provided by an embodiment of the present disclosure. The difference between this method and the method in FIG. 1 is that after step S104, the following S105 is also included:

S105. Output first prompt information if it is determined that the camera is blocked.

Exemplarily, if it is determined that the camera is blocked, then output the first prompt information that the camera is blocked, so that the driver can be prompted that the camera is blocked, so that the driver can more clearly understand the blocking state of the camera, and Process the blocked camera device, and then pay attention to your own driving behavior and adjust it, so as to improve driving safety and improve the driver's car experience.

Wherein, the prompt information includes but not limited to voice prompt information, sound (such as alarm sound) prompt information, graphic prompt information, and the like.

For the above S105, see FIG. 7 , which is a schematic flow chart of a method for outputting prompt information based on an image-based judgment result provided by an embodiment of the present disclosure. The method includes the following S1051 and S1052:

S1051. Determine the continuous shielding time of the camera according to the detection result of the camera in each frame of image in the video data.

S1052. Output the first prompt information when the continuous shielding time reaches a preset time.

Exemplarily, according to the camera detection results of each frame of image in the video data, the continuous occlusion time of the camera is determined, and if the continuous occlusion time reaches a preset time, a first prompt message is output, thus, in When the camera continues to be blocked, the first prompt information is output to reduce the occurrence of frequent output of prompt information.

In this embodiment, when the current image frame determines that the camera is blocked, it is also necessary to determine whether the judgment result of the subsequent frame image is also blocked. It just flashes, and at this time, the first prompt information is not output; if the judgment result of the frame image after at least one frame is also that the camera is blocked, that is, it is judged that the camera is blocked for multiple consecutive frames of images, it means that the camera is blocked at this time. Continuous occlusion is intentional occlusion, not false occlusion, and when the continuous occlusion time reaches a preset time (for example, 5 seconds), first prompt information is output for prompting.

In some possible implementations, taking five frames of images as an example, if the first four frames of images are all blocked, and by accumulating the blocking time, it is found that the blocking time is 4s, and the preset time is 3s, then it is determined that the camera lasts is blocked, at this time, output the first prompt information that the camera is blocked.

In some other possible implementations, taking five frames of images as an example, if the first two frames of images are blocked and the third frame of images is not blocked, by accumulating the blocking time of the first two frames, it is found that the blocking time is 2s, and the preset time is 3s, it means that the occlusion is a temporary occlusion, and at this time, the first prompt message will not be output.

In some possible implementations, when the scene area includes the driving area of the vehicle, in the case that the camera is detected to be unobstructed according to the above method, the state information of the vehicle may be further acquired, and the current image frame may be determined according to the state information of the vehicle Whether the vehicle is in a driving state at the corresponding moment. If it is determined that the vehicle is in a driving state, since no human face is detected from the current image frame at this time and the camera is not blocked, it can be considered that the driver of the vehicle has left the driving area or the driver turned his head towards the rear of the car. At this time, the second prompt information for the driver leaving the post may be generated. In order to accurately detect and prompt the driver to leave the driving area or turn his head towards the rear while the vehicle is running. In some embodiments, if the vehicle is in a non-driving state, such as when the vehicle is parked, the driver may leave the driving area, and if it is determined that the camera is not blocked, it can No alarm or prompt is issued.

Those skilled in the art can understand that in the above method of specific implementation, the writing order of each step does not mean a strict execution order and constitutes any limitation on the implementation process. The specific execution order of each step should be based on its function and possible The inner logic is OK.

Based on the same inventive concept, the embodiment of the present disclosure also provides a camera device occlusion detection device corresponding to the camera device occlusion detection method, because the principle of solving the problem of the device in the embodiment of the present disclosure is the same as the above-mentioned camera device occlusion detection method in the embodiment of the present disclosure Similarly, the implementation of the device can refer to the implementation of the method.

Referring to FIG. 8 , which is a schematic structural diagram of an occlusion detection device for a camera device provided by an embodiment of the present disclosure, the device 500 includes:

The first acquiring module 501 is configured to acquire the video data of the scene area through the camera device;

The detection module 502 is configured to perform face detection on the current image frame, and determine the pixel average value of the current image frame when no face is detected;

The processing module 503 is configured to perform inversion processing on the pixel values of the pixels in the current image frame when the average value of the pixels in the current image frame is lower than a preset reference threshold, to obtain the inversion processing image;

The judging module 504 is configured to determine whether the camera is blocked based on the inverted image.

In a possible implementation manner, the processing module 503 is further configured to:

determining a target pixel point from pixels in the current image frame, wherein the pixel value of the target pixel point is greater than a first preset pixel value and smaller than a second preset pixel value;

Negative processing is performed on the pixel value of the target pixel point.

In a possible implementation manner, the first preset pixel value and the second preset pixel value are determined by imaging parameters of the camera device.

In a possible implementation manner, the judging module 504 is further configured to:

Determining the largest connected domain in the image after the inversion process;

In the case that the area of the largest connected domain is greater than a preset area threshold, it is determined that the occlusion detection result of the current image frame is that the camera is occluded.

In the case where the area of the largest connected domain is greater than the preset area threshold, determine at least one frame of image in the video data before and after at least one of the current image frame in the image after inversion processing The largest connected domain of ;

Determining the average value of the area of the largest connected domain in the image after the inverse processing of the current image frame, and at least one frame of image before and after the current image frame after inversion processing;

In a case where the average value is greater than the preset area threshold, it is determined that the occlusion detection result of the current image frame is that the camera is occluded.

Referring to Figure 9, in a possible implementation manner, the device further includes:

The first output module 505 is configured to output first prompt information when it is determined that the camera is blocked.

In a possible implementation manner, the first output module 505 is further configured to:

According to the camera detection results of each frame of image in the video data, determine the continuous shielding time of the camera; when the shielding continuous shielding time reaches a preset time, output the first prompt information.

In a possible implementation manner, the detection module 502 is further configured to:

When no face is detected, perform noise reduction processing on the current image frame;

Determine the pixel average value of the current image frame after the noise reduction processing.

In a possible implementation manner, the scene area includes a vehicle driving area, and the device further includes:

The second acquisition module is configured to acquire the status information of the vehicle when it is determined that the camera device is not blocked;

The second output module is configured to generate second prompt information when it is determined according to the state information of the vehicle that the vehicle is in a driving state at the moment corresponding to the current image frame.

For the description of the processing flow of each module in the device and the interaction flow between the modules, reference may be made to the relevant description in the above method embodiment, and details will not be described here.

Based on the same technical idea, an embodiment of the present disclosure also provides an electronic device. Referring to FIG. 10 , it is a schematic structural diagram of an electronic device 700 provided by an embodiment of the present disclosure, including a processor 701 , a memory 702 and a bus 703 . Among them, the memory 702 is used to store execution instructions, including a memory 7021 and an external memory 7022; the memory 7021 here is also called an internal memory, and is used to temporarily store calculation data in the processor 701 and exchange data with an external memory 7022 such as a hard disk. The processor 701 exchanges data with the external memory 7022 through the memory 7021 .

In the embodiment of the present disclosure, the memory 702 is used to store application program codes for executing the solutions of the present disclosure, and the execution is controlled by the processor 701 . That is, when the electronic device 700 is running, the processor 701 communicates with the memory 702 through the bus 703, so that the processor 701 executes the application program code stored in the memory 702, and then executes the methods disclosed in any of the foregoing embodiments.

Wherein, memory 702 can be, but not limited to, random access memory (Random Access Memory, RAM), read-only memory (Read Only Memory, ROM), programmable read-only memory (Programmable Read-Only Memory, PROM), can Erasable Programmable Read-Only Memory (EPROM), Electric Erasable Programmable Read-Only Memory (EEPROM), etc.

The processor 701 may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor can be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; it can also be a digital signal processor (Digital Signal Processing, DSP), dedicated integrated Circuit (Application-Specific Integrated Circuit, ASIC), Field Programmable Gate Array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. Various methods, steps and logic block diagrams disclosed in the embodiments of the present disclosure may be implemented or executed. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.

It can be understood that, the structure illustrated in the embodiment of the present disclosure does not constitute a specific limitation on the electronic device 700 . In other embodiments of the present disclosure, the electronic device 700 may include more or fewer components than shown in the illustration, or combine certain components, or separate certain components, or arrange different components. The illustrated components can be realized in hardware, software or a combination of software and hardware.

An embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processor, the steps of the camera-device occlusion detection method described in the foregoing method embodiments are executed. . Wherein, the storage medium may be a volatile or non-volatile computer-readable storage medium.

The computer program product of the access verification method provided by the embodiments of the present disclosure includes a computer-readable storage medium storing program codes, and the instructions included in the program codes can be used to execute the steps of the camera device occlusion detection method in the above method embodiments , refer to the above method embodiment.

Wherein, the above-mentioned computer program product may be implemented by means of hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) and the like.

Those skilled in the art can clearly understand that for the convenience and brevity of description, for the working process of the above-described system and device, reference may be made to the corresponding process in the foregoing method embodiments. In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods may be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.

If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present disclosure. The aforementioned storage medium includes: various media capable of storing program codes such as U disk, mobile hard disk, read-only memory, random access memory, magnetic disk or optical disk.

If the disclosed technical solution involves personal information, the products applying the disclosed technical solution have clearly notified the personal information processing rules and obtained the individual's independent consent before processing personal information. If the disclosed technical solution involves sensitive personal information, the products applying the disclosed technical solution have obtained individual consent before processing sensitive personal information, and at the same time meet the requirement of "express consent". For example, at a personal information collection device such as a camera, a clear and prominent sign is set up to inform that it has entered the scope of personal information collection, and personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed to agree to the collection of his personal information; or On the personal information processing device, when the personal information processing rules are informed with obvious signs/information, personal authorization is obtained through pop-up information or by asking individuals to upload their personal information; among them, the personal information processing rules may include Information processor, purpose of personal information processing, processing method, type of personal information processed and other information.

Finally, it should be noted that: the above-described embodiments are implementations of the present disclosure, and are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto, although referring to the foregoing embodiments The present disclosure has been described in detail, and those of ordinary skill in the art should understand that: within the technical scope of the present disclosure, any person familiar with the art can still modify the technical solutions described in the foregoing embodiments or can Changes are easily thought of, or equivalent replacements are made to some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be covered by the protection of the present disclosure. within range. Therefore, the protection scope of the present disclosure should be defined by the protection scope of the claims.

Claims

A camera occlusion detection method, comprising:

Obtain video data of the scene area through the camera device;

Perform face detection on the current image frame in the video data, and determine the pixel average value of the current image frame if no face is detected;

In the case that the pixel average value of the current image frame is lower than the preset reference threshold, performing inversion processing on the pixel values of the pixels in the current image frame to obtain an image after inversion processing;

Based on the inverted image, it is determined whether the camera is blocked.
The method according to claim 1, wherein said inverting the pixel values of the pixels in the current image frame comprises:

determining a target pixel point from pixels in the current image frame, wherein the pixel value of the target pixel point is greater than a first preset pixel value and smaller than a second preset pixel value;

Negative processing is performed on the pixel value of the target pixel point.
The method according to claim 2, wherein the first preset pixel value and the second preset pixel value are determined by imaging parameters of the camera device.
The method according to any one of claims 1 to 3, wherein the determining whether the camera is blocked based on the inversely processed image comprises:

Determining the largest connected domain in the image after the inversion process;

In a case where the area of the largest connected domain is greater than a preset area threshold, it is determined that the camera is blocked as a result of the occlusion detection of the current image frame.
The method according to claim 4, wherein in the case where the area of the largest connected domain is greater than a preset area threshold, determining that the occlusion detection result of the current image frame is that the camera is occluded comprises:

In the case where the area of the largest connected domain is greater than the preset area threshold, determine at least one frame of image in the video data before and after at least one of the current image frame in the image after inversion processing The largest connected domain of ;

Determining the average value of the area of the largest connected domain in the image after the inverse processing of the current image frame, and at least one frame of image before and after the current image frame after inversion processing;

In a case where the average value is greater than the preset area threshold, it is determined that the occlusion detection result of the current image frame is that the camera is occluded.
The method according to any one of claims 1 to 5, wherein the method further comprises:

If it is determined that the camera is blocked, first prompt information is output.
The method according to claim 6, wherein, when it is determined that the camera is blocked, outputting first prompt information includes:

According to the camera detection results of each frame image in the video data, determine the continuous occlusion time of the camera;

When the continuous shielding time reaches a preset time, the first prompt information is output.
The method according to any one of claims 1 to 7, wherein said determining the pixel average value of said current image frame in the case of no human face is detected comprises:

When no face is detected, perform noise reduction processing on the current image frame;

Determine the pixel average value of the current image frame after the noise reduction processing.
The method according to any one of claims 1 to 8, wherein the scene area includes a vehicle driving area, and the method further comprises:

When it is determined that the camera device is not blocked, obtain the state information of the vehicle;

When it is determined according to the state information of the vehicle that the vehicle is in a driving state at the time corresponding to the current image frame, second prompt information is generated.
A camera blocking detection device, comprising:

The first acquisition module is configured to acquire the video data of the scene area through the camera device;

A detection module configured to perform face detection on the current image frame in the video data, and determine the pixel average value of the current image frame if no face is detected;

A processing module configured to perform inversion processing on the pixel values of the pixels in the current image frame when the average value of the pixels in the current image frame is lower than a preset reference threshold, to obtain an image after inversion processing ;

The judging module is configured to determine whether the camera is blocked based on the inverted image.
The device according to claim 10, wherein the processing module is further configured to: determine a target pixel point from the pixels in the current image frame, wherein the pixel value of the target pixel point is greater than a first preset The pixel value is smaller than a second preset pixel value; performing inversion processing on the pixel value of the target pixel point.
The device according to claim 11, wherein the first preset pixel value and the second preset pixel value are determined by imaging parameters of the camera device.
The device according to any one of claims 10 to 12, wherein the judging module is further configured to: determine the maximum connected domain in the image after inversion processing; the area of the maximum connected domain is greater than a preset In the case of an area threshold, it is determined that the occlusion detection result of the current image frame is that the camera is occluded.
The device according to claim 13, wherein the judging module is further configured to: when the area of the largest connected domain is greater than the preset area threshold, determine the current image frame in the video data The largest connected domain in the image after at least one frame image of at least one of and after is reversed; determine the image after the current image frame is reversed, and at least one of before and after the current image frame One of at least one frame of image takes the average value of the area of the largest connected domain in the image after inversion processing; in the case where the average value is greater than the preset area threshold, determine the occlusion detection result of the current image frame For the camera is blocked.
The device according to any one of claims 10 to 14, wherein the device further comprises:

The first output module is configured to output first prompt information when it is determined that the camera is blocked.
The device according to claim 15, wherein the first output module is further configured to: determine the continuous occlusion time of the camera device according to the detection results of the camera device in each frame of image in the video data; When the continuous occlusion time reaches a preset time, the first prompt information is output.
The device according to any one of claims 10 to 16, wherein the detection module is further configured to: perform noise reduction processing on the current image frame when no face is detected; determine the noise reduction Pixel average of the current image frame after processing.
The device according to any one of claims 10 to 17, wherein the scene area includes a vehicle driving area, and the device further includes: a second acquisition module configured to determine that the camera is not blocked , acquiring state information of the vehicle; a second output module configured to generate second prompt information when it is determined according to the state information of the vehicle that the vehicle is in a driving state at a time corresponding to the current image frame.
An electronic device, comprising: a processor, a memory, and a bus, the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory through the bus , when the machine-readable instructions are executed by the processor, the camera device occlusion detection method according to any one of claims 1 to 9 is executed.
A computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the camera-device occlusion detection method according to any one of claims 1 to 9 is executed.
A computer program product, comprising a computer-readable storage medium storing program code, when the instructions included in the program code are executed by the processor of the computer device, the steps in the method described in any one of claims 1 to 9 are realized .