WO2023103298A1 - 遮挡检测方法及装置、电子设备、存储介质及计算机程序产品 - Google Patents

遮挡检测方法及装置、电子设备、存储介质及计算机程序产品 Download PDF

Info

Publication number
WO2023103298A1
WO2023103298A1 PCT/CN2022/095516 CN2022095516W WO2023103298A1 WO 2023103298 A1 WO2023103298 A1 WO 2023103298A1 CN 2022095516 W CN2022095516 W CN 2022095516W WO 2023103298 A1 WO2023103298 A1 WO 2023103298A1
Authority
WO
WIPO (PCT)
Prior art keywords
video frame
image block
detected
image
target image
Prior art date
Application number
PCT/CN2022/095516
Other languages
English (en)
French (fr)
Inventor
宗泽亮
吴佳飞
张广程
张炜
Original Assignee
上海商汤智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海商汤智能科技有限公司 filed Critical 上海商汤智能科技有限公司
Publication of WO2023103298A1 publication Critical patent/WO2023103298A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/98—Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects

Definitions

  • the embodiment of the present disclosure is based on the Chinese patent application with the application number 202111512423.8, the application date is December 08, 2021, and the application name is "occlusion detection method and device, electronic equipment and storage medium", and the priority of the Chinese patent application is required Right, the entire content of this Chinese patent application is hereby incorporated into this disclosure as a reference.
  • the present disclosure relates to but not limited to the technical field of computer vision, and in particular relates to an occlusion detection method and device, electronic equipment, storage media and computer program products.
  • Intelligent monitoring system has been widely used in various fields of national life.
  • the monitoring area of the intelligent monitoring system is initially set and adjusted. If the image acquisition equipment is blocked during use, the intelligent monitoring system will not be able to To achieve the expected monitoring effect. Therefore, occlusion detection of image acquisition equipment is very important for the stable operation of intelligent monitoring system. In related technologies, performing occlusion detection calculations on image acquisition devices is time-consuming and has low accuracy.
  • the present disclosure proposes a technical solution of an occlusion detection method and device, electronic equipment, a storage medium, and a computer program product.
  • An embodiment of the present disclosure provides an occlusion detection method, including: determining the Whether there is a target to-be-detected image block in the current video frame relative to the reference video frame with an abnormal change in the image characteristic graph; if it is determined that the target to-be-detected image block exists in the current video frame, the Perform edge detection on the target image block to be detected to obtain an edge detection result of the target image block to be detected; according to the edge detection result, determine whether the image acquisition device that collects the current video frame is blocked.
  • An embodiment of the present disclosure also provides an occlusion detection device, including: a first determining part configured to, according to the image feature statistics map of a plurality of image blocks to be detected in the current video frame, and a plurality of reference image blocks in a reference video frame An image feature statistical map for determining whether there is a target image block to be detected that has an abnormal change in the image feature statistical map relative to the reference video frame in the current video frame; the edge detection part is configured to determine the current video frame In the case where the target image block to be detected exists, edge detection is performed on the target image block to be detected to obtain an edge detection result of the target image block to be detected; the second determining part is configured to As a result of the detection, it is determined whether the image acquisition device that acquires the current video frame is occluded.
  • An embodiment of the present disclosure also provides an electronic device, including: a processor; a memory configured to store processor-executable instructions; wherein the processor is configured to call the instructions stored in the memory to perform the above method .
  • An embodiment of the present disclosure also provides a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the foregoing method is implemented.
  • An embodiment of the present disclosure also provides a computer program product, the computer program product includes a computer program or an instruction, and when the computer program or instruction is run on an electronic device, the electronic device is made to execute the above-mentioned occlusion detection method in the steps.
  • preliminary occlusion detection is performed on the current video frame , to determine whether there is a target image block to be detected in the current video frame that has an abnormal change in the image feature statistical map compared to the reference video frame; then, only perform edge detection on the target image block to be detected that has an abnormal image feature statistical map change, and according to The edge detection result further determines whether there is an occlusion in the image acquisition device that captures the current video frame.
  • the embodiment of the present disclosure can determine the target image block to be detected that has an abnormal change in the image characteristic statistical map in the current video frame through preliminary occlusion detection, that is, it is initially determined that the image acquisition device that collects the current video frame may be occluded, and then detect In the step, it is further determined according to the edge detection result that the image acquisition device collecting the current video frame has occlusion, therefore, the accuracy of occlusion detection to the image acquisition device can be improved; at the same time, because the edge detection is only aimed at the target with abnormal changes in the image feature statistics map Detect image blocks, therefore, while improving the accuracy of occlusion detection, it can also effectively reduce the computing power requirements of occlusion detection.
  • FIG. 1 is a schematic flowchart of an occlusion detection method provided by an embodiment of the present disclosure
  • FIG. 2 is a schematic diagram of multiple image blocks in a video frame provided by an embodiment of the present disclosure
  • FIG. 3A is a schematic diagram of an image block to be detected provided by an embodiment of the present disclosure.
  • 3B is a schematic diagram of an image histogram of an image block to be detected provided by an embodiment of the present disclosure
  • FIG. 4 is a schematic flowchart of an occlusion detection method provided by an embodiment of the present disclosure
  • FIG. 5 is a schematic diagram of the composition and structure of an occlusion detection device provided by an embodiment of the present disclosure
  • FIG. 6 is a hardware entity block diagram of an electronic device provided by an embodiment of the present disclosure.
  • FIG. 7 is a physical block diagram of hardware of an electronic device provided by an embodiment of the present disclosure.
  • occlusion detection for image acquisition equipment mainly includes the following four methods:
  • the first one is to use the background modeling technology to extract suspicious foreground from the video collected by the image acquisition device, and judge whether the image acquisition device is occluded based on the extracted suspicious foreground.
  • this detection method is greatly disturbed by the environment, for example, the detection accuracy is low in crowd flow scenes with large changes in the foreground;
  • the second is to judge the occlusion based on the motion vector, that is, divide the video frame into a predetermined number of image blocks, estimate one or more candidate motion vectors for each image block, and determine the motion vector of the image block from at least one candidate motion vector , and then determine whether the image block is an image block in an occlusion area or an image block in an exposure area according to the motion vector of the image block to detect the occlusion type of the image block.
  • this method needs to continuously detect new video frames to determine whether the image acquisition device is blocked in real time, so the calculation is time-consuming and greatly affected by light;
  • the third method is to establish an image feature histogram for the video frame, and then detect whether there is an image acquisition device occlusion problem through the change of the image feature histogram.
  • this detection method is also greatly affected by the environment;
  • the fourth type is to use the deep learning method to train the occlusion classification model, and use the occlusion classification model to perform occlusion detection.
  • the model training process of this detection method needs to depend on the quality and quantity of training samples, and the calculation load is relatively large.
  • an embodiment of the present disclosure provides a occlusion detection method, and the occlusion detection method may be executed by an electronic device such as a terminal device or a server.
  • the terminal device may be user equipment (User Equipment, UE), mobile device, user terminal, cellular phone, cordless phone, personal digital assistant (Personal Digital Assistant, PDA), handheld device, computing device, vehicle-mounted device, wearable device, etc.
  • the occlusion detection method may be implemented by a processor invoking computer-readable instructions stored in a memory, or the occlusion detection method may be executed by a server.
  • Fig. 1 is a schematic flow chart of a occlusion detection method provided by an embodiment of the present disclosure. As shown in Fig. 1 , the occlusion detection method includes steps S11 to S13, wherein:
  • Step S11 according to the image characteristic statistical diagram of multiple image blocks to be detected in the current video frame and the image characteristic statistical diagram of multiple reference image blocks in the reference video frame, determine whether there is an image appearing in the current video frame relative to the reference video frame The target image block to be detected with abnormal changes in the feature statistics map.
  • the image acquisition device may be a camera in the intelligent monitoring system, a video camera, a terminal with an image acquisition function, etc., which is not specifically limited in the present disclosure.
  • the current video frame and the reference video frame are two video frames in the video stream collected by the image acquisition device, wherein the current video frame is the video frame collected by the image acquisition device at the current moment, and the acquisition time of the reference video frame is before the current video frame .
  • each video frame used for occlusion detection can be divided into image blocks based on the same image block rule, so that the partial image of the video frame can be determined according to the image block Features (for example, the image feature statistical map of a local area), and then use the local image features of the video frame to effectively detect whether there is a local occlusion in the image acquisition device.
  • image block Features for example, the image feature statistical map of a local area
  • the image feature statistical graphs of multiple image blocks to be detected in the current video frame and the image feature statistical graphs of multiple reference image blocks in the reference video frame it can be determined whether there are image feature statistics in the current video frame relative to the reference video frame.
  • the image block of the target to be detected with an abnormal change in the map can be used to perform preliminary occlusion detection on the image acquisition device to determine whether the image acquisition device is occluded at the current moment relative to the sampling time of the reference video frame.
  • the determination of the image feature statistics graph of multiple image blocks to be detected in the current video frame the determination of the image feature statistics graph of multiple reference image blocks in the reference video frame, and the determination of the image feature statistics graph of the current video frame.
  • Step S12 when it is determined that the target image block to be detected exists in the current video frame, edge detection is performed on the target image block to be detected, and an edge detection result of the target image block to be detected is obtained.
  • Step S13 according to the edge detection result, it is determined whether the image acquisition device that acquires the current video frame is occluded.
  • the edge detection result of the target image block to be detected with abnormal changes in the image characteristic statistical map it can be further determined whether the image acquisition device is blocked, and the detection accuracy can be improved.
  • the specific process of how to determine whether the image acquisition device capturing the current video frame is occluded according to the edge detection result will be described in detail later in combination with the implementation of the present disclosure.
  • preliminary occlusion detection is performed on the current video frame , to determine whether there is a target image block to be detected in the current video frame that has an abnormal change in the image feature statistical map compared to the reference video frame; then, only perform edge detection on the target image block to be detected that has an abnormal image feature statistical map change, and according to The edge detection result further determines whether there is an occlusion in the image acquisition device that captures the current video frame.
  • the embodiment of the present disclosure can determine the target image block to be detected that has an abnormal change in the image characteristic statistical map in the current video frame through preliminary occlusion detection, that is, it is initially determined that the image acquisition device that collects the current video frame may be occluded, and then detect In the step, it is further determined according to the edge detection result that the image acquisition device collecting the current video frame has occlusion, therefore, the accuracy of occlusion detection to the image acquisition device can be improved; at the same time, because the edge detection is only aimed at the target with abnormal changes in the image feature statistics map Detect image blocks, therefore, while improving the accuracy of occlusion detection, it can also effectively reduce the computing power requirements of occlusion detection.
  • the same image segmentation rule is used to perform image segmentation on each video frame in the video stream collected by the image acquisition device, and multiple video frames corresponding to each video frame are obtained.
  • Image blocks In this way, since the same image block rule is used to block the image of each video frame, the image blocks at the same position in different video frames are comparable, therefore, the image blocks at the same position in different video frames can be The comparison is performed to realize the comparison of local image features in different video frames, so as to effectively detect whether there is a local occlusion in the image acquisition device.
  • the image block rule may be to equally divide each video frame into a preset number of image blocks. For example, the image block rule is to divide each video frame into 9 image blocks on average. At this time, divide the length and width of the video frame by 3 to obtain 9 blocks. Assuming that the size of the video frame is 10 ⁇ 10, the length and width cannot be divisible by 3, that is, there is a remainder of 1, then the 1 row and 1 column of the remainder are discarded, and 9 image blocks after equalization are obtained, and the size of each image block is 3 ⁇ 3.
  • the image block rule may be the above-mentioned average division of the video frame into a preset number of image blocks, and other image block rules may also be set according to actual needs, which is not specifically limited in the present disclosure.
  • the number of image blocks obtained by dividing the video frame into image blocks using the image block rules may also be set according to actual requirements, which is not specifically limited in the present disclosure.
  • Fig. 2 is a schematic diagram of multiple image blocks in a video frame provided by an embodiment of the present disclosure.
  • the current video frame 2 is divided into 9 image blocks according to the standard of equal division, that is, the length and width of the current video frame 2 are divided by 3, respectively, to obtain 9 image blocks with the same size, that is, image block 21 to image block 29 .
  • the video stream collected by the image collection device is acquired, and the occlusion detection process for the image collection device is started.
  • the first video frame collected by the image acquisition device is determined as the reference video frame.
  • the reference video frame is divided into image blocks according to a preset image block rule to obtain a plurality of reference image blocks in the reference video frame.
  • the image feature statistical map and the edge detection result of each reference image block in the reference video frame are used as a reference for occlusion detection of subsequent video frames.
  • the image feature map may be an image histogram.
  • the image histogram may be a statistical map of image color features, for example, the image histogram may be an image color histogram based on a red green blue (Red Green Blue, RGB) color space.
  • the image histogram may be an image color histogram based on the Hue Saturation Value (HSV) color space.
  • the image histogram may also be a statistical map of image grayscale features, for example, the image histogram may be an image grayscale histogram based on grayscale space.
  • HSV Hue Saturation Value
  • the initial occlusion detection of the current video frame can be quickly performed by using the image histogram.
  • the image feature statistical map may be an image histogram, and other forms of statistical maps may also be selected according to actual conditions, which is not specifically limited in the present disclosure.
  • the edge detection in the embodiment of the present disclosure may be implemented based on an edge detection operator, where the edge detection operator may be a Sobel operator, a Laplacian operator, etc., which is not specifically limited in the present disclosure.
  • the occlusion detection process is described in detail by taking the image feature statistical graph as an image histogram and edge detection based on an edge detection operator as an example.
  • the image feature statistics map changes abnormally, that is, the image histogram changes abnormally.
  • Each reference image block in the reference video frame is converted into a grayscale image, edge detection is performed on the grayscale image of each reference image block by an edge detection operator, and the image block edge value of each reference image block is obtained.
  • any reference image block first calculate the gray value f(x, y) of the pixel point (x, y) in the reference image block, and then according to the gray value f(x, y) of each pixel point and The average gray value ⁇ corresponding to the grayscale image of the reference image block, based on the following formula (1-1), determines the edge value variance D(f) of the reference image block:
  • the edge value variance of each reference image block is determined as the edge detection result of the corresponding reference image block.
  • occlusion detection is sequentially performed on subsequent video frames of the reference video frame in the video stream in time sequence.
  • the current video frame For the current video frame whose image acquisition time in the video stream is after the reference video frame, the current video frame is divided into image blocks according to the same preset image block rules as the reference video frame, and multiple to-be-detected frames in the current video frame are obtained.
  • Image blocks Wherein, the plurality of image blocks to be detected in the current video frame are in one-to-one correspondence with the plurality of reference image blocks in the reference video frame in terms of image block position distribution.
  • each image block to be detected in the current video frame into a grayscale image, and use the grayscale image of each image block to be detected in the current video frame and the grayscale image of the corresponding reference image block in the reference video frame to The current video frame performs preliminary occlusion detection.
  • FIG. 3A is a schematic diagram of an image block to be detected provided by an embodiment of the present disclosure
  • FIG. 3B is a schematic diagram of an image histogram of an image block to be detected provided by an embodiment of the present disclosure.
  • the current video frame is divided into blocks according to predetermined image block rules to obtain at least one image block to be detected, such as the detected image block A11 in Figure 3A; Detect the color feature of the image block, determine the image color histogram of the image block to be detected based on the RGB color space, and obtain the histograms of three color channels (i.e. R color channel, G color channel and B color channel), as shown in Figure 3B
  • step S11 includes step S111 to step S112, wherein:
  • Step S111 determining the similarity between the image feature statistical map of each image block to be detected in the current video frame and the image feature statistical map of the corresponding reference image block in the reference video frame;
  • Step S112 if there are image blocks to be detected whose similarity is lower than the similarity threshold, determine the image block to be detected whose similarity is lower than the similarity threshold as the target image block to be detected.
  • the histogram similarity between the image histogram of the image block to be detected and the image histogram of the corresponding reference image block in the reference video frame is determined, and then based on The histogram similarity can effectively determine whether there is an abnormal change in the image histogram.
  • the histogram similarity between two image histograms may be determined based on a correlation comparison (Correlation) algorithm.
  • the image histogram of an image block to be detected in the current video frame is H 1
  • the image histogram of the corresponding reference image block in the reference video frame is H 2
  • the following correlation comparison algorithm formula (1-2 ) determine the image histogram H 1 of the image block to be detected, and the histogram similarity d(H 1 , H 2 ) between the image histogram H 2 of the corresponding reference image block in the reference video frame:
  • H k (I) is the statistical value of the color feature I in the image block (the image block to be detected or the reference image block), and the value range of the color feature I is 1 to N.
  • the histogram similarity between two image histograms can also be determined based on the Chi-Square algorithm, the Intersection algorithm, the Bhattacharyya distance algorithm, etc. , which is not specifically limited in the present disclosure.
  • the image block to be detected Based on the image histogram of each image block to be detected in the current video frame, and the histogram similarity of the image histogram of the corresponding reference image block in the reference video frame, determine whether there is a histogram similarity lower than the similarity in the current video frame
  • the image block to be detected with a degree threshold.
  • the specific value of the similarity threshold can be set according to actual conditions, for example, the similarity threshold is 0.7, which is not specifically limited in the present disclosure.
  • the occlusion detection method further includes step S14 to step S15, wherein:
  • Step S14 in the case of determining that there is no target image block to be detected in the current video frame, determine whether there is no image characteristic statistical diagram relative to the reference video frame in each video frame within the second acquisition duration before the current video frame An abnormally changing target image block to be detected;
  • Step S15 In each video frame within the second acquisition duration before the current video frame is determined, if there is no target image block to be detected that has an abnormal change in the image characteristic statistical map relative to the reference video frame, determine the current video frame is the updated reference video frame.
  • each image block to be detected in the current video frame is relative to the corresponding reference image block in the reference video frame, There was no abnormal change in the image histogram. That is to say, the detection result of performing preliminary occlusion detection on the current video frame by using the image histogram is that there is no occlusion in the image acquisition device.
  • the specific value of the second collection duration may be set according to actual conditions, and this disclosure does not specifically limit it.
  • the current video frame may be updated as a reference video frame, which is used as a reference for performing occlusion detection on subsequent video frames.
  • the influence of factors such as illumination changes on the reference video frame can be reduced, so as to improve the referenceability of the reference video frame, thereby effectively improving the accuracy of occlusion detection.
  • the image block to be detected if there is an image block to be detected in the current video frame whose histogram similarity is lower than the similarity threshold, it indicates that the image block to be detected in the current video frame is relative to the corresponding reference image block in the reference video frame. If there is an abnormal change in the image histogram of the image block, the image block to be detected is determined as the target image block to be detected.
  • the current video frame there is an image block to be detected whose image histogram changes abnormally compared with the reference video frame, indicating that the detection result of the preliminary occlusion detection of the current video frame by using the image histogram is that the image block to be detected in the image acquisition device
  • the corresponding acquisition area may be blocked.
  • the detection results of preliminary occlusion detection based on image histogram may have image distortion caused by foreground changes or short-term illumination changes.
  • the histogram changes abnormally, resulting in false occlusion detection. Therefore, when the preliminary occlusion detection is performed based on the image histogram, and it is determined that there is a target image block to be detected that has an abnormal change in the image histogram relative to the reference video frame in the current video frame, edge detection can be used to detect the target image block to be detected. Occlusion detection to improve the accuracy of detection results.
  • the embodiments of the present disclosure perform preliminary occlusion detection based on the image histogram, extract target image blocks that may be occluded in the current video frame, and then only use edge detection Performing occlusion detection on target image blocks to be detected can improve the accuracy of occlusion detection while effectively reducing the computing power requirements of occlusion detection and improving the efficiency of occlusion detection.
  • edge detection is performed on the target image block to be detected to obtain an edge detection result of the target image block to be detected, including steps S121 to S124, wherein:
  • Step S121 performing image grayscale conversion on the target image block to be detected to obtain a grayscale image of the target image block to be detected
  • Step S122 performing edge detection on the grayscale image, and determining the image block edge value of the target image block to be detected
  • Step S123 according to the edge value of the image block and the average gray value corresponding to the gray image, determine the edge value variance of the target image block to be detected;
  • Step S124 determining the edge value variance of the target image block to be detected as the edge detection result.
  • the same edge detection operator is used as when performing edge detection on the corresponding reference image block in the reference image frame.
  • the edge value variance D'(f) of the target image block to be detected is the edge detection result of the target image block to be detected.
  • the edge value variance of the target image block to be detected can reflect the image blur of the target image block to be detected
  • the edge value variance of the target image block to be detected can be determined as the edge detection result of the target image block to be detected for subsequent Determine whether image blurring occurs in the target image block to be detected due to the occlusion of the image acquisition device.
  • the step S13 includes steps S131 to S132, wherein:
  • Step S131 according to the edge value variance of the target image block to be detected and the edge value variance of the corresponding reference image block in the reference video frame, determine whether the target image block to be detected has an edge value relative to the corresponding reference image block in the reference video frame Variance sharply reduced;
  • Step S132 in a case where it is determined that the variance of the edge value of the target image block to be detected has decreased sharply, determine that the acquisition area corresponding to the target image block to be detected in the image acquisition device is blocked.
  • the edge value of the target image block to be detected is sharply reduced relative to the corresponding reference image block in the reference video frame, it can be explained that compared with the corresponding reference image block in the reference frame, the target image to be detected in the current video frame At this time, using edge detection to perform occlusion detection on the target image block to be detected results in partial occlusion in the acquisition area corresponding to the target image block to be detected in the image acquisition device.
  • the edge value of the target image block to be detected does not decrease sharply relative to the corresponding reference image block in the reference video frame. It can be explained that the target image to be detected in the current video frame is At this time, the result of further occlusion detection using edge detection is that there is no local occlusion in the acquisition area corresponding to the target image block to be detected in the image acquisition device.
  • the step S131 includes steps S141 to S142, wherein:
  • Step S141 determining the edge value variance ratio between the edge value variance of the target image block to be detected and the edge value variance of the corresponding reference image block in the reference video frame;
  • Step S142 in the case that the edge value variance ratio is lower than the variance ratio threshold, it is determined that the edge value variance of the target image block to be detected has sharply decreased.
  • the variance ratio threshold may be determined according to actual conditions, for example, the variance ratio threshold is 0.2, which is not specifically limited in the present disclosure.
  • the edge value variance of the target image block to be detected is D'(f)
  • the edge value variance of the corresponding reference image block in the reference video frame is D(f)
  • the edge value variance ratio between the two If it is lower than the variance ratio threshold, it is determined that the variance of the edge value of the target to-be-detected image block is sharply reduced relative to the corresponding reference image block in the reference video frame.
  • the method of determining whether the marginal value variance sharply decreases may be the method of whether the above-mentioned marginal value variance ratio is lower than the variance ratio threshold, or other methods selected according to the actual situation, which is not specifically limited in the present disclosure.
  • the image acquisition device has a lens abnormal state, that is, when it is determined that the edge value variance of the target image block to be detected sharply decreases, but the target in the image acquisition device If there is no occlusion in the acquisition area corresponding to the image block to be detected, it can be considered whether the image acquisition device has lens abnormalities such as lens out of focus, blurred screen, and black screen.
  • the occlusion detection method further includes steps S133 to S134, wherein:
  • Step S133 determine whether there are target image blocks to be detected that have abnormal changes in the image feature statistical graph and sharply reduced edge value variances compared to the reference video frame in each video frame within the first acquisition duration after the current video frame;
  • Step S134 In each video frame within the first acquisition duration after the current video frame is determined, there are target image blocks to be detected that have abnormal changes in the image characteristic statistical map and sharply reduced edge value variance relative to the reference video frame, Generate occlusion warning information.
  • each video collected within the first acquisition time period after the current video frame is collected
  • the frame is continuously detected to determine whether the image acquisition device is continuously detected to be occluded based on each video frame collected within the first acquisition period after the current video frame.
  • the specific value of the first collection duration may be determined according to actual conditions, for example, the first collection duration is 10s, which is not specifically limited in the present disclosure.
  • the step S134 includes steps S151 to S152, wherein:
  • Step S151 Determine the proportion of the occlusion area of the image acquisition device according to the target image block to be detected with abnormal changes in the image characteristic statistical map and a sharp decrease in the variance of the edge value;
  • Step S152 generating occlusion warning information when the proportion of the occlusion area is higher than the threshold of the occlusion area proportion.
  • the proportion of the occlusion area is higher than the threshold of the occlusion area proportion, it can reflect that the current occlusion has affected the normal operation of the image acquisition device. At this time, an occlusion warning message is generated, so as to realize an effective occlusion alarm for the occlusion of the image acquisition device. To avoid the waste of maintenance resources caused by the occlusion alarm that does not affect the normal operation of the image acquisition device due to the current occlusion.
  • the specific value of the occlusion area proportion threshold may be determined according to actual conditions, for example, the occlusion area proportion threshold is 30%, which is not specifically limited in the present disclosure.
  • the reference video frame is no longer regularly updated, so as to avoid determining the video frame captured by the image acquisition device with occlusion as the reference video frame until the alarm is cleared.
  • the alarm release indicates that the image acquisition device with occlusion has been maintained, and the image acquisition device after the alarm is cleared has no occlusion.
  • the image acquisition device in the initial stage of occlusion detection, after the first video frame collected by the image acquisition device is determined as the reference video frame, if in each video frame within the third acquisition duration after the first video frame, each If the image histogram, image block edge value, and image block edge value variance of each image block to be detected remain unchanged, it can be determined that the image acquisition device has been occluded before the occlusion detection starts.
  • FIG. 4 is a schematic diagram of an execution flow of an occlusion detection system provided by an embodiment of the present disclosure. As shown in Figure 4, the execution flow of the occlusion detection system includes:
  • Step S401 acquiring the video stream collected by the image acquisition device
  • Step S402 determining the reference video frame, and the image histogram and edge value variance of each reference image block in the reference video frame;
  • Step S403 performing occlusion detection in real time on the current video frame whose acquisition time in the video stream is after the acquisition time of the reference video frame;
  • Step S404 determine whether there is a target image block to be detected in the current video frame that has an abnormal change in the image histogram relative to the reference video frame, if yes, execute step S405, if not, execute step S409;
  • Step S405 extracting the edge value variance of the target image block to be detected
  • Step S406 according to the edge value variance of the target image block to be detected and the edge value variance of the corresponding reference image block in the reference video frame, determine whether the target image block to be detected has an edge value relative to the corresponding reference image block in the reference video frame Variance decreases sharply, if yes, then execute step S407, if not, then execute step S411;
  • Step S407 determine whether there is a target image block to be detected that has an abnormal change in image histogram and a sharp decrease in the variance of the edge value relative to the reference video frame in each video frame within the first acquisition period after the current video frame, and if so, execute Step S408, if not, execute step S411;
  • Step S408 generating alarm information
  • Step S409 determine whether there is no target image block to be detected whose image histogram changes abnormally relative to the reference video frame in each video frame within the second acquisition duration before the current video frame, if yes, then perform step S410, if not , then execute step S411;
  • Step S410 updating the current video frame to a reference video frame, and executing the above step S403;
  • Step S411 determine the video frame at the next acquisition time as the current video frame, and execute the above step S404.
  • the present disclosure also provides an occlusion detection device, electronic equipment, a computer-readable storage medium, and a computer program product, all of which can be used to implement any of the occlusion detection methods provided in the present disclosure, and refer to the corresponding technical solutions and descriptions in the method section record accordingly.
  • FIG. 5 is a schematic diagram of the composition and structure of a occlusion detection device provided by an embodiment of the present disclosure. As shown in Figure 5, the occlusion detection device 50 includes:
  • the first determining part 51 is configured to determine whether there is a relative A target image block to be detected with an abnormal change in the image feature statistical map in the reference video frame;
  • the edge detection part 52 is configured to perform edge detection on the target image block to be detected when it is determined that there is a target image block to be detected in the current video frame, and obtain an edge detection result of the target image block to be detected;
  • the second determination part 53 is configured to determine whether the image capture device that captures the current video frame is occluded according to the edge detection result.
  • the first determining part 51 is further configured to: determine the image feature statistical map of each image block to be detected in the current video frame, and the image feature statistical map of the corresponding reference image block in the reference video frame The similarity between them; in the case that there is an image block to be detected whose similarity is lower than the similarity threshold, the image block to be detected whose similarity is lower than the similarity threshold is determined as the target image block to be detected.
  • the edge detection part 52 is further configured to: perform image grayscale conversion on the target image block to be detected to obtain a grayscale image of the target image block to be detected; perform edge detection on the grayscale image to determine the target image block to be detected. Detect the image block edge value of the image block; determine the edge value variance of the target image block to be detected according to the image block edge value and the corresponding average gray value of the grayscale image; determine the edge value variance of the target image block to be detected as the edge Test results.
  • the second determination part 53 includes: a first determination subsection configured to determine according to the edge value variance of the target image block to be detected and the edge value variance of the corresponding reference image block in the reference video frame Whether the target image block to be detected has a sharp decrease in the variance of the edge value relative to the corresponding reference image block in the reference video frame; the second determining subsection is configured to determine that the target image block to be detected has a sharp decrease in the variance of the edge value, It is determined that there is occlusion in the acquisition area corresponding to the target image block to be detected in the image acquisition device.
  • the first determining subpart is further configured to: determine the edge value variance ratio between the edge value variance of the target image block to be detected and the edge value variance of the corresponding reference image block in the reference video frame; In the case that the variance ratio of the edge value is lower than the variance ratio threshold, it is determined that the variance of the edge value of the target image block to be detected sharply decreases.
  • the occlusion detection device 50 further includes: a third determining part configured to determine whether there are image feature statistics that appear relative to the reference video frame in each video frame within the first acquisition duration after the current video frame
  • the generation part is configured to have image feature statistics relative to the reference video frame in each video frame within the first acquisition duration after determining the current video frame
  • an occlusion warning message is generated.
  • the generating part includes: a third determining sub-part configured to determine the proportion of the occlusion area of the image acquisition device according to the target image block to be detected with an abnormal change in the image characteristic statistical map and a sharp decrease in the variance of the edge value ;
  • the generation subsection is configured to generate occlusion warning information when the proportion of the occlusion area is higher than the threshold of the occlusion area proportion.
  • the occlusion detection device 50 further includes: a fourth determining part, configured to determine the current video frame within the second acquisition duration before the current video frame when it is determined that there is no target image block to be detected In each video frame, whether there is no target image block to be detected that has an abnormal change in the image feature statistical graph relative to the reference video frame; , when there is no target image block to be detected that has an abnormal change in the image feature statistical graph relative to the reference video frame, the current video frame is determined as the updated reference video frame.
  • a fourth determining part configured to determine the current video frame within the second acquisition duration before the current video frame when it is determined that there is no target image block to be detected In each video frame, whether there is no target image block to be detected that has an abnormal change in the image feature statistical graph relative to the reference video frame; , when there is no target image block to be detected that has an abnormal change in the image feature statistical graph relative to the reference video frame, the current video frame is determined as the updated reference video frame.
  • the image feature statistics map is an image histogram.
  • the functions or parts included in the apparatus provided by the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments, and for specific implementation, refer to the descriptions of the above method embodiments.
  • a "part" may be a part of a circuit, a part of a processor, a part of a program or software, etc., of course it may also be a unit, a module or a non-modular one.
  • Embodiments of the present disclosure also provide a computer-readable storage medium, on which computer program instructions are stored, and the above-mentioned method is implemented when the computer program instructions are executed by a processor.
  • Computer readable storage media may be volatile or nonvolatile computer readable storage media.
  • An embodiment of the present disclosure also proposes an electronic device, including: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to invoke the instructions stored in the memory to execute the above method.
  • An embodiment of the present disclosure also provides a computer program product, including computer-readable codes, or a non-volatile computer-readable storage medium carrying computer-readable codes, when the computer-readable codes are stored in a processor of an electronic device When running in the electronic device, the processor in the electronic device executes the above method.
  • Electronic devices may be provided as terminals, servers, or other forms of devices.
  • FIG. 6 is a physical block diagram of hardware of an electronic device provided by an embodiment of the present disclosure.
  • the electronic device 600 may be a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle Devices, wearable devices and other terminal equipment.
  • UE User Equipment
  • PDA personal digital assistant
  • electronic device 600 may include one or more of the following components: processing component 602, memory 604, power supply component 606, multimedia component 608, audio component 610, input/output (I/O) interface 612, sensor component 614, and communication component 616 .
  • the processing component 602 generally controls the overall operations of the electronic device 600, such as those associated with display, telephone calls, data communications, camera operations, and recording operations.
  • the processing component 602 may include one or more processors 618 to execute computer instructions to complete all or part of the steps of the above method.
  • processing component 602 may include one or more modules that facilitate interaction between processing component 602 and other components.
  • processing component 602 may include a multimedia module to facilitate interaction between multimedia component 608 and processing component 602 .
  • the memory 604 is configured to store various types of data to support operations at the electronic device 600 . Some examples of such data include instructions for any application or method operating on the electronic device 600, contact data, phonebook data, messages, pictures, videos, and the like.
  • the memory 604 can be realized by any type of volatile or non-volatile storage device or their combination, such as Static Random Access Memory (Static Random Access Memory, SRAM), Electrically Erasable Programmable Read-Only Memory (Electrically Erasable Programmable read only memory, EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, EPROM), programmable read-only memory (Programmable read-only memory, PROM), read-only memory (Read-only memory , ROM), magnetic memory, flash memory, magnetic disk or optical disk.
  • Static Random Access Memory Static Random Access Memory
  • SRAM Static Random Access Memory
  • EEPROM Electrically Erasable Programmable Read-Only Memory
  • EPROM erasable programm
  • the power supply component 606 provides power to various components of the electronic device 600 .
  • Power components 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for electronic device 600 .
  • the multimedia component 608 includes a screen providing an output interface between the electronic device 600 and the user.
  • the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user.
  • the touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may not only sense a boundary of a touch or swipe action, but also detect duration and pressure associated with the touch or swipe action.
  • the multimedia component 608 includes a front camera and/or a rear camera. When the electronic device 600 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capability.
  • the audio component 610 is configured to output and/or input audio signals.
  • the audio component 610 includes a microphone (MIC), which is configured to receive external audio signals when the electronic device 600 is in operation modes, such as call mode, recording mode and voice recognition mode. Received audio signals may be further stored in memory 604 or sent via communication component 616 .
  • the audio component 610 also includes a speaker for outputting audio signals.
  • the I/O interface 612 provides an interface between the processing component 602 and a peripheral interface module.
  • the peripheral interface module may be a keyboard, a click wheel, a button, and the like. These buttons may include, but are not limited to: a home button, volume buttons, start button, and lock button.
  • Sensor assembly 614 includes one or more sensors for providing various aspects of status assessment for electronic device 600 .
  • the sensor assembly 614 can detect the open/close state of the electronic device 600, the relative positioning of the components, such as the display and the keypad of the electronic device 600, the sensor assembly 614 can also detect the electronic device 600 or one of the electronic device 600 The position of components changes, the presence or absence of user contact with the electronic device 600 , the orientation or acceleration/deceleration of the electronic device 600 and the temperature of the electronic device 600 change.
  • the sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact.
  • Sensor assembly 614 may also include an optical sensor, such as a complementary metal-oxide-semiconductor (CMOS) or charge-coupled device (CCD) image sensor, for use in imaging applications.
  • CMOS complementary metal-oxide-semiconductor
  • CCD charge-coupled device
  • the sensor component 614 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.
  • the communication component 616 is configured to facilitate wired or wireless communication between the electronic device 600 and other devices.
  • the electronic device 600 can access wireless networks based on communication standards, such as wireless networks (Wi-Fi), second-generation mobile communication technologies (2G), third-generation mobile communication technologies (3G), fourth-generation mobile communication technologies (4G ), the long-term evolution (LTE) of the universal mobile communication technology, the fifth generation mobile communication technology (5G) or their combination.
  • the communication component 616 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel.
  • the communication component 616 also includes a near field communication (Near Field Communication, NFC) module to facilitate short-range communication.
  • NFC Near Field Communication
  • the NFC module can be based on Radio Frequency Identification (RFID) technology, Infrared Data Association (Infrared Data Association, IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (bluetooth, BT) technology and other technology to achieve.
  • RFID Radio Frequency Identification
  • IrDA Infrared Data Association
  • UWB Ultra Wide Band
  • Bluetooth bluetooth, BT
  • the electronic device 600 may be implemented by one or more application-specific integrated circuits (Application Specific Integrated Circuit, ASIC), digital signal processors (Digital Signal Processing, DSP), digital signal processing equipment (Digital Signal Processing Device, DSPD), Programmable Logic Device (Programmable Logic Device, PLD), Field Programmable Gate Array (Field Programmable Gate Array, FPGA), controller, microcontroller, microprocessor or other electronic components are implemented for performing the above method .
  • ASIC Application Specific Integrated Circuit
  • DSP Digital Signal Processing
  • DSPD digital signal processing equipment
  • PLD Programmable Logic Device
  • FPGA Field Programmable Gate Array
  • controller microcontroller, microprocessor or other electronic components are implemented for performing the above method .
  • non-volatile computer-readable storage medium such as the memory 604 including computer program instructions, which can be executed by the processor 618 of the electronic device 600 to implement the above method.
  • This disclosure relates to the field of augmented reality.
  • acquiring the image information of the target object in the real environment and then using various visual correlation algorithms to detect or identify the relevant features, states and attributes of the target object, and thus obtain the image information that matches the specific application.
  • AR effect combining virtual and reality.
  • the target object may involve faces, limbs, gestures, actions, etc. related to the human body, or markers and markers related to objects, or sand tables, display areas or display items related to venues or places, etc.
  • Vision-related algorithms can involve visual positioning, SLAM, 3D reconstruction, image registration, background segmentation, object key point extraction and tracking, object pose or depth detection, etc.
  • Specific applications can not only involve interactive scenes such as guided tours, navigation, explanations, reconstructions, virtual effect overlays and display related to real scenes or objects, but also special effects processing related to people, such as makeup beautification, body beautification, special effect display, virtual Interactive scenarios such as model display.
  • the relevant features, states and attributes of the target object can be detected or identified through the convolutional neural network.
  • the above-mentioned convolutional neural network is a network model obtained by performing model training based on a deep learning framework.
  • FIG. 7 is a physical block diagram of hardware of an electronic device provided by an embodiment of the present disclosure.
  • an electronic device 700 may be provided as a server.
  • electronic device 700 includes processing component 702 , which further includes one or more processors, and a memory resource represented by memory 704 for storing instructions executable by processing component 702 , such as application programs.
  • the application program stored in memory 704 may include one or more modules each corresponding to a set of instructions.
  • the processing component 702 is configured to execute instructions to perform the above method.
  • Electronic device 700 may also include a power supply component 706 configured to perform power management of electronic device 700, a wired or wireless network interface 708 configured to connect electronic device 700 to a network, and an input-output (I/O) interface 710 .
  • the electronic device 700 can operate based on the operating system stored in the memory 704, such as the Microsoft server operating system (Windows ServerTM), the graphical user interface-based operating system (Mac OS XTM) introduced by Apple Inc., the multi-user and multi-process computer operating system (UnixTM) ), a free and open source Unix-like operating system (LinuxTM), an open source Unix-like operating system (FreeBSDTM), or similar.
  • Microsoft server operating system Windows ServerTM
  • Mac OS XTM graphical user interface-based operating system
  • UnixTM multi-user and multi-process computer operating system
  • LinuxTM free and open source Unix-like operating system
  • FreeBSDTM open source Unix-like operating system
  • non-volatile computer-readable storage medium such as the memory 704 including computer program instructions, which can be executed by the processing component 702 of the electronic device 700 to complete the above method.
  • the present disclosure can be a system, method and/or computer program product.
  • a computer program product may include a computer readable storage medium having computer readable program instructions thereon for causing a processor to implement various aspects of the present disclosure.
  • a computer readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device.
  • a computer readable storage medium may be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
  • Computer-readable storage media include: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or flash memory), static random access memory (SRAM), compact disc read only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanically encoded device, such as a printer with instructions stored thereon A hole card or a raised structure in a groove, and any suitable combination of the above.
  • RAM random access memory
  • ROM read-only memory
  • EPROM erasable programmable read-only memory
  • flash memory static random access memory
  • SRAM static random access memory
  • CD-ROM compact disc read only memory
  • DVD digital versatile disc
  • memory stick floppy disk
  • mechanically encoded device such as a printer with instructions stored thereon
  • a hole card or a raised structure in a groove and any suitable combination of the above.
  • computer-readable storage media are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., pulses of light through fiber optic cables), or transmitted electrical signals.
  • Computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or downloaded to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, and/or a wireless network.
  • the network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers.
  • a network adapter card or a network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .
  • Computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or Source or object code written in any combination, including object-oriented programming languages—such as Smalltalk, C++, etc., and conventional procedural programming languages—such as the “C” language or similar programming languages.
  • Computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement.
  • the remote computer can be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as via the Internet using an Internet service provider). connect).
  • LAN local area network
  • WAN wide area network
  • an electronic circuit such as a programmable logic circuit, field programmable gate array (FPGA), or programmable logic array (PLA)
  • FPGA field programmable gate array
  • PDA programmable logic array
  • These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine such that when executed by the processor of the computer or other programmable data processing apparatus , producing an apparatus for realizing the functions/actions specified in one or more blocks in the flowchart and/or block diagram.
  • These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause computers, programmable data processing devices and/or other devices to work in a specific way, so that the computer-readable medium storing instructions includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks in flowcharts and/or block diagrams.
  • each block in a flowchart or block diagram may represent a module, a portion of a program segment, or an instruction that includes one or more Executable instructions.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations can be implemented by a dedicated hardware-based system that performs the specified function or action , or may be implemented by a combination of dedicated hardware and computer instructions.
  • the computer program product can be specifically realized by means of hardware, software or a combination thereof.
  • the computer program product is embodied as a computer storage medium, and in other embodiments, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) and the like.
  • the present disclosure relates to an occlusion detection method and device, electronic equipment, a storage medium, and a computer program product.
  • the method includes: according to the image feature statistical graph of multiple image blocks to be detected in the current video frame, and multiple image blocks in the reference video frame Referring to the image feature statistical map of the image block, determining whether there is a target image block to be detected that has an abnormal change in the image feature statistical map relative to the reference video frame in the current video frame; In the case of the target image block to be detected, edge detection is performed on the target image block to be detected to obtain an edge detection result of the target image block to be detected; according to the edge detection result, it is determined to acquire an image of the current video frame Whether the acquisition device is blocked.
  • the above technical solution can improve the accuracy of the occlusion detection of the image acquisition device, and at the same time reduce the computing power requirement of the occlusion detection.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)
  • Studio Devices (AREA)

Abstract

本公开涉及一种遮挡检测方法及装置、电子设备、存储介质和计算机程序产品，所述方法包括：根据当前视频帧中多个待检测图像块的图像特征统计图，与参考视频帧中多个参考图像块的图像特征统计图，确定所述当前视频帧中是否存在相对于所述参考视频帧出现图像特征统计图变化异常的目标待检测图像块；在确定所述当前视频帧中存在所述目标待检测图像块的情况下，对所述目标待检测图像块进行边缘检测，得到所述目标待检测图像块的边缘检测结果；根据所述边缘检测结果，确定采集所述当前视频帧的图像采集设备是否存在遮挡。

Description

遮挡检测方法及装置、电子设备、存储介质及计算机程序产品
相关申请的交叉引用
本公开实施例基于申请号为202111512423.8、申请日为2021年12月08日、申请名称为“遮挡检测方法及装置、电子设备和存储介质”的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本公开作为参考。
技术领域
本公开涉及但不限于计算机视觉技术领域，尤其涉及一种遮挡检测方法及装置、电子设备、存储介质及计算机程序产品。
背景技术
智能监控系统目前已广泛应用于国民生活的各个领域。在安装智能监控系统中的图像采集设备(例如，摄像头)时，即对智能监控系统的监控区域做了初始设定和调整，如果在使用过程中图像采集设备被遮挡，将导致智能监控系统无法达到预期的监控效果。因此，对图像采集设备进行遮挡检测对于智能监控系统的稳定运行来说至关重要。在相关技术中，对图像采集设备进行遮挡检测计算耗时且准确度低。
发明内容
本公开提出了一种遮挡检测方法及装置、电子设备、存储介质及计算机程序产品的技术方案。
本公开实施例提供了一种遮挡检测方法，包括：根据当前视频帧中多个待检测图像块的图像特征统计图，与参考视频帧中多个参考图像块的图像特征统计图，确定所述当前视频帧中是否存在相对于所述参考视频帧出现图像特征统计图变化异常的目标待检测图像块；在确定所述当前视频帧中存在所述目标待检测图像块的情况下，对所述目标待检测图像块进行边缘检测，得到所述目标待检测图像块的边缘检测结果；根据所述边缘检测结果，确定采集所述当前视频帧的图像采集设备是否存在遮挡。
本公开实施例还提供了一种遮挡检测装置，包括：第一确定部分，被配置为根据当前视频帧中多个待检测图像块的图像特征统计图，与参考视频帧中多个参考图像块的图像特征统计图，确定所述当前视频帧中是否存在相对于所述参考视频帧出现图像特征统计图变化异常的目标待检测图像块；边缘检测部分，被配置为在确定所述当前视频帧中存在所述目标待检测图像块的情况下，对所述目标待检测图像块进行边缘检测，得到所述目标待检测图像块的边缘检测结果；第二确定部分，被配置为根据所述边缘检测结果，确定采集所述当前视频帧的图像采集设备是否存在遮挡。
本公开实施例还提供了一种电子设备，包括：处理器；被配置为存储处理器可执行指令的存储器；其中，所述处理器被配置为调用所述存储器存储的指令，以执行上述方法。
本公开实施例还提供了一种计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令被处理器执行时实现上述方法。
本公开实施例还提供了一种计算机程序产品，所述计算机程序产品包括计算机程序或指令，在所述计算机程序或指令在电子设备上运行的情况下，使得所述电子设备执行上述遮挡检测方法中的步骤。
在本公开实施例中，首先，根据当前视频帧中多个待检测图像块的图像特征统计图，与参考视频帧 中多个参考图像块的图像特征统计图，对当前视频帧进行初步遮挡检测，确定当前视频帧中是否存在相对于参考视频帧出现图像特征统计图变化异常的目标待检测图像块；然后，仅对出现图像特征统计图变化异常的目标待检测图像块进行边缘检测，并根据边缘检测结果进一步确定采集当前视频帧的图像采集设备是否存在遮挡情况。这样，本公开实施例通过初步遮挡检测可以确定当前视频帧中存在图像特征统计图变化异常的目标待检测图像块，即，初步确定采集当前视频帧的图像采集设备可能存在遮挡，然后在边缘检测步骤中进一步根据边缘检测结果确定采集当前视频帧的图像采集设备存在遮挡，因此，可以提高对图像采集设备遮挡检测的准确度；同时，由于边缘检测仅针对出现图像特征统计图变化异常的目标待检测图像块，因此，在提高遮挡检测准确度的同时还可以有效降低遮挡检测的算力需求。
应当理解的是，以上的一般描述和后文的细节描述是示例性和解释性的，而非限制本公开。根据下面参考附图对示例性实施例的详细说明，本公开的其它特征及方面将变得清楚。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，这些附图示出了符合本公开的实施例，并与说明书一起用于说明本公开的技术方案。
图1为本公开实施例提供的一种遮挡检测方法的流程示意程图；
图2为本公开实施例提供的视频帧中多个图像块的示意图；
图3A为本公开实施例提供的待检测图像块的示意图；
图3B为本公开实施例提供的待检测图像块的图像直方图的示意图；
图4为本公开实施例提供的一种遮挡检测方法的流程示意图；
图5为本公开实施例提供的一种遮挡检测装置的组成结构示意图；
图6为本公开实施例提供的一种电子设备的一种硬件实体框图；
图7为本公开实施例提供的一种电子设备的一种硬件实体框图。
具体实施方式
以下将参考附图详细说明本公开的各种实施例、特征和方面。附图中相同的附图标记表示功能相同或相似的元件。尽管在附图中示出了实施例的各种方面，但是除非特别指出，不必按比例绘制附图。
在这里专用的词“示例性”意为“用作例子、实施例或说明性”。这里作为“示例性”所说明的任何实施例不必解释为优于或好于其它实施例。
本文中术语“和/或”，是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中术语“至少一种”表示多种中的任意一种或多种中的至少两种的任意组合，例如，包括A、B、C中的至少一种，可以表示包括从A、B和C构成的集合中选择的任意一个或多个元素。
另外，为了更好地说明本公开，在下文的具体实施方式中给出了众多的具体细节。本领域技术人员应当理解，没有某些具体细节，本公开同样可以实施。在一些实例中，对于本领域技术人员熟知的方法、手段、元件和电路未作详细描述，以便于凸显本公开的主旨。
相关技术中，对图像采集设备进行遮挡检测主要包括以下四种方法：
第一种，利用背景建模技术从图像采集设备采集的视频中提取可疑前景，以所提取的可疑前景为依据判断图像采集设备是否存在遮挡。但是，该检测方法受环境干扰较大，例如，前景变化较大的人群流动场景中，检测准确性较低；
第二种，基于运动向量判断遮挡，即，将视频帧划分为预定数量的图像块，估计每一图像块的一个或多个候选运动向量，从至少一个候选运动向量中确定图像块的运动向量，然后根据图像块的运动向量来确定图像块是遮挡区域图像块还是显露区域图像块来检测图像块的遮挡类型。但是，该方法需要不断检测新的视频帧以实时确定图像采集设备是否被遮挡，因此计算耗时且受光照影响较大；
第三种，为视频帧建立图像特征直方图，然后通过图像特征直方图的变化来检测是否存在图像采集设备遮挡问题。但是，该检测方法同样受环境影响较大；
第四种，采用深度学习方法训练得到遮挡分类模型，利用该遮挡分类模型进行遮挡检测。但是，该检测方法的模型训练过程需要依赖于训练样本的质量和数量，并且运算量较大。
基于此，本公开实施例提供了一种遮挡检测方法，该遮挡检测方法可以由终端设备或服务器等电子设备执行。终端设备可以为用户设备(User Equipment，UE)、移动设备、用户终端、蜂窝电话、无绳电话、个人数字助理(Personal Digital Assistant，PDA)、手持设备、计算设备、车载设备、可穿戴设备等。该遮挡检测方法可以通过处理器调用存储器中存储的计算机可读指令的方式来实现，或者，可通过服务器执行该遮挡检测方法。
下面，将结合本公开实施例中的附图，对本公开实施例中的技术方案进行清楚、完整地描述。
图1为本公开实施例提供的一种遮挡检测方法的流程示意图，如图1所示，该遮挡检测方法包括步骤S11至步骤S13，其中：
步骤S11，根据当前视频帧中多个待检测图像块的图像特征统计图，与参考视频帧中多个参考图像块的图像特征统计图，确定当前视频帧中是否存在相对于参考视频帧出现图像特征统计图变化异常的目标待检测图像块。
这里，获取需要进行遮挡检测的图像采集设备采集的视频流，对视频流中的各视频帧按照时序进行遮挡检测。图像采集设备可以是智能监控系统中的摄像头、摄像机、具有图像采集功能的终端等，本公开对此不作具体限定。
当前视频帧和参考视频帧是图像采集设备采集的视频流中的两个视频帧，其中，当前视频帧是图像采集设备当前时刻采集得到的视频帧，参考视频帧的采集时刻在当前视频帧之前。
为了有效检测图像采集设备，例如相机，摄像头等是否存在局部遮挡，可以基于相同的图像分块规则对用于遮挡检测的各视频帧进行图像分块，从而可以根据图像块确定视频帧的局部图像特征(例如，局部区域的图像特征统计图)，进而利用视频帧的局部图像特征有效检测图像采集设备是否存在局部遮挡。
后文会结合本公开的多个实现方式，利用相同的图像分块规则，对用于遮挡检测的各视频帧进行图像分块的具体过程作详细描述。
根据当前视频帧中多个待检测图像块的图像特征统计图，与参考视频帧中多个参考图像块的图像特征统计图，可以确定当前视频帧中是否存在相对于参考视频帧出现图像特征统计图变化异常的目标待检 测图像块，从而可以对图像采集设备进行初步的遮挡检测，以确定相对于参考视频帧的采样时刻，图像采集设备在当前时刻是否出现遮挡。
后文会结合本公开可能的实现方式，对确定当前视频帧中多个待检测图像块的图像特征统计图、确定参考视频帧中多个参考图像块的图像特征统计图，以及确定当前视频帧中是否存在相对于参考视频帧出现图像特征统计图变化异常的目标待检测图像块的具体过程作详细描述。
步骤S12，在确定当前视频帧中存在目标待检测图像块的情况下，对目标待检测图像块进行边缘检测，得到目标待检测图像块的边缘检测结果。
这里，在确定当前视频帧中存在相对于参考视频帧出现图像特征统计图变化异常的目标待检测图像块的情况下，可以初步确定图像采集设备中存在遮挡，为了进一步提高检测的准确度，降低出现误判的概率，对出现图像特征统计图变化异常的目标待检测图像块进行边缘检测。后文会结合本公开的实现方式，对边缘检测的具体过程作详细描述。
步骤S13，根据边缘检测结果，确定采集当前视频帧的图像采集设备是否存在遮挡。
这里，根据对出现图像特征统计图变化异常的目标待检测图像块进行边缘检测的边缘检测结果，可以进一步确定图像采集设备是否存在遮挡，提高检测的准确度。后文会结合本公开的实现方式，对如何根据边缘检测结果，确定采集当前视频帧的图像采集设备是否存在遮挡的具体过程作详细描述。
在本公开实施例中，首先，根据当前视频帧中多个待检测图像块的图像特征统计图，与参考视频帧中多个参考图像块的图像特征统计图，对当前视频帧进行初步遮挡检测，确定当前视频帧中是否存在相对于参考视频帧出现图像特征统计图变化异常的目标待检测图像块；然后，仅对出现图像特征统计图变化异常的目标待检测图像块进行边缘检测，并根据边缘检测结果进一步确定采集当前视频帧的图像采集设备是否存在遮挡情况。这样，本公开实施例通过初步遮挡检测可以确定当前视频帧中存在图像特征统计图变化异常的目标待检测图像块，即，初步确定采集当前视频帧的图像采集设备可能存在遮挡，然后在边缘检测步骤中进一步根据边缘检测结果确定采集当前视频帧的图像采集设备存在遮挡，因此，可以提高对图像采集设备遮挡检测的准确度；同时，由于边缘检测仅针对出现图像特征统计图变化异常的目标待检测图像块，因此，在提高遮挡检测准确度的同时还可以有效降低遮挡检测的算力需求。
在一些实现方式中，对图像采集设备进行遮挡检测的过程中，利用相同的图像分块规则对图像采集设备采集的视频流中的各视频帧进行图像分块，得到各视频帧对应的多个图像块。这样，由于采用相同的图像分块规则对各视频帧进行图像分块，使得不同视频帧中的相同位置处的图像块具有可比性，因此，可以通过对不同视频帧中相同位置处的图像块进行对比来实现对不同视频帧中局部图像特征的比对，从而有效地对图像采集设备是否出现局部遮挡进行检测。
在一些实现方式中，图像分块规则可以是将各视频帧平均分成预设数目的图像块。例如，图像分块规则是将各视频帧平均分成9个图像块，此时，将视频帧的长和宽分别除以3得到9个分块。假设视频帧的大小为10×10，长和宽不能被3整除，即存在余数1，则舍弃掉余数的1行和1列，得到均分后的9个图像块，每个图像块大小为3×3。
图像分块规则可以是上述将视频帧平均分成预设数目的图像块，还可以根据实际需求，设置其它的图像分块规则，本公开对此不作具体限定。
利用图像分块规则对视频帧进行图像分块得到的图像块的块数，也可以根据实际需求进行设置，本公开对此不作具体限定。
图2为本公开实施例提供的视频帧中多个图像块的示意图。如图2所示，针对图像采集设备采集的当前视频帧2，按照均分的标准将当前视频帧2分成9个图像块，即，将当前视频帧2的长和宽分别除以3，得到大小相同的9个图像块，即图像块21至图像块29。
在一些实现方式中，获取图像采集设备采集的视频流，启动对图像采集设备的遮挡检测流程。在遮挡检测的初始阶段，将图像采集设备采集的第1个视频帧确定为参考视频帧。按照预设的图像分块规则，对该参考视频帧进行图像分块，得到参考视频帧中的多个参考图像块。
确定参考视频帧中每个参考图像块的图像特征统计图，以及对每个参考图像块进行边缘检测，得到每个参考图像块的边缘检测结果。将参考视频帧中每个参考图像块的图像特征统计图以及边缘检测结果，用于作为对后续视频帧进行遮挡检测的参考。
在一些实现方式中，图像特征统计图可以是图像直方图。其中，图像直方图可以是对图像颜色特征的统计图，例如，图像直方图可以是基于红绿蓝(Red Green Blue，RGB)颜色空间的图像颜色直方图。又例如，图像直方图可以是基于色调饱和度明度(Hue Saturation Value，HSV)颜色空间的图像颜色直方图。还例如，图像直方图还可以是对图像灰度特征的统计图，比如，图像直方图可以是基于灰度空间的图像灰度直方图。在实施时，本领域技术人员可以根据实际需求自主确定图像直方图，本公开对此不作具体限定。
由于图像直方图可以直观地反映图像特征的频数分布，因此，利用图像直方图可以快速对当前视频帧进行初步遮挡检测。
在一些实现方式中，图像特征统计图可以是图像直方图，还可以根据实际情况选择其它形式的统计图，本公开对此不作具体限定。
本公开实施例的边缘检测可以是基于边缘检测算子实现的，其中，边缘检测算子可以是索贝尔sobel算子、拉普拉斯Laplacian算子等，本公开对此不作具体限定。
下面以图像特征统计图为图像直方图、边缘检测是基于边缘检测算子为例，对遮挡检测过程作详细描述。此时，图像特征统计图变化异常，也即图像直方图变化异常。
将参考视频帧中的每个参考图像块转换为灰度图，利用边缘检测算子对每个参考图像块的灰度图进行边缘检测，得到每个参考图像块的图像块边缘值。
针对任意一个参考图像块，首先计算该参考图像块中的像素点(x,y)的灰度值f(x,y)，然后根据每个像素点的灰度值f(x,y)以及该参考图像块的灰度图对应的平均灰度值μ，基于下述公式(1-1)，确定该参考图像块的边缘值方差D(f)：
D(f)＝∑ y∑ x|f(x,y)-μ| 2        (1-1)；
将每个参考图像块的边缘值方差确定为对应参考图像块的边缘检测结果。
基于参考视频帧，对视频流中参考视频帧的后续视频帧在时序上依次进行遮挡检测。
针对视频流中的图像采集时刻在参考视频帧之后的当前视频帧，按照与参考视频帧相同的预设图像 分块规则对当前视频帧进行图像分块，得到当前视频帧中的多个待检测图像块。其中，当前视频帧中的多个待检测图像块与参考视频帧中的多个参考图像块，在图像块位置分布上一一对应。
将当前视频帧中的每个待检测图像块转换为灰度图，利用当前视频帧中的每个待检测图像块的灰度图与参考视频帧中对应的参考图像块的灰度图，对当前视频帧进行初步遮挡检测。
下面结合图3A和图3B说明将当前视频帧中的每个待检测图像块转换为灰度图的过程。
图3A为本公开实施例提供的待检测图像块的示意图，图3B为本公开实施例提供的待检测图像块的图像直方图的示意图。首先，按照预定的图像分块规则对当前视频帧进行分块，得到至少一个待检测图像块，如图3A中的检测图像块A11；然后，按照红、绿、蓝三个颜色通道分别提取待检测图像块的颜色特征，确定该待检测图像块基于RGB颜色空间的图像颜色直方图，得到三个颜色通道(即R颜色通道、G颜色通道和B颜色通道)的直方图，如图3B中的B颜色通道的直方图B11、G颜色通道的直方图B12和R颜色通道的直方图B13。
将当前视频帧中每个待检测图像块的图像直方图，与参考视频帧中对应的参考图像块的图像直方图进行对比，确定当前视频帧中是否存在相对于参考视频帧出现图像直方图变化异常的目标待检测图像块。
在一些实现方式中，所述步骤S11包括步骤S111至步骤S112，其中：
步骤S111、确定当前视频帧中每个待检测图像块的图像特征统计图，与参考视频帧中对应的参考图像块的图像特征统计图之间的相似度；
步骤S112、在存在相似度低于相似度阈值的待检测图像块的情况下，将相似度低于相似度阈值的待检测图像块确定为目标待检测图像块。
这里，针对当前视频帧中的任意一个待检测图像块，确定该待检测图像块的图像直方图，与参考视频帧中对应的参考图像块的图像直方图之间的直方图相似度，进而基于直方图相似度，可以有效确定是否出现图像直方图变化异常。
在一些实现方式中，可以基于相关性比较(Correlation)算法确定两个图像直方图之间的直方图相似度。
例如，当前视频帧中一个待检测图像块的图像直方图是H 1，参考视频帧中对应的参考图像块的图像直方图是H 2，则可以利用下述相关性比较算法公式(1-2)，确定待检测图像块的图像直方图H 1，与参考视频帧中对应的参考图像块的图像直方图H 2之间的直方图相似度d(H 1,H 2)：
Figure PCTCN2022095516-appb-000001
其中，
Figure PCTCN2022095516-appb-000002
k是1或2，H k(I)是图像块(待检测图像块或参考图像块)中颜色特征I的统计值，颜色特征I的取值范围是1至N。
在一些实现方式中，还可以基于卡方比较(Chi-Square)算法、十字交叉性(Intersection)算法、巴氏距离(Bhattacharyya distance)算法等，确定两个图像直方图之间的直方图相似度，本公开对此不作具体限定。
基于当前视频帧中每个待检测图像块的图像直方图，与参考视频帧中对应的参考图像块的图像直方图的直方图相似度，确定当前视频帧中是否存在直方图相似度低于相似度阈值的待检测图像块。相似度阈值的具体取值可以根据实际情况设置，例如，相似度阈值是0.7，本公开对此不作具体限定。
在一些实现方式中，该遮挡检测方法还包括步骤S14至步骤S15，其中：
步骤S14、在确定当前视频帧中不存在目标待检测图像块的情况下，确定当前视频帧之前第二采集时长内的各视频帧内，是否均不存在相对于参考视频帧出现图像特征统计图变化异常的目标待检测图像块；
步骤S15、在确定当前视频帧之前第二采集时长内的各视频帧内，均不存在相对于参考视频帧出现图像特征统计图变化异常的目标待检测图像块的情况下，将当前视频帧确定为更新后的参考视频帧。
在当前视频帧中不存在直方图相似度低于相似度阈值的待检测图像块的情况下，可以说明当前视频帧中的每个待检测图像块相对于参考视频帧中对应的参考图像块，均未出现图像直方图变化异常。也就是说，利用图像直方图对当前视频帧进行初步遮挡检测的检测结果是，图像采集设备不存在遮挡。
进而，确定当前视频帧之前第二采集时长内的各视频帧内，是否均不存在相对于参考视频帧出现直方图变化异常的目标待检测图像块，也即利用图像直方图对当前时刻之前第二采集时长内的各视频帧进行初步遮挡检测的结果是，图像采集设备在当前时刻之前第二采集时长内均不存在遮挡。第二采集时长的具体取值可以根据实际情况设置，本公开对此不作具体限定。
此时，可以将当前视频帧更新为参考视频帧，用于作为对后续视频帧进行遮挡检测的参考。通过定期更新参考视频帧，可以降低由于光照变化等因素对参考视频帧的影响，以提高参考视频帧的可参考性，进而有效提高遮挡检测的准确度。
在一些实现方式中，在当前视频帧中存在直方图相似度低于相似度阈值的待检测图像块的情况下，说明当前视频帧中的该待检测图像块相对于参考视频帧中对应的参考图像块，出现了图像直方图变化异常，则将该待检测图像块确定为目标待检测图像块。
当前视频帧中存在相对于参考视频帧出现图像直方图变化异常的目标待检测图像块，说明利用图像直方图对当前视频帧进行初步遮挡检测的检测结果是，图像采集设备中目标待检测图像块对应的采集区域可能存在遮挡。
但是，由于图像直方图可以反映图像特征的频数统计分布，无法反映图像特征的空间位置分布，因此，基于图像直方图进行初步遮挡检测的检测结果可能存在由于前景变化或短时间光照变化导致的图像直方图变化异常，从而造成遮挡误检测。因此，在基于图像直方图进行初步遮挡检测，确定当前视频帧中存在相对于参考视频帧出现图像直方图变化异常的目标待检测图像块的情况下，可以利用边缘检测对目标待检测图像块进行遮挡检测，以提高检测结果的准确度。
相比于直接利用边缘检测对当前视频帧进行遮挡检测的方式，本公开实施例基于图像直方图进行初步遮挡检测，提取当前视频帧中可能存在遮挡的目标待检测图像块，进而仅利用边缘检测对目标待检测图像块进行遮挡检测，可以在提高遮挡检测的准确性的同时，有效降低遮挡检测的算力需求，提高遮挡检测效率。
在一些实现方式中，对目标待检测图像块进行边缘检测，得到目标待检测图像块的边缘检测结果，包括步骤S121至步骤S124，其中：
步骤S121、对目标待检测图像块进行图像灰度转换，得到目标待检测图像块的灰度图；
步骤S122、对灰度图进行边缘检测，确定目标待检测图像块的图像块边缘值；
步骤S123、根据图像块边缘值和灰度图对应的平均灰度值，确定目标待检测图像块的边缘值方差；
步骤S124、将目标待检测图像块的边缘值方差，确定为边缘检测结果。
其中，对目标待检测图像块的灰度图进行边缘检测时，采用与对参考图像帧中对应的参考图像块进行边缘检测时相同的边缘检测算子。
在一些实现方式中，在确定目标待检测图像块的图像块边缘值f'(x,y)，以及目标待检测图像块的灰度图对应的平均灰度值μ'之后，可以基于上述公式(1-2)，确定目标待检测图像块的边缘值方差D'(f)。目标待检测图像块的边缘值方差D'(f)，是目标待检测图像块的边缘检测结果。
由于目标待检测图像块的边缘值方差可以反映目标待检测图像块的图像模糊度，因此，可以将目标待检测图像块的边缘值方差确定为目标待检测图像块的边缘检测结果，用于后续确定是否由于图像采集设备存在遮挡而导致目标待检测图像块出现图像模糊。
在一些实现方式中，所述步骤S13包括步骤S131至步骤S132，其中：
步骤S131、根据目标待检测图像块的边缘值方差，以及参考视频帧中对应的参考图像块的边缘值方差，确定目标待检测图像块是否相对于参考视频帧中对应的参考图像块出现边缘值方差剧减；
步骤S132、在确定目标待检测图像块出现边缘值方差剧减的情况下，确定图像采集设备中目标待检测图像块对应的采集区域存在遮挡。
这里，在目标待检测图像块相对于参考视频帧中对应的参考图像块出现边缘值剧减的情况下，可以说明相对于参考帧中对应的参考图像块，当前视频帧中的目标待检测图像块出现了图像模糊，此时，利用边缘检测对目标待检测图像块进行遮挡检测的结果是图像采集设备中目标待检测图像块对应的采集区域存在局部遮挡。
在目标待检测图像块相对于参考视频帧中对应的参考图像块并未出现边缘值剧减的情况下，可以说明相对于参考帧中对应的参考图像块，当前视频帧中的目标待检测图像块并未出现图像模糊，此时，利用边缘检测进一步进行遮挡检测的结果是图像采集设备中目标待检测图像块对应的采集区域不存在局部遮挡。
针对当前视频帧，在当前视频帧中不存在出现边缘值剧减的目标待检测图像块的情况下，可以说明利用边缘检测进一步进行遮挡检测的结果是图像采集设备整体上均不存在遮挡。此时，继续对当前时刻之后的下一采集时刻的视频帧进行遮挡检测。
在一些实现方式中，所述步骤S131包括步骤S141至步骤S142，其中：
步骤S141、确定目标待检测图像块的边缘值方差，与参考视频帧中对应的参考图像块的边缘值方差之间的边缘值方差比例；
步骤S142、在边缘值方差比例低于方差比例阈值的情况下，确定目标待检测图像块出现边缘值方差剧减。
这里，利用目标待检测图像块的边缘值方差与参考视频帧中对应的参考图像块的边缘值方差之间的 边缘值方差比例，以及方差比例阈值，可以快速确定目标待检测图像块是否出现边缘值方差剧减。方差比例阈值的具体取值可以根据实际情况确定，例如，方差比例阈值是0.2，本公开对此不作具体限定。
例如，目标待检测图像块的边缘值方差是D'(f)，参考视频帧中对应的参考图像块的边缘值方差是D(f)，在二者之间的边缘值方差比例
Figure PCTCN2022095516-appb-000003
低于方差比例阈值的情况下，确定目标待检测图像块相对于参考视频帧中对应的参考图像块出现边缘值方差剧减。
确定是否出现边缘值方差剧减的方式可以是上述边缘值方差比例是否低于方差比例阈值的方式，还可以是根据实际情况选择的其它方式，本公开对此不作具体限定。
在一些实现方式中，基于上述边缘方差剧减的判断方式，还可以检测图像采集设备是否出现镜头异常状态，即，在确定目标待检测图像块出现边缘值方差剧减，但是图像采集设备中目标待检测图像块对应的采集区域并不存在遮挡的情况下，可以考虑图像采集设备是否出现镜头失焦、花屏、黑屏等镜头异常状态。
在一些实现方式中，该遮挡检测方法还包括步骤S133至步骤S134，其中：
步骤S133、确定当前视频帧之后第一采集时长内的各视频帧内，是否均存在相对于参考视频帧出现图像特征统计图变化异常且边缘值方差剧减的目标待检测图像块；
步骤S134、在确定当前视频帧之后第一采集时长内的各视频帧内，均存在相对于参考视频帧出现图像特征统计图变化异常且边缘值方差剧减的目标待检测图像块的情况下，生成遮挡告警信息。
这里，由于单次检测可能存在误检，为了降低误检率，在基于当前视频帧进行遮挡检测并确定图像采集设备存在遮挡之后，对采集当前视频帧之后的第一采集时长内采集的各视频帧持续进行检测，确定基于当前视频帧之后第一采集时长内采集的各视频帧，是否持续检测到图像采集设备存在遮挡，若持续检测到图像采集设备存在遮挡，则生成遮挡告警信息，从而实现对图像采集设备存在遮挡的有效遮挡告警，精准触发后续的图像采集设备维护流程。
其中，第一采集时长的具体取值可以根据实际情况确定，例如，第一采集时长是10s，本公开对此不作具体限定。
在一些实现方式中，所述步骤S134包括步骤S151至步骤S152，其中：
步骤S151、根据出现图像特征统计图变化异常且边缘值方差剧减的目标待检测图像块，确定图像采集设备的遮挡区域占比；
步骤S152、在遮挡区域占比高于遮挡区域占比阈值的情况下，生成遮挡告警信息。
其中，遮挡区域占比可以是目标待检测图像块的数量与待检测图像块的数量之间的比值。例如，当前视频帧包括9个待检测图像块，其中包括2个出现图像直方图变化异常且边缘值方差剧减的目标待检测图像块，则可以确定图像采集设备的遮挡区域占比是2/9＝22.2％。
在遮挡区域占比高于遮挡区域占比阈值的情况下，可以反映当前遮挡已经影响图像采集设备的正常工作，此时，生成遮挡告警信息，从而实现对图像采集设备存在遮挡的有效遮挡告警，避免当前遮挡并未影响图像采集设备的正常工作而进行遮挡告警造成的维护资源浪费。
其中，遮挡区域占比阈值的具体取值可以根据实际情况确定，例如，遮挡区域占比阈值是30％，本公开对此不作具体限定。
在一些实现方式中，在生成告警信息之后，表明图像采集设备当前存在遮挡，因此，不再对参考视频帧进行定期更新，以避免将存在遮挡的图像采集设备采集的视频帧确定为参考视频帧，直至告警解除。
其中，告警解除表示已经对存在遮挡的图像采集设备进行维护，告警解除后的图像采集设备不存在遮挡。
在一些实现方式中，遮挡检测的初始阶段，将图像采集设备采集的第1个视频帧确定为参考视频帧之后，如果在第1个视频帧之后第三采集时长内的各视频帧中，每个待检测图像块的图像直方图、图像块边缘值、图像块边缘值方差等均持续不变，则可以确定在遮挡检测开始之前，图像采集设备已经存在遮挡。
图4为本公开实施例提供的一种遮挡检测系统的执行流程的示意图。如图4所示，该遮挡检测系统的执行流程包括：
步骤S401，获取图像采集设备采集的视频流；
步骤S402，确定参考视频帧，以及参考视频帧中每个参考图像块的图像直方图和边缘值方差；
这里，对参考视频帧进行图像分块，以及确定参考视频帧中每个参考图像块的图像直方图和边缘值方差的过程，可以参考上述实施例中相关部分的详细描述。
步骤S403，对视频流中的采集时刻在参考视频帧的采集时刻之后的当前视频帧实时进行遮挡检测；
步骤S404，确定当前视频帧中是否存在相对于参考视频帧出现图像直方图变化异常的目标待检测图像块，若是，则执行步骤S405，若否，则执行步骤S409；
这里，确定当前视频帧中是否存在相对于参考视频帧出现图像直方图变化异常的目标待检测图像块的过程，可以参考上述实施例中相关部分的详细描述。
步骤S405，提取目标待检测图像块的边缘值方差；
这里，提取目标待检测图像块的边缘值方差的过程，可以参考上述实施例中相关部分的详细描述。
步骤S406，根据目标待检测图像块的边缘值方差，以及参考视频帧中对应的参考图像块的边缘值方差，确定目标待检测图像块是否相对于参考视频帧中对应的参考图像块出现边缘值方差剧减，若是，则执行步骤S407，若否，则执行步骤S411；
这里，确定目标待检测图像块是否相对于参考视频帧中对应的参考图像块出现边缘值方差剧减的过程，可以参考上述实施例中相关部分的详细描述。
步骤S407，确定当前视频帧之后第一采集时长内的各视频帧内，是否持续存在相对于参考视频帧出现图像直方图变化异常且边缘值方差剧减的目标待检测图像块，若是，则执行步骤S408，若否，则执行步骤S411；
步骤S408，生成告警信息；
步骤S409，确定当前视频帧之前第二采集时长内的各视频帧内，是否持续不存在相对于参考视频帧出现图像直方图变化异常的目标待检测图像块，若是，则执行步骤S410，若否，则执行步骤S411；
步骤S410，将当前视频帧更新为参考视频帧，并执行上述步骤S403；
步骤S411，将下一采集时刻的视频帧确定为当前视频帧，并执行上述步骤S404。
可以理解，本公开提及的上述各个方法实施例，在不违背原理逻辑的情况下，均可以彼此相互结合形成结合后的实施例。本领域技术人员可以理解，在具体实施方式的上述方法中，各步骤的具体执行顺序应当以其功能和可能的内在逻辑确定。
此外，本公开还提供了遮挡检测装置、电子设备、计算机可读存储介质和计算机程序产品，上述均可用来实现本公开提供的任一种遮挡检测方法，相应技术方案和描述和参见方法部分的相应记载。
图5为本公开实施例提供的一种遮挡检测装置的组成结构示意图。如图5所示，遮挡检测装置50包括：
第一确定部分51，被配置为根据当前视频帧中多个待检测图像块的图像特征统计图，与参考视频帧中多个参考图像块的图像特征统计图，确定当前视频帧中是否存在相对于参考视频帧出现图像特征统计图变化异常的目标待检测图像块；
边缘检测部分52，被配置为在确定当前视频帧中存在目标待检测图像块的情况下，对目标待检测图像块进行边缘检测，得到目标待检测图像块的边缘检测结果；
第二确定部分53，被配置为根据边缘检测结果，确定采集当前视频帧的图像采集设备是否存在遮挡。
在一些实现方式中，第一确定部分51，还被配置为：确定当前视频帧中每个待检测图像块的图像特征统计图，与参考视频帧中对应的参考图像块的图像特征统计图之间的相似度；在存在相似度低于相似度阈值的待检测图像块的情况下，将相似度低于相似度阈值的待检测图像块，确定为目标待检测图像块。
在一些实现方式中，边缘检测部分52，还被配置为：对目标待检测图像块进行图像灰度转换，得到目标待检测图像块的灰度图；对灰度图进行边缘检测，确定目标待检测图像块的图像块边缘值；根据图像块边缘值和灰度图对应的平均灰度值，确定目标待检测图像块的边缘值方差；将目标待检测图像块的边缘值方差，确定为边缘检测结果。
在一些实现方式中，第二确定部分53，包括：第一确定子部分，被配置为根据目标待检测图像块的边缘值方差，以及参考视频帧中对应的参考图像块的边缘值方差，确定目标待检测图像块是否相对于参考视频帧中对应的参考图像块出现边缘值方差剧减；第二确定子部分，被配置为在确定目标待检测图像块出现边缘值方差剧减的情况下，确定图像采集设备中目标待检测图像块对应的采集区域存在遮挡。
在一些实现方式中，第一确定子部分，还被配置为：确定目标待检测图像块的边缘值方差，与参考视频帧中对应的参考图像块的边缘值方差之间的边缘值方差比例；在边缘值方差比例低于方差比例阈值的情况下，确定目标待检测图像块出现边缘值方差剧减。
在一些实现方式中，遮挡检测装置50，还包括：第三确定部分，被配置为确定当前视频帧之后第一采集时长内的各视频帧内，是否均存在相对于参考视频帧出现图像特征统计图变化异常且边缘值方差剧减的目标待检测图像块；生成部分，被配置为在确定当前视频帧之后第一采集时长内的各视频帧内，均存在相对于参考视频帧出现图像特征统计图变化异常且边缘值方差剧减的目标待检测图像块的情况下，生成遮挡告警信息。
在一些实现方式中，生成部分，包括：第三确定子部分，被配置为根据出现图像特征统计图变化异常且边缘值方差剧减的目标待检测图像块，确定图像采集设备的遮挡区域占比；生成子部分，被配置为 在遮挡区域占比高于遮挡区域占比阈值的情况下，生成遮挡告警信息。
在一些实现方式中，遮挡检测装置50，还包括：第四确定部分，被配置为在确定当前视频帧中不存在目标待检测图像块的情况下，确定当前视频帧之前第二采集时长内的各视频帧内，是否均不存在相对于参考视频帧出现图像特征统计图变化异常的目标待检测图像块；更新部分，被配置为在确定当前视频帧之前第二采集时长内的各视频帧内，均不存在相对于参考视频帧出现图像特征统计图变化异常的目标待检测图像块的情况下，将当前视频帧确定为更新后的参考视频帧。
在一些实现方式中，图像特征统计图为图像直方图。
在一些实施例中，本公开实施例提供的装置具有的功能或包含的部分可以用于执行上文方法实施例描述的方法，其具体实现可以参照上文方法实施例的描述。
在本公开实施例以及其他的实施例中，“部分”可以是部分电路、部分处理器、部分程序或软件等等，当然也可以是单元，还可以是模块也可以是非模块化的。
本公开实施例还提出一种计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令被处理器执行时实现上述方法。计算机可读存储介质可以是易失性或非易失性计算机可读存储介质。
本公开实施例还提出一种电子设备，包括：处理器；用于存储处理器可执行指令的存储器；其中，所述处理器被配置为调用所述存储器存储的指令，以执行上述方法。
本公开实施例还提供了一种计算机程序产品，包括计算机可读代码，或者承载有计算机可读代码的非易失性计算机可读存储介质，当所述计算机可读代码在电子设备的处理器中运行时，所述电子设备中的处理器执行上述方法。
电子设备可以被提供为终端、服务器或其它形态的设备。
图6为本公开实施例提供的一种电子设备的一种硬件实体框图。参照图6，电子设备600可以是用户设备(User Equipment，UE)、移动设备、用户终端、终端、蜂窝电话、无绳电话、个人数字处理(Personal Digital Assistant，PDA)、手持设备、计算设备、车载设备、可穿戴设备等终端设备。
参照图6，电子设备600可以包括以下一个或多个组件：处理组件602，存储器604，电源组件606，多媒体组件608，音频组件610，输入/输出(I/O)接口612，传感器组件614，以及通信组件616。
处理组件602通常控制电子设备600的整体操作，诸如与显示，电话呼叫，数据通信，相机操作和记录操作相关联的操作。处理组件602可以包括一个或多个处理器618来执行计算机指令，以完成上述的方法的全部或部分步骤。此外，处理组件602可以包括一个或多个模块，便于处理组件602和其他组件之间的交互。例如，处理组件602可以包括多媒体模块，以方便多媒体组件608和处理组件602之间的交互。
存储器604被配置为存储各种类型的数据以支持在电子设备600的操作。这些数据的一些实施例包括用于在电子设备600上操作的任何应用程序或方法的指令，联系人数据，电话簿数据，消息，图片，视频等。存储器604可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(Static Random Access Memory,SRAM)，电可擦除可编程只读存储器(Electrically Erasable Programmable read only memory,EEPROM)，可擦除可编程只读存储器(Erasable Programmable Read-Only Memory,EPROM)，可编程只读存储器(Programmable read-only memory，PROM)，只读存储器(Read-only  memory,ROM)，磁存储器，快闪存储器，磁盘或光盘。
电源组件606为电子设备600的各种组件提供电力。电源组件606可以包括电源管理系统，一个或多个电源，及其他与为电子设备600生成、管理和分配电力相关联的组件。
多媒体组件608包括在所述电子设备600和用户之间的提供一个输出接口的屏幕。在一些实施例中，屏幕可以包括液晶显示器(LCD)和触摸面板(TP)。如果屏幕包括触摸面板，屏幕可以被实现为触摸屏，以接收来自用户的输入信号。触摸面板包括一个或多个触摸传感器以感测触摸、滑动和触摸面板上的手势。所述触摸传感器可以不仅感测触摸或滑动动作的边界，而且还检测与所述触摸或滑动操作相关的持续时间和压力。在一些实施例中，多媒体组件608包括一个前置摄像头和/或后置摄像头。当电子设备600处于操作模式，如拍摄模式或视频模式时，前置摄像头和/或后置摄像头可以接收外部的多媒体数据。每个前置摄像头和后置摄像头可以是一个固定的光学透镜系统或具有焦距和光学变焦能力。
音频组件610被配置为输出和/或输入音频信号。例如，音频组件610包括一个麦克风(MIC)，当电子设备600处于操作模式，如呼叫模式、记录模式和语音识别模式时，麦克风被配置为接收外部音频信号。所接收的音频信号可以被进一步存储在存储器604或经由通信组件616发送。在一些实施例中，音频组件610还包括一个扬声器，用于输出音频信号。
I/O接口612为处理组件602和外围接口模块之间提供接口，上述外围接口模块可以是键盘，点击轮，按钮等。这些按钮可包括但不限于：主页按钮、音量按钮、启动按钮和锁定按钮。
传感器组件614包括一个或多个传感器，用于为电子设备600提供各个方面的状态评估。例如，传感器组件614可以检测到电子设备600的打开/关闭状态，组件的相对定位，例如所述组件为电子设备600的显示器和小键盘，传感器组件614还可以检测电子设备600或电子设备600一个组件的位置改变，用户与电子设备600接触的存在或不存在，电子设备600方位或加速/减速和电子设备600的温度变化。传感器组件614可以包括接近传感器，被配置用来在没有任何的物理接触时检测附近物体的存在。传感器组件614还可以包括光传感器，如互补金属氧化物半导体(CMOS)或电荷耦合装置(CCD)图像传感器，用于在成像应用中使用。在一些实施例中，该传感器组件614还可以包括加速度传感器，陀螺仪传感器，磁传感器，压力传感器或温度传感器。
通信组件616被配置为便于电子设备600和其他设备之间有线或无线方式的通信。电子设备600可以接入基于通信标准的无线网络，如无线网络(Wi-Fi)、第二代移动通信技术(2G)、第三代移动通信技术(3G)、第四代移动通信技术(4G)、通用移动通信技术的长期演进(LTE)、第五代移动通信技术(5G)或它们的组合。在一些实施例中，通信组件616经由广播信道接收来自外部广播管理系统的广播信号或广播相关信息。在一些实施例中，所述通信组件616还包括近场通信(Near Field Communication,NFC)模块，以促进短程通信。例如，在NFC模块可基于射频识别(Radio Frequency Identification,RFID)技术，红外数据协会(Infrared Data Association,IrDA)技术，超宽带(Ultra Wide Band,UWB)技术，蓝牙(bluetooth,BT)技术和其他技术来实现。
在一些实施例中，电子设备600可以被一个或多个应用专用集成电路(Application Specific Integrated Circuit,ASIC)、数字信号处理器(Digital Signal Processing,DSP)、数字信号处理设备(Digital Signal Processing Device,DSPD)、可编程逻辑器件(Programmable Logic Device,PLD)、现场可编程门阵列(Field  Programmable Gate Array,FPGA)、控制器、微控制器、微处理器或其他电子元件实现，用于执行上述方法。
在一些实施例中，还提供了一种非易失性计算机可读存储介质，例如包括计算机程序指令的存储器604，上述计算机程序指令可由电子设备600的处理器618执行以完成上述方法。
本公开涉及增强现实领域，通过获取现实环境中的目标对象的图像信息，进而借助各类视觉相关算法实现对目标对象的相关特征、状态及属性进行检测或识别处理，从而得到与具体应用匹配的虚拟与现实相结合的AR效果。在一些实施例中，目标对象可涉及与人体相关的脸部、肢体、手势、动作等，或者与物体相关的标识物、标志物，或者与场馆或场所相关的沙盘、展示区域或展示物品等。视觉相关算法可涉及视觉定位、SLAM、三维重建、图像注册、背景分割、对象的关键点提取及跟踪、对象的位姿或深度检测等。具体应用不仅可以涉及跟真实场景或物品相关的导览、导航、讲解、重建、虚拟效果叠加展示等交互场景，还可以涉及与人相关的特效处理，比如妆容美化、肢体美化、特效展示、虚拟模型展示等交互场景。可通过卷积神经网络，实现对目标对象的相关特征、状态及属性进行检测或识别处理。上述卷积神经网络是基于深度学习框架进行模型训练而得到的网络模型。
图7为本公开实施例提供的一种电子设备的一种硬件实体框图。参照图7，电子设备700可以被提供为一服务器。参照图7，电子设备700包括处理组件702，其进一步包括一个或多个处理器，以及由存储器704所代表的存储器资源，用于存储可由处理组件702的执行的指令，例如应用程序。存储器704中存储的应用程序可以包括一个或一个以上的每一个对应于一组指令的模块。此外，处理组件702被配置为执行指令，以执行上述方法。
电子设备700还可以包括一个电源组件706被配置为执行电子设备700的电源管理，一个有线或无线网络接口708被配置为将电子设备700连接到网络，和一个输入输出(I/O)接口710。电子设备700可以操作基于存储在存储器704的操作系统，例如微软服务器操作系统(Windows ServerTM)，苹果公司推出的基于图形用户界面操作系统(Mac OS XTM)，多用户多进程的计算机操作系统(UnixTM),自由和开放原代码的类Unix操作系统(LinuxTM)，开放原代码的类Unix操作系统(FreeBSDTM)或类似。
在一些实施例中，还提供了一种非易失性计算机可读存储介质，例如包括计算机程序指令的存储器704，上述计算机程序指令可由电子设备700的处理组件702执行以完成上述方法。
本公开可以是系统、方法和/或计算机程序产品。计算机程序产品可以包括计算机可读存储介质，其上载有用于使处理器实现本公开的各个方面的计算机可读程序指令。
计算机可读存储介质可以是可以保持和存储由指令执行设备使用的指令的有形设备。计算机可读存储介质例如可以是(但不限于)电存储设备、磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、静态随机存取存储器(SRAM)、便携式压缩盘只读存储器(CD-ROM)、数字多功能盘(DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。这里所使用的计算机可读存储介质不被解释为瞬时信号本身，诸如无线电波或者其他自由传播的电磁波、通过波导或其他传输媒介传播的电磁波(例如，通过光纤电缆的光脉冲)、或者通过电线传输 的电信号。
这里所描述的计算机可读程序指令可以从计算机可读存储介质下载到各个计算/处理设备，或者通过网络、例如因特网、局域网、广域网和/或无线网下载到外部计算机或外部存储设备。网络可以包括铜传输电缆、光纤传输、无线传输、路由器、防火墙、交换机、网关计算机和/或边缘服务器。每个计算/处理设备中的网络适配卡或者网络接口从网络接收计算机可读程序指令，并转发该计算机可读程序指令，以供存储在各个计算/处理设备中的计算机可读存储介质中。
用于执行本公开操作的计算机程序指令可以是汇编指令、指令集架构(ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码，所述编程语言包括面向对象的编程语言—诸如Smalltalk、C++等，以及常规的过程式编程语言—诸如“C”语言或类似的编程语言。计算机可读程序指令可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络—包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。在一些实施例中，通过利用计算机可读程序指令的状态信息来个性化定制电子电路，例如可编程逻辑电路、现场可编程门阵列(FPGA)或可编程逻辑阵列(PLA)，该电子电路可以执行计算机可读程序指令，从而实现本公开的各个方面。
这里参照根据本公开实施例的方法、装置(系统)和计算机程序产品的流程图和/或框图描述了本公开的各个方面。应当理解，流程图和/或框图的每个方框以及流程图和/或框图中各方框的组合，都可以由计算机可读程序指令实现。
这些计算机可读程序指令可以提供给通用计算机、专用计算机或其它可编程数据处理装置的处理器，从而生产出一种机器，使得这些指令在通过计算机或其它可编程数据处理装置的处理器执行时，产生了实现流程图和/或框图中的一个或多个方框中规定的功能/动作的装置。也可以把这些计算机可读程序指令存储在计算机可读存储介质中，这些指令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个方框中规定的功能/动作的各个方面的指令。
也可以把计算机可读程序指令加载到计算机、其它可编程数据处理装置、或其它设备上，使得在计算机、其它可编程数据处理装置或其它设备上执行一系列操作步骤，以产生计算机实现的过程，从而使得在计算机、其它可编程数据处理装置、或其它设备上执行的指令实现流程图和/或框图中的一个或多个方框中规定的功能/动作。
附图中的流程图和框图显示了根据本公开的多个实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或指令的一部分，所述模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的 功能或动作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
该计算机程序产品可以具体通过硬件、软件或其结合的方式实现。在一些实施例中，所述计算机程序产品具体体现为计算机存储介质，在另一些实施例中，计算机程序产品具体体现为软件产品，例如软件开发包(Software Development Kit，SDK)等等。
以上已经描述了本公开的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术的改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。
工业实用性
本公开涉及一种遮挡检测方法及装置、电子设备、存储介质和计算机程序产品，所述方法包括：根据当前视频帧中多个待检测图像块的图像特征统计图，与参考视频帧中多个参考图像块的图像特征统计图，确定所述当前视频帧中是否存在相对于所述参考视频帧出现图像特征统计图变化异常的目标待检测图像块；在确定所述当前视频帧中存在所述目标待检测图像块的情况下，对所述目标待检测图像块进行边缘检测，得到所述目标待检测图像块的边缘检测结果；根据所述边缘检测结果，确定采集所述当前视频帧的图像采集设备是否存在遮挡。上述技术方案能够提高对图像采集设备遮挡检测的准确度，同时降低遮挡检测的算力需求。

Claims (21)

  1. 一种遮挡检测方法，包括：
    根据当前视频帧中多个待检测图像块的图像特征统计图，与参考视频帧中多个参考图像块的图像特征统计图，确定所述当前视频帧中是否存在相对于所述参考视频帧出现图像特征统计图变化异常的目标待检测图像块；
    在确定所述当前视频帧中存在所述目标待检测图像块的情况下，对所述目标待检测图像块进行边缘检测，得到所述目标待检测图像块的边缘检测结果；
    根据所述边缘检测结果，确定采集所述当前视频帧的图像采集设备是否存在遮挡。
  2. 根据权利要求1所述的方法，其中，所述根据当前视频帧中多个待检测图像块的图像特征统计图，与参考视频帧中多个参考图像块的图像特征统计图，确定所述当前视频帧中是否存在相对于所述参考视频帧出现图像特征统计图变化异常的目标待检测图像块，包括：
    确定所述当前视频帧中每个待检测图像块的图像特征统计图，与所述参考视频帧中对应的参考图像块的图像特征统计图之间的相似度；
    在存在相似度低于相似度阈值的待检测图像块的情况下，将相似度低于所述相似度阈值的待检测图像块，确定为所述目标待检测图像块。
  3. 根据权利要求1或2所述的方法，其中，所述对所述目标待检测图像块进行边缘检测，得到所述目标待检测图像块的边缘检测结果，包括：
    对所述目标待检测图像块进行图像灰度转换，得到所述目标待检测图像块的灰度图；
    对所述灰度图进行边缘检测，确定所述目标待检测图像块的图像块边缘值；
    根据所述图像块边缘值和所述灰度图对应的平均灰度值，确定所述目标待检测图像块的边缘值方差；
    将所述目标待检测图像块的边缘值方差，确定为所述边缘检测结果。
  4. 根据权利要求3所述的方法，其中，所述根据所述边缘检测结果，确定采集所述当前视频帧的图像采集设备是否存在遮挡，包括：
    根据所述目标待检测图像块的边缘值方差，以及所述参考视频帧中对应的参考图像块的边缘值方差，确定所述目标待检测图像块是否相对于所述参考视频帧中对应的参考图像块出现边缘值方差剧减；
    在确定所述目标待检测图像块出现边缘值方差剧减的情况下，确定所述图像采集设备中所述目标待检测图像块对应的采集区域存在遮挡。
  5. 根据权利要求4所述的方法，其中，所述根据所述目标待检测图像块的边缘值方差，以及所述参考视频帧中对应的参考图像块的边缘值方差，确定所述目标待检测图像块是否相对于所述参考视频帧中对应的参考图像块出现边缘值方差剧减，包括：
    确定所述目标待检测图像块的边缘值方差，与所述参考视频帧中对应的参考图像块的边缘值方差之间的边缘值方差比例；
    在所述边缘值方差比例低于方差比例阈值的情况下，确定所述目标待检测图像块出现边缘值方差剧减。
  6. 根据权利要求4或5所述的方法，其中，所述方法还包括：
    确定所述当前视频帧之后第一采集时长内的各视频帧内，是否均存在相对于所述参考视频帧出现图像特征统计图变化异常且边缘值方差剧减的所述目标待检测图像块；
    在确定所述当前视频帧之后所述第一采集时长内的各视频帧内，均存在相对于所述参考视频帧出现图像特征统计图变化异常且边缘值方差剧减的所述目标待检测图像块的情况下，生成遮挡告警信息。
  7. 根据权利要求6所述的方法，其中，所述在确定所述当前视频帧之后所述第一采集时长内的各视频帧内，均存在相对于所述参考视频帧出现图像特征统计图变化异常且边缘值方差剧减的所述目标待检测图像块的情况下，生成遮挡告警信息，包括：
    根据出现图像特征统计图变化异常且边缘值方差剧减的所述目标待检测图像块，确定所述图像采集设备的遮挡区域占比；
    在所述遮挡区域占比高于遮挡区域占比阈值的情况下，生成所述遮挡告警信息。
  8. 根据权利要求1至7中任一项所述的方法，其中，所述方法还包括：
    在确定所述当前视频帧中不存在所述目标待检测图像块的情况下，确定所述当前视频帧之前第二采集时长内的各视频帧内，是否均不存在相对于所述参考视频帧出现图像特征统计图变化异常的所述目标待检测图像块；
    在确定所述当前视频帧之前所述第二采集时长内的各视频帧内，均不存在相对于所述参考视频帧出现图像特征统计图变化异常的所述目标待检测图像块的情况下，将所述当前视频帧确定为更新后的参考视频帧。
  9. 根据权利要求1至8中任意一项所述的方法，其中，所述图像特征统计图为图像直方图。
  10. 一种遮挡检测装置，包括：
    第一确定部分，被配置为根据当前视频帧中多个待检测图像块的图像特征统计图，与参考视频帧中多个参考图像块的图像特征统计图，确定所述当前视频帧中是否存在相对于所述参考视频帧出现图像特征统计图变化异常的目标待检测图像块；
    边缘检测部分，被配置为在确定所述当前视频帧中存在所述目标待检测图像块的情况下，对所述目标待检测图像块进行边缘检测，得到所述目标待检测图像块的边缘检测结果；
    第二确定部分，被配置为根据所述边缘检测结果，确定采集所述当前视频帧的图像采集设备是否存在遮挡。
  11. 根据权利要求10所述的装置，其中，所述第一确定部分还被配置为：
    确定所述当前视频帧中每个待检测图像块的图像特征统计图，与所述参考视频帧中对应的参考图像块的图像特征统计图之间的相似度；
    在存在相似度低于相似度阈值的待检测图像块的情况下，将相似度低于所述相似度阈值的待检测图像块，确定为所述目标待检测图像块。
  12. 根据权利要求10或11所述的装置，其中，所述边缘检测部分还被配置为：
    对所述目标待检测图像块进行图像灰度转换，得到所述目标待检测图像块的灰度图；
    对所述灰度图进行边缘检测，确定所述目标待检测图像块的图像块边缘值；
    根据所述图像块边缘值和所述灰度图对应的平均灰度值，确定所述目标待检测图像块的边缘值方差；
    将所述目标待检测图像块的边缘值方差，确定为所述边缘检测结果。
  13. 根据权利要求12所述的装置，其中，所述第二确定部分包括：
    第一确定子部分，被配置为根据所述目标待检测图像块的边缘值方差，以及所述参考视频帧中对应的参考图像块的边缘值方差，确定所述目标待检测图像块是否相对于所述参考视频帧中对应的参考图像块出现边缘值方差剧减；
    第二确定子部分，被配置为在确定所述目标待检测图像块出现边缘值方差剧减的情况下，确定所述图像采集设备中所述目标待检测图像块对应的采集区域存在遮挡。
  14. 根据权利要求13所述的装置，其中，所述第一确定子部分还被配置为：
    确定所述目标待检测图像块的边缘值方差，与所述参考视频帧中对应的参考图像块的边缘值方差之间的边缘值方差比例；
    在所述边缘值方差比例低于方差比例阈值的情况下，确定所述目标待检测图像块出现边缘值方差剧减。
  15. 根据权利要求13或14所述的装置，其中，所述装置还包括：
    第三确定部分，被配置为确定所述当前视频帧之后第一采集时长内的各视频帧内，是否均存在相对于所述参考视频帧出现图像特征统计图变化异常且边缘值方差剧减的所述目标待检测图像块；
    生成部分，被配置为在确定所述当前视频帧之后所述第一采集时长内的各视频帧内，均存在相对于所述参考视频帧出现图像特征统计图变化异常且边缘值方差剧减的所述目标待检测图像块的情况下，生成遮挡告警信息。
  16. 根据权利要求15所述的装置，其中，所述生成部分还包括：
    第三确定子部分，被配置为根据出现图像特征统计图变化异常且边缘值方差剧减的所述目标待检测图像块，确定所述图像采集设备的遮挡区域占比；
    生成子部分，被配置为在所述遮挡区域占比高于遮挡区域占比阈值的情况下，生成所述遮挡告警信息。
  17. 根据权利要求10至16中任一项所述的装置，其中，所述装置还包括：
    第四确定部分，被配置为在确定所述当前视频帧中不存在所述目标待检测图像块的情况下，确定所述当前视频帧之前第二采集时长内的各视频帧内，是否均不存在相对于所述参考视频帧出现图像特征统计图变化异常的所述目标待检测图像块；
    更新部分，被配置为在确定所述当前视频帧之前所述第二采集时长内的各视频帧内，均不存在相对于所述参考视频帧出现图像特征统计图变化异常的所述目标待检测图像块的情况下，将所述当前视频帧确定为更新后的参考视频帧。
  18. 根据权利要求10至17中任意一项所述的装置，其中，所述图像特征统计图为图像直方图。
  19. 一种电子设备，包括：
    处理器；
    用于存储处理器可执行指令的存储器；
    其中，所述处理器被配置为调用所述存储器存储的指令，以执行权利要求1至9中任意一项所述的 方法。
  20. 一种计算机可读存储介质，其上存储有计算机程序指令，其中，所述计算机程序指令被处理器执行时实现权利要求1至9中任意一项所述的方法。
  21. 一种计算机程序产品，所述计算机程序产品包括计算机程序或指令，在所述计算机程序或指令在电子设备上运行的情况下，使得所述电子设备执行权利要求1至9中任意一项所述的方法。
PCT/CN2022/095516 2021-12-08 2022-05-27 遮挡检测方法及装置、电子设备、存储介质及计算机程序产品 WO2023103298A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111512423.8A CN114187498A (zh) 2021-12-08 2021-12-08 遮挡检测方法及装置、电子设备和存储介质
CN202111512423.8 2021-12-08

Publications (1)

Publication Number Publication Date
WO2023103298A1 true WO2023103298A1 (zh) 2023-06-15

Family

ID=80604538

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/095516 WO2023103298A1 (zh) 2021-12-08 2022-05-27 遮挡检测方法及装置、电子设备、存储介质及计算机程序产品

Country Status (2)

Country Link
CN (1) CN114187498A (zh)
WO (1) WO2023103298A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114187498A (zh) * 2021-12-08 2022-03-15 上海商汤智能科技有限公司 遮挡检测方法及装置、电子设备和存储介质
CN114782984B (zh) * 2022-03-25 2022-11-25 珠海视熙科技有限公司 一种基于tof相机的坐姿识别遮挡判定方法及智能台灯
CN116156157B (zh) * 2023-04-24 2023-08-18 长沙海信智能系统研究院有限公司 一种摄像头遮挡异常的检测方法及电子设备

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102231223A (zh) * 2011-06-02 2011-11-02 深圳中兴力维技术有限公司 一种用于摄像机的异物遮挡告警方法
US20150003741A1 (en) * 2013-07-01 2015-01-01 Here Global B.V. Occlusion Resistant Image Template Matching Using Distance Transform
CN107316312A (zh) * 2017-06-30 2017-11-03 深圳信路通智能技术有限公司 一种视频图像遮挡检测方法及系统
CN111027398A (zh) * 2019-11-14 2020-04-17 深圳市有为信息技术发展有限公司 行车记录仪视频遮挡检测方法
CN111723644A (zh) * 2020-04-20 2020-09-29 北京邮电大学 一种监控视频遮挡检测方法及系统
CN113052147A (zh) * 2021-04-30 2021-06-29 北京邮电大学 一种行为识别方法及装置
CN113379705A (zh) * 2021-06-09 2021-09-10 苏州智加科技有限公司 图像处理方法、装置、计算机设备及存储介质
CN114187498A (zh) * 2021-12-08 2022-03-15 上海商汤智能科技有限公司 遮挡检测方法及装置、电子设备和存储介质

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102231223A (zh) * 2011-06-02 2011-11-02 深圳中兴力维技术有限公司 一种用于摄像机的异物遮挡告警方法
US20150003741A1 (en) * 2013-07-01 2015-01-01 Here Global B.V. Occlusion Resistant Image Template Matching Using Distance Transform
CN107316312A (zh) * 2017-06-30 2017-11-03 深圳信路通智能技术有限公司 一种视频图像遮挡检测方法及系统
CN111027398A (zh) * 2019-11-14 2020-04-17 深圳市有为信息技术发展有限公司 行车记录仪视频遮挡检测方法
CN111723644A (zh) * 2020-04-20 2020-09-29 北京邮电大学 一种监控视频遮挡检测方法及系统
CN113052147A (zh) * 2021-04-30 2021-06-29 北京邮电大学 一种行为识别方法及装置
CN113379705A (zh) * 2021-06-09 2021-09-10 苏州智加科技有限公司 图像处理方法、装置、计算机设备及存储介质
CN114187498A (zh) * 2021-12-08 2022-03-15 上海商汤智能科技有限公司 遮挡检测方法及装置、电子设备和存储介质

Also Published As

Publication number Publication date
CN114187498A (zh) 2022-03-15

Similar Documents

Publication Publication Date Title
WO2023103298A1 (zh) 遮挡检测方法及装置、电子设备、存储介质及计算机程序产品
TWI766286B (zh) 圖像處理方法及圖像處理裝置、電子設備和電腦可讀儲存媒介
CN108804980B (zh) 视频场景切换检测方法及装置
US10007841B2 (en) Human face recognition method, apparatus and terminal
WO2021035812A1 (zh) 一种图像处理方法及装置、电子设备和存储介质
TWI702544B (zh) 圖像處理方法、電子設備和電腦可讀儲存介質
WO2020062969A1 (zh) 动作识别方法及装置、驾驶员状态分析方法及装置
CN109784164B (zh) 前景识别方法、装置、电子设备及存储介质
EP2998960A1 (en) Method and device for video browsing
TW202209254A (zh) 圖像分割方法、電子設備和電腦可讀儲存介質
TW202105202A (zh) 影片處理方法及裝置、電子設備、儲存媒體和電腦程式
CN111553864A (zh) 图像修复方法及装置、电子设备和存储介质
CN110532956B (zh) 图像处理方法及装置、电子设备和存储介质
EP3975046B1 (en) Method and apparatus for detecting occluded image and medium
CN107025441B (zh) 肤色检测方法及装置
CN109509195B (zh) 前景处理方法、装置、电子设备及存储介质
EP3933675A1 (en) Method and apparatus for detecting finger occlusion image, and storage medium
CN112184787A (zh) 图像配准方法及装置、电子设备和存储介质
CN108171222B (zh) 一种基于多流神经网络的实时视频分类方法及装置
CN113139471A (zh) 目标检测方法及装置、电子设备和存储介质
WO2023050690A1 (zh) 图像处理方法、装置、电子设备、存储介质和程序
CN111680646A (zh) 动作检测方法及装置、电子设备和存储介质
WO2022141969A1 (zh) 图像分割方法及装置、电子设备、存储介质和程序
CN113660531A (zh) 视频处理方法及装置、电子设备和存储介质
CN110110742B (zh) 多特征融合方法、装置、电子设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22902736

Country of ref document: EP

Kind code of ref document: A1