WO2022142414A1 - 高空抛物的监测方法、装置、电子设备及存储介质 - Google Patents
高空抛物的监测方法、装置、电子设备及存储介质 Download PDFInfo
- Publication number
- WO2022142414A1 WO2022142414A1 PCT/CN2021/114803 CN2021114803W WO2022142414A1 WO 2022142414 A1 WO2022142414 A1 WO 2022142414A1 CN 2021114803 W CN2021114803 W CN 2021114803W WO 2022142414 A1 WO2022142414 A1 WO 2022142414A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- mask
- motion
- image
- area
- difference
- Prior art date
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 72
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000004364 calculation method Methods 0.000 claims abstract description 22
- 230000003068 static effect Effects 0.000 claims description 55
- 238000010276 construction Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 13
- 238000012806 monitoring device Methods 0.000 claims description 6
- 230000000877 morphologic effect Effects 0.000 claims description 4
- 230000000694 effects Effects 0.000 abstract description 9
- 238000010586 diagram Methods 0.000 description 13
- 230000006399 behavior Effects 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/254—Analysis of motion involving subtraction of images
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/14—Picture signal circuitry for video frequency region
- H04N5/144—Movement detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/44—Event detection
Definitions
- the invention relates to the field of artificial intelligence, and in particular, to a monitoring method, device, electronic device and storage medium for high-altitude parabolic objects.
- Embodiments of the present invention provide a method for monitoring high-altitude parabolas, which can improve the monitoring effect of high-altitude parabolic behavior.
- an embodiment of the present invention provides a method for monitoring a high-altitude parabola, the method comprising:
- the high-altitude parabola is monitored.
- the first motion mask image includes a motion mask value and a static mask value
- the first motion mask image between adjacent frame images is obtained by calculating the difference between the frame images, including :
- a first motion mask map is obtained.
- constructing a current background image according to the first motion mask image and the frame image corresponding to the first motion mask image including:
- the first motion mask image is divided into blocks to obtain a plurality of mask regions
- the mask area and the frame image corresponding to the first motion mask construct a plurality of background areas with the same shape as the mask area;
- the mask area includes a motion mask value and a static mask value
- the construction shape is the same as the mask area according to the mask area and the frame image corresponding to the first motion mask image.
- multiple background areas including:
- the corresponding image area in the frame image corresponding to the first motion mask image is selected as the background area.
- the number of the first motion mask image is n frames
- the number of frame images corresponding to the first motion mask image is also n frames, where n is a positive integer greater than 0, and the Calculate the motion state of the mask area according to the motion mask value and the static mask value of the mask area, including:
- the first motion mask map of n frames extract the mask value sequence of each mask region in the first motion mask map, and the dimension of the mask value sequence is n;
- a current background image is constructed based on the frame image corresponding to the first motion mask image and the target image area index.
- calculate the second motion mask image between the current background image and the current frame image including:
- a second motion mask map is obtained.
- the monitoring of the high-altitude parabola based on the second motion mask map includes:
- the monitoring of the high-altitude parabola based on the second motion mask map includes:
- the interference information in the second motion mask image is removed, and a third motion mask image is obtained, and the third motion mask image includes a motion mask value and a static mask value;
- the high-altitude parabola is monitored based on the third motion mask.
- the method further includes:
- a preset number of before and after frame images are acquired as a sequence of forensic frame images.
- an embodiment of the present invention further provides a monitoring device for a high-altitude parabola, the device comprising:
- the first acquisition module is used to acquire the frame image sequence of the current monitoring scene
- the first calculation module is used to calculate the first motion mask between adjacent frame images through the difference between the frame images;
- a background construction module configured to construct a current background image according to the first motion mask image and the frame image corresponding to the first motion mask image
- the second calculation module is used to calculate the second motion mask image between the current background image and the current frame image through the difference between the current background image and the current frame image;
- a monitoring module configured to monitor the high-altitude parabola based on the second motion mask map.
- an embodiment of the present invention provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, when the processor executes the computer program
- the steps in the high-altitude parabolic monitoring method provided by the embodiments of the present invention are implemented.
- an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method for monitoring high-altitude parabola provided by the embodiment of the present invention is implemented steps in .
- the frame image sequence of the current monitoring scene is obtained; the difference between the frame images is used to calculate the first motion mask between adjacent frame images; according to the first motion mask and the The frame image corresponding to the first motion mask image, constructs the current background image; Calculate the second motion mask image between the current background image and the current frame image by the difference between the current background image and the current frame image; Based on the second motion mask, the high-altitude parabola is monitored.
- the constructed current background image is a real-time background image, which can improve the effect of real-time monitoring.
- the second mask image pair is obtained through the current background image and the current frame image.
- the monitoring of high-altitude parabolas reduces the numerical complexity and improves the calculation speed, so that the presence of high-altitude parabolas can be judged in real time, thereby further improving the monitoring effect of high-altitude parabolas.
- FIG. 1 is a flowchart of a monitoring method for a high-altitude parabola provided by an embodiment of the present invention
- FIG. 2 is a flowchart of a method for constructing a background image provided by an embodiment of the present invention
- FIG. 3 is a schematic diagram of a first motion mask image block provided by an embodiment of the present invention.
- FIG. 5 is a schematic structural diagram of a monitoring device for a high-altitude parabola provided by an embodiment of the present invention
- FIG. 6 is a schematic structural diagram of a first computing module provided by an embodiment of the present invention.
- FIG. 7 is a schematic structural diagram of a background building module provided by an embodiment of the present invention.
- FIG. 8 is a schematic structural diagram of a first building submodule provided by an embodiment of the present invention.
- FIG. 9 is a schematic structural diagram of a computing unit provided by an embodiment of the present invention.
- FIG. 10 is a schematic structural diagram of a second computing module provided by an embodiment of the present invention.
- FIG. 11 is a schematic structural diagram of a monitoring module provided by an embodiment of the present invention.
- FIG. 12 is a schematic structural diagram of another monitoring module provided by an embodiment of the present invention.
- FIG. 13 is a schematic structural diagram of another high-altitude parabolic monitoring device provided by an embodiment of the present invention.
- FIG. 14 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention.
- FIG. 1 is a flow chart of a method for monitoring high-altitude parabolas provided by an embodiment of the present invention. As shown in FIG. 1, the method is used to monitor high-altitude parabolas in timing or in real time, and includes the following steps:
- the above-mentioned current monitoring scene may be a building scene such as a residential building, a commercial building, or an office building being monitored by the camera.
- the above-mentioned camera monitoring range can be all floors of the building or floors above a certain number of floors, such as floors above the 4th floor. Monitor the floor of the corresponding range.
- the above-mentioned preset frame image sequence may be the image frame sequence of the preset number of frames captured by the current camera in the video stream, the above-mentioned sequence refers to the sequence arrangement, and the above-mentioned video stream may be a real-time video stream or a certain period of time. video stream.
- the difference between frame images may be understood as the difference between pixel values of corresponding pixel points between two frame images. Since it is a video stream captured by the same camera, the resolution and size of each frame image in the video are the same, so the frame image of the Nth frame can be subtracted from the frame image of the N+1th frame to obtain the difference map, In the difference map, the value of each difference pixel point is the difference value of the corresponding pixel point pair.
- K1 is a 3*3 image
- K2 is also a 3*3 image
- the difference between corresponding pixel pairs between adjacent frame images can be calculated, and it is determined whether the difference between corresponding pixel pairs between adjacent frame images is greater than or equal to a preset first difference threshold; if the adjacent frame images are If the difference between the corresponding pixel pairs is greater than or equal to the preset first difference threshold, the corresponding first mask pixel is assigned as the motion mask value; if the difference between the corresponding pixel pairs between adjacent frame images If it is less than the preset first difference threshold, the corresponding first mask pixel is assigned as a static mask value; based on the assigned first mask pixel, a first motion mask image is obtained.
- the first motion mask map between adjacent frame images can be obtained by calculating the absolute value of the difference between the frame images.
- the above-mentioned first motion mask map may be a binary mask map.
- the motion mask map is represented by two preset mask values.
- One mask value can be used to represent the motion state, that is, the motion mask value, and the other mask value can represent the static state. , the static mask value. Specifically, it can be shown in the following formula:
- motion_mask(i,j) represents the mask value of the first mask pixel (i,j)
- frame_diff(i,j) represents the difference between the pixel (i,j) in adjacent frame images
- motion_thr1 represents the preset first difference threshold
- b represents the motion mask value
- c represents the static mask value.
- the above motion mask value in order to increase the visibility of the first motion mask image, may be 255, and the visual result of outputting the motion mask value 255 in the form of pixels 255 is white,
- the above-mentioned static mask value may be 0, and the visible result of outputting the static mask value 0 in the form of pixel 0 is black.
- the frame image sequence in order to improve the calculation speed between frame images, may be preprocessed, and the frame images in the frame image sequence may be scaled to a preset size, for example, width*height is 720* Pixel size of 480, 480*320, etc.
- the mask value in the above-mentioned first motion mask map may represent the motion state of the corresponding pixel.
- the above-mentioned mask value may be a binary mask, that is, through two different mask values , the different motion states of each pixel are represented by the motion mask value and the static mask value.
- the above motion state can be moving or static, corresponding to the motion mask value and the static mask value respectively.
- pixels whose motion state is static they can be used as background pixels.
- a frame image corresponding to when it is static in the frame image sequence can be found, and the pixel is determined as a background pixel in the frame image.
- the number of the above-mentioned first motion mask images is n frames
- the number of frame images corresponding to the first motion mask images is also n frames.
- the motion state of corresponding pixels in the frame image is judged by using the pixels in the first motion mask image, so as to determine which pixel in which frame image can be used as the background pixel.
- a first data set can be maintained to accommodate the latest frame images sampled from the preset image sequence in sequence, for example, the t+5th frame is sampled , the t+5th frame is added to the first dataset.
- a second data set to accommodate the corresponding motion mask map. For example, after adding the frame image of the t+5th frame in the first data set, the second data set will be added with the t+5th frame.
- the motion mask map corresponding to the frame image are both set to n frames. After n frames are exceeded, the frame image or motion mask image added first will be removed.
- the frame images or motion masks in the dataset exceed 11
- the frame images or motion masks first added to the dataset will be removed, and the frame images or motion masks in the dataset will not exceed 11.
- the removed frame images or motion masks can be stored separately for data reuse, or can be directly deleted.
- the frame images of the first dataset and the motion mask images of the second dataset are in a one-to-one correspondence.
- the motion state of the pixel point whose motion state is static may be selected as the background pixel point.
- the construction of the current background image is completed.
- the difference between the current background image and the current frame image may be understood as the difference between the pixel values of the corresponding pixel points between the current background image and the current frame image. Since the current background image is constructed in real time, and the constructed current background image has the same resolution and size as the corresponding frame image, the current background image can be subtracted from the current frame image to obtain the corresponding difference image.
- the value of each difference pixel is the difference between the current background image and the corresponding pixel pair in the current frame image.
- the difference between the pixel pair corresponding to the current background image and the current frame image can be calculated, and it is determined whether the difference between the pixel pair is greater than or equal to a preset second difference threshold; if the current background image corresponds to the current frame image If the difference value of the pixel point pair is greater than or equal to the preset first difference value threshold, the corresponding second mask pixel point is assigned as the motion mask value; if the difference value of the pixel point pair corresponding to the current background image and the current frame image If it is less than the preset second difference threshold, the corresponding second mask pixel is assigned as a static mask value; based on the assigned second mask pixel, a second motion mask image is obtained.
- the second motion mask image between the current background image and the current frame image can be obtained by calculating the absolute value of the difference between the current background image and the current frame image.
- the above-mentioned second motion mask map may be a binary mask map.
- the motion mask map is represented by two preset mask values.
- One mask value can be used to represent the motion state, that is, the motion mask value, and the other mask value can represent the static state. , the static mask value. Specifically, it can be shown in the following formula:
- motion_mask(i,j) represents the mask value of the second mask pixel point (i,j)
- frame_diff(i,j) represents the pixel point (i,j) in the current background image and the current frame image
- the absolute value of the difference between, motion_thr2 represents the preset second difference threshold
- b represents the motion mask value
- c represents the static mask value.
- K2 is shown in the following formula:
- the above motion mask value in order to increase the visibility of the second motion mask image, may be 255, and the visual result of outputting the motion mask value 255 in the form of pixels 255 is white,
- the above-mentioned static mask value may be 0, and the visible result of outputting the static mask value 0 in the form of pixel 0 is black.
- the second motion mask map includes a motion mask value and a static mask value. It can be understood that in the case of no high-altitude parabola, there is only a static mask in the second motion mask map value, when a certain area of motion mask value appears in the second motion mask image, it means that there is a moving object in the current frame image, and there is a suspicion of high-altitude parabolic behavior.
- the second motion mask map includes motion mask values and static mask values.
- the second motion mask map By visualizing the second motion mask map, it is easier for the staff to see the moving objects, and it is convenient for the staff to monitor the monitoring area. .
- the white object can be separated from the white building background by the mask value, which increases the recognition degree.
- the motion mask value of 255 and the static mask value of 0 Take the motion mask value of 255 and the static mask value of 0 as an example, the visual result of outputting a motion mask value of 255 in the form of pixel 255 is white, and the visual result of outputting a static mask value of 0 in the form of pixel 0. is black; at this time, the white building is the static background, which is black after the second mask image is visualized, and the white object is the moving object, which is white after the second mask image is visualized.
- the second motion mask image corresponding to the current frame image if a large-area motion mask value is detected, it means that there is a moving object, and the motion analysis of the moving object can be performed to determine the moving direction of the moving object. If the direction of motion is downward, it can be considered that the behavior of a high-altitude parabola is detected.
- the monitoring workload and labor cost can be reduced.
- the second motion mask image since the second motion mask image is updated along with the current frame image, the second motion mask image has real-time performance, which satisfies the real-time performance of video surveillance.
- the frame image sequence of the current monitoring scene is obtained; the difference between the frame images is used to calculate the first motion mask between adjacent frame images; according to the first motion mask and the The frame image corresponding to the first motion mask image, constructs the current background image; Calculate the second motion mask image between the current background image and the current frame image by the difference between the current background image and the current frame image; Based on the second motion mask, the high-altitude parabola is monitored.
- the constructed current background image is a real-time background image, which can improve the effect of real-time monitoring.
- the second mask image pair is obtained through the current background image and the current frame image.
- the monitoring of high-altitude parabolas reduces the numerical complexity and improves the calculation speed, so that the presence of high-altitude parabolas can be judged in real time, thereby further improving the monitoring effect of high-altitude parabolas.
- FIG. 2 is a flowchart of a method for constructing a background image provided by an embodiment of the present invention. As shown in FIG. 2, the method includes the following steps:
- the first motion mask image may be divided into blocks according to a preset rule, or the first motion mask image may be divided into blocks according to image recognition.
- the above preset rule may be an average division, and specifically, the first motion mask image may be divided into M*N blocks of the same size, each block corresponding to a mask area, so as to obtain M*N masks code area, the size of each mask area is: (img_width/M, img_height/N), wherein, the above img_width is the total width of the first motion mask image, and img_height is the total height of the first motion mask image.
- the mask area and the frame image corresponding to the first motion mask image construct multiple background areas with the same shape as the mask area.
- the motion state of the mask area can be calculated according to the mask value of the mask area; and then according to the motion state of the mask area, an image corresponding to the frame image corresponding to the first motion mask image is selected area as background area.
- the above-mentioned mask area includes a motion mask value and a static mask value
- the motion state of a mask area can be judged according to the motion mask value and the static mask value
- the motion state of a mask area is judged to be static
- the image area corresponding to the mask area may be taken out from the frame image corresponding to the first motion mask image as the background area.
- the motion state of the mask area is judged to be motion
- the motion state of the mask area corresponding to the first motion mask image of the next frame is judged again.
- the motion state of the mask region can be judged according to the average value of the motion mask value of the mask region and the static mask value, or the mask can be judged according to the number or proportion of the motion mask values of the mask region.
- the motion state of the area can be judged according to the average value of the motion mask value of the mask region and the static mask value, or the mask can be judged according to the number or proportion of the motion mask values of the mask region.
- the number of the first motion mask image is n frames
- the number of frame images corresponding to the first motion mask image is also n frames, where n is a positive integer greater than 0, and can be
- the first motion mask image of n frames extract the mask value sequence of each mask area in the first motion mask image, and the dimension of the mask value sequence is n; extract the corresponding mask value sequence according to the mask value sequence corresponding to the mask area.
- the frame image corresponding to the first motion mask map and the target image area index construct a corresponding background area.
- the frame image sequence includes n+1 frame images, correspondingly, n first motion mask images can be obtained by calculation, and the nth first motion mask image corresponds to the n+1th frame image.
- the first motion mask map includes M*N mask regions.
- the latest n first motion mask images can be put into an array motion_value_array(n, N, M), and the latest n frame images can be put into another array frame_patch_array(n, height, width).
- the storage quantity of each array is n. If n is exceeded, the first motion mask image or frame image added to the array will be deleted.
- the corresponding frame image also includes M*N images. area, each image area corresponds to a mask area, you can put the latest n frame images into another array frame_patch_array(n, N, M).
- the motion state of the masked region can be calculated by:
- motion_value(pi, pj) in the formula represents the motion state of the mask area (pi, pj)
- M*N represents the number of mask areas
- img_width is the total width of the first motion mask image
- img_height is the first motion
- img_height*img_width represents the pixel area of the first motion mask image
- (i, j) represents the first mask pixel in the mask area (pi, pj)
- patch(pi, pj) (i, j) represents the mask value of the first mask pixel (i, j) in the mask region (pi, pj).
- each mask area there is an n-dimensional array representing the motion state of the block in the last n frames, take the index min_index with the smallest value in the array as the target image area index, and take the latest n frame images
- the entire image area corresponding to the index min_index position block in the array frame_patch_array(n, height, width) is used as the background area.
- the first motion mask image includes 4 mask areas, namely, mask area 1, mask area 2, mask area 3, and mask area 4, as shown in the figure
- the value of the mask area 1 of the first motion mask image in the second frame is the smallest, then the value of the first motion mask image in the second frame is the smallest.
- the corresponding index min_index is (frame 2, mask area 1), then according to the index (frame 2, mask area 1) to the array frame_patch_array(n, height, width) to find the frame image of the second frame, and take the area corresponding to the mask area 1 in the frame image of the second frame as the background area 1;
- the first motion mask image array motion_value_array(n, N, M ) the value of the mask area 2 of the first motion mask image of the 4th frame is the smallest, then the mask area 2 of the first motion mask image of the 4th frame is judged to be in a static state, then the corresponding index min_index is (4th frame, mask area 2), then according to the index (frame 4, mask area 2) to the frame image array frame_patch_array(n, height, width) to find the frame image of the 4th frame, and put the frame image of the 4th frame
- the area corresponding to the mask area 2 in the frame image is used as the background area 2; similarly
- the background regions obtained by corresponding calculation of the mask regions have a corresponding positional relationship with each mask region, and the background regions are spliced according to the positions of the mask regions in the first motion mask map, then The current background image can be obtained.
- the frame image sequence participates in the real-time construction of the current background image.
- the number n of frame images in the frame image sequence may be limited, for example, set a smaller Numbers, such as 20, 30, etc.
- the acquisition of video stream image frames is about 30 frames per second. Therefore, n can be set to a positive integer less than 30, thereby improving the speed of constructing the current background image and improving the real-time monitoring.
- the current background image is constructed in the form of blocks, and the reconstruction calculation is not performed pixel by pixel, which further improves the construction speed of the real-time background image.
- FIG. 4 is a flowchart of another high-altitude parabolic monitoring method provided by an embodiment of the present invention.
- the second motion mask So as to realize the monitoring of high-altitude parabola, as shown in Figure 4, it includes the following steps:
- the above-mentioned first motion region is composed of motion mask values.
- the area of the first motion area is larger than the preset area, which is a detectable object, and the probability of noise interference is small.
- the above-mentioned second motion region is composed of motion mask values.
- the area of the second movement area is larger than the preset area, which is a detectable object, and the probability of noise interference is small.
- first motion area and the second motion area are in the relationship between the frames before and after, it can be determined whether the first motion area and the second motion area are motion areas corresponding to the same object through similarity calculation.
- the above-mentioned first motion region and second motion region may be frame-shaped regions obtained by contour fitting based on motion mask values. If there are multiple first motion areas and multiple second motion areas, calculate the similarity between the frame image areas corresponding to the multiple first motion areas and the frame image areas corresponding to the multiple second motion areas one by one, using The structural similarity ssim is used as the similarity calculation method.
- the structural similarity ssim of the image area corresponding to the first motion area and the image area corresponding to the second motion area is greater than the preset similarity threshold, for example, the structural similarity ssim is greater than 0.8 , it can be determined that the first motion area and the second motion area are motion areas of the same object.
- the frame image area corresponding to the first motion area is the image area in the current frame image
- the frame image area corresponding to the second motion area is the image area in the next frame image.
- the similarity between the frame image area corresponding to the first motion area and the frame image area corresponding to the second motion area is greater than a preset similarity threshold, it means that the object in the first motion area and the second motion area are similar.
- the object in is the same object, and the object is in motion. According to the motion trajectory of the object, it can be judged whether the object is a high-altitude parabola.
- the vector distance between the center position of the first motion area and the center position of the second motion area may be calculated as the motion trajectory.
- the direction of the motion track (which may be the vector distance) is downward, and the value of the motion track (which may be the vector distance) is greater than the preset value, it can be determined to be a high-altitude parabola, and in other cases, it can be determined that it has not occurred High-altitude parabola.
- the difference value obtained by subtracting the vertical coordinate cur_roi_y of the center position of the first motion area and the vertical coordinate next_roi_y of the center position of the second motion area may also be calculated, if the difference value is positive, and the numerical value If it is greater than the preset value, it can be determined as a high-altitude parabola, and in other cases, it can be determined that no high-altitude parabola has occurred; it can also be determined whether next_roi_y is greater than acur_roi_y, where a is greater than 1, for example, determine whether next_roi_y is greater than 1.2acur_roi_y, if it is greater, it can be determined as High-altitude parabola, otherwise it can be determined that no high-altitude parabola has occurred.
- the second motion mask image may be preprocessed to eliminate noise interference in the second motion mask image, so as to obtain a more accurate and clear second motion mask image as the second motion mask image.
- the third motion mask map.
- the high-altitude parabola can be monitored based on the third motion mask, so that the detection of the high-altitude parabola is more accurate. Monitoring the high-altitude parabola based on the third motion mask can refer to steps 401 and 402.
- a preset number of frame images before and after may be obtained as a sequence of forensic frame images. For example, the first 100 frames and the last 100 frames at the time of occurrence are retained as a sequence of evidence images, and then manual confirmation is performed to find out which resident created the high-altitude parabola, and carry out follow-up processing, such as warning, punishment, and accountability.
- a prompt alarm of high-altitude parabola can be automatically issued to the current monitoring scene and/or the management department.
- the above-mentioned current monitoring scene refers to the scene where the corresponding camera is deployed, such as Unit C, Building B, Residential Area A.
- a danger alarm will be issued at Unit C, Building B, Building A, Residential Area A to alert people near Unit C, Building B, Residential Area A.
- the above-mentioned management department may be a property management department or a city management department or other organizations with management authority, other organizations with management authority such as owners' committees, garden clubs, and so on.
- the prompt alarm sent to the management department also includes video information of the current monitoring scene, and the video information includes continuous frame images of high-altitude parabolic occurrences, and the alarm can be sent through various contact methods.
- the information is sent to the management department or the contact terminal of relevant personnel, such as email, mobile phone APP or WeChat public account push, etc.
- FIG. 5 is a schematic structural diagram of a monitoring device for a high-altitude parabola provided by an embodiment of the present invention. As shown in FIG. 5, the device includes:
- the first acquisition module 501 is used to acquire the frame image sequence of the current monitoring scene
- the first calculation module 502 is used for calculating the first motion mask between adjacent frame images through the difference between the frame images;
- a background construction module 503 is configured to construct a current background image according to the first motion mask image and the frame image corresponding to the first motion mask image;
- the second calculation module 504 is used to calculate the second motion mask image between the current background image and the current frame image through the difference between the current background image and the current frame image;
- the monitoring module 505 is configured to monitor the high-altitude parabola based on the second motion mask map.
- the first motion mask map includes a motion mask value and a static mask value
- the first calculation module 502 includes:
- the first calculation sub-module 5021 is used to calculate the difference between the corresponding pixel pairs between adjacent frame images, and determine whether the difference between the corresponding pixel pairs between the adjacent frame images is greater than or equal to a preset first difference threshold;
- the first mask sub-module 5022 is configured to assign the corresponding first mask pixel as a motion mask if the difference between the corresponding pixel pairs between the adjacent frame images is greater than or equal to a preset first difference threshold. code value;
- the second mask sub-module is configured to assign the corresponding first mask pixel as a static mask value if the difference between the corresponding pixel pairs between the adjacent frame images is less than the preset first difference threshold 5023 ;
- the first processing sub-module 5024 is configured to obtain a first motion mask map based on the assigned first mask pixels.
- the background building module 503 includes:
- a block submodule 5031 configured to block the first motion mask image to obtain a plurality of mask regions
- the first construction sub-module 5032 is used to construct a plurality of background areas with the same shape as the mask area according to the mask area and the frame image corresponding to the first motion mask image;
- the second construction sub-module 5033 constructs the current background image based on the background area.
- the mask area includes a motion mask value and a static mask value
- the first construction submodule 5032 includes:
- Calculation unit 5032 for calculating the motion state of the mask area according to the motion mask value and the static mask value of the mask area;
- the selecting unit 50322 is configured to select an image area corresponding to the frame image corresponding to the first motion mask image as a background area according to the motion state of the mask area.
- the number of the first motion mask image is n frames
- the number of frame images corresponding to the first motion mask image is also n frames, where n is greater than 0
- the first extraction subunit 503211 is used to extract the mask value sequence of each mask region in the first motion mask image according to the n frames of the first motion mask image, and the dimension of the mask value sequence is n;
- the second extraction subunit 503212 is used to extract the corresponding target image region index according to the mask value sequence corresponding to the mask region;
- the construction subunit 503213 is configured to construct a current background image based on the frame image corresponding to the first motion mask image and the target image area index.
- the second computing module 504 includes:
- the second calculation sub-module 5041 is used to calculate the difference between the pixel pairs corresponding to the current background image and the current frame image, and determine whether the difference between the pixel pairs is greater than or equal to a preset second difference threshold;
- the third mask sub-module 5042 is configured to assign the corresponding second mask pixel as motion mask value
- the fourth mask sub-module 5043 is used to assign the corresponding second mask pixel as a static mask if the difference between the pixel pair corresponding to the current background image and the current frame image is less than the preset second difference threshold. code value;
- the second processing sub-module 5044 is configured to obtain a second motion mask map based on the assigned second mask pixels.
- the monitoring module 505 includes:
- the first monitoring sub-module 5051 is used to monitor whether a first motion area with an area larger than a preset area threshold appears in the second motion mask image corresponding to the current frame image, and the first motion area is composed of a motion mask value;
- the third calculation sub-module 5052 is configured to monitor whether a second motion area with an area larger than a preset area threshold appears in the second motion mask corresponding to the next frame of image, if it exists, and when the second motion area appears Next, calculate the similarity between the frame image area corresponding to the first motion area and the frame image area corresponding to the second motion area, and the second motion area is composed of a motion mask value;
- the fourth calculation sub-module 5053 is configured to calculate the first motion area if the similarity between the frame image area corresponding to the first motion area and the frame image area corresponding to the second motion area is greater than a preset similarity threshold with the motion trajectory of the second motion area;
- the judgment sub-module 5054 is used for judging whether the motion trajectory complies with the condition of the high-altitude parabola.
- the monitoring module 505 further includes:
- the preprocessing sub-module 5055 is used for removing the interference information in the second motion mask image through the morphological opening operation to obtain a third motion mask image, and the third motion mask image includes the motion mask value and static mask value;
- the second monitoring sub-module 5056 is configured to monitor the high-altitude parabola based on the third motion mask map.
- the device further includes:
- the second acquisition module 506 is configured to acquire a preset number of front and rear frame images as a sequence of forensic frame images when the occurrence of high-altitude parabolas is monitored.
- the device for monitoring high-altitude parabolas provided by the embodiments of the present invention can be applied to devices such as mobile phones, monitors, computers, and servers that can monitor high-altitude parabolas.
- the high-altitude parabolic monitoring device provided by the embodiment of the present invention can realize the various processes implemented by the high-altitude parabolic monitoring method in the above method embodiments, and can achieve the same beneficial effects. In order to avoid repetition, details are not repeated here.
- FIG. 14 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention, as shown in FIG. 14, including: a memory 1402, a processor 1401, and a memory 1402 and a processor A computer program running on 1401, which:
- the processor 1401 is configured to call the computer program stored in the memory 1402, and perform the following steps:
- the high-altitude parabola is monitored.
- the first motion mask map includes a motion mask value and a still mask value, and the difference between the frame images performed by the processor 1401 is calculated to obtain the first motion between adjacent frame images.
- Mask map including:
- a first motion mask map is obtained.
- the construction of the current background image according to the first motion mask image and the frame image corresponding to the first motion mask image performed by the processor 1401 includes:
- the first motion mask image is divided into blocks to obtain a plurality of mask regions
- the mask area and the frame image corresponding to the first motion mask construct a plurality of background areas with the same shape as the mask area;
- the processor 1401 performs the construction of multiple background areas with the same shape as the mask area, including:
- the corresponding image area in the frame image corresponding to the first motion mask image is selected as the background area.
- the number of the first motion mask image is n frames
- the number of frame images corresponding to the first motion mask image is also n frames, where n is a positive integer greater than 0, and the processor
- calculating the motion state of the mask area including:
- the first motion mask map of n frames extract the mask value sequence of each mask region in the first motion mask map, and the dimension of the mask value sequence is n;
- a current background image is constructed based on the frame image corresponding to the first motion mask image and the target image area index.
- the calculation of the second motion mask image between the current background image and the current frame image by the difference between the current background image and the current frame image performed by the processor 1401 includes:
- a second motion mask map is obtained.
- the monitoring of the high-altitude parabola performed by the processor 1401 based on the second motion mask map includes:
- the monitoring of the high-altitude parabola performed by the processor 1401 based on the second motion mask map includes:
- the interference information in the second motion mask image is removed, and a third motion mask image is obtained, and the third motion mask image includes a motion mask value and a static mask value;
- the high-altitude parabola is monitored based on the third motion mask.
- processor 1401 further executes the steps of:
- a preset number of before and after frame images are acquired as a sequence of forensic frame images.
- the above-mentioned electronic device may be a mobile phone, a monitor, a computer, a server and other devices that can be applied to the monitoring of high-altitude parabolic objects.
- the electronic device provided by the embodiment of the present invention can realize the various processes realized by the monitoring method for high-altitude parabola in the above method embodiments, and can achieve the same beneficial effect, which is not repeated here in order to avoid repetition.
- Embodiments of the present invention further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, each process of the high-altitude parabolic monitoring method provided by the embodiment of the present invention is implemented, and The same technical effect can be achieved, and in order to avoid repetition, details are not repeated here.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Signal Processing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
一种高空抛物的监测方法、装置、电子设备及存储介质，所述方法包括：获取当前监测场景的帧图像序列（101）；通过帧图像之间的差值，计算得到相邻帧图像间的第一运动掩码图（102）；根据所述第一运动掩码图以及与所述第一运动掩码图对应的帧图像，构建当前背景图（103）；通过当前背景图与当前帧图像的差值，计算当前背景图与当前帧图像之间的第二运动掩码图（104）；基于第二运动掩码图，对高空抛物进行监测（105）。可以提高实时监测的效果，同时，通过当前背景图以及当前帧图像得到第二掩码图对高空抛物进行监测，降低了数值的复杂性，提高计算速度，从而可以实时的判断是否存在高空抛物的情况，进一步提高了高空抛物的监测效果。
Description
本申请要求于2020年12月30日提交中国专利局，申请号为202011627329.2、发明名称为“高空抛物的监测方法、装置、电子设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
本发明涉及人工智能领域，尤其涉及一种高空抛物的监测方法、装置、电子设备及存储介质。
随着房地产的发展，新建居民小区的楼层越来越高，高空抛物的问题越来越突出。现在的居民小区中大多安装有监控相机对小区内的情况进行监控，在发生高空抛物的事件时，相关人员可以根据采集到该高空抛物行为的监控视频进行调用查看，但是具体的高空抛物情况需要人工逐帧进行查看或通过慢放镜头进行查看，不仅工作量大，还容易发生遗漏，而且，无法及时的发现高空抛物情况。而且，由于相机的安装及相机分辨率的原因，有时高空抛物的物体较小，在监控视频中也不容易被肉眼发现。因此，现有的高空抛物事件的监测效果不好。
发明内容
本发明实施例提供一种高空抛物的监测方法，能够提高高空抛物行为的监测效果。
第一方面，本发明实施例提供一种高空抛物的监测方法，所述方法包括：
获取当前监测场景的帧图像序列；
通过帧图像之间的差值，计算得到相邻帧图像间的第一运动掩码图；
根据所述第一运动掩码图以及与所述第一运动掩码图对应的帧图像，构建当前背景图；
通过当前背景图与当前帧图像的差值，计算所述当前背景图与当前帧图像之间的第二运动掩码图；
基于所述第二运动掩码图，对所述高空抛物进行监测。
可选的，所述第一运动掩码图包括运动掩码值与静止掩码值，所述通过帧图像之间的差值，计算得到相邻帧图像间的第一运动掩码图，包括：
计算相邻帧图像间对应像素点对的差值，并判断所述相邻帧图像间对应像素点对的差值是否大于或等于预设的第一差值阈值；
若所述相邻帧图像间对应像素点对的差值大于或等于预设的第一差值阈值，则将对应的第一掩码像素点赋值为运动掩码值；
若所述相邻帧图像间对应像素点对的差值小于预设的第一差值阈值，则将对应的第一掩码像素点赋值为静止掩码值；
基于赋值后的第一掩码像素点，得到第一运动掩码图。
可选的，所述根据所述第一运动掩码图以及与所述第一运动掩码图对应的帧图像，构建当前背景图，包括：
对所述第一运动掩码图进行分块，得到多个掩码区域；
根据所述掩码区域以及与所述第一运动掩码图对应的帧图像，构建形状与掩码区域相同的多个背景区域；
基于所述背景区域，构建当前背景图。
可选的，所述掩码区域包括运动掩码值以及静止掩码值，所述根据所述掩码区域以及与所述第一运动掩码图对应的帧图像，构建形状与掩码区域相同的多个背景区域，包括：
根据所述掩码区域的运动掩码值以及静止掩码值，计算所述掩码区域的运动状态；
根据所述掩码区域的运动状态，选取与所述第一运动掩码图对应的帧图像中对应的图像区域作为背景区域。
可选的，所述第一运动掩码图的数量为n帧，与所述第一运动掩码图对应的帧图像的数量也为n帧，其中，n为大于0的正整数，所述根据所述掩码区域的运动掩码值以及静止掩码值，计算所述掩码区域的运动状态，包括：
根据n帧第一运动掩码图，提取第一运动掩码图中各个掩码区域的掩码值序列，所述掩码值序列的维度为n；
根据掩码区域对应的掩码值序列，提取对应的目标图像区域索引；
基于所述与所述第一运动掩码图对应的帧图像以及所述目标图像区域索引，构建当前背景图。
可选的，所述通过当前背景图与当前帧图像的差值，计算所述当前背景图 与当前帧图像之间的第二运动掩码图，包括：
计算所述当前背景图与当前帧图像对应像素点对的差值，并判断所述像素点对的差值是否大于或等于预设的第二差值阈值；
若所述当前背景图与当前帧图像对应像素点对的差值大于或等于预设的第一差值阈值，则将对应的第二掩码像素点赋值为运动掩码值；
若所述当前背景图与当前帧图像对应像素点对的差值小于预设的第二差值阈值，则将对应的第二掩码像素点赋值为静止掩码值；
基于赋值后的第二掩码像素点，得到第二运动掩码图。
可选的，所述基于所述第二运动掩码图，对所述高空抛物进行监测，包括：
监测当前帧图像对应的第二运动掩码图中是否出现面积大于预设面积阈值的第一运动区域，所述第一运动区域由运动掩码值构成；
若存在，则监测下一帧图像对应的第二运动掩码图中是否出现面积大于预设面积阈值的第二运动区域，并在出现第二运动区域的情况下，计算所述第一运动区域对应帧图像区域与所述第二运动区域对应帧图像区域的相似度，所述第二运动区域由运动掩码值构成；
若所述第一运动区域对应帧图像区域与所述第二运动区域对应帧图像区域的相似度大于预设的相似度阈值，则计算所述第一运动区域与所述第二运动区域的运动轨迹；
判断所述运动轨迹是否符合高空抛物的条件。
可选的，所述基于所述第二运动掩码图，对所述高空抛物进行监测，包括：
通过形态学开运算，去除所述第二运动掩码图中的干扰信息，得到第三运动掩码图，所述第三运动掩码图包括运动掩码值以及静止掩码值；
基于所述第三运动掩码图，对所述高空抛物进行监测。
可选的，所述方法还包括：
在监测到高空抛物发生时，获取预设数量的前后帧图像作为取证帧图像序列。
第二方面，本发明实施例还提供一种高空抛物的监测装置，所述装置包括：
第一获取模块，用于获取当前监测场景的帧图像序列；
第一计算模块，用于通过帧图像之间的差值，计算得到相邻帧图像间的第一运动掩码图；
背景构建模块，用于根据所述第一运动掩码图以及与所述第一运动掩码图对应的帧图像，构建当前背景图；
第二计算模块，用于通过当前背景图与当前帧图像的差值，计算所述当前背景图与当前帧图像之间的第二运动掩码图；
监测模块，用于基于所述第二运动掩码图，对所述高空抛物进行监测。
第三方面，本发明实施例提供一种电子设备，包括：存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现本发明实施例提供的高空抛物的监测方法中的步骤。
第四方面，本发明实施例提供一种计算机可读存储介质，所述计算机可读存储介质上存储有计算机程序，所述计算机程序被处理器执行时实现发明实施例提供的高空抛物的监测方法中的步骤。
本发明实施例中，获取当前监测场景的帧图像序列；通过帧图像之间的差值，计算得到相邻帧图像间的第一运动掩码图；根据所述第一运动掩码图以及与所述第一运动掩码图对应的帧图像，构建当前背景图；通过当前背景图与当前帧图像的差值，计算所述当前背景图与当前帧图像之间的第二运动掩码图；基于所述第二运动掩码图，对所述高空抛物进行监测。通过相邻帧图像间的第一运动掩码图，构建得到的当前背景图为实时的背景图，可以提高实时监测的效果，同时，通过当前背景图以及当前帧图像得到第二掩码图对高空抛物进行监测，降低了数值的复杂性，提高计算速度，从而可以实时的判断是否存在高空抛物的情况，从而进一步提高了高空抛物的监测效果。
图1是本发明实施例提供的一种高空抛物的监测方法的流程图；
图2是本发明实施例提供的一种背景图构建方法的流程图；
图3是本发明实施例提供的一种第一运动掩码图分块的示意图；
图4是本发明实施例提供的另一种高空抛物的监测方法的流程图；
图5是本发明实施例提供的一种高空抛物的监测装置的结构示意图；
图6是本发明实施例提供的一种第一计算模块的结构示意图；
图7是本发明实施例提供的一种背景构建模块的结构示意图；
图8是本发明实施例提供的一种第一构建子模块的结构示意图；
图9是本发明实施例提供的一种计算单元的结构示意图；
图10是本发明实施例提供的一种第二计算模块的结构示意图；
图11是本发明实施例提供的一种监测模块的结构示意图；
图12是本发明实施例提供的另一种监测模块的结构示意图；
图13是本发明实施例提供的另一种高空抛物的监测装置的结构示意图；
图14是本发明实施例提供的一种电子设备的结构示意图。
请参见图1，图1是本发明实施例提供的一种高空抛物的监测方法的流程图，如图1所示，该方法用于定时或实时进行高空抛物的监测，包括以下步骤：
101、获取当前监测场景的帧图像序列。
在本发明实施例中，上述当前监测场景可以是相机正在监控的居民楼、商业楼或办公楼等楼栋场景。上述的相机监控范围可以是楼栋的全部楼层或一定层数以上的楼层，比如4楼以上的楼层，可以在安装相机时，根据需要进行确定，并调整相机的拍摄角度，以使该相机能够监控对应范围的楼层。
上述预设帧图像序列可以是当前相机拍摄到视频流中的预设帧数量的图像帧序列，上述序列指的是按时序排列，上述的视频流可以是实时视频流，也可以是某一段时间的视频流。
102、通过帧图像之间的差值，计算得到相邻帧图像间的第一运动掩码图。
在本发明实施例中，帧图像之间的差值可以理解为两个帧图像间对应像素点的像素值之差。由于是同一相机拍摄到的视频流，视频中各个帧图像的分辨率和大小均是相同的，因此可以用第N+1帧的帧图像减去第N帧的帧图像，得到差值图，在差值图中，每个差值像素点的值为对应像素点对的差值。
举例来说，假设两个相邻两帧图像分别K1与K2，设K1为3*3的图像，K2也为3*3的图像，如下述矩阵式子所示：
相邻帧图像间的差值图K＝K2－K1为：
在差值图K中，差值的绝对值越大，说明该像素点像素值的变化越大，而对于作为背景的背景像素点来说，变化一般很小。因此，可以根据相邻帧图像之间对应像素点的像素值变化，来确定对应像素点是处于运动状态或静止状态。
具体的，可以计算相邻帧图像间对应像素点对的差值，并判断相邻帧图像间对应像素点对的差值是否大于或等于预设的第一差值阈值；若相邻帧图像间对应像素点对的差值大于或等于预设的第一差值阈值，则将对应的第一掩码像素点赋值为运动掩码值；若相邻帧图像间对应像素点对的差值小于预设的第一差值阈值，则将对应的第一掩码像素点赋值为静止掩码值；基于赋值后的第一掩码像素点，得到第一运动掩码图。
进一步的，可以通过帧图像之间的差值绝对值来计算得到相邻帧图像间的第一运动掩码图。上述第一运动掩码图可以是二值掩码图。在二值掩码图中，通过两个预设的掩码值来对运动掩码图进行表示，可以用一个掩码值代表运动状态，即运动掩码值，另一个掩码值代表静止状态，即静止掩码值。具体可以如下述式子所示：
其中，上述式中motion_mask(i,j)表示第一掩码像素点(i,j)的掩码值，frame_diff(i,j)表示像素点(i,j)在相邻帧图像间的差值绝对值，motion_thr1表示预设的第一差值阈值，b表示运动掩码值，c表示静止掩码值。第一运动掩码图motion_mask1记录了相邻帧图像之间对应像素点的运动变化。比如，假设第一差值阈值motion_thr1设置为30，第一运动掩码图motion_mask1＝motion_mask(|K|)，则有第一运动掩码图motion_mask1如下所示：
在一种可能的实施例中，为了增加第一运动掩码图的可视性，上述运动掩码值可以是255，将运动掩码值255以像素255形式进行输出的可视结果为白 色，上述的静止掩码值可以是0，将静止掩码值0以像素0形式进行输出的可视结果为黑色。
在一种可能的实施例中，为了提高帧图像之间的计算速度，可以对帧图像序列进行预处理，将帧图像序列中的帧图像缩放到预设的尺寸，比如宽*高为720*480、480*320等的像素尺寸。
103、根据第一运动掩码图以及与第一运动掩码图对应的帧图像，构建当前背景图。
在本发明实施例中，上述第一运动掩码图中的掩码值可以表示对应像素点的运动状态，比如，上述掩码值可以是二值掩码，即通过两个不同的掩码值，通过运动掩码值与静止掩码值来表示各个像素点不同的运动状态，上述运动状态可以是移动或静止，分别对应于运动掩码值与静止掩码值。
对于运动状态为静止的像素点，可以作为背景像素点来进行使用。对于运动状态为移动的像素点，则可以寻找其在帧图像序列中处于静态时对应的帧图像，并在该帧图像中确定该像素点作为背景像素点。
进一步的，上述第一运动掩码图的数量为n帧，与第一运动掩码图对应的帧图像的数量也为n帧。通过第一运动掩码图中的像素点对帧图像中对应像素点进行运动状态的判断，从而确定哪一帧图像中的哪一像素点可以作为背景像素点。
在根据预设帧图像序列获取对应的运动掩码图时，可以维护一个第一数据集，用于容纳依次从预设图像序列中采样的最新的帧图像，比如，采样到第t+5帧时，则将该第t+5帧添加到第一数据集中。并维护一个第二数据集，用于容纳对应的运动掩码图，比如，在第一数据集中添加第t+5帧的帧图像后，会在第二数据集中添加与第t+5帧的帧图像对应的运动掩码图。其中，第一数据集与第二数据集的容纳能力均设置为n帧，在超出n帧后，则会将最先添加的帧图像或运动掩码图进行移除，比如，设n为11，则数据集中帧图像或运动掩码图超过11时，则将最先添加到数据集中的帧图像或运动掩码图进行移除，保持数据集中帧图像或运动掩码图不超过11。移除后的帧图像或运动掩码图可以另行存储，以供数据复用，也可以直接进行删除。第一数据集的帧图像与第二数据集中的运动掩码图为一一对应的关系。
可以根据n帧第一运动掩码图中各个像素点的运动状态，选取n帧对应的帧图像中，运动状态为静止的像素点作为背景像素点。当得到大小与帧图像或运动掩码图相同的背景图像，则完成当前背景图像的构建。
104、通过当前背景图与当前帧图像的差值，计算当前背景图与当前帧图像之间的第二运动掩码图。
在本发明实施例中，当前背景图与当前帧图像的差值可以理解为当前背景图与当前帧图像间对应像素点的像素值之差。由于当前背景图为实时构建得到，且构建得到的当前背景图与对应的帧图像具有相同的分辨率和大小，因此可以用当前帧图像减去当前背景图，得到对应的差值图，在该差值图中，每个差值像素点的值为当前背景图与当前帧图像中对应像素点对的差值。
在差值图中，差值的绝对值越大，说明该像素点像素值的变化越大，而对于作为背景的背景像素点来说，变化一般很小。因此，可以根据当前背景图与当前帧图像之间对应像素点的像素值变化，来确定当前帧图中各个像素点是处于运动状态或静止状态。
具体的，可以计算当前背景图与当前帧图像对应像素点对的差值，并判断像素点对的差值是否大于或等于预设的第二差值阈值；若当前背景图与当前帧图像对应像素点对的差值大于或等于预设的第一差值阈值，则将对应的第二掩码像素点赋值为运动掩码值；若当前背景图与当前帧图像对应像素点对的差值小于预设的第二差值阈值，则将对应的第二掩码像素点赋值为静止掩码值；基于赋值后的第二掩码像素点，得到第二运动掩码图。
与步骤102相似的，可以通过当前背景图与当前帧图像之间的差值绝对值来计算得到当前背景图与当前帧图像间的第二运动掩码图。上述第二运动掩码图可以是二值掩码图。在二值掩码图中，通过两个预设的掩码值来对运动掩码图进行表示，可以用一个掩码值代表运动状态，即运动掩码值，另一个掩码值代表静止状态，即静止掩码值。具体可以如下述式子所示：
其中，上述式中motion_mask(i,j)表示第二掩码像素点(i,j)的掩码值，frame_diff(i,j)表示像素点(i,j)在当前背景图与当前帧图像间的差值绝对值，motion_thr2表示预设的第二差值阈值，b表示运动掩码值，c表示静止掩码值。第二运动掩码图motion_mask2记录了当前背景图与当前帧图像间之间对应像素点的运动变化。比如，假设第二差值阈值motion_thr2设置为30，第一运动掩码图motion_mask2＝motion_mask(|K2|)，K2为当前背景图与当前帧图像间差值图，假设K2如下式所示：
则有第二运动掩码图motion_mask如下所示：
在一种可能的实施例中，为了增加第二运动掩码图的可视性，上述运动掩码值可以是255，将运动掩码值255以像素255形式进行输出的可视结果为白色，上述的静止掩码值可以是0，将静止掩码值0以像素0形式进行输出的可视结果为黑色。
105、基于第二运动掩码图，对高空抛物进行监测。
在本发明实施例中，第二运动掩码图中包括运动掩码值与静止掩码值，可以理解的是，在没有高空抛物的情况下，第二运动掩码图中只存在静止掩码值，当第二运动掩码图中出现一定面积的运动掩码值，则说明当前帧图像中有运动的物体，存在高空抛物行为的嫌疑。
同时，第二运动掩码图中包括运动掩码值与静止掩码值，通过对第二运动掩码图进行可视化，可以使工作人员更容易看出运动物体，方便工作人员对监测区域的监控。比如，白色大楼有一个白色物体抛出，通过肉眼对监控视频进行观察，很难观察出来。而通过第二运动掩码图，则可以将白色物体通过掩码值与白色大楼背景进行分开，增加了辨识度。以运动掩码值为255，静止掩码值为0进行举例，运动掩码值255以像素255形式进行输出的可视结果为白色，静止掩码值0以像素0形式进行输出的可视结果为黑色；此时，白色大楼为静止的背景，在第二掩码图可视化后为黑色，而白色物体为运动对象，在第二掩码图可视化后为白色。
在当前帧图像对应的第二运动掩码图中，若检测出大面积的运动掩码值，则说明存在运动物体，可以对运动物体进行运动分析，判断运动物体的运动方向，若运动物体的运动方向是向下的，则可以认为检测到高空抛物的行为。通过自动检测第二运动掩码图中运动掩码值出现的情况，对监测区域自动进行监控与检测，可以减少监测工作量，降低人力成本。另外，由于第二运动掩码图是随着当前帧图像进行更新的，所以第二运动掩码图具有实时性，满足视频监控的实时性。
本发明实施例中，获取当前监测场景的帧图像序列；通过帧图像之间的差值，计算得到相邻帧图像间的第一运动掩码图；根据所述第一运动掩码图以及与所述第一运动掩码图对应的帧图像，构建当前背景图；通过当前背景图与当前帧图像的差值，计算所述当前背景图与当前帧图像之间的第二运动掩码图；基于所述第二运动掩码图，对所述高空抛物进行监测。通过相邻帧图像间的第一运动掩码图，构建得到的当前背景图为实时的背景图，可以提高实时监测的效果，同时，通过当前背景图以及当前帧图像得到第二掩码图对高空抛物进行监测，降低了数值的复杂性，提高计算速度，从而可以实时的判断是否存在高空抛物的情况，从而进一步提高了高空抛物的监测效果。
可选的，请参见图2，图2是本发明实施例提供的一种背景图构建方法的流程图，如图2所示，包括以下步骤：
201、对第一运动掩码图进行分块，得到多个掩码区域。
在本发明实施例中，可以根据预先设定的规则对第一运动掩码图进行分块，也可以根据图像识别来对第一运动掩码图进行分块。
具体的，上述预先设定的规则可以是平均划分，具体可以是将第一运动掩码图分成M*N个相同大小的块，每个块对应一个掩码区域，从而得到M*N个掩码区域，每个掩码区域的大小为：(img_width/M，img_height/N),其中，上述img_width为第一运动掩码图的总宽度，img_height为第一运动掩码图的总高度。
202、根据掩码区域以及与第一运动掩码图对应的帧图像，构建形状与掩码区域相同的多个背景区域。
在本发明实施例中，可以根据掩码区域的掩码值来计算掩码区域的运动状态；再根据掩码区域的运动状态，选取与第一运动掩码图对应的帧图像中对应的图像区域作为背景区域。
具体的，上述掩码区域包括运动掩码值与静止掩码值，可以根据运动掩码值与静止掩码值判断一个掩码区域的运动状态，在一个掩码区域的运动状态被判断为静止时，则可以在与第一运动掩码图对应的帧图像中取出与该掩码区域对应的图像区域作为背景区域。当掩码区域的运动状态被判断为运动时，则对下一帧第一运动掩码图对应的掩码区域再进行运动状态的判断。可以是根据掩码区域运动掩码值与静止掩码值的平均值来判断该个掩码区域的运动状态，也可以是根据掩码区域运动掩码值的数量或占比来判断该掩码区域的运动状态。
进一步的，本发明实施例中，第一运动掩码图的数量为n帧，与第一运动 掩码图对应的帧图像的数量也为n帧，其中，n为大于0的正整数，可以根据n帧第一运动掩码图，提取第一运动掩码图中各个掩码区域的掩码值序列，掩码值序列的维度为n；根据掩码区域对应的掩码值序列，提取对应的目标图像区域索引；根据述与第一运动掩码图对应的帧图像以及所述目标图像区域索引，构建对应的背景区域。
更进一步的，帧图像序列中包括n+1个帧图像，对应可以计算得到n个第一运动掩码图，第n个第一运动掩码图与第n+1个帧图像对应，每个第一运动掩码图包括M*N个掩码区域。可以将最新的n个第一运动掩码图放入一个数组motion_value_array(n，N，M)，将最新的n个帧图像放入另一个数组frame_patch_array(n,height,width)。每个数组的存入数量为n，超过n则将最早加入数组的第一运动掩码图或帧图像进行删除，在一种可能的实施例中，帧图像对应的也包括M*N个图像区域，每个图像区域与一个掩码区域对应，可以将最新的n个帧图像放入另一个数组frame_patch_array(n，N，M)。
掩码区域的运动状态可以通过下式进行计算：
其中，式中的motion_value(pi,pj)表示掩码区域(pi,pj)的运动状态，M*N表示掩码区域数量，img_width为第一运动掩码图的总宽度，img_height为第一运动掩码图的总高度，img_height*img_width表示第一运动掩码图的像素面积，(i,j)表示掩码区域(pi,pj)中的第一掩码像素点，patch(pi,pj)(i,j)表示掩码区域(pi,pj)中的第一掩码像素点(i,j)的掩码值。
对于每一个掩码区域的运动状态motion_value，存在有一个n维的数组表示此块在最近n帧的运动状态，取该数组中数值最小的索引min_index作为目标图像区域索引,取最新n个帧图像的数组frame_patch_array(n,height,width)中对应索引min_index位置块的整块图像区域，作为背景区域。比如，当M＝2，N＝2时，则第一运动掩码图包括4个掩码区域，分别为掩码区域1、掩码区域2、掩码区域3、掩码区域4，如图3所示，当第一运动掩码图数组motion_value_array(n，N，M)中，第2帧第一运动掩码图的掩码区域1数值最小，则第2帧第一运动掩码图的掩码区域1被判断为静止状态，则对应得到索引min_index为(第2帧，掩码区域1)，则根据该索引(第2帧，掩码区域1)到帧图像的数组frame_patch_array(n,height,width)中查找第2帧的帧图像，并将第2帧的帧图像中与掩码区域1对应的区域作为背景区域1； 当第一运动掩码图数组motion_value_array(n，N，M)中，第4帧第一运动掩码图的掩码区域2数值最小，则第4帧第一运动掩码图的掩码区域2被判断为静止状态，则对应得到索引min_index为(第4帧，掩码区域2)，则根据该索引(第4帧，掩码区域2)到帧图像的数组frame_patch_array(n,height,width)中查找第4帧的帧图像，并将第4帧的帧图像中与掩码区域2对应的区域作为背景区域2；同理可以得到背景区域3和背景区域4。
203、基于背景区域，构建当前背景图。
在本发明实施例中，通过掩码区域对应计算得到的背景区域，与各个掩码区域具有对应的位置关系，将背景区域根据掩码区域在第一运动掩码图中的位置进行拼接，则可以得到当前背景图。
在一种可能的实施例中，帧图像序列参与当前背景图的实时构建，为了加快当前背景图的构建，可以对帧图像序列中帧图像的数量n进行限制，比如将设置为一个较小的数，比如20、30等。对于普通相机来说，视频流图像帧的采集约为每秒30帧，因此，n可以设置为小于30的正整数，从而提高构建当前背景图的速度，进而提高监测的实时性。
在本发明实施例中，通过对第一运动掩码图进行分块，通过分块的形式构建当前背景图，不用逐像素点进行重建计算，进一步提高了实时背景图像的构建速度。
可选的，请参见图4，图4是本发明实施例提供的另一种高空抛物的监测方法的流程图，在图1实施例的基础上，通过对第二运动掩码图进行检测，从而实现对高空抛物的监测，如图4所示，包括以下步骤：
401、监测当前帧图像对应的第二运动掩码图中是否出现面积大于预设面积阈值的第一运动区域。
在本发明实施例中，上述第一运动区域由运动掩码值构成。第一运动区域的面积大于预设面积，是可被检测的物体，是噪声干扰的概率较小。
402、若存在，则监测下一帧图像对应的第二运动掩码图中是否出现面积大于预设面积阈值的第二运动区域，并在出现第二运动区域的情况下，计算第一运动区域对应帧图像区域与第二运动区域对应帧图像区域的相似度。
在本发明实施例中，上述第二运动区域由运动掩码值构成。第二运动区域的面积大于预设面积，是可被检测的物体，是噪声干扰的概率较小。
由于第一运动区域与第二运动区域为前后帧的关系，可以通过相似度计算，判断第一运动区域与第二运动区域是否是同一个物体所对应的运动区域。
在一种可能的实施例中，上述第一运动区域与第二运动区域可以是基于运动掩码值轮廓拟合得到的框形区域。若存在多个第一运动区域和多个第二运动区域，则将多个第一运动区域所对应的帧图像区域与多个第二运动区域所对应的帧图像区域一一计算相似度，使用结构相似度ssim作为相似度计算方法，当第一运动区域所对应的图像区域与第二运动区域所对应的图像区域的结构相似度ssim大于预设的相似度阈值，比如结构相似度ssim大于0.8，则可以判定第一运动区域与第二运动区域为同一个物体的运动区域。需要说明的是，第一运动区域所对应的帧图像区域为当前帧图像中的图像区域，则第二运动区域所对应的帧图像区域为下一帧图像中的图像区域。
403、若第一运动区域对应帧图像区域与第二运动区域对应帧图像区域的相似度大于预设的相似度阈值，则计算第一运动区域与第二运动区域的运动轨迹。
在本发明实施例中，若第一运动区域对应帧图像区域与第二运动区域对应帧图像区域的相似度大于预设的相似度阈值，则说明第一运动区域中的物体与第二运动区域中的物体为同一物体，该物体正处于运动状态中，根据该物体运动轨迹，则可以判断该物体是否为高空抛物。
具体的，可以计算第一运动区域中心位置与第二运动区域中心位置的矢量距离作为运动轨迹。
404、判断运动轨迹是否符合高空抛物的条件。
在本发明实施例中，若运动轨迹(可以是矢量距离)的方向向下，运动轨迹(可以是矢量距离)的数值大于预设数值，则可以确定为高空抛物，其他情况则可以确定没有发生高空抛物。
在一些可能的实施例中，也可以计算第一运动区域中心位置的竖直坐标cur_roi_y减去第二运动区域中心位置的竖直坐标next_roi_y所得到的差值，若该差值为正，且数值大于预设数值，则可以确定为高空抛物，其他情况则可以确定没有发生高空抛物；还可以判断next_roi_y是否大于acur_roi_y，其中a大于1，比如判断next_roi_y是否大于1.2acur_roi_y，若大于，则可以确定为高空抛物，其他情况则可以确定没有发生高空抛物。
可选的，在步骤401与步骤402之前，可以先对第二运动掩码图进行预处理，消除第二运动掩码图中的噪声干扰，从而得到更加准确清晰的第二运动掩码图作为第三运动掩码图。这样，可以基于第三运动掩码图对高空抛物进行监测，使得高空抛物的检测更为准确。基于第三运动掩码图对高空抛物进行监测 可以参照步骤401与402。
可选的，在监测到高空抛物发生时，可以获取预设数量的前后帧图像作为取证帧图像序列。比如，将发生时刻的前100帧和后100帧进行保留，作为证据图像序列，再进行人工确认，找到是哪一户居民制造的高空抛物，进行后续处理，比如警告，处罚以及追责等。
可选的，在监测到高空抛物发生时，可以自动向当前监测场景和/或管理部门发出高空抛物的提示警报。
上述的当前监测场景指的是对应摄像头部署的所在地场景，比如A居民区B栋C单元。当对A居民区B栋C单元的摄像头检测到高空抛物时，则会在A居民区B栋C单元处发出危险警报，以向在A居民区B栋C单元附近的人员进行提示。
上述的管理部门可以是物业管理部门或城市管理部门或其他具有管理权限的组织，其他具有管理权限的组织比如业主委员会、游园同好会等。在一种可能的实施方式中，在发送到管理部门的提示警报中，还包括当前监测场景的视频信息，该视频信息中包括高空抛物发生的连续帧图像，可以通过各种联系方式将该警报信息发送到管理部门或相关人员的联系终端，比如邮件、手机APP或微信公众号推送等。
请参见图5，图5是本发明实施例提供的一种高空抛物的监测装置的结构示意图，如图5所示，所述装置包括：
第一获取模块501，用于获取当前监测场景的帧图像序列；
第一计算模块502，用于通过帧图像之间的差值，计算得到相邻帧图像间的第一运动掩码图；
背景构建模块503，用于根据所述第一运动掩码图以及与所述第一运动掩码图对应的帧图像，构建当前背景图；
第二计算模块504，用于通过当前背景图与当前帧图像的差值，计算所述当前背景图与当前帧图像之间的第二运动掩码图；
监测模块505，用于基于所述第二运动掩码图，对所述高空抛物进行监测。
可选的，如图6所示，所述第一运动掩码图包括运动掩码值与静止掩码值，所述第一计算模块502，包括：
第一计算子模块5021，用于计算相邻帧图像间对应像素点对的差值，并判断所述相邻帧图像间对应像素点对的差值是否大于或等于预设的第一差值阈值；
第一掩码子模块5022，用于若所述相邻帧图像间对应像素点对的差值大于或等于预设的第一差值阈值，则将对应的第一掩码像素点赋值为运动掩码值；
第二掩码子模块，用于若所述相邻帧图像间对应像素点对的差值小于预设的第一差值阈值5023，则将对应的第一掩码像素点赋值为静止掩码值；
第一处理子模块5024，用于基于赋值后的第一掩码像素点，得到第一运动掩码图。
可选的，如图7所示，所述背景构建模块503，包括：
分块子模块5031，用于对所述第一运动掩码图进行分块，得到多个掩码区域；
第一构建子模块5032，用于根据所述掩码区域以及与所述第一运动掩码图对应的帧图像，构建形状与掩码区域相同的多个背景区域；
第二构建子模块5033，基于所述背景区域，构建当前背景图。
可选的，如图8所示，所述掩码区域包括运动掩码值以及静止掩码值，所述第一构建子模块5032，包括：
计算单元50321，用于根据所述掩码区域的运动掩码值以及静止掩码值，计算所述掩码区域的运动状态；
选取单元50322，用于根据所述掩码区域的运动状态，选取与所述第一运动掩码图对应的帧图像中对应的图像区域作为背景区域。
可选的，如图9所示，所述第一运动掩码图的数量为n帧，与所述第一运动掩码图对应的帧图像的数量也为n帧，其中，n为大于0的正整数，所述计算单元50321，包括：
第一提取子单元503211，用于根据n帧第一运动掩码图，提取第一运动掩码图中各个掩码区域的掩码值序列，所述掩码值序列的维度为n；
第二提取子单元503212，用于根据掩码区域对应的掩码值序列，提取对应的目标图像区域索引；
构建子单元503213，用于基于所述与所述第一运动掩码图对应的帧图像以及所述目标图像区域索引，构建当前背景图。
可选的，如图10所示，所述第二计算模块504，包括：
第二计算子模块5041，用于计算所述当前背景图与当前帧图像对应像素点对的差值，并判断所述像素点对的差值是否大于或等于预设的第二差值阈值；
第三掩码子模块5042，用于若所述当前背景图与当前帧图像对应像素点对的差值大于或等于预设的第一差值阈值，则将对应的第二掩码像素点赋值为运动掩码值；
第四掩码子模块5043，用于若所述当前背景图与当前帧图像对应像素点对的差值小于预设的第二差值阈值，则将对应的第二掩码像素点赋值为静止掩码值；
第二处理子模块5044，用于基于赋值后的第二掩码像素点，得到第二运动掩码图。
可选的，如图11所示，所述监测模块505，包括：
第一监测子模块5051，用于监测当前帧图像对应的第二运动掩码图中是否出现面积大于预设面积阈值的第一运动区域，所述第一运动区域由运动掩码值构成；
第三计算子模块5052，用于若存在，则监测下一帧图像对应的第二运动掩码图中是否出现面积大于预设面积阈值的第二运动区域，并在出现第二运动区域的情况下，计算所述第一运动区域对应帧图像区域与所述第二运动区域对应帧图像区域的相似度，所述第二运动区域由运动掩码值构成；
第四计算子模块5053，用于若所述第一运动区域对应帧图像区域与所述第二运动区域对应帧图像区域的相似度大于预设的相似度阈值，则计算所述第一运动区域与所述第二运动区域的运动轨迹；
判断子模块5054，用于判断所述运动轨迹是否符合高空抛物的条件。
可选的，如图12所示，所述监测模块505，还包括：
预处理子模块5055，用于通过形态学开运算，去除所述第二运动掩码图中的干扰信息，得到第三运动掩码图，所述第三运动掩码图包括运动掩码值以及静止掩码值；
第二监测子模块5056，用于基于所述第三运动掩码图，对所述高空抛物进行监测。
可选的，如图13所示，所述装置还包括：
第二获取模块506，用于在监测到高空抛物发生时，获取预设数量的前后帧图像作为取证帧图像序列。
需要说明的是，本发明实施例提供的高空抛物的监测装置可以应用于可以进行高空抛物的监测的手机、监控器、计算机、服务器等设备。
本发明实施例提供的高空抛物的监测装置能够实现上述方法实施例中高 空抛物的监测方法实现的各个过程，且可以达到相同的有益效果。为避免重复，这里不再赘述。
参见图14，图14是本发明实施例提供的一种电子设备的结构示意图，如图14所示，包括：存储器1402、处理器1401及存储在所述存储器1402上并可在所述处理器1401上运行的计算机程序，其中：
处理器1401用于调用存储器1402存储的计算机程序，执行如下步骤：
获取当前监测场景的帧图像序列；
通过帧图像之间的差值，计算得到相邻帧图像间的第一运动掩码图；
根据所述第一运动掩码图以及与所述第一运动掩码图对应的帧图像，构建当前背景图；
通过当前背景图与当前帧图像的差值，计算所述当前背景图与当前帧图像之间的第二运动掩码图；
基于所述第二运动掩码图，对所述高空抛物进行监测。
可选的，所述第一运动掩码图包括运动掩码值与静止掩码值，处理器1401执行的所述通过帧图像之间的差值，计算得到相邻帧图像间的第一运动掩码图，包括：
计算相邻帧图像间对应像素点对的差值，并判断所述相邻帧图像间对应像素点对的差值是否大于或等于预设的第一差值阈值；
若所述相邻帧图像间对应像素点对的差值大于或等于预设的第一差值阈值，则将对应的第一掩码像素点赋值为运动掩码值；
若所述相邻帧图像间对应像素点对的差值小于预设的第一差值阈值，则将对应的第一掩码像素点赋值为静止掩码值；
基于赋值后的第一掩码像素点，得到第一运动掩码图。
可选的，处理器1401执行的所述根据所述第一运动掩码图以及与所述第一运动掩码图对应的帧图像，构建当前背景图，包括：
对所述第一运动掩码图进行分块，得到多个掩码区域；
根据所述掩码区域以及与所述第一运动掩码图对应的帧图像，构建形状与掩码区域相同的多个背景区域；
基于所述背景区域，构建当前背景图。
可选的，处理器1401执行的所述根据所述掩码区域以及与所述第一运动掩码图对应的帧图像，构建形状与掩码区域相同的多个背景区域，包括：
根据所述掩码区域的掩码值，计算所述掩码区域的运动状态；
根据所述掩码区域的运动状态，选取与所述第一运动掩码图对应的帧图像中对应的图像区域作为背景区域。
可选的，所述第一运动掩码图的数量为n帧，与所述第一运动掩码图对应的帧图像的数量也为n帧，其中，n为大于0的正整数，处理器1401执行的所述根据所述掩码区域的运动掩码值以及静止掩码值，计算所述掩码区域的运动状态，包括：
根据n帧第一运动掩码图，提取第一运动掩码图中各个掩码区域的掩码值序列，所述掩码值序列的维度为n；
根据掩码区域对应的掩码值序列，提取对应的目标图像区域索引；
基于所述与所述第一运动掩码图对应的帧图像以及所述目标图像区域索引，构建当前背景图。
可选的，处理器1401执行的所述通过当前背景图与当前帧图像的差值，计算所述当前背景图与当前帧图像之间的第二运动掩码图，包括：
计算所述当前背景图与当前帧图像对应像素点对的差值，并判断所述像素点对的差值是否大于或等于预设的第二差值阈值；
若所述当前背景图与当前帧图像对应像素点对的差值大于或等于预设的第一差值阈值，则将对应的第二掩码像素点赋值为运动掩码值；
若所述当前背景图与当前帧图像对应像素点对的差值小于预设的第二差值阈值，则将对应的第二掩码像素点赋值为静止掩码值；
基于赋值后的第二掩码像素点，得到第二运动掩码图。
可选的，处理器1401执行的所述基于所述第二运动掩码图，对所述高空抛物进行监测，包括：
监测当前帧图像对应的第二运动掩码图中是否出现面积大于预设面积阈值的第一运动区域，所述第一运动区域由运动掩码值构成；
若存在，则监测下一帧图像对应的第二运动掩码图中是否出现面积大于预设面积阈值的第二运动区域，并在出现第二运动区域的情况下，计算所述第一运动区域对应帧图像区域与所述第二运动区域对应帧图像区域的相似度，所述第二运动区域由运动掩码值构成；
若所述第一运动区域对应帧图像区域与所述第二运动区域对应帧图像区域的相似度大于预设的相似度阈值，则计算所述第一运动区域与所述第二运动区域的运动轨迹；
判断所述运动轨迹是否符合高空抛物的条件。
可选的，处理器1401执行的所述基于所述第二运动掩码图，对所述高空抛物进行监测，包括：
通过形态学开运算，去除所述第二运动掩码图中的干扰信息，得到第三运动掩码图，所述第三运动掩码图包括运动掩码值以及静止掩码值；
基于所述第三运动掩码图，对所述高空抛物进行监测。
可选的，处理器1401还执行包括：
在监测到高空抛物发生时，获取预设数量的前后帧图像作为取证帧图像序列。
需要说明的是，上述电子设备可以是可以应用于可以进行高空抛物的监测的手机、监控器、计算机、服务器等设备。
本发明实施例提供的电子设备能够实现上述方法实施例中高空抛物的监测方法实现的各个过程，且可以达到相同的有益效果，为避免重复，这里不再赘述。
本发明实施例还提供一种计算机可读存储介质，计算机可读存储介质上存储有计算机程序，该计算机程序被处理器执行时实现本发明实施例提供的高空抛物的监测方法的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
Claims (12)
- 一种高空抛物的监测方法，其特征在于，包括以下步骤：获取当前监测场景的帧图像序列；通过帧图像之间的差值，计算得到相邻帧图像间的第一运动掩码图；根据所述第一运动掩码图以及与所述第一运动掩码图对应的帧图像，构建当前背景图；通过当前背景图与当前帧图像的差值，计算所述当前背景图与当前帧图像之间的第二运动掩码图；基于所述第二运动掩码图，对所述高空抛物进行监测。
- 如权利要求1所述的方法，其特征在于，所述第一运动掩码图包括运动掩码值与静止掩码值，所述通过帧图像之间的差值，计算得到相邻帧图像间的第一运动掩码图，包括：计算相邻帧图像间对应像素点对的差值，并判断所述相邻帧图像间对应像素点对的差值是否大于或等于预设的第一差值阈值；若所述相邻帧图像间对应像素点对的差值大于或等于预设的第一差值阈值，则将对应的第一掩码像素点赋值为运动掩码值；若所述相邻帧图像间对应像素点对的差值小于预设的第一差值阈值，则将对应的第一掩码像素点赋值为静止掩码值；基于赋值后的第一掩码像素点，得到第一运动掩码图。
- 如权利要求1所述的方法，其特征在于，所述根据所述第一运动掩码图以及与所述第一运动掩码图对应的帧图像，构建当前背景图，包括：对所述第一运动掩码图进行分块，得到多个掩码区域；根据所述掩码区域以及与所述第一运动掩码图对应的帧图像，构建形状与掩码区域相同的多个背景区域；基于所述背景区域，构建当前背景图。
- 如权利要求3所述的方法，其特征在于，所述掩码区域包括运动掩码值以及静止掩码值，所述根据所述掩码区域以及与所述第一运动掩码图对应的帧图像，构建形状与掩码区域相同的多个背景区域，包括：根据所述掩码区域的运动掩码值以及静止掩码值，计算所述掩码区域的运动状态；根据所述掩码区域的运动状态，选取与所述第一运动掩码图对应的帧图像中对应的图像区域作为背景区域。
- 如权利要求4所述的方法，其特征在于，所述第一运动掩码图的数量为 n帧，与所述第一运动掩码图对应的帧图像的数量也为n帧，其中，n为大于0的正整数，所述根据所述掩码区域的运动掩码值以及静止掩码值，计算所述掩码区域的运动状态，包括：根据n帧第一运动掩码图，提取第一运动掩码图中各个掩码区域的掩码值序列，所述掩码值序列的维度为n；根据掩码区域对应的掩码值序列，提取对应的目标图像区域索引；基于所述与所述第一运动掩码图对应的帧图像以及所述目标图像区域索引，构建当前背景图。
- 如权利要求1所述的方法，其特征在于，所述通过当前背景图与当前帧图像的差值，计算所述当前背景图与当前帧图像之间的第二运动掩码图，包括：计算所述当前背景图与当前帧图像对应像素点对的差值，并判断所述像素点对的差值是否大于或等于预设的第二差值阈值；若所述当前背景图与当前帧图像对应像素点对的差值大于或等于预设的第一差值阈值，则将对应的第二掩码像素点赋值为运动掩码值；若所述当前背景图与当前帧图像对应像素点对的差值小于预设的第二差值阈值，则将对应的第二掩码像素点赋值为静止掩码值；基于赋值后的第二掩码像素点，得到第二运动掩码图。
- 如权利要求1所述的方法，其特征在于，所述基于所述第二运动掩码图，对所述高空抛物进行监测，包括：监测当前帧图像对应的第二运动掩码图中是否出现面积大于预设面积阈值的第一运动区域，所述第一运动区域由运动掩码值构成；若存在，则监测下一帧图像对应的第二运动掩码图中是否出现面积大于预设面积阈值的第二运动区域，并在出现第二运动区域的情况下，计算所述第一运动区域对应帧图像区域与所述第二运动区域对应帧图像区域的相似度，所述第二运动区域由运动掩码值构成；若所述第一运动区域对应帧图像区域与所述第二运动区域对应帧图像区域的相似度大于预设的相似度阈值，则计算所述第一运动区域与所述第二运动区域的运动轨迹；判断所述运动轨迹是否符合高空抛物的条件。
- 如权利要求1所述的方法，其特征在于，所述基于所述第二运动掩码图，对所述高空抛物进行监测，包括：通过形态学开运算，去除所述第二运动掩码图中的干扰信息，得到第三运动掩码图，所述第三运动掩码图包括运动掩码值以及静止掩码值；基于所述第三运动掩码图，对所述高空抛物进行监测。
- 如权利要求1至8任一所述的方法，其特征在于，所述方法还包括：在监测到高空抛物发生时，获取预设数量的前后帧图像作为取证帧图像序列。
- 一种高空抛物的监测装置，其特征在于，所述装置包括：第一获取模块，用于获取当前监测场景的帧图像序列；第一计算模块，用于通过帧图像之间的差值，计算得到相邻帧图像间的第一运动掩码图；背景构建模块，用于根据所述第一运动掩码图以及与所述第一运动掩码图对应的帧图像，构建当前背景图；第二计算模块，用于通过当前背景图与当前帧图像的差值，计算所述当前背景图与当前帧图像之间的第二运动掩码图；监测模块，用于基于所述第二运动掩码图，对所述高空抛物进行监测。
- 一种电子设备，其特征在于，包括：存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现如权利要求1至9中任一项所述的高空抛物的监测方法中的步骤。
- 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质上存储有计算机程序，所述计算机程序被处理器执行时实现如权利要求1至9中任一项所述的高空抛物的监测方法中的步骤。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011627329.2 | 2020-12-30 | ||
| CN202011627329.2A CN112800846A (zh) | 2020-12-30 | 2020-12-30 | 高空抛物的监测方法、装置、电子设备及存储介质 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2022142414A1 true WO2022142414A1 (zh) | 2022-07-07 |
Family
ID=75807876
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2021/114803 WO2022142414A1 (zh) | 2020-12-30 | 2021-08-26 | 高空抛物的监测方法、装置、电子设备及存储介质 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN112800846A (zh) |
| WO (1) | WO2022142414A1 (zh) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117423058A (zh) * | 2023-11-02 | 2024-01-19 | 江苏三棱智慧物联发展股份有限公司 | 基于城市安全眼的高空抛物检测系统及方法 |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112800846A (zh) * | 2020-12-30 | 2021-05-14 | 深圳云天励飞技术股份有限公司 | 高空抛物的监测方法、装置、电子设备及存储介质 |
| CN113297949B (zh) * | 2021-05-20 | 2024-02-20 | 科大讯飞股份有限公司 | 高空抛物检测方法、装置、计算机设备和存储介质 |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2430830A (en) * | 2005-09-28 | 2007-04-04 | Univ Dundee | Image sequence movement analysis system using object model, likelihood sampling and scoring |
| CN110647822A (zh) * | 2019-08-30 | 2020-01-03 | 重庆博拉智略科技有限公司 | 高空抛物行为识别方法、装置、存储介质及电子设备 |
| CN111476163A (zh) * | 2020-04-07 | 2020-07-31 | 浙江大华技术股份有限公司 | 一种高空抛物监控方法、装置以及计算机存储介质 |
| CN111723654A (zh) * | 2020-05-12 | 2020-09-29 | 中国电子系统技术有限公司 | 基于背景建模、YOLOv3与自优化的高空抛物检测方法及装置 |
| CN112800846A (zh) * | 2020-12-30 | 2021-05-14 | 深圳云天励飞技术股份有限公司 | 高空抛物的监测方法、装置、电子设备及存储介质 |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107330916A (zh) * | 2017-06-15 | 2017-11-07 | 精伦电子股份有限公司 | 一种运动物体检测方法及系统 |
| CN111079663B (zh) * | 2019-12-19 | 2022-01-11 | 深圳云天励飞技术股份有限公司 | 高空抛物的监测方法、装置、电子设备及存储介质 |
| CN111260684A (zh) * | 2020-03-02 | 2020-06-09 | 成都信息工程大学 | 基于帧差法和背景差分法结合的前景像素提取方法及系统 |
| CN112016414A (zh) * | 2020-08-14 | 2020-12-01 | 熵康(深圳)科技有限公司 | 一种检测高空抛物事件的方法、装置及楼面智能监控系统 |
-
2020
- 2020-12-30 CN CN202011627329.2A patent/CN112800846A/zh active Pending
-
2021
- 2021-08-26 WO PCT/CN2021/114803 patent/WO2022142414A1/zh active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2430830A (en) * | 2005-09-28 | 2007-04-04 | Univ Dundee | Image sequence movement analysis system using object model, likelihood sampling and scoring |
| CN110647822A (zh) * | 2019-08-30 | 2020-01-03 | 重庆博拉智略科技有限公司 | 高空抛物行为识别方法、装置、存储介质及电子设备 |
| CN111476163A (zh) * | 2020-04-07 | 2020-07-31 | 浙江大华技术股份有限公司 | 一种高空抛物监控方法、装置以及计算机存储介质 |
| CN111723654A (zh) * | 2020-05-12 | 2020-09-29 | 中国电子系统技术有限公司 | 基于背景建模、YOLOv3与自优化的高空抛物检测方法及装置 |
| CN112800846A (zh) * | 2020-12-30 | 2021-05-14 | 深圳云天励飞技术股份有限公司 | 高空抛物的监测方法、装置、电子设备及存储介质 |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117423058A (zh) * | 2023-11-02 | 2024-01-19 | 江苏三棱智慧物联发展股份有限公司 | 基于城市安全眼的高空抛物检测系统及方法 |
| CN117423058B (zh) * | 2023-11-02 | 2024-05-03 | 江苏三棱智慧物联发展股份有限公司 | 基于城市安全眼的高空抛物检测系统及方法 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112800846A (zh) | 2021-05-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2022142414A1 (zh) | 高空抛物的监测方法、装置、电子设备及存储介质 | |
| US11532156B2 (en) | Methods and systems for fire detection | |
| WO2021120867A1 (zh) | 高空抛物的监测方法、装置、电子设备及存储介质 | |
| US20190228233A1 (en) | Video tracking systems and methods employing cognitive vision | |
| CN109299703B (zh) | 对鼠情进行统计的方法、装置以及图像采集设备 | |
| US20060056702A1 (en) | Image processing apparatus and image processing method | |
| US20230005176A1 (en) | Throwing position acquisition method and apparatus, computer device and storage medium | |
| CN104966304A (zh) | 基于卡尔曼滤波与非参数背景模型的多目标检测跟踪方法 | |
| GB2443739A (en) | Detecting image regions of salient motion | |
| CN104820995A (zh) | 一种面向大型公共场所的人流密度监测预警方法 | |
| Kumar et al. | Real time target tracking with pan tilt zoom camera | |
| JPH0844874A (ja) | 画像変化検出装置 | |
| CN108460319B (zh) | 异常人脸检测方法及装置 | |
| JP2010015469A (ja) | 静止領域検出方法とその装置、プログラム及び記録媒体 | |
| CN111401239A (zh) | 一种视频分析方法、装置、系统、设备及存储介质 | |
| CN113920585A (zh) | 行为识别方法及装置、设备和存储介质 | |
| CN113505643A (zh) | 违章目标物检测方法及相关装置 | |
| JP2012212238A (ja) | 物品検出装置および静止人物検出装置 | |
| JP2023036054A (ja) | 目標カウント方法、装置、デバイス及び記憶媒体 | |
| CN113947795B (zh) | 口罩佩戴检测方法、装置、设备及存储介质 | |
| CN112232113B (zh) | 人员识别方法、人员识别装置、存储介质与电子设备 | |
| KR20230039468A (ko) | 영상의 객체 간 상호작용행위 검출 장치 및 방법 | |
| CN111091024B (zh) | 一种基于视频识别结果的小目标过滤方法和系统 | |
| KR20030018487A (ko) | 영상을 이용한 출입 이동 물체 계수 방법 및 장치 | |
| CN114699702A (zh) | 消防设备检测方法及相关装置 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21913161 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 21913161 Country of ref document: EP Kind code of ref document: A1 |