CN112365524A

CN112365524A - High-altitude parabolic real-time alarm system based on time sequence images

Info

Publication number: CN112365524A
Application number: CN202011248813.4A
Authority: CN
Inventors: 邬松渊; 许济海; 赵捷
Original assignee: Ningbo Boden Intelligent Technology Co ltd
Current assignee: Ningbo Boden Intelligent Technology Co ltd
Priority date: 2020-11-10
Filing date: 2020-11-10
Publication date: 2021-02-12
Anticipated expiration: 2040-11-10
Also published as: CN112365524B

Abstract

The invention discloses a high-altitude parabolic real-time alarm system based on time sequence images, which relates to the technical field of high-altitude security protection of middle and high-rise buildings. The invention has the advantages of high alarm detection rate, high alarm response speed, wide detection range of the network camera, small using quantity, good privacy protection, good expansibility and very convenient operation.

Description

High-altitude parabolic real-time alarm system based on time sequence images

Technical Field

The invention relates to the technical field of high-altitude security of middle and high-rise buildings, in particular to a high-altitude parabolic real-time alarm system based on time sequence images.

Background

With the rapid growth of the population and the increase of the economic level, the height of buildings is also higher and higher. Homes are also transitioning from multiple stories to medium to high rise. With the increasing number of middle and high-rise residences, the high-altitude parabolic events are also gradually increased.

The high-altitude parabolic model brings great potential safety hazards to low-rise residents and pavement personnel, and regarding the problem at present, the main method in community management is to post warning slogans and oral propaganda education, so that the awareness of people on the high-altitude parabolic hazard is improved, and the occurrence of the event is reduced. In the law and regulation, the eighty-seven rules of the infringement liability act specify that the owners of the whole building undertake compensation together in the case that no thrower can be found. At present, the law responsibility of high-altitude parabolic is further stipulated by the civil infringement responsibility compilation, a term of 'the building user can recoup after reimbursement' is newly added, and the condition that a management and management organization needs to fulfill the obligation of precaution, the obligation of investigation under the authorities such as public security and the like is clarified.

However, the related measures cannot completely solve the problems of falling objects and throwing objects from the source, and corresponding injury cases still occur. When public property and personal safety are damaged, the law responsibility needs to be traced to a troublemaker, however, due to the fact that the floor is too high, the time difference exists when an object is thrown to the ground, the falling speed of the object is high, and the like, the throwing behavior is rarely witnessed by witnessers, and therefore people in charge are difficult to trace by related departments.

The ' people's daily newspaper ' talks about the ' high-altitude parabolic ' renovation, and shows that legal and technical means are indispensable. The Guangzhou city residence bureau has also issued "prevent falling objects" notifications. The notice is that in the aspect of a long-acting mechanism, a monitoring camera can be additionally arranged, so that the outside of a building is brought into a monitoring range, and the specific position of a high-altitude object and related responsible persons can be locked.

The scheme of erecting the upward camera can solve the problem of lack of witnesses, and helps to accurately position the parabolic source floor and house number. However, new problems are also brought about:

1. when the accurate time of the high-altitude parabolic event is not determined, a manager needs to check a historical video, particularly when the drop object size is small, the time of 1-2 seconds appears in the video, the video needs to be checked very carefully, and sometimes, the floor can be determined even after the video is checked repeatedly for several times;

2. timely alarming of the high-altitude parabolic event cannot be obtained, and tracing the parabolic event at unknown time consumes time and labor;

3. the camera has small installation elevation angle and small coverage area. Because the camera is installed right above the floor, if the installation elevation angle is small and the installation distance far enough away from the building is not available, the privacy of residents is easily invaded, and the coverage surface of the camera on the floor is small.

For the defects, a scholars provides a high-altitude parabolic evidence obtaining system based on digital image processing, a data processing module of the system adopts a frame difference method to extract foreground points of a moving object, and then determines the position of the moving object in an image according to color information, so that the speed and the acceleration of the moving object are calculated, and finally a complete motion track is obtained. However, this solution has the following problems:

1. the existing high-altitude parabolic image processing evidence obtaining system extracts a motion foreground by using a frame difference method, however, a few-frame difference method is not detailed, if a traditional two-frame difference method or a three-frame difference method is simply used, the applicability to a complex environment is poor, and a foreground point can be greatly interfered by noise. Such as raindrops falling, camera shake, foliage shaking, etc.;

2. the existing high-altitude parabolic detection method generally obtains a complete falling track of an object by calculating the speed and the acceleration of the object, has high requirement on the continuity of the foreground points of the object, and cannot obtain accurate speed and acceleration information once the foreground points are interrupted in the falling process, for example, the foreground points are influenced by the change of ambient light.

Therefore, those skilled in the art are devoted to developing a high altitude parabolic real-time warning system based on time series images.

Disclosure of Invention

In view of the above defects in the prior art, the technical problems to be solved by the invention are that the coverage area of the camera is small, the privacy of residents is safe, the background modeling has the problems of foreground abrupt change and more noise interference in a complex environment, and the traditional pure camera scheme cannot give an alarm in real time.

The method comprises the steps that firstly, the problems existing in the existing camera monitoring scheme are analyzed, the camera installation and deployment scheme for detecting the high altitude parabolic object is redesigned, the camera installation distance is optimized, the upward elevation angle of the camera is adjusted, and the monitoring full coverage of the building is realized by using the cameras as few as possible; in order to extract a moving target, a self-designed band-pass filtering frame difference method is adopted, and the interference of camera shake, cloud layer movement, light change and the like can be effectively avoided aiming at a high-altitude scene; in order to alarm in real time, a high-altitude parabolic detection algorithm based on a time sequence image is designed, a second-order curve is randomly sampled and fitted through different position information of a moving target foreground point in the time dimension, the fitting score of the second-order curve is calculated according to a designed loss function, whether the alarm condition of the high-altitude parabolic is met or not is finally judged, and real-time alarm is conducted on the condition that the alarm condition is met.

The invention provides a high-altitude parabolic real-time alarm system based on time sequence images, which comprises a camera acquisition module, an algorithm detection module and a client alarm module, wherein the camera acquisition module is connected with and communicated with a local area network, monitors all-weather outer wall surfaces and outer wall sky of buildings, sends image data acquired in real time to the algorithm detection module, and is stored locally for backup; the algorithm detection module carries out modeling detection on the image data to obtain a second-order curve fitting result and a corresponding score of the falling object track, and sends alarm information and an alarm video to the client alarm module; the client alarm module realizes the operation of the user on the control of the camera acquisition module and the algorithm detection module, and checks alarm information and alarm video sent by the algorithm detection module.

Optionally, in the high altitude parabolic real-time alarm system based on the time sequence image in the above embodiment, the camera acquisition module includes a plurality of network cameras and POE switches, and POE (power Over ethernet) refers to a switch supporting power Over ethernet, and the IP-based terminal provides direct current while transmitting a data signal to an IP-based terminal (such as an IP telephone, a wireless local area network access point AP, a network camera, and the like) without any change to an existing ethernet cat.5 wiring infrastructure.

Further, in the high-altitude parabolic real-time alarm system based on the time sequence image in the embodiment, the network camera faces the outer wall of the building in front, is 5-15 meters away from the building, has an upward elevation angle of 60-80 degrees, and covers 4-33 floors of one building in two network camera detection areas, so that on one hand, the visual field of the network camera can only see the outer wall surface and the outer wall space of the building, and the problem of invading the privacy of residents is avoided; on the other hand, the network cameras with less quantity can realize the coverage of more floors, the hardware cost is reduced, and the camera management is also convenient.

Further, in the high altitude parabolic real-time alarm system based on the time sequence image in the above embodiment, the resolution of the network camera selects 500 ten thousand pixels, and the system has a starlight night vision function, supports POE network cable power supply, works outdoors, and is waterproof and sun-proof.

Further, in the high altitude parabolic real-time alarm system based on the time sequence image in the above embodiment, the POE switch connects all the network cameras, the algorithm detection module, and the client alarm module to form a local area network, and supplies power to all the network cameras.

Further, in the high-altitude parabolic real-time alarm system based on the time-series image in the above embodiment, the local area network is configured to ensure that all the network cameras, the algorithm detection module and the client alarm module are in the same network and can communicate with each other.

Optionally, in the high-altitude parabolic real-time alarm system based on the time-series image in the above embodiment, the local area network is consistent with an IP address network segment, a subnet mask and a gateway used by a cell where the user is located.

Further, in the high-altitude parabolic real-time alarm system based on the time-series image in the above embodiment, each network camera is pulled to send a video stream to the algorithm detection module through a static IP address.

Optionally, in the high-altitude parabolic real-time alarm system based on the time-series image in any of the embodiments, the detection algorithm of the algorithm detection module operates in an algorithm server, the algorithm server is an operation carrier of the high-altitude parabolic algorithm, and the calculation performance meets the requirement that multiple network cameras simultaneously detect and operate in real time.

Optionally, in the high-altitude parabolic real-time alarm system based on the time-series image in any of the embodiments above, the flow of the detection algorithm includes data acquisition, data processing, data storage, and alarm sending:

data acquisition, namely pulling a video stream with a resolution of 1920 pixels by 1080 pixels from a network camera according to an RTSP (real time streaming protocol) to obtain real-time image data for detection;

data processing, namely performing foreground detection on moving objects on real-time image data, then randomly sampling and fitting a second-order curve to foreground points, evaluating the fitting effect and finally judging whether the moving objects are thrown objects or not;

data storage, namely storing alarm video clips with falling objects and storing all-weather complete original video data;

and sending an alarm, namely sending alarm information at the moment when the object is thrown to a client alarm module, wherein the alarm information comprises the time when the object throwing event is detected, a camera static IP and an alarm video storage path.

Further, in the high altitude parabolic real-time alarm system based on time sequence images in the above embodiment, the data processing includes the following steps:

s100, foreground images are extracted, foreground images of moving targets are extracted from continuous frame images by using a band-pass filtering frame difference algorithm, wherein three continuous frame images are subjected to frame difference averaging to extract high-frequency signals, seven continuous frame images are subjected to frame difference averaging to extract low-frequency signals, and the high-frequency signals are subtracted from the low-frequency signals to obtain final foreground images. The band-pass filtering frame difference method can effectively reduce foreground noise points generated by lens shake and raindrop of the network camera, and reduce the influence of shadows generated by light change;

s200, foreground image preprocessing, namely filtering discrete noise points in the final foreground image by Gaussian filtering, morphological open operation and threshold segmentation in image processing, then counting the area of each connected domain by using connected domain operation, removing the connected domain if the area of the connected domain is larger than a threshold value, and filtering abnormal large-area connected domains in the foreground image, wherein all pixel points in the remaining connected domains are foreground points, and the foreground points are pixel points of a next frame of image in continuous images which are changed relative to a previous frame of image;

s300, performing three-point sampling by using RANSAC, and performing random three-point sampling on all foreground points along the x direction of an image and the y direction of the image respectively in a time dimension by using the RANSAC method to obtain (x, t) and (y, t) distribution;

s400, fitting a second-order curve, wherein random three-point sampling is used for fitting the second-order curve by using a RANSAC method;

s500, counting support points, wherein points close to the second-order curve are counted as support points in all foreground points;

s600, screening a second-order curve, and screening the second-order curve meeting the three conditions according to the three conditions of whether the number of support points is greater than a threshold value, whether the acceleration change range of the center point of the support point at the adjacent moment is smaller than the threshold value and whether the moment type of the support point is greater than the threshold value;

s700, judging whether the maximum sampling frequency is reached, continuing to perform the next step S800 when the three-point sampling frequency of RANSAC reaches the maximum sampling frequency, otherwise, repeating the steps S300-S700;

and S800, outputting a parabolic fitting result, wherein the fitting result is a fitting score of a second-order curve, and judging whether an alarm condition is met according to the fitting score. The fitting score is the percentage of the above support points to all foreground points.

Optionally, in the high-altitude parabolic real-time alarm system based on the time-series image in any of the embodiments above, the operation of the user on the camera acquisition module includes logging in, playing, stopping, grouping, renaming, and capturing a screen, and the operation of the user on the algorithm detection module includes turning on or off an algorithm.

The invention extracts the foreground by using a band-pass filtering frame difference algorithm, combines an innovative foreground point second-order curve fitting algorithm based on time sequence, has the alarm detection rate of over 90 percent, and supports the object detection of 8 pixels by 8 pixels with the minimum resolution; the alarm response speed is high, once a high-altitude parabolic event occurs, the alarm message can be received within one second, and the stored alarm video can be checked at the alarm module of the client for verification; the detection of a color scene in the daytime and an infrared scene at night is supported, and uninterrupted work for 24 hours all day is guaranteed; the network camera has wide detection range and small using quantity, can realize floor coverage of two units from 4 floors to 33 floors by only 2 cameras, and has good privacy protection; the invention has good expansibility, can work by supporting the access of an additional network camera to the local area network, and is very convenient to operate.

The conception, the specific structure and the technical effects of the present invention will be further described with reference to the accompanying drawings to fully understand the objects, the features and the effects of the present invention.

Drawings

FIG. 1 is a topological diagram illustrating a time-sequenced image-based high altitude parabolic real-time alert system in accordance with an exemplary embodiment;

FIG. 2 is a schematic diagram illustrating camera deployment according to an exemplary embodiment;

FIG. 3 is a flow chart illustrating a detection algorithm according to an exemplary embodiment.

Fig. 4 is a flowchart illustrating data processing according to an exemplary embodiment.

FIG. 5 is a flowchart illustrating extracting a foreground image in accordance with an illustrative embodiment;

fig. 6 is a flowchart illustrating foreground image preprocessing according to an exemplary embodiment.

Detailed Description

The technical contents of the preferred embodiments of the present invention will be more clearly and easily understood by referring to the drawings attached to the specification. The present invention may be embodied in many different forms of embodiments and the scope of the invention is not limited to the embodiments set forth herein.

In the drawings, structurally identical elements are represented by like reference numerals, and structurally or functionally similar elements are represented by like reference numerals throughout the several views. The size and thickness of each component shown in the drawings are arbitrarily illustrated, and the present invention is not limited to the size and thickness of each component. The thickness of the components is exaggerated somewhat schematically and appropriately in order to make the illustration clearer.

The inventor designs a high-altitude parabolic real-time alarm system based on time sequence images, as shown in fig. 1, the high-altitude parabolic real-time alarm system comprises a camera acquisition module, an algorithm detection module and a client alarm module, wherein the modules are connected through a local area network and communicate.

The camera acquisition module monitors all weather of the outer wall surface and the outer wall sky of the building, sends image data acquired in real time to the algorithm detection module, and simultaneously stores the image data in the local for backup; the camera acquisition module designed by the inventor comprises a plurality of network cameras and a POE (power Over Ethernet) switch, wherein the POE switch supports Ethernet power supply, and under the condition that the existing Ethernet Cat.5 wiring infrastructure is not changed, the IP-based terminal (such as an IP telephone, a wireless local area network Access Point (AP), a network camera and the like) is provided with direct current while transmitting data signals. The network cameras are arranged to face the outer wall of a building, as shown in fig. 2, the distance from the network cameras to the building is 5-15 meters, the upward elevation angle is 60-80 degrees, the two network camera detection areas cover 4-33 layers of the building, and the design is based on the fact that on one hand, the visual field of the network cameras can only see the outer wall surface and the outer wall space of the building, and the problem of invasion of resident privacy is avoided; on the other hand, the network cameras with less quantity can realize the coverage of more floors, the hardware cost is reduced, and the camera management is also convenient. The network camera preferably selects 500 ten thousand pixels in resolution, has a starlight night vision function, supports POE network cable power supply, works outdoors, and is waterproof and sun-proof; and the POE switch is connected with all the network cameras, the algorithm detection module and the client alarm module to form a local area network and supplies power to all the network cameras. The local area network is used for ensuring that all network cameras, the algorithm detection module and the client alarm module can communicate with each other in the same network, the local area network is consistent with an IP address network segment, a subnet mask and a gateway used by a cell where a user is located, and each network camera is pulled to send video streams to the algorithm detection module through a static IP address.

And the algorithm detection module carries out modeling detection on the image data to obtain a second-order curve fitting result and a corresponding score of the falling object track, and sends alarm information and an alarm video to the client alarm module. The detection algorithm of the algorithm detection module operates in an algorithm server, the algorithm server is an operation carrier of a high-altitude parabolic algorithm, and the calculation performance meets the requirements of simultaneous detection and real-time operation of multiple paths of network cameras.

The inventor designs a flow of a detection algorithm, as shown in fig. 3, which comprises data acquisition, data processing, data storage and alarm sending, and specifically comprises the following steps:

data processing, namely performing foreground detection on moving objects on real-time image data, then randomly sampling and fitting a second-order curve to foreground points, evaluating the fitting effect and finally judging whether the moving objects are thrown objects or not; the method specifically comprises the following steps as shown in fig. 4:

s100, foreground images are extracted, as shown in FIG. 5, foreground images of the moving target are extracted from continuous frame images by using a band-pass filtering frame difference algorithm, wherein frame differences of three continuous frame images are averaged to extract a high-frequency signal, frame differences of seven continuous frame images are averaged to extract a low-frequency signal, and the high-frequency signal subtracts the low-frequency signal to obtain the foreground images. The band-pass filtering frame difference method can effectively reduce foreground noise points generated by lens shake and raindrop of the network camera, and reduce the influence of shadows generated by light change;

s200, foreground image preprocessing, as shown in FIG. 6, firstly filtering some discrete noise points in the final foreground image by using Gaussian filtering, morphological open operation and threshold segmentation in image processing, then counting the area of each connected domain by using connected domain operation, if the area of each connected domain is larger than a threshold, removing the connected domain for filtering abnormal large-area connected domains in the foreground image, and taking all pixel points in the remaining connected domains as foreground points;

s300, performing three-point sampling by using RANSAC, and performing random three-point sampling on the foreground point along the x direction of the image and the y direction of the image respectively in the time dimension by using the RANSAC method to obtain (x, t) and (y, t) distribution;

and S800, outputting a parabolic fitting result, wherein the fitting result is a fitting score of a second-order curve, judging whether an alarm condition is met according to the fitting score, and the fitting score is the percentage of the support points in all foreground points.

The client alarm module realizes the operation of the user on the control of the camera acquisition module and the algorithm detection module, and checks alarm information and alarm video sent by the algorithm detection module. The operation of the user on the camera acquisition module comprises login, playing, stopping, grouping, renaming and screen capturing, and the operation of the user on the algorithm detection module comprises algorithm starting or algorithm closing.

The foregoing detailed description of the preferred embodiments of the invention has been presented. It should be understood that numerous modifications and variations could be devised by those skilled in the art in light of the present teachings without departing from the inventive concepts. Therefore, the technical solutions available to those skilled in the art through logic analysis, reasoning and limited experiments based on the prior art according to the concept of the present invention should be within the scope of protection defined by the claims.

Claims

1. A high-altitude parabolic real-time alarm system based on time sequence images is characterized by comprising a camera acquisition module, an algorithm detection module and a client alarm module, wherein the camera acquisition module, the algorithm detection module and the client alarm module are connected through a local area network and communicate with each other, the camera acquisition module monitors all weather of the outer wall surface and the sky of the outer wall of a building, sends image data acquired in real time to the algorithm detection module and is stored locally for backup; the algorithm detection module carries out modeling detection on the image data to obtain a second-order curve fitting result and a corresponding score of the trajectory of the falling object, and sends alarm information and an alarm video to the client alarm module; and the client alarm module realizes the operation of the user on the control of the camera acquisition module and the algorithm detection module, and checks the alarm information and the alarm video.

2. The time-series image-based high-altitude parabolic real-time alarm system as claimed in claim 1, wherein the camera acquisition module comprises a plurality of webcams and a POE switch.

3. The high-altitude parabolic real-time alarm system based on time sequence images as claimed in claim 2, wherein the network cameras face towards the outer wall of the building, are 5-15 meters away from the building, have upward elevation angles of 60-80 degrees, and cover 4-33 floors of one building in two network camera detection areas.

4. The high altitude parabolic real-time alarm system based on time sequence image as claimed in claim 2 or 3, characterized in that the resolution of the network camera selects 500 ten thousand pixels, and the system has starlight night vision function, supports POE network cable power supply, works outdoors and is waterproof and sun-proof.

5. The time-series image-based high-altitude parabolic real-time alarm system according to claim 2, wherein the POE switch is connected with all the webcams, the algorithm detection module and the client alarm module to form the local area network, and supplies power to all the webcams.

6. The time-series image-based high-altitude parabolic real-time alarm system according to claim 5, wherein the local area network ensures that all the network cameras, the algorithm detection module and the client alarm module can communicate with each other within the same network.

7. The high altitude parabolic real-time warning system according to claim 5, wherein the local area network is consistent with an IP address network segment, a subnet mask and a gateway used by a cell where the user is located.

8. The time-series image-based high-altitude parabolic real-time alarm system as claimed in claim 1, wherein the detection algorithm of the algorithm detection module is operated on an algorithm server, and the algorithm server is an operation carrier of the high-altitude parabolic algorithm.

9. The high altitude parabolic real-time warning system based on time series images as claimed in claim 8, wherein the flow of the detection algorithm comprises data acquisition, data processing, data storage and warning transmission.

10. The time-series image-based high altitude parabolic real-time warning system according to claim 9, wherein the data processing comprises the steps of:

s100, extracting foreground images of a moving target from continuous frame images by using a band-pass filtering frame difference algorithm, wherein frame differences of three continuous frame images are averaged to extract a high-frequency signal, frame differences of seven continuous frame images are averaged to extract a low-frequency signal, and the high-frequency signal subtracts the low-frequency signal to obtain the foreground images;

s200, foreground image preprocessing, namely filtering discrete noise points in the final foreground image by Gaussian filtering, morphological open operation and threshold segmentation in image processing, then counting the area of each connected domain by using connected domain operation, removing the connected domains if the area of each connected domain is larger than a threshold value, filtering abnormal large-area connected domains in the foreground image, and taking all pixel points in the rest connected domains as foreground points;

s600, screening a second-order curve, and screening the second-order curve meeting the three conditions according to the three conditions of whether the number of the support points is greater than a threshold value, whether the acceleration change range of the center point of the support point at the adjacent moment is smaller than the threshold value and whether the moment type of the support point is greater than the threshold value;

and S800, outputting a parabolic fitting result, wherein the fitting result is a fitting score of a second-order curve, and judging whether an alarm condition is met according to the fitting score.