CN115601684A

CN115601684A - Emergency early warning method and device, electronic equipment and storage medium

Info

Publication number: CN115601684A
Application number: CN202211392157.4A
Authority: CN
Inventors: 蒋志文; 曾彦; 彭长生; 谢莎
Original assignee: Shenzhen Jitong Intelligent Technology Co ltd
Current assignee: Shenzhen Jitong Intelligent Technology Co ltd
Priority date: 2022-11-08
Filing date: 2022-11-08
Publication date: 2023-01-13

Abstract

The invention relates to an artificial intelligence technology, and discloses an emergency early warning method, which comprises the following steps: the method comprises the steps of dividing a monitoring image into a plurality of image sub-blocks, extracting motion features, size features and texture features in the image sub-blocks respectively, constructing feature probabilities corresponding to the motion features and the size features, calculating matching correlation coefficients between the texture features and a preset feature codebook, carrying out initial abnormal behavior detection and secondary abnormal behavior detection according to the matching correlation coefficients and the feature probabilities, judging whether an emergency occurs or not by comparing a standard abnormal behavior detection result with an emergency early warning library, and carrying out early warning. The invention also provides an emergency early warning device, electronic equipment and a computer readable storage medium. The invention can solve the problem of low accuracy of the early warning of the emergency.

Description

Emergency early warning method and device, electronic equipment and storage medium

Technical Field

The invention relates to the technical field of artificial intelligence, in particular to an emergency early warning method and device, electronic equipment and a computer readable storage medium.

Background

With the continuous promotion of Chinese urbanization, the urban population grows year by year. Accordingly, there are more and more large public buildings providing various social life services to people to meet the continuously improved demands of the people on the material culture life. However, public buildings are more and more, so that the public buildings also become a place for gathering people, and in order to ensure that accidents can be early warned in time when people gather, an emergency early warning method needs to be provided.

Disclosure of Invention

The invention provides an emergency early warning method, an emergency early warning device and a computer readable storage medium, and mainly aims to solve the problem of low accuracy of emergency early warning.

In order to achieve the above object, the invention provides an emergency early warning method, which comprises the following steps:

acquiring monitoring images from video data recorded by a plurality of monitoring devices based on a preset distributed framework;

dividing the monitoring image into a plurality of image sub-blocks which accord with preset dividing conditions, and respectively extracting motion characteristics, size characteristics and texture characteristics from the image sub-blocks;

respectively constructing feature probabilities corresponding to the motion features and the size features, and calculating matching correlation coefficients between the texture features and a preset feature codebook;

constructing an abnormal behavior detection system according to a first classifier and a second classifier which are constructed in advance, and performing initial abnormal behavior detection on the feature probability and the matching correlation coefficient based on the abnormal behavior detection system to obtain an initial abnormal behavior detection result;

acquiring an abnormal behavior image corresponding to the initial abnormal behavior detection result, detecting a behavior human body in the abnormal behavior image and human body position information corresponding to the behavior human body by using a target detection algorithm, and inputting the human body position information into a symmetric space transformation network to obtain a plurality of human body targets to generate a skeleton map;

extracting a plurality of human body skeleton points in the human body target generation skeleton diagram, calculating skeleton transformation characteristic values corresponding to the human body skeleton points, and performing abnormal behavior detection again according to the skeleton transformation characteristic values to obtain a standard abnormal behavior detection result;

and comparing the standard abnormal behavior detection result with a pre-constructed emergency early warning library to judge whether an emergency occurs or not, and early warning.

Optionally, the extracting the motion feature in the image sub-block includes:

acquiring a foreground part in the image sub-blocks, and extracting a transverse optical flow corresponding to any pixel point in the foreground part in the abscissa axis direction and a longitudinal optical flow corresponding to any pixel point in the foreground part in the ordinate axis direction;

and substituting the transverse optical flow and the longitudinal optical flow into a preset motion characteristic calculation formula to obtain the motion characteristics of the image sub-blocks.

Optionally, the preset motion characteristic calculation formula is as follows:

where M is the motion characteristic in the image sub-block, N _f Is the total number of pixels of the foreground portion in the image sub-block,

representing the corresponding transverse optical flow of any pixel point in the foreground part in the direction of the abscissa axis,

and representing a longitudinal optical flow corresponding to any pixel point in the foreground part in the direction of the ordinate axis, wherein n is the nth pixel in the foreground part.

Optionally, the extracting the size features in the image sub-blocks includes:

acquiring a preset Gaussian template and a preset first parameter and a preset second parameter, and calculating the initial characteristics of the image subblocks according to the Gaussian template;

and substituting the initial characteristic and the foreground ratio obtained by calculation according to the first parameter and the second parameter into a preset size characteristic calculation formula to obtain the size characteristic in the image sub-block.

Optionally, the extracting the texture features from the image sub-blocks includes:

acquiring a plurality of preset target directions, and extracting a plurality of amplitude values in the image sub-blocks from the plurality of target directions by using a two-dimensional filter;

and summarizing the amplitude values to obtain the texture features in the image subblocks.

Optionally, the performing, by the abnormal behavior detection system, initial abnormal behavior detection on the feature probability and the matching correlation coefficient to obtain an initial abnormal behavior detection result includes:

carrying out speed judgment on the feature probability corresponding to the motion feature by using a first classifier in the abnormal behavior detection system to obtain a speed judgment result;

when the speed judgment result is that the speed is abnormal, judging the target monitoring image as an abnormal image;

and when the speed judgment result is that the speed is normal, utilizing a second classifier in the abnormal behavior detection system to judge the characteristic probability corresponding to the size characteristic and the matching correlation coefficient, and when the characteristic judgment fails, judging the target monitoring image as an abnormal image and taking the abnormal image as an initial abnormal behavior detection result.

Optionally, the determining whether an emergency occurs before comparing the standard abnormal behavior detection result with a pre-constructed emergency early warning library, includes:

identifying historical abnormal behaviors in a pre-acquired historical reference image, and analyzing abnormal behavior categories corresponding to the historical abnormal behaviors;

obtaining a corresponding emergency type according to the abnormal behavior category and performance analysis corresponding to the abnormal behavior category;

and summarizing the multiple emergency types to obtain an emergency early warning library.

In order to solve the above problems, the present invention further provides an emergency warning device, including:

the system comprises a characteristic extraction module, a motion characteristic extraction module and a texture characteristic extraction module, wherein the characteristic extraction module is used for acquiring a monitoring image from video data recorded by a plurality of monitoring devices based on a preset distributed framework, dividing the monitoring image into a plurality of image sub-blocks meeting preset division conditions, and respectively extracting motion characteristics, size characteristics and texture characteristics from the image sub-blocks;

the initial abnormal behavior detection module is used for respectively constructing feature probabilities corresponding to the motion features and the size features, calculating matching correlation coefficients between the texture features and a preset feature codebook, constructing an abnormal behavior detection system according to a first classifier and a second classifier which are constructed in advance, and performing initial abnormal behavior detection on the feature probabilities and the matching correlation coefficients based on the abnormal behavior detection system to obtain an initial abnormal behavior detection result;

a standard abnormal detection module, configured to obtain an abnormal behavior image corresponding to the initial abnormal behavior detection result, detect a behavior body in the abnormal behavior image and body position information corresponding to the behavior body by using a target detection algorithm, input the body position information into a symmetric space transformation network, obtain a plurality of body target generated skeleton maps, extract body skeleton points in the plurality of body target generated skeleton maps, calculate skeleton transformation characteristic values corresponding to the body skeleton points, and perform abnormal behavior detection again according to the skeleton transformation characteristic values, so as to obtain a standard abnormal behavior detection result;

and the event comparison module is used for comparing the standard abnormal behavior detection result with a pre-constructed emergency early warning library to judge whether an emergency occurs and carrying out early warning.

In order to solve the above problem, the present invention also provides an electronic device, including:

a memory storing at least one instruction; and

and the processor executes the instructions stored in the memory to realize the emergency early warning method.

In order to solve the above problem, the present invention further provides a computer-readable storage medium, where at least one instruction is stored in the computer-readable storage medium, and the at least one instruction is executed by a processor in an electronic device to implement the emergency early warning method described above.

In the embodiment of the invention, the monitoring image is divided into a plurality of image sub-blocks which accord with the preset division condition, the motion characteristic, the size characteristic and the texture characteristic in the image sub-blocks are respectively extracted, and different image-related characteristics are obtained from the perspective of local characteristics to carry out initial abnormal behavior detection and standard abnormal behavior detection, so that the detection accuracy is higher. And comparing the abnormal behavior detection result with an emergency early warning library based on the emergency early warning library, wherein the emergency early warning library comprises a plurality of emergency events, and when the comparison is consistent, determining the event to which the target monitoring image belongs as the emergency event and early warning. Therefore, the emergency early warning method, the emergency early warning device, the electronic equipment and the computer readable storage medium provided by the invention can solve the problem of low accuracy of the emergency early warning.

Drawings

Fig. 1 is a schematic flow chart illustrating an emergency early warning method according to an embodiment of the present invention;

fig. 2 is a functional block diagram of an emergency warning apparatus according to an embodiment of the present invention;

fig. 3 is a schematic structural diagram of an electronic device for implementing the emergency warning method according to an embodiment of the present invention.

The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.

Detailed Description

It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

The embodiment of the application provides an emergency early warning method. The execution subject of the emergency early warning method includes, but is not limited to, at least one of electronic devices, such as a server and a terminal, which can be configured to execute the method provided by the embodiment of the present application. In other words, the emergency early warning method may be performed by software or hardware installed in the terminal device or the server device, and the software may be a block chain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like.

Fig. 1 is a schematic flow chart of an emergency early warning method according to an embodiment of the present invention. In this embodiment, the emergency early warning method includes:

s1, acquiring monitoring images from video data recorded by a plurality of monitoring devices based on a preset distributed framework.

In the embodiment of the present invention, the monitoring device may refer to a plurality of cameras in different positions, or other devices with a photographing function. Assuming that monitoring is performed on an arbitrarily selected target square, in order to achieve full-coverage monitoring of the target square without dead angles, a plurality of monitoring cameras may be deployed at different viewing angles of the target square, and a plurality of monitoring cameras may also be deployed at a plurality of viewing angles of a plurality of intersections leading to the target square. The massive monitoring cameras form a video monitoring network based on the Internet of things, and can obtain video data recorded by massive monitoring equipment.

Furthermore, since the positions of the monitoring cameras are different, in order to relieve the network transmission pressure, the video data recorded by the monitoring devices can be transmitted to a distributed system formed by a local area network, and the video data can be analyzed by using a distributed framework in the distributed system. The distributed framework refers to a MapReduce framework, and the MapReduce framework is a framework of distributed computing and has the characteristics of effectiveness, fault tolerance and large-scale parallel capability.

In detail, the MapReduce framework can implement parallel processing on video data recorded by different monitoring devices, and output processed effective video images as standard monitoring images.

Further, after the monitoring image is obtained, denoising processing can be performed on the monitoring image. The denoising process refers to a process of reducing noise in an image. Noise is an important cause of image interference, various noises may exist in one image in practical application, and the noise may be generated in transmission or quantization and other processes, so that denoising processing is required, and a standard monitoring image subjected to denoising processing is more accurate.

Preferably, the denoising process may be implemented by a filter-based method, a model-based method, and a learning-based method.

The filter-based method is to remove image noise by using a designed low-pass filter. For example, the low-pass filter may be a median filter, which is a commonly used nonlinear smoothing filter, or an adaptive wiener filter, and the basic principle is to substitute the value of a point in the digital image by the median of the values of the points in a field of the point. The adaptive wiener filter adjusts the output of the filter according to the local variance of the image. The model-based method is used for solving the problem that a denoising task is defined as an optimization problem based on the maximum posterior. The learning-based approach is implemented according to a deep network.

S2, dividing the standard monitoring image into a plurality of image sub-blocks which accord with preset dividing conditions, and respectively extracting motion characteristics, size characteristics and texture characteristics from the image sub-blocks.

In the embodiment of the present invention, the preset segmentation condition refers to that the standard monitoring image is segmented into a plurality of non-overlapping image sub-blocks according to a preset image size and the fixed images are not overlapped. The image size set in advance can be adjusted according to application situations, and the image subblocks divided into non-overlapping image subblocks can perform feature extraction on each local region, so that the detection result of each subblock is obtained, irrelevant backgrounds can be screened and eliminated, and the feature aggregation mode is more flexible. And the global condition can be judged according to the detection result of each image subblock, so that the global early warning detection is more accurate.

Further, the motion features in the image sub-blocks can be extracted by the following method:

acquiring a foreground part in the image sub-block, and extracting a transverse optical flow corresponding to any pixel point in the foreground part in the abscissa axis direction and a longitudinal optical flow corresponding to any pixel point in the foreground part in the ordinate axis direction;

In detail, the optical flow refers to a concept in motion detection of an object in a field of view, which describes a motion of an observation target, a surface, or an edge caused with respect to a motion of an observer. Wherein, the horizontal optical flow of any pixel point in the foreground part in the direction of the abscissa axis is

Longitudinal optical flow corresponding to any pixel point in the foreground part in the direction of the ordinate axis

The preset motion characteristic calculation formula is as follows:

where M is the motion characteristic in the image sub-block, N _f For foreground in said image sub-blockThe total number of pixel points of the part,

Further, the size features in the image sub-blocks are extracted by the following method:

In detail, the preset gaussian template is G, which may be a gaussian template of 3 × 3 in the present embodiment, the first parameter is a, and the second parameter is b.

Preferably, since the motion characteristics do not sufficiently express information in the image, for example, people and objects with the same motion speed cannot be distinguished, in order to improve the detection effect, the size of the target in the foreground needs to be considered.

Specifically, the preset size characteristic calculation formula is as follows:

S＝∑∑G(a-i+1,b-j+1)o(a,b)

wherein S is the size feature, G is the Gaussian template, a is the first parameter, b is the second parameter, o (a, b) is the foreground proportion, G (a-i +1, b-j + 1) is the initial feature, and i and j are the image positions of the image sub-blocks.

Further, the texture features in the image sub-blocks are extracted by the following method:

In detail, the target directions may be 0, 45, 90, and 135 degrees, and then the picture may be filtered in four directions of 0, 45, 90, and 135 degrees by using a two-dimensional Gabor filter, so as to obtain a plurality of amplitude values m ₀ ，m ₄₅ ，m ₉₀ And m ₁₃₅ . Summarizing the amplitude values to obtain that the textural features in the image subblock are T = [ m ] ₀ m ₄₅ m ₉₀ m ₁₃₅ ]。

Preferably, the introduction of texture features may use textures inside the sub-blocks to screen the target.

And S3, respectively constructing feature probabilities corresponding to the motion features and the size features, and calculating a matching correlation coefficient between the texture features and a preset feature codebook.

In the embodiment of the present invention, for the two basic features, namely the motion feature and the size feature, since the sizes of the motion and the target size change under different scenes, and the semi-parameterization can just adapt to the change, a relatively smooth model needs to be established.

Specifically, in the embodiment of the present invention, the feature probability corresponding to the motion feature is:

wherein f (s Δ x) is a feature probability corresponding to the motion feature, h is a bandwidth of a gaussian kernel, s Δ x represents an effective upper limit of the motion feature, N is the number of the image subblocks, Δ x is a motion gap, x _n Representing the motion parameters of the nth image sub-block.

Further, the feature probability corresponding to the size feature is consistent with the feature probability corresponding to the motion feature, and only the parameters are different, which is not described herein again.

Specifically, the calculating a matching correlation coefficient between the texture feature and a preset feature codebook includes:

wherein, p (T, c) _k ) Is a matching correlation coefficient between the texture feature and a preset feature codebook, T is the texture feature, c _k For entries in a predetermined feature codebook, μ _T Is the element mean value corresponding to the texture feature,

and the element mean value corresponding to the item.

S4, an abnormal behavior detection system is built according to the first classifier and the second classifier which are built in advance, and initial abnormal behavior detection is carried out on the feature probability and the matching correlation coefficient based on the abnormal behavior detection system to obtain an initial abnormal behavior detection result.

In the embodiment of the invention, abnormal behavior judgment is carried out according to the acquired first classifier and the acquired second classifier, wherein different classifiers are used for abnormal detection of different dimensions, the first classifier is used for speed detection, the abnormal behavior can be judged when the speed is slower or faster than the normal speed, and a plurality of normal speeds can be set in advance to be used as reference. The second classifier is for considering target size and texture features.

In detail, since there is no discrimination capability for a large target object and a small target group only with a target size, resulting in false detection, while since texture of a background is mainly extracted when texture containing few foreground pixels is extracted, using texture features alone to determine an abnormality also results in high false detection. Therefore, it is necessary to consider the complementation between the two, thereby improving the detection performance.

Specifically, the performing, by the abnormal behavior detection system, initial abnormal behavior detection on the feature probability and the matching correlation coefficient to obtain an initial abnormal behavior detection result includes:

and when the speed judgment result is that the speed is normal, performing characteristic judgment on the characteristic probability corresponding to the size characteristic and the matching correlation coefficient by using a second classifier in the abnormal behavior detection system, and when the characteristic judgment does not pass, judging the target monitoring image as an abnormal image and taking the abnormal image as an initial abnormal behavior detection result.

In detail, the abnormal behavior judgment is performed by combining different characteristics, so that the accuracy of abnormal behavior detection can be improved.

S5, acquiring an abnormal behavior image corresponding to the initial abnormal behavior detection result, detecting a behavior human body in the abnormal behavior image and human body position information corresponding to the behavior human body by using a target detection algorithm, and inputting the human body position information into a symmetric space transformation network to obtain a plurality of human body target generation skeleton diagrams.

In the embodiment of the invention, the initial abnormal behavior detection result is used for initially detecting a standard monitoring image, judging whether abnormal behaviors occur in the standard monitoring image from a plurality of angles such as motion characteristics, size characteristics, texture characteristics and the like, positioning to an abnormal behavior image corresponding to the initial abnormal behavior detection result, detecting the abnormal behavior image again, further determining whether the abnormal behaviors exist or not, and improving the accuracy of judging the abnormal behaviors.

Specifically, a behavioral human body in the abnormal behavior image is detected by using a target detection algorithm, wherein the target detection algorithm can be a convolutional neural network based on a sampling region, image recognition is mainly realized through window selection, candidate frames are generated firstly, then the candidate frames are classified, whether a target exists in each candidate frame is judged, the obtained target is the behavioral human body in the abnormal behavior image, human body position information corresponding to the behavioral human body is extracted, and the human body position information in the image is input into a symmetric space transformation network to generate a plurality of human body target skeleton maps.

The symmetric space transformation network is composed of a space transformation network, a single posture detector and an inverse space transformation network, the space transformation network extracts human body region information in the candidate frame to serve as input of the single posture detector, the single posture detector realizes single posture estimation in the candidate frame, and the inverse space transformation network re-maps the human body posture detected by the single posture detector back to an image to obtain a plurality of human body target generated skeleton maps.

S6, extracting a plurality of human body skeleton points in the human body target generation skeleton diagram, calculating skeleton transformation characteristic values corresponding to the human body skeleton points, and performing abnormal behavior detection again according to the skeleton transformation characteristic values to obtain a standard abnormal behavior detection result.

In the embodiment of the invention, a Non-Maximum Suppression (NMS) strategy is used for extracting a plurality of human body skeleton points in the human body target generation skeleton map, wherein the Non-Maximum Suppression is to suppress elements which are not Maximum values, and can be understood as local Maximum search. The local representation is a neighborhood, and the neighborhood has two variable parameters, namely the dimension of the neighborhood and the size of the neighborhood.

Specifically, the calculating of the skeleton transformation characteristic value corresponding to the human skeleton point includes:

mapping the human body skeleton points to a preset two-dimensional rectangular coordinate system, and acquiring a first frame coordinate and a last frame coordinate corresponding to the human body skeleton points on the preset two-dimensional rectangular coordinate system;

calculating to obtain the corresponding movement speed of the human body skeleton point according to the first frame coordinate, the last frame coordinate and a preset movement speed calculation formula;

splitting the movement speed into a horizontal speed and a vertical speed, and calculating according to the horizontal speed, the vertical speed and a preset acceleration calculation formula to obtain a movement acceleration corresponding to the human skeleton point;

and summarizing the movement speed and the movement acceleration into a skeleton transformation characteristic value corresponding to the human skeleton point.

Further, the preset movement speed calculation formula is as follows:

wherein, V _p The movement speed corresponding to the human skeleton point,

in order to be the horizontal velocity,

is the vertical velocity, f represents the acquisition frequency frame, i is the first frame value, x _i，p Is the abscissa value, y, in the first frame coordinate _i，p Is the longitudinal coordinate value, x, in the first frame coordinate _i+f，p Is the abscissa value, y, in the coordinates of the end frame _i+f，p And the coordinate value is the longitudinal coordinate value in the tail frame coordinate.

Specifically, the preset acceleration calculation formula is as follows:

wherein, a _p The motion acceleration corresponding to the human skeleton point is obtained,

in order to be the horizontal acceleration, the acceleration,

is the vertical acceleration, f represents the acquisition frequency frame,

is the horizontal velocity of the i + f-th frame,

is the horizontal velocity of the i-th frame,

is the vertical velocity of the (i + f) th frame,

the vertical velocity of the ith frame.

Further, performing abnormal behavior detection again according to the skeleton transformation characteristic value to obtain a standard abnormal behavior detection result, namely, judging whether an abnormal condition occurs according to the motion speed and the motion acceleration in the skeleton transformation characteristic value and the size between the speed threshold and the acceleration threshold. This is because when abnormal behaviors occur, hand joints, elbow joints, foot joints, knee joints or other skeleton points of the human body are closely related to common abnormal behaviors.

And S7, comparing the standard abnormal behavior detection result with a pre-constructed emergency early warning library to judge whether an emergency occurs and early warning.

In the embodiment of the present invention, the determining whether an emergency occurs by comparing the standard abnormal behavior detection result with a pre-constructed emergency early warning library includes:

obtaining a corresponding emergency type according to the abnormal behavior category and the performance analysis corresponding to the abnormal behavior category;

In detail, the historical reference image refers to a monitoring image corresponding to a past time period in which an emergency occurs. The abnormal behavior category may be a movement speed change, a crowd density change, a human body gravity center change, a movement amplitude change, a movement direction change, and the like. The corresponding performance of the abnormal behavior category refers to the actual specific behavior of human beings.

For example, the emergency event generally includes a natural disaster, an accident disaster, a public health event and a social security event, and the scheme focuses on processing the social security event and can perform early warning on the emergency event in an indoor environment such as a shopping mall or an office building.

Further, the emergency early warning library comprises a plurality of different types of emergency events, comparison is performed on the basis of the emergency early warning library and the abnormal behavior detection result, and when the comparison is consistent, the event to which the target monitoring image belongs is determined as the emergency and early warning is performed.

In the embodiment of the invention, the monitoring image is divided into a plurality of image sub-blocks which accord with the preset division condition, the motion characteristic, the size characteristic and the texture characteristic in the image sub-blocks are respectively extracted, and different image-related characteristics are obtained from the perspective of local characteristics to carry out initial abnormal behavior detection and standard abnormal behavior detection, so that the detection accuracy is higher. And comparing the abnormal behavior detection result with an emergency early warning library based on the emergency early warning library, wherein the emergency early warning library comprises a plurality of emergency events, and when the comparison is consistent, determining the event to which the target monitoring image belongs as the emergency event and early warning. Therefore, the emergency early warning method provided by the invention can solve the problem of low accuracy of the emergency early warning.

Fig. 2 is a functional block diagram of an emergency warning apparatus according to an embodiment of the present invention.

The emergency early warning apparatus 100 according to the present invention may be installed in an electronic device. According to the realized functions, the emergency early warning apparatus 100 may include a feature extraction module 101, an initial anomaly detection module 102, a standard anomaly detection module 103, and an event comparison module 104. The module of the present invention, which may also be referred to as a unit, refers to a series of computer program segments that can be executed by a processor of an electronic device and that can perform a fixed function, and that are stored in a memory of the electronic device.

In the present embodiment, the functions regarding the respective modules/units are as follows:

the feature extraction module 101 is configured to obtain a monitoring image from video data recorded by multiple monitoring devices based on a preset distributed framework, divide the monitoring image into multiple image sub-blocks meeting preset division conditions, and extract motion features, size features, and texture features in the image sub-blocks respectively;

the initial abnormal behavior detection module 102 is configured to respectively construct feature probabilities corresponding to the motion features and the size features, calculate matching correlation coefficients between the texture features and a preset feature codebook, construct an abnormal behavior detection system according to a first classifier and a second classifier that are constructed in advance, and perform initial abnormal behavior detection on the feature probabilities and the matching correlation coefficients based on the abnormal behavior detection system to obtain an initial abnormal behavior detection result;

the standard abnormal detection module 103 is configured to obtain an abnormal behavior image corresponding to the initial abnormal behavior detection result, detect a behavioral human body in the abnormal behavior image and human body position information corresponding to the behavioral human body by using a target detection algorithm, input the human body position information into a symmetric space transformation network, obtain a plurality of human body target generation skeleton maps, extract human body skeleton points in the human body target generation skeleton maps, calculate skeleton transformation characteristic values corresponding to the human body skeleton points, and perform abnormal behavior detection again according to the skeleton transformation characteristic values to obtain a standard abnormal behavior detection result;

the event comparison module 104 is configured to compare the standard abnormal behavior detection result with a pre-constructed emergency early warning library to determine whether an emergency occurs, and perform early warning.

In detail, the specific implementation of each module of the emergency early warning apparatus 100 is as follows:

firstly, acquiring monitoring images from video data recorded by a plurality of monitoring devices based on a preset distributed framework.

Furthermore, since the positions of the monitoring cameras are different, in order to relieve the network transmission pressure, the video data recorded by the multiple monitoring devices can be transmitted to a distributed system formed by a local area network, and the video data can be analyzed by using a distributed framework in the distributed system. The distributed framework refers to a MapReduce framework, and the MapReduce framework is a framework of distributed computing and has the characteristics of effectiveness, fault tolerance and large-scale parallel capability.

Further, after the monitoring image is obtained, denoising processing can be performed on the monitoring image. The denoising process refers to a process of reducing noise in an image. Noise is an important cause of image interference, various noises may exist in an image in practical application, and the noises may be generated in transmission or quantization and other processes, so that denoising processing is required, and a standard monitoring image subjected to denoising processing is more accurate.

The filter-based method is to remove image noise by using a designed low-pass filter. For example, the low-pass filter may be a median filter or an adaptive wiener filter, the median filter is a commonly used nonlinear smoothing filter, and the basic principle is to substitute the value of a point in the digital image by the median of the values of the points in a field of the point. The adaptive wiener filter adjusts the output of the filter according to the local variance of the image. The model-based method is used for solving the problem that a denoising task is defined as an optimization problem based on the maximum posterior. The learning-based approach is implemented according to a deep network.

And secondly, dividing the standard monitoring image into a plurality of image sub-blocks which accord with preset dividing conditions, and respectively extracting the motion characteristics, the size characteristics and the texture characteristics in the image sub-blocks.

In the embodiment of the present invention, the preset segmentation condition refers to that the standard monitoring image is segmented into a plurality of non-overlapping image sub-blocks according to a preset image size and the fixed images are not overlapped. The image size set in advance can be adjusted according to application situations, and each local area can be subjected to feature extraction by dividing the image sub-blocks into a plurality of non-overlapping image sub-blocks, so that the detection result of each sub-block is obtained, irrelevant backgrounds can be screened and eliminated, and meanwhile, the feature aggregation mode is more flexible. And the global condition can be judged according to the detection result of each image subblock, so that the global early warning detection is more accurate.

The preset motion characteristic calculation formula is as follows:

wherein M is the motion characteristic in the image sub-block, N _f Is the total number of pixels of the foreground portion in the image sub-block,

Preferably, since the motion characteristics do not sufficiently express information in the image, for example, people and objects with the same motion speed cannot be distinguished, in order to improve the detection effect, the size of the object in the foreground needs to be considered.

Specifically, the preset size characteristic calculation formula is as follows:

S＝∑∑G(a-i+1,b-j+1)o(a,b)

In detail, the target directions may be 0, 45, 90, and 135 degrees, and then the picture may be filtered by using a two-dimensional Gabor filter in four directions of 0, 45, 90, and 135 degrees, so as to obtain a plurality of amplitude values m ₀ ，m ₄₅ ，m ₉₀ And m ₁₃₅ . Will be manySummarizing the amplitude values to obtain texture features T = [ m ] (in the image subblocks) ₀ m ₄₅ m ₉₀ m ₁₃₅ ]。

Preferably, the introduction of texture features may employ textures inside the sub-blocks to screen the target.

And step three, respectively constructing feature probabilities corresponding to the motion features and the size features, and calculating a matching correlation coefficient between the texture features and a preset feature codebook.

wherein f (s Δ x) is a feature probability corresponding to the motion feature, h is a bandwidth of a Gaussian kernel, s Δ x represents an effective upper limit of the motion feature, N is the number of the image subblocks, Δ x is a motion gap, x is _n Representing the motion parameters of the nth image sub-block.

Further, the feature probability corresponding to the size feature is the same as the feature probability corresponding to the motion feature, but the parameters are different, which is not described herein again.

wherein, p (T, c) _k ) Is the matching correlation coefficient between the texture feature and a preset feature codebook, T is the texture feature, c _k To prepareLet us consider the entry in the codebook of features, μ _T Is the element mean value corresponding to the texture feature,

and the element mean value corresponding to the item.

And fourthly, constructing an abnormal behavior detection system according to a first classifier and a second classifier which are constructed in advance, and carrying out initial abnormal behavior detection on the feature probability and the matching correlation coefficient based on the abnormal behavior detection system to obtain an initial abnormal behavior detection result.

Specifically, the performing, by the abnormal behavior detection system, an initial abnormal behavior detection on the feature probability and the matching correlation coefficient to obtain an initial abnormal behavior detection result includes:

And fifthly, acquiring an abnormal behavior image corresponding to the initial abnormal behavior detection result, detecting a behavior human body in the abnormal behavior image and human body position information corresponding to the behavior human body by using a target detection algorithm, and inputting the human body position information into a symmetric space transformation network to obtain a plurality of human body target generation skeleton maps.

In the embodiment of the invention, the initial abnormal behavior detection result is to perform initial detection on a standard monitoring image, judge whether an abnormal behavior occurs in the standard monitoring image from a plurality of angles such as motion characteristics, size characteristics and texture characteristics, position the standard monitoring image to an abnormal behavior image corresponding to the initial abnormal behavior detection result, detect the abnormal behavior image again, further determine whether an abnormality exists, and improve the accuracy of judging the abnormal behavior.

The symmetric space transformation network is composed of a space transformation network, a single posture detector and a reverse space transformation network, the space transformation network can extract human body region information in the candidate frame to serve as input of the single posture detector, the single posture detector can achieve single posture estimation in the candidate frame, the reverse space transformation network re-maps human body postures detected by the single posture detector back to an image, and a plurality of human body target generation skeleton diagrams are obtained.

And step six, extracting a plurality of human body skeleton points in the human body target generation skeleton diagram, calculating skeleton transformation characteristic values corresponding to the human body skeleton points, and performing abnormal behavior detection again according to the skeleton transformation characteristic values to obtain a standard abnormal behavior detection result.

Further, the preset movement speed calculation formula is as follows:

wherein, V _p The movement speed corresponding to the human skeleton point,

in order to be the horizontal velocity,

is the vertical velocity, f represents the acquisition frequency frame, i is the first frame value, x _i，p Is the abscissa value, y, in the first frame coordinate _i，p Is a longitudinal coordinate value, x, in the first frame coordinate _i+f，p Is the abscissa value, y, in the coordinates of the end frame _i+f，p And the coordinate value is a longitudinal coordinate value in the tail frame coordinate.

Specifically, the preset acceleration calculation formula is as follows:

wherein, a _p Is the motion acceleration corresponding to the human skeleton point,

in order to be the horizontal acceleration, the acceleration,

is the vertical acceleration, f represents the acquisition frequency frame,

is the horizontal velocity of the i + f-th frame,

is the horizontal velocity of the i-th frame,

is the vertical velocity of the i + f-th frame,

the vertical velocity of the ith frame.

Further, performing abnormal behavior detection again according to the skeleton transformation characteristic value to obtain a standard abnormal behavior detection result, namely determining whether an abnormal condition occurs according to the motion speed and the motion acceleration in the skeleton transformation characteristic value and the size between the speed threshold and the acceleration threshold. This is because when abnormal behaviors occur, the hand joints, elbow joints, foot joints, knee joints or other skeleton points of the human body are closely related to the common abnormal behaviors.

And step seven, comparing the standard abnormal behavior detection result with a pre-constructed emergency early warning library to judge whether an emergency occurs and carrying out early warning.

In detail, the historical reference image refers to a monitoring image corresponding to a past time period in which an emergency occurs. The abnormal behavior category may be a movement speed change, a crowd density change, a human body gravity center change, a movement amplitude change, a movement direction change, and the like. And the corresponding performance of the abnormal behavior category refers to the real specific behavior of human beings.

For example, the emergency event generally includes a natural disaster, an accident disaster, a public health event and a social security event, and the present solution focuses on handling the social security event, and may be used for early warning the emergency event in an indoor environment, such as a mall or an office building.

In the embodiment of the invention, the monitoring image is divided into a plurality of image sub-blocks which accord with the preset division condition, the motion characteristic, the size characteristic and the texture characteristic in the image sub-blocks are respectively extracted, and different image-related characteristics are obtained from the perspective of local characteristics to carry out initial abnormal behavior detection and standard abnormal behavior detection, so that the detection accuracy is higher. And comparing the abnormal behavior detection result with an emergency early warning library based on the emergency early warning library, wherein the emergency early warning library comprises a plurality of emergency events, and judging the event to which the target monitoring image belongs as an emergency and early warning when the comparison is consistent. Therefore, the emergency early warning device provided by the invention can solve the problem of low accuracy of the emergency early warning.

Fig. 3 is a schematic structural diagram of an electronic device for implementing an emergency early warning method according to an embodiment of the present invention.

The electronic device may include a processor 10, a memory 11, a communication interface 12 and a bus 13, and may further include a computer program, such as an emergency warning program, stored in the memory 11 and executable on the processor 10.

The memory 11 includes at least one type of readable storage medium, which includes flash memory, removable hard disk, multimedia card, card type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, etc. The memory 11 may in some embodiments be an internal storage unit of the electronic device, for example a removable hard disk of the electronic device. The memory 11 may also be an external storage device of the electronic device in other embodiments, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, provided on the electronic device. Further, the memory 11 may also include both an internal storage unit and an external storage device of the electronic device. The memory 11 may be used to store not only application software installed in the electronic device and various types of data, such as codes of an emergency warning program, but also temporarily store data that has been output or will be output.

The processor 10 may be formed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or may be formed of a plurality of integrated circuits packaged with the same function or different functions, including one or more Central Processing Units (CPUs), microprocessors, digital Processing chips, graphics processors, and combinations of various control chips. The processor 10 is a Control Unit (Control Unit) of the electronic device, connects various components of the electronic device by using various interfaces and lines, and executes various functions and processes data of the electronic device by running or executing programs or modules (e.g., emergency warning programs, etc.) stored in the memory 11 and calling data stored in the memory 11.

The communication interface 12 is used for communication between the electronic device and other devices, and includes a network interface and a user interface. Optionally, the network interface may include a wired interface and/or a wireless interface (e.g., WI-FI interface, bluetooth interface, etc.), which are typically used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a Display (Display), an input unit such as a Keyboard (Keyboard), and optionally a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable, among other things, for displaying information processed in the electronic device and for displaying a visualized user interface.

The bus 13 may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus 13 may be divided into an address bus, a data bus, a control bus, etc. The bus 13 is arranged to enable connection communication between the memory 11 and at least one processor 10 or the like.

Fig. 3 shows only an electronic device having components, and those skilled in the art will appreciate that the structure shown in fig. 3 does not constitute a limitation of the electronic device, and may include fewer or more components than those shown, or some components may be combined, or a different arrangement of components.

For example, although not shown, the electronic device may further include a power supply (such as a battery) for supplying power to each component, and preferably, the power supply may be logically connected to the at least one processor 10 through a power management device, so that functions such as charge management, discharge management, and power consumption management are implemented through the power management device. The power supply may also include any component of one or more dc or ac power sources, recharging devices, power failure detection circuitry, power converters or inverters, power status indicators, and the like. The electronic device may further include various sensors, a bluetooth module, a Wi-Fi module, and the like, which are not described herein again.

Further, the electronic device may further include a network interface, and optionally, the network interface may include a wired interface and/or a wireless interface (such as a WI-FI interface, a bluetooth interface, etc.), which are generally used to establish a communication connection between the electronic device and another electronic device.

Optionally, the electronic device may further comprise a user interface, which may be a Display (Display), an input unit (such as a Keyboard), and optionally a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable, among other things, for displaying information processed in the electronic device and for displaying a visualized user interface.

It is to be understood that the embodiments described are illustrative only and are not to be construed as limiting the scope of the claims.

The memory 11 in the electronic device stores an emergency alert program which is a combination of instructions, and when executed in the processor 10, can implement:

dividing the monitoring image into a plurality of image sub-blocks which accord with preset division conditions, and respectively extracting motion characteristics, size characteristics and texture characteristics in the image sub-blocks;

acquiring an abnormal behavior image corresponding to the initial abnormal behavior detection result, detecting a behavior human body in the abnormal behavior image and human body position information corresponding to the behavior human body by using a target detection algorithm, and inputting the human body position information into a symmetric space transformation network to obtain a plurality of human body target generation skeleton maps;

and comparing the standard abnormal behavior detection result with a pre-constructed emergency early warning library to judge whether an emergency occurs and early warning.

Specifically, the specific implementation method of the processor 10 for the instruction may refer to the description of the relevant steps in the embodiment corresponding to fig. 1, which is not repeated herein.

Further, the electronic device integrated module/unit, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in a computer readable storage medium. The computer readable storage medium may be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying said computer program code, a recording medium, a usb-disk, a removable hard disk, a magnetic diskette, an optical disk, a computer Memory, a Read-Only Memory (ROM).

The present invention also provides a computer-readable storage medium storing a computer program which, when executed by a processor of an electronic device, implements:

respectively constructing feature probabilities corresponding to the motion features and the size features, and calculating a matching correlation coefficient between the texture features and a preset feature codebook;

extracting human body skeleton points in a skeleton map generated by a plurality of human body targets, calculating skeleton transformation characteristic values corresponding to the human body skeleton points, and performing abnormal behavior detection again according to the skeleton transformation characteristic values to obtain a standard abnormal behavior detection result;

In the several embodiments provided in the present invention, it should be understood that the disclosed apparatus, device and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is only one logical functional division, and other divisions may be realized in practice.

The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.

In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional module.

It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof.

The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.

The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.

Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the system claims may also be implemented by one unit or means in software or hardware. The terms second, etc. are used to denote names, but not to denote any particular order.

Finally, it should be noted that the above embodiments are only intended to illustrate the technical solutions of the present invention and not to limit the same, and although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. An emergency early warning method, characterized in that the method comprises:

2. The method for early warning of emergency events according to claim 1, wherein the extracting the motion features from the image sub-blocks comprises:

3. The emergency early warning method according to claim 2, wherein the preset motion characteristic calculation formula is:

4. The emergency early warning method according to claim 1, wherein the extracting of the size features in the image sub-blocks comprises:

5. The emergency early warning method according to claim 1, wherein the extracting the texture features from the image sub-blocks comprises:

acquiring a plurality of preset target directions, and extracting a plurality of amplitude values in the image subblocks from the plurality of target directions by using a two-dimensional filter;

6. The emergency early warning method according to claim 1, wherein the performing initial abnormal behavior detection on the feature probability and the matching correlation coefficient based on the abnormal behavior detection system to obtain an initial abnormal behavior detection result comprises:

7. The emergency early warning method according to claim 1, wherein the determining whether an emergency occurs by comparing the standard abnormal behavior detection result with a pre-constructed emergency early warning library comprises:

8. An emergency early warning apparatus, comprising:

the system comprises a feature extraction module, a motion feature extraction module and a texture feature extraction module, wherein the feature extraction module is used for acquiring a monitoring image from video data recorded by a plurality of monitoring devices based on a preset distributed framework, dividing the monitoring image into a plurality of image sub-blocks meeting preset division conditions, and respectively extracting a motion feature, a size feature and a texture feature in the image sub-blocks;

9. An electronic device, characterized in that the electronic device comprises:

at least one processor; and (c) a second step of,

a memory communicatively coupled to the at least one processor; wherein, the first and the second end of the pipe are connected with each other,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the emergency alert method according to any one of claims 1 to 7.