CN112347906A

CN112347906A - Detection method of abnormal aggregation behavior in buses

Info

Publication number: CN112347906A
Application number: CN202011218749.5A
Authority: CN
Inventors: 王玉全; 费玉婧楠; 王力; 何忠贺; 刘鹏; 徐龙
Original assignee: North China University of Technology
Current assignee: North China University of Technology
Priority date: 2020-11-04
Filing date: 2020-11-04
Publication date: 2021-02-09
Anticipated expiration: 2040-11-04
Also published as: CN112347906B

Abstract

The invention relates to a method for detecting abnormal aggregation behaviors in a bus, which comprises the following steps: s1: arranging an infrared camera for shooting videos in the bus; s2: acquiring a video segment in the bus by using an infrared camera in the bus in S1, preprocessing a video frame image of the acquired video segment, and converting the video frame image into a data set; s3: and performing affine transformation processing on the video frame image in the S2 to model the internal structure of the bus. The detection device for the abnormal aggregation behaviors in the bus comprises a video acquisition module, a video analysis module, an abnormality judgment module and an abnormality alarm module, wherein the video acquisition module is electrically connected with the video analysis module, the video analysis module is electrically connected with the abnormality judgment module, and the abnormality judgment module is electrically connected with the abnormality alarm module. The method aims at the problem that the detection algorithm in the bus lacks consideration of the actual background of the bus, really considers the actual scene in the bus, and accurately judges the abnormal gathering behavior.

Description

Method for detecting abnormal aggregation behaviors in bus

Technical Field

The invention relates to a method for detecting abnormal aggregation behaviors in a bus.

Background

In recent years, the country has made public travel the first choice of many citizens by vigorously advocating green transportation. The bus, as the most common green travel mode, has the characteristics of high passenger flow mobility, high density, complex crowd and the like while being convenient, and is a high-occurrence scene of a group event. Meanwhile, the danger degree of the occurrence of group events is more prominent due to the relative sealing and no other safety monitoring means.

In the conventional research, attention is paid only to the use of surveillance video for driver boarding and disembarking dispatch or for location tracking of criminal agents. And the abnormal behavior of passengers has not been widely noticed. Due to the characteristic of strong influence of the population, the detection of abnormal gathering behaviors in the bus can prevent major violent events and treading events. If the danger in the crowd is found in time, the early warning of the emergent crowd event can be carried out in time, and the life safety of more people is guaranteed. Therefore, the method has important practical significance for detecting and early warning the abnormal aggregation behavior in the bus.

At present, the research on the aspect of abnormal behavior detection in the bus has certain limitations. Firstly, the traditional processing mainly relies on manpower to perform follow-up tracking and analysis, lacks real-time analysis and identification on a monitoring scene, and does not have the function of warning the abnormal event; secondly, the detection algorithm used in the existing bus does not consider the actual background of the bus, but mostly uses an algorithm model trained by a public action data set, does not consider the characteristics of a closed structure in the bus, and the actual effect cannot be well guaranteed.

In summary, in terms of practical application of the research on detecting abnormal aggregation behaviors in the bus, the design of the method considering practical scenes in the bus needs to be carried out practically.

Disclosure of Invention

Technical problems to be solved by the invention

The invention provides a method and a device for detecting abnormal aggregation behaviors in a bus, aiming at the technical problem that the detection algorithm in the bus in the prior art is lack of consideration of the actual background of the bus.

Technical scheme

In order to solve the problems, the technical scheme provided by the invention is as follows:

the method for detecting the abnormal aggregation behavior in the bus comprises the following steps:

s1: arranging infrared cameras for shooting videos at a plurality of positions in a bus;

s2: acquiring a video segment in the bus by using an infrared camera in the bus in S1, preprocessing a video frame image of the acquired video segment, and converting the video frame image into a data set;

s3: carrying out affine transformation processing on the video frame image in the S2, modeling the internal structure of the bus, and dividing the internal space of the bus into four spaces, namely the front part, the tail part, the left side and the right side of the carriage;

s4: training a human head target detection algorithm on the data set in the S2 by using yolov5-S algorithm, and coding and identifying the human head in the video band;

s5: the bus has two motion states, namely a driving state between two stations and a state of arriving at the station and stopping for getting on and off the bus, time values of the two motion states of the bus are set, the driving state time of the bus between the two stations is set to be t1, and the time of the bus in the state of arriving at the station and stopping for getting on and off the bus is set to be t 2;

s6: according to results of S3, S4 and S5, four motion characteristic indexes of crowd average kinetic energy, crowd motion direction entropy, crowd individual distance potential energy and individual average acceleration of different spaces in the bus at the time of t1 and different spaces in the bus at the time of t2 are calculated;

s7: setting an adjustable time threshold according to the division of the internal space of the bus and the motion state of the bus;

s8: calculating a comprehensive weight a according to the four motion characteristic indexes in the S6, and performing potential aggregation set judgment of abnormal aggregation;

s9: when the potential aggregation set with abnormal aggregation is judged, triggering prompt information to an information platform at a bus driver, lighting a yellow light and carrying out voice prompt, and if the potential aggregation set without abnormal aggregation is judged, continuing to execute from S2;

s10: and if the duration of the potential aggregation set of the abnormal aggregation in the S9 exceeds the adjustable time threshold set in the S7, a red light is turned on, voice prompt is carried out, and meanwhile, an abnormal video segment and a danger signal are uploaded to a cloud platform and a traffic safety department for real-time information transmission.

Optionally, when preprocessing is performed on each frame of image in S2, Mosaic data enhancement, random scaling, random clipping, and random arrangement are used for splicing.

Optionally, the head code identifier in S4 includes a head code ID, location information and time information, and the location information is obtained by using yolov5 algorithm.

Optionally, in S3, the bus space information is modeled by using infrared emission thermography, the structural modeling of the car is performed by using 3dmax, and the positions of the facilities such as seats and the like are calibrated therein.

Alternatively, the internal space of the bus in S3 is defined as: the front door to the back door of the bus is arranged at the front part of the carriage, the back door to the last row of seats of the bus is arranged at the tail part of the carriage, the left side window of the bus is provided with a left side window of the carriage, and the right side window of the bus is provided with a right side window of the right side vertical handrail.

Optionally, for the determination of the abnormal aggregation behavior in S8, the specific steps are:

s81: extracting information data of the current head real-time position, time and head code ID from S4 according to the time value set in S5;

s82: calculating the distance between the current human head real-time positions in the S81;

s83: setting the grade of the head gathering distance, and setting head gathering distance thresholds of different grades;

s84: dividing the current head real-time position data into a plurality of head position sets according to the value comparison of S82 and S83;

s85: determining the regional distribution of the positions of each head position set in the S84, wherein the positions of each head position set are positioned at the front part, the tail part, the left side and the right side of the bus compartment;

s86: setting a personnel gathering quantity threshold value of the head position set;

s87: screening potential people gathering sets with the number of people in each set of S85 larger than the threshold number of people gathering of S86;

s88: according to the spatial position information and the time information of the potential people gathering set screened by the S87, matching the threshold values under different conditions to further screen the potential people gathering set;

s89: and calculating comprehensive weights of four motion characteristic indexes of the crowd average kinetic energy, the crowd motion direction entropy, the inter-individual distance potential energy and the individual average acceleration of the people gathering set screened in the S88 to judge the potential gathering set of abnormal gathering.

Optionally, the threshold of the number of people gathered set in S86 is defined as:

when the bus is in a driving state between two stations: the threshold value of the number of people gathered at the front part of the carriage is 5, the threshold value of the number of people gathered at the rear part of the carriage is 4, and the threshold values of the number of people gathered at the left side and the right side of the carriage are 3 respectively;

when the bus is in a state of arriving at a station, stopping and getting on or off the bus: the threshold value of the number of people gathered at the front part of the carriage is 8 people, the threshold value of the number of people gathered at the rear part of the carriage is 6 people, and the threshold values of the number of people gathered at the left side and the right side of the carriage are 5 people respectively.

Optionally, the average kinetic energy of the crowd in S6 or S89 is a ratio of the sum of the crowd motion energies to the number of the crowd, the sum of the crowd motion energies being represented by calculating optical flow energy within the region of interest;

the human motion direction entropy in the S6 or S89 comprises an optical flow vector direction histogram, a direction probability distribution graph and a direction entropy, and the larger the human motion direction entropy is, the larger the chaos degree of the human motion direction is;

the inter-individual distance potential of the population in the S6 or S89 can reflect the dispersion degree between every two individuals of the population, and if the distance potential suddenly increases or suddenly decreases, the probability of abnormal conditions is indicated;

the average acceleration of the individuals in the S6 or S89 reflects the degree of violence of the movement of the population, and the higher the value, the more abnormal.

Has the beneficial effects of

Compared with the prior art, the technical scheme provided by the invention has the following beneficial effects:

the invention provides a method and a device for detecting abnormal aggregation behaviors in a bus, which practically consider the actual scene in the bus and accurately judge the abnormal aggregation behaviors.

Drawings

FIG. 1 is a logic flow diagram of a method for detecting anomalous aggregation behavior in a bus in accordance with the present invention;

fig. 2 is a logic block diagram of the detecting device for abnormal aggregation behavior in a bus according to the present invention.

Detailed Description

For a further understanding of the present invention, reference is made to the following detailed description of the invention taken in conjunction with the accompanying FIG. 1.

With reference to the attached drawing 1, the method for detecting the abnormal aggregation behavior in the bus provided by the invention comprises the following steps:

The detection steps S3, S4 and S5 of the method for detecting the abnormal aggregation behaviors in the bus are not in time sequence. In the S3, the space information of the bus is modeled by adopting infrared emission thermography imaging, the structural modeling of a carriage is carried out by utilizing 3dmax, and the positions of facilities such as seats and the like are calibrated in the carriage. The definition of the bus interior space in S3 is: the front door to the back door of the bus is arranged at the front part of the carriage, the back door to the last row of seats of the bus is arranged at the tail part of the carriage, the left side window of the bus is provided with a left side window of the carriage, and the right side window of the bus is provided with a right side window of the right side vertical handrail. In S3, affine transformation processing is performed on the video frame data, and mapping conversion between the head position and the actual distance position of the car space structure is performed.

The head code identification in the S4 comprises a head code ID, position information and time information, wherein the position information is obtained by utilizing yolov5-S algorithm.

Setting an abnormal aggregation threshold value in the algorithm, judging the magnitude of the comprehensive weight a and a field aggregation threshold value when judging the potential aggregation set of abnormal aggregation of the comprehensive weight a in S8, and judging that the potential aggregation of abnormal aggregation exists in the current bus if judging that a is greater than the set abnormal aggregation threshold value; and if the judgment a is smaller than the set abnormal aggregation threshold value, judging that the current bus has no abnormal aggregation potential aggregation.

Furthermore, each frame of image obtained by shooting by the video camera is preprocessed, so that the problems of image blurring, image deformation and the like caused by the environment or shooting azimuth angle and the like of the image are solved. And when preprocessing each frame image in the S2, splicing by adopting a method of Mosaic data enhancement, random scaling, random cutting and random arrangement to realize accurate identification of small targets.

Further, in the step S4, training a human head target detection algorithm, and finally training to obtain a pyrtch model; the human head target detection algorithm needs to be embedded into a xavier NX algorithm board, the xavier NX algorithm board is installed in a bus, model conversion needs to be carried out when the pitorch model is deployed on the xavier NX, and the conversion steps are as follows:

s41: converting the pytoch model to an ONNX model;

s42: converting the ONNX model in S41 into a tensorrT model;

s43: calling a tentrorT model in S42 by using C + +;

s44: c + + in S43 is embedded into the xavier NX algorithm board.

Further, for the determination of the abnormal aggregation behavior in S8, the specific steps are:

Further, the threshold of the number of people gathered set in S86 is defined as:

Further, the average kinetic energy of the crowd in S6 or S89 is a ratio of the sum of the crowd motion energies, which is represented by calculating the optical flow energy in the region of interest, to the number of the crowd, and the average kinetic energy of the crowd reflects the speed and intensity of the crowd motion. The human motion direction entropy in S6 or S89 includes an optical flow vector direction histogram, a direction probability distribution map, and a direction entropy, and a larger human motion direction entropy indicates a larger degree of confusion of the human motion direction. In a bus, when the bus is in a driving state between two stations, the entropy of the crowd moving direction is mostly in a stable value, which indicates that the crowd moves or does not move in the same direction; when the bus is in a state of arriving at a station, stopping and getting on or off the bus, the entropy of the movement direction of the crowd is in a floating numerical value, and the crowd movement is concentrated to flow into or flow out of the front door and the rear door.

The inter-individual distance potential of the crowds in the S6 or S89 can reflect the dispersion degree between every two individuals of the crowds, and the head position information obtained by the yolov5-S algorithm is combined for calculation. If the distance potential energy between the individual groups of the crowd suddenly increases or suddenly decreases, the possibility of abnormal conditions is indicated. The average acceleration of the individuals in the S6 or S89 reflects the degree of violence of the movement of the population, and the higher the value, the more abnormal.

With reference to fig. 2, the device for detecting abnormal aggregation behavior in a bus provided by the invention comprises a video acquisition module, a video analysis module, an abnormality judgment module and an abnormality alarm module, wherein the video acquisition module is electrically connected with the video analysis module, the video analysis module is electrically connected with the abnormality judgment module, and the abnormality judgment module is electrically connected with the abnormality alarm module.

The video acquisition module is used for acquiring video data in the bus, preprocessing the video data and modeling the internal space of the bus; the video acquisition module transmits the preprocessed video data to the video analysis module, and the video analysis module carries out coding identification on the head in the received video data; the video analysis module transmits the head coding identification information to the abnormality judgment module, the abnormality judgment module calculates abnormality judgment indexes, namely four motion characteristic indexes of population average kinetic energy, population motion direction entropy, population inter-individual distance potential energy and individual average acceleration according to the received head identification information, calculates comprehensive weight a according to the four motion characteristic indexes, and judges the potential aggregation set of abnormal aggregation; if the set is judged to be a potential aggregation set, the abnormity judging module transmits an alarm signal to the abnormity alarm module, and the abnormity alarm module receives the alarm signal and gives an alarm.

The present invention and its embodiments have been described above schematically, without limitation, and what is shown in the drawings is only one of the embodiments of the present invention, and the actual structure is not limited thereto. Therefore, if the person skilled in the art receives the teaching, without departing from the spirit of the invention, the person skilled in the art shall not inventively design the similar structural modes and embodiments to the technical solution, but shall fall within the scope of the invention.

Claims

1. A method for detecting abnormal aggregation behavior in a bus, characterized in that it comprises the following steps:

S1: Infrared cameras for shooting video are arranged at multiple positions in the bus;

S2: Use the infrared camera in the bus in S1 to obtain the video segment in the bus, and perform video frame image preprocessing on the acquired video segment, and convert the video frame image into a data set;

S3: Perform affine transformation on the video frame images in S2, model the internal structure of the bus, and divide the internal space of the bus into four spaces: front, rear, left and right;

S4: Use the yolov5-s algorithm to train the human head target detection algorithm on the data set in S2, and encode and identify the human head in the video segment;

S5: The bus has two motion states, namely the running state between two stops and the state of getting on and off at the stop, set the time value of the two motion states of the bus, and set the time of the bus in the running state between the two stops as t1, the time when the bus is in the state of getting on and off at the station is t2;

S6: According to the results of S3, S4 and S5, calculate the average kinetic energy of the crowd, the entropy of the movement direction of the crowd, the distance potential energy between individuals and the average acceleration of the crowd in different spaces inside the bus at time t1 and different spaces inside the bus at time t2 Four sports characteristic indicators;

S7: Set an adjustable time threshold according to the division of the internal space of the bus and the movement state of the bus;

S8: Calculate the comprehensive weight a according to the four motion characteristic indicators in S6, and judge the potential aggregation set of abnormal aggregation;

S9: when it is determined that there is a potential aggregation set of abnormal aggregation, trigger the prompt information to the information platform at the bus driver, turn on the yellow light and give a voice prompt, if it is determined that there is no potential aggregation set of abnormal aggregation, continue to execute from S2;

S10: If the duration of the potential aggregation set of abnormal aggregation in S9 exceeds the adjustable time threshold set in S7, the red light will be turned on and a voice prompt will be given, and the abnormal video segment and danger signal will be uploaded to the cloud platform and the traffic safety department. Real-time information transmission.

2. The method for detecting abnormal aggregation behavior in a bus according to claim 1, wherein when preprocessing each frame of image in the S2, Mosaic data enhancement, random scaling, random cropping, random Arranged to splicing.

3. the detection method of abnormal aggregation behavior in the bus according to claim 1, is characterized in that, the human head coding mark in described S4 comprises human head coding ID, position information and time information, and described position information utilizes yolov5 algorithm to obtain .

4. The method for detecting abnormal aggregation behavior in a bus according to claim 1, characterized in that in said S3, infrared emission thermal image imaging is used to model the spatial information of the bus, and 3dmax is used to construct the structure of the carriage. model, and calibrate the position of the seat and other facilities in it.

5 . The method for detecting abnormal aggregation behavior in a bus according to claim 1 , wherein the internal space of the bus in S3 is defined as: the position from the front door of the bus to the rear door is set as the front of the carriage, and the position of the bus The rear door to the last row of seats is the rear of the car, the vertical handrail on the left side of the bus to the left window is the left side of the car, and the vertical handrail on the right side of the bus to the right window is the right side of the car.

6. The method for detecting abnormal aggregation behavior in a bus according to claim 3, wherein, for the determination of abnormal aggregation behavior in the S8, the specific steps are:

S81: According to the time value set in S5, extract the information data of the current real-time position of the head, time and head code ID from S4;

S82: Calculate the distance between the real-time positions of the current head in S81;

S83: Set the level of the head gathering distance, and set different levels of the head gathering distance threshold;

S84: According to the value comparison between S82 and S83, divide the current real-time head position data into a plurality of head position sets;

S85: Determine that the positions of each head position set in S84 are distributed in the front, rear, left and right sides of the bus compartment;

S86: Set the threshold of the number of people gathered in the head position set;

S87: Screening the potential personnel aggregation sets in which the number of personnel in each set of S85 is greater than the threshold of the number of personnel aggregation in S86;

S88: According to the spatial location information and time information of the potential personnel gathering set screened in S87, and matching thresholds under different conditions to further screen the potential personnel gathering set;

S89: Calculate the comprehensive weight of the four movement characteristic indexes of the crowd aggregated set screened in S88, the crowd movement direction entropy, the inter-individual distance potential energy and the individual average acceleration in the crowd, to judge the potential aggregation set of abnormal aggregation.

7. The method for detecting abnormal aggregation behavior in a bus according to claim 6, wherein the threshold of the number of personnel aggregation set in the S86 is defined as:

When the bus is running between two stations: the threshold for the number of people gathered in the front of the car is 5, the threshold for the number of people in the rear of the car is 4, and the threshold for the number of people on the left and right of the car is 3 each;

When the bus is in the state of stopping and getting off the bus: the threshold for the number of people gathered in the front of the car is 8, the threshold for the number of people in the back of the car is 6, and the threshold for the number of people on the left and right of the car is 5.

8. The method for detecting abnormal aggregation behavior in a bus according to claim 6, wherein the average kinetic energy of the crowd in the S6 or S89 is the ratio of the sum of the crowd movement energy to the crowd quantity, and the crowd movement energy The sum is represented by calculating the optical flow energy within the region of interest;

The crowd movement direction entropy in the described S6 or S89 includes the optical flow vector direction histogram, the direction probability distribution diagram and the direction entropy, and the larger the crowd movement direction entropy is, the greater the chaotic degree of the crowd movement direction;

The inter-individual distance potential energy of the population in the S6 or S89 reflects the dispersion degree between two individuals in the population, and if the distance potential energy increases or decreases suddenly, it indicates that there is a possibility of an abnormal situation;

The individual average acceleration in S6 or S89 reflects the intensity of crowd movement, and the higher the value, the greater the abnormality.