CN115620228A - A video analysis-based early warning method for passengers entering the subway screen door - Google Patents

Abstract

The invention discloses a video analysis-based early warning method for door violation of passengers who close a subway shield door, which comprises the following steps: starting a passenger door-rushing early warning terminal system installed at a shielding door head at the moment of closing a door, and continuously acquiring video frames by a video acquisition module according to a given frame rate; the video analysis module continuously calculates the motion track of the pedestrian target in the front door area based on YOLOv5 and Deepsort; the door-rushing risk evaluation module calculates the total passenger door-rushing risk faced by the door based on the motion tracks of all the pedestrian targets; and performing secondary door-violation early warning treatment including acousto-optic alarm and door control linkage on the condition that the evaluation risk exceeds a given threshold value. The invention provides an effective scheme for reducing adverse effects of door-break behaviors of passengers on driving and passenger safety.

Description

A video analysis-based early warning method for passengers entering the subway screen door

technical field

The invention relates to an urban rail transit smart station, in particular to a method for pre-warning a screen door for a passenger entering the door when the screen door is about to close based on video analysis.

Background technique

Due to many advantages such as safety, punctuality, fastness, comfort, and environmental protection, the subway has become the main tool and vigorously developed field of public transportation in many cities. The essence of smart subway is to intelligently empower train vehicles, network stations, station yards, dispatch management, operation and maintenance guarantee and other systems to achieve the goals of improving operational efficiency, reducing operational risks, and improving passenger service satisfaction. The development of smart subway technology and applications based on technologies such as cloud computing, Internet of Things and artificial intelligence is in full swing. In recent years, systems such as face recognition for non-inductive payment gates, voice recognition smart customer service, and station operation based on intelligent video analysis have been demonstrated and applied in major cities in China and have achieved practical results.

In the smart station scenario, intelligent video analysis through CCTV connected to the signal system can detect abnormal scenes such as escalator retrograde, crowded stairs, delivery of objects on the fence, passengers falling, and missing items, which improves the initiative of abnormal event discovery and timeliness. During the whole process of passengers riding, the behavior of passengers breaking into the door near the closing time is a high-risk behavior that endangers the safety of passengers and driving. Usually screen doors and train doors are equipped with anti-bracketing functions, but these functions only have feedback on the clamping of rigid and large objects, which makes a large number of passenger break-in events directly cause the doors to fail to close normally, and the vehicle cannot operate normally. Accidents, more seriously cause passengers to be clamped between the screen door and the car door and cause fatal major accidents. In recent years, some cities have carried out beneficial exploration of new equipment based on sensors such as visible light vision and lidar in the detection of foreign objects between screen doors and train doors, but they still can only solve the problem of people and objects that have occurred. There has always been a lack of proactive means of early warning for door breaks in emergency and linkage processing after the event. The traditional CCTV monitoring at the platform level usually uses several sets of cameras parallel to the train tracks. The unfavorable field of view and the high-latency video analysis at the station level cannot cope with the early warning of the rapid movement of passengers entering each door.

The present invention provides a video analysis-based early warning method for passengers entering the door of a subway screen door, based on a camera with an oblique downward viewing angle installed at the head of each screen door and the on-site computing capability of the passenger entry warning terminal system. In order to reduce the adverse impact of passengers' door-breaking behavior on driving and passenger safety.

Contents of the invention

Purpose of the invention: The purpose of the present invention is to provide a video analysis-based early warning method for passengers entering the subway screen door when it is about to close, so as to reduce the adverse effects of passenger breaking behavior on driving and passenger safety through active door-breaking warnings.

Technical solution: In order to solve the above-mentioned technical problems, the technical solution adopted in the present invention is:

In the first aspect, a video analysis-based early warning method for passengers entering the subway screen door when it is about to close is provided, including:

Step S1: With T seconds left before the closing time, continuously acquire the video frames I ₁ , I ₂ , ... I _i , ... of the field of view F according to the preset frame rate R;

Step S2: For the current video frame I _i , detect the pedestrian target based on the YOLOv5 neural network and associate it with the detected target in the previous video frame based on _Deepsort , so as to realize the detection of the target O ₁ , O ₂ , ..., On Continuous tracking, where n is the number of all pedestrian targets, and the movement trajectory of the target in the field of view F is converted into the continuous movement trajectory Tr ₁ (t) and Tr ₂ (t) of the target in the area H in front of the corresponding door through perspective transformation 、...Tr _n (t);

Step S3: Use the movement trajectory _Tri (t) data of pedestrian targets O ₁ , O ₂ ,..., O _n in the area H in front of the door to conduct single-target door-breaking risk assessment, and calculate the continuous risk assessment value of each target R ₁ (t), R ₂ (t), ..., R _n (t), and synthesize the single-objective risk to generate the total risk RH(t) of passengers entering the door;

Step S4: Based on the total passenger entry risk RH(t) faced by the door and the preset secondary threshold, carry out hierarchical door entry warning treatment.

In some embodiments, in the step S1, the method for collecting video frames of the field of view F includes:

According to the time plan of the train dispatching signal system, the passenger entry warning terminal system installed at the head of the screen door will start the warning function of the passenger entry warning function when there are T seconds left before the closing time of the door. The video acquisition module of the system utilizes the The camera with an oblique downward viewing angle right above the screen door collects video frames.

In some embodiments, the step S2 includes:

S201: For the current video frame I _i collected in step S1, detect pedestrian targets based on the YOLOv5 neural network, and output n pedestrian targets O ₁ , O ₂ , ..., O _n , if no pedestrian targets are detected, output NULL;

S202: If the output of S201 is NULL, execute step S201 in a loop to perform pedestrian target detection operation on the next frame I _i+1 output by S1 until the current moment t=T;

If the pedestrian target is detected in step S201, perform the Deepsort target association tracking operation according to the target detection result recorded in the last frame I _i-1 , and form O ₁ , O ₂ ,..., O _n target movements in the field of view F trajectories K ₁ (t), K ₂ (t), . . . K _n (t);

S203: Transform K ₁ (t), K ₂ (t), ... K _n (t) into trajectories Tr ₁ (t), Tr ₂ (t), ... Tr _n (t).

In some embodiments, the step S3 includes:

S301: For all targets O _i that have detected and generated motion trajectories T _i (t), continuously estimate the angle u _i between their motion direction and the Y axis, and define the coordinate system of the area H in front of the door as taking the downward direction of the center line as Y-axis, the X-axis is the rightward direction of the upper edge of the H area;

S302: For all targets O _i that have detected and generated motion trajectories Tri (t), continuously estimate the Y-axis component v _i of their motion speeds _, and define the coordinate system of the area H in front of the door as the Y-axis in the downward direction of the center line , taking the rightward direction of the upper edge of the H area as the X-axis;

S303: For all detected and generated objects O _i with motion trajectories Tri (t ₎ , record the distance between the objects O _i and the bottom line of the H area in the Y direction as D _i ;

According to R _i (t)=max(cos(u _i ),0)×max(1.5-v _i ×(Tt)/D _i ,0), calculate the risk of the target O _i breaking through the door at the current moment t;

S304: Calculate the total risk of passengers breaking through the door: RH(t)=max(R ₁ (t), R ₂ (t), . . . , R _n (t)).

In some embodiments, the step S4 includes:

The first-level response to the total passenger door-breaking risk RH(t) exceeding the first threshold Th ₁ is to remind passengers through the sound and light alarm device installed above the screen door;

The secondary response to the total passenger entry risk RH(t) exceeding the second threshold Th ₂ is to keep the acousto-optic alarm and at the same time link the screen door and the car door control system to keep the screen door and the car door open until the total number of passengers enter the door After the risk RH(t) returns to zero, the shielded door and the car door control system are linked to close the shielded door and the car door, wherein the second threshold Th ₂ is greater than the first threshold Th ₁ .

The step S4 also includes:

Record and generate and save a video recording of a passenger entering a door while performing a primary or secondary response.

In some embodiments, the first threshold Th ₁ is 0.25, and the second threshold Th ₂ is 0.5.

In a second aspect, the present invention provides a video analysis-based early warning device for passengers entering the door of a subway screen door that is about to close, including a processor and a storage medium;

The storage medium is used to store instructions;

The processor is configured to operate in accordance with the instructions to perform the steps of the method according to the first aspect.

In a third aspect, the present invention provides a storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

Beneficial effect: compared with the prior art, the present invention has the following advantages:

1. Passenger breaking the door has caused huge risks to the safe operation of the station and the safe and orderly operation of the vehicles. The present invention solves the problem that the current stations and trains lack an early warning mechanism for passenger breaking the door, and provides a method to reduce the incidence of passenger breaking the door and extreme risk. An effective solution for the risk of entrapment under certain circumstances;

2. Compared with the abnormal behavior detection of passengers based on the existing video surveillance in the station hall, the present invention can provide information for each single door based on the favorable field of view and real-time computing capabilities of the intelligent computing unit installed on the screen door. Passengers break the door early warning, with higher real-time and reliability.

Description of drawings

Fig. 1 is the flow chart of the method for early warning of passenger break-in of the subway screen door near closing door based on the video analysis of the embodiment of the present invention;

Fig. 2 is a schematic diagram of the moving direction and speed of the pedestrian target in the area in front of the door in the embodiment.

detailed description

In order to make the technical means, creative features, goals and effects achieved by the present invention easy to understand, the present invention will be further described below in conjunction with specific embodiments.

In the description of the present invention, several means more than one, and multiple means more than two. Greater than, less than, exceeding, etc. are understood as not including the original number, and above, below, within, etc. are understood as including the original number. If the description of the first and second is only for the purpose of distinguishing the technical features, it cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features relation.

In the description of the present invention, reference to the terms "one embodiment," "some embodiments," "exemplary embodiments," "examples," "specific examples," or "some examples" is intended to mean that the embodiments are A specific feature, structure, material, or characteristic described by or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

Example 1

A video analysis-based early warning method for passengers entering a subway screen door when it is about to close, including:

Step S1: With T seconds left before the closing time, continuously acquire the video frames I ₁ , I ₂ , ... I _i , ... of the field of view F according to the preset frame rate R;

Step S2: For the current video frame I _i , detect the pedestrian target based on the YOLOv5 neural network and associate it with the detected target in the previous video frame based on _Deepsort , so as to realize the detection of the target O ₁ , O ₂ , ..., On Continuous tracking, where n is the number of all pedestrian targets, and the movement trajectory of the target in the field of view F is converted into the continuous movement trajectory Tr ₁ (t) and Tr ₂ (t) of the target in the area H in front of the corresponding door through perspective transformation 、...Tr _n (t);

Step S3: Use the movement trajectory _Tri (t) data of pedestrian targets O ₁ , O ₂ ,..., O _n in the area H in front of the door to conduct single-target door-breaking risk assessment, and calculate the continuous risk assessment value of each target R ₁ (t), R ₂ (t), ..., R _n (t), and synthesize the single-objective risk to generate the total risk RH(t) of passengers entering the door;

Step S4: Based on the total passenger entry risk RH(t) faced by the door and the preset secondary threshold, carry out hierarchical door entry warning treatment.

In some embodiments, as shown in FIG. 1 , a video analysis-based method for early warning of passenger entry of a subway screen door when it is about to close. System, the video acquisition module continuously collects video frames at a given frame rate; the video analysis module continuously calculates the movement trajectory of pedestrian targets in the area in front of the door based on YOLOv5 and Deepsort; The total risk of passengers entering the door; when the assessed risk exceeds a given threshold, a door entry warning including sound and light alarms and door control linkage is carried out.

In this embodiment, the specific steps are as follows:

S1: According to the time plan of the train dispatching signal system, the passenger entry warning terminal system installed at the head of the screen door has T seconds left before the door closing time (according to the specific configuration of the station opening entry and early warning, the T value is recommended to be set 10S) to enable the pre-warning function of the passenger entering the door when the door is closing. The video acquisition module of the system uses the camera installed directly above the screen door with an obliquely downward angle of view to follow the frame rate R (in order to ensure the real-time performance of motion detection, it is recommended that the frame rate be lower than lower than 720@60fps) to continuously acquire the video frames I ₁ , I ₂ , ... of the field of view F;

S2: For the current frame I _i , detect the pedestrian target based on the YOLOv5 neural network and associate it with the detected target in the previous frame based on _Deepsort , so as to realize the continuous tracking of the target O ₁ , O ₂ ,..., On, Among them, n is the number of all pedestrian targets, through perspective transformation (the perspective transformation matrix is calculated in advance based on parameters such as camera lens parameters and installation positions, and the whole line shares a unified perspective transformation matrix under the condition of ensuring installation accuracy) to place the target in the field of view The motion trajectory in F is converted into the target continuous motion trajectory Tr ₁ (t), Tr ₂ (t), ... Tr _n (t) in the area H in front of the corresponding door;

S3: Use the movement trajectory _Tri (t) data of pedestrian targets O ₁ , O ₂ ,..., O _n in the area H in front of the door to conduct single-target door-breaking risk assessment, and calculate the continuous risk assessment value R of each target ₁ (t), R ₂ (t), ..., R _n (t), and the single-objective risk is combined to generate the total passenger entry risk RH(t) faced by the door;

S4: Based on the total passenger entry risk RH(t) of the door and the set secondary threshold, carry out hierarchical door entry warning treatment including sound and light alarm and door control linkage.

In this embodiment, step S2 specifically includes the following steps:

S201: For the current frame I _i collected in step S1, detect pedestrian targets based on the YOLOv5 neural network, and output n pedestrian targets O ₁ , O ₂ , ..., O _n in total. If no pedestrian target is detected, output NULL ;

S202: If the output of S201 is NULL, execute step S201 in a loop to detect the pedestrian target on the next frame I _{i +1} output by S1 until t=T; ₁ The recorded target detection results are subjected to the Deepsort target association tracking operation to form the motion trajectories K ₁ (t), K ₂ (t), ... of O ₁ , O ₂ , ..., O _n targets in the field of view F .K _n (t);

S203: Transform K ₁ (t), K ₂ (t), ... K _n (t) into trajectories Tr ₁ (t), Tr ₂ (t), ... Tr _n (t);

In this embodiment, step S2 specifically includes the following steps:

S301: For all targets O _i that have detected and generated motion trajectories T _i (t), continuously estimate the angle u _i between their motion direction and the Y axis, as shown in Figure 2, the coordinate system of the area H in front of the door is defined as Take the downward direction of the midline as the Y axis, and take the rightward direction of the upper edge of the H area as the X axis;

S302: For all targets O _i that have detected and generated motion trajectories T _i (t), continuously estimate the Y-axis component v _i of their motion speeds. As shown in Figure 2, the coordinate system of the area H in front of the door is defined as the center line The downward direction is the Y axis, and the rightward direction of the upper edge of the H area is the X axis;

S303: For all objects O _i that have detected and generated motion trajectories Tri (t), record the distance between the objects O _i and the bottom line of the H area in the Y direction as D _i , according to R _{i (} t)=max(cos(u _i ),0)×max(1.5-v _i ×(Tt)/D _i ,0) calculates the door-breaking risk of the target O _i at the current moment t.

S304: Calculate the total risk of passengers entering the door at the door position as RH(t)=max(R ₁ (t), R ₂ (t), . . . , R _n (t)).

In this embodiment, step S4 specifically includes the following steps:

S401: When the total passenger entry risk RH(t) exceeds the threshold Th ₁ (according to the sensitivity requirements of the entry risk assessment, Th ₁ is recommended to be set to 0.25), remind through the sound and light alarm device installed above the screen door;

S402: When the total passenger entry risk RH(t) exceeds the threshold Th ₂ which is higher than the threshold Th ₁ (according to the sensitivity requirements of the door entry risk assessment, Th ₂ is recommended to be set to 0.5), maintain audible and visual alarms, and link them The screen door and car door control system keeps the car door open until the total passenger entry risk RH(t) of the door returns to zero, and then the screen door and car door control system is linked to close the screen door and car door;

S403: Recording and generating and saving a video record of the passenger breaking the door while performing the primary and secondary responses.

Example 2

In the second aspect, the present embodiment provides a video analysis-based early warning device for passengers entering the subway screen door when the door is about to close, including a processor and a storage medium;

The storage medium is used to store instructions;

The processor is configured to operate according to the instructions to execute the steps of the method according to Embodiment 1.

Example 3

In a third aspect, this embodiment provides a storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described in Embodiment 1 are implemented.

Those skilled in the art should understand that the embodiments of the present application may be provided as methods, systems, or computer program products. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowcharts and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow diagram procedure or procedures and/or block diagram procedures or blocks.

It can be known from common technical knowledge that the present invention can be realized through other embodiments without departing from its spirit or essential features. Accordingly, the above-disclosed embodiments are, in all respects, illustrative and not exclusive. All changes within the scope of the present invention or within the scope equivalent to the present invention are embraced by the present invention.

Claims (9)

1. A video analysis-based subway screen door near closing door passenger breaks the door early warning method, it is characterized in that, comprises: Step S1: With T seconds left before the closing time, continuously acquire the video frames I ₁ , I ₂ , ... I _i , ... of the field of view F according to the preset frame rate R; Step S2: For the current video frame I _i , detect the pedestrian target based on the YOLOv5 neural network and associate it with the detected target in the previous video frame based on _Deepsort , so as to realize the detection of the target O ₁ , O ₂ , ..., On Continuous tracking, where n is the number of all pedestrian targets, and the movement trajectory of the target in the field of view F is converted into the continuous movement trajectory Tr ₁ (t) and Tr ₂ (t) of the target in the area H in front of the corresponding door through perspective transformation 、...Tr _n (t); Step S3: Use the movement trajectory _Tri (t) data of pedestrian targets O ₁ , O ₂ ,..., O _n in the area H in front of the door to conduct single-target door-breaking risk assessment, and calculate the continuous risk assessment value of each target R ₁ (t), R ₂ (t), ..., R _n (t), and synthesize the single-objective risk to generate the total risk RH(t) of passengers entering the door; Step S4: Based on the total passenger entry risk RH(t) faced by the door and the preset secondary threshold, carry out hierarchical door entry warning treatment. 2. a kind of subway screen door based on video analysis according to claim 1 closes the passenger breaks the door early warning method, it is characterized in that, in described step S1, the acquisition method of the video frame of field of view F comprises: According to the time plan of the train dispatching signal system, the passenger entry warning terminal system installed at the head of the screen door will start the warning function of the passenger entry warning function when there are T seconds left before the closing time of the door. The video acquisition module of the system utilizes the The camera with an oblique downward viewing angle right above the screen door collects video frames. 3. a kind of video analysis-based subway screen door close door passenger breaks the door pre-warning method according to claim 1, it is characterized in that, described step S2 comprises: S201: For the current video frame I _i collected in step S1, detect pedestrian targets based on the YOLOv5 neural network, and output n pedestrian targets O ₁ , O ₂ , ..., O _n , if no pedestrian targets are detected, output NULL; S202: If the output of S201 is NULL, execute step S201 in a loop to perform pedestrian target detection operation on the next frame I _i+1 output by S1 until the current moment t=T; If the pedestrian target is detected in step S201, perform the Deepsort target association tracking operation according to the target detection result recorded in the last frame I _i-1 , and form O ₁ , O ₂ ,..., O _n target movements in the field of view F trajectories K ₁ (t), K ₂ (t), . . . K _n (t); S203: Transform K ₁ (t), K ₂ (t), ... K _n (t) into trajectories Tr ₁ (t), Tr ₂ (t), ... Tr _n (t). 4. A kind of video analysis-based subway screen door impending closing door passenger breaks the door early warning method according to claim 1, it is characterized in that, described step S3 comprises: S301: For all targets O _i that have detected and generated motion trajectories T _i (t), continuously estimate the angle u _i between their motion direction and the Y axis, and define the coordinate system of the area H in front of the door as taking the downward direction of the center line as Y-axis, the X-axis is the rightward direction of the upper edge of the H area; S302: For all targets O _i that have detected and generated motion trajectories Tri (t), continuously estimate the Y-axis component v _i of their motion speeds _, and define the coordinate system of the area H in front of the door as the Y-axis in the downward direction of the center line , taking the rightward direction of the upper edge of the H area as the X-axis; S303: For all detected and generated objects O _i with motion trajectories Tri (t ₎ , record the distance between the objects O _i and the bottom line of the H area in the Y direction as D _i ; According to R _i (t)=max(cos(u _i ),0)×max(1.5-v _i ×(Tt)/D _i ,0), calculate the risk of the target O _i breaking through the door at the current moment t; S304: Calculate the total risk of passengers breaking through the door: RH(t)=max(R ₁ (t), R ₂ (t), . . . , R _n (t)). 5. A kind of video analysis-based subway screen door impending closing door passenger breaks the door early warning method according to claim 1, it is characterized in that, described step S4 comprises: The first-level response to the total passenger door-breaking risk RH(t) exceeding the first threshold Th ₁ is to remind passengers through the sound and light alarm device installed above the screen door; The secondary response to the total passenger entry risk RH(t) exceeding the second threshold Th ₂ is to keep the acousto-optic alarm and at the same time link the screen door and the car door control system to keep the screen door and the car door open until the total number of passengers enter the door After the risk RH(t) returns to zero, the shielded door and the car door control system are linked to close the shielded door and the car door, wherein the second threshold Th ₂ is greater than the first threshold Th ₁ . 6. A kind of video analysis-based subway screen door impending closing door passenger breaks the door early warning method according to claim 5, it is characterized in that, described step S4 also comprises: Record and generate and save a video recording of a passenger entering a door while performing a primary or secondary response. 7. a kind of video analysis-based subway screen door near-closing passenger breaks the door early warning method according to claim 5, it is characterized in that, the first threshold Th ₁ is 0.25, and the second threshold Th ₂ is 0.5. 8. A video analysis-based early warning device for passengers entering the door of a subway screen door, which is close to closing, is characterized in that it includes a processor and a storage medium; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to perform the steps of the method according to any one of claims 1-7. 9. A storage medium on which a computer program is stored, wherein the computer program implements the steps of the method according to any one of claims 1 to 7 when the computer program is executed by a processor.

Images