CN111582171B

CN111582171B - Pedestrian red light running monitoring method, device and system and readable storage medium

Info

Publication number: CN111582171B
Application number: CN202010382657.4A
Authority: CN
Inventors: 余永龙; 谢会斌; 李聪廷
Original assignee: Jinan Boguan Intelligent Technology Co Ltd
Current assignee: Jinan Boguan Intelligent Technology Co Ltd
Priority date: 2020-05-08
Filing date: 2020-05-08
Publication date: 2024-04-09
Anticipated expiration: 2040-05-08
Also published as: CN111582171A

Abstract

The application discloses a pedestrian red light running monitoring method, device, system and computer readable storage medium, wherein the method comprises the following steps: acquiring a human body region image to be detected and a human face region image to be detected from the pedestrian scene image according to the human body region frame to be detected and the human face region frame to be detected; determining positions of a human body target and a human face target in a pedestrian scene image; acquiring the running track of the pedestrian according to the human body target and the human face target belonging to the same pedestrian; and judging whether the pedestrians run the red light according to the running track and the signal lamp state. According to the technical scheme, the human body region frame to be detected is located in the upper partial region of the pedestrian scene image, the human face region frame to be detected is located in the lower partial region of the image, and the human face region frame to be detected is equivalent to far attention to human bodies and near attention to human faces.

Description

Pedestrian red light running monitoring method, device and system and readable storage medium

Technical Field

The present application relates to the field of intelligent traffic technologies, and in particular, to a method, an apparatus, a system, and a computer readable storage medium for monitoring pedestrian red light running.

Background

The pedestrian red light running monitoring is paid more attention to as a component part in the intelligent traffic implementation, and the pedestrian red light running monitoring can play roles of finding, reminding and the like. When a pedestrian runs the red light, most zebra stripes enter a monitoring picture, so that the picture at a distance cannot be pulled too close.

Currently, pedestrian red light running monitoring is performed through face target monitoring, specifically, a depth intelligent camera (intelligent camera with deep learning capability) performs image shooting at the front end, processes the shot whole image to obtain a face target, realizes detection and tracking of the face target according to the image, and judges whether pedestrians form red light running behaviors according to signal lamp information. However, since the distance depth intelligent camera is far when the pedestrian just enters the picture or just steps on the farthest zebra crossing, the human face of the pedestrian is smaller, and the situation of unstable snapshot occurs due to the influence of the number of pedestrians, walking states (overlapping, crossing and the like) and the like during middle shooting, the tracking is interrupted, a complete evidence chain cannot be formed, and the accuracy of the judgment of the behavior of the pedestrian running the red light is reduced. In addition, because the whole image is processed and detected, the timeliness of the judgment of the pedestrian running the red light is reduced.

In summary, how to improve the accuracy and timeliness of the determination of the pedestrian running the red light is a technical problem to be solved by those skilled in the art.

Disclosure of Invention

In view of the foregoing, an object of the present application is to provide a method, apparatus, system and computer readable storage medium for monitoring a pedestrian running a red light, which are used for improving accuracy and timeliness of the determination of the pedestrian running the red light.

In order to achieve the above object, the present application provides the following technical solutions:

a pedestrian red light running monitoring method, comprising:

acquiring a pedestrian scene image on a zebra crossing, and acquiring a human body region image to be detected and a human face region image to be detected from the pedestrian scene image according to a preset human body region frame to be detected and a preset human face region frame to be detected; the human body region frame to be detected is positioned in the upper part region of the pedestrian scene image, the human face region frame to be detected is positioned in the lower part region of the pedestrian scene image, and an overlapping region exists between the human body region frame to be detected and the human face region frame to be detected;

determining the position of a human target in the human region image to be detected in the pedestrian scene image, and determining the position of a human face target in the human face region image to be detected in the pedestrian scene image;

Determining a human body target and a human face target belonging to the same pedestrian, acquiring the coordinate position of the pedestrian according to the human body target and the human face target belonging to the same pedestrian, and acquiring the running track of the pedestrian according to the coordinate position of the pedestrian in a plurality of frames of pedestrian scene images;

judging whether the pedestrian runs the red light or not according to the running track and the signal lamp state.

Preferably, determining the position of the human target in the human region to be detected image in the pedestrian scene image, and determining the position of the human face target in the human face region to be detected image in the pedestrian scene image includes:

inputting the human body region image to be detected into a human body detection model obtained by pre-training to obtain a human body target and the position of the human body target in the human body region image to be detected, and inputting the human face region image to be detected into a human face detection model obtained by pre-training to obtain a human face target and the position of the human face target in the human face region image to be detected;

the human body target and the position thereof in the human body area to be detected image are converted into the pedestrian scene image to determine the position of the human body target in the pedestrian scene image, and the human face target and the position thereof in the human face area to be detected image are converted into the pedestrian scene image to determine the position of the human face target in the pedestrian scene image.

Preferably, before inputting the image of the region to be detected of the human body into a human body detection model obtained by training in advance, the method further comprises:

the human body detection model is used for carrying out edge repair on the human body to-be-detected region image according to the requirement of the human body detection model on the aspect ratio of the input image, so that the aspect ratio of the human body to-be-detected region image after edge repair meets the requirement of the human body detection model on the aspect ratio of the input image and the content in the human body to-be-detected region image is not distorted;

before the image of the area to be detected of the human face is input into the human face detection model obtained by training in advance, the method further comprises the following steps:

and carrying out edge repair on the face to-be-detected region image according to the requirement of the face detection model on the aspect ratio of the input image so that the aspect ratio of the edge repaired face to-be-detected region image meets the requirement of the face detection model on the aspect ratio of the input image and the content in the face to-be-detected region image is not distorted.

Preferably, the setting process of the human body region frame to be detected and the human face region frame to be detected is as follows:

determining an upper boundary line of a human face region frame to be detected and a lower boundary line of a human body region frame to be detected in advance according to shooting parameters of an intelligent camera; the intelligent camera is used for acquiring a pedestrian scene image on a zebra crossing, the shooting parameters comprise a lower limit value of the size of a human face which can be detected by the intelligent camera, and a lower boundary line of a human body region frame to be detected is positioned below an upper boundary line of the human face region frame to be detected and is separated from the upper boundary line of the human face region frame to be detected by a preset distance;

Determining the face to-be-detected area frame according to the upper boundary line of the face to-be-detected area frame, the lower boundary line of the image acquired by the intelligent camera and the pre-drawn zebra crossing boundary line;

and determining the human body region frame to be detected according to the lower boundary line of the human body region frame to be detected, the upper boundary line of the image acquired by the intelligent camera and the zebra crossing boundary line.

Preferably, when determining that the pedestrian runs the red light, the method further comprises:

and sending the face target of the pedestrian to a back-end server, displaying the face target by the back-end server, and identifying and storing the identity information of the pedestrian according to the face target.

Preferably, the method further comprises:

receiving a plurality of mixing lines which are sent by a camera Web end or a camera configuration terminal and are arranged on the pedestrian image in advance; wherein the plurality of mixing lines are sequentially arranged from far to near in the pedestrian scene image;

correspondingly, judging whether the pedestrian runs the red light or not according to the running track and the signal lamp state comprises the following steps:

judging whether the pedestrian runs the red light or not according to the signal lamp state when the running track is intersected with the plurality of mixing lines.

Preferably, the method further comprises:

If the pedestrian crosses the first mixed line when the signal lamp state is the red lamp state according to the pedestrian scene image, a signal that the pedestrian possibly runs the red lamp is sent to the back-end server, so that the back-end server sends out early warning, if the pedestrian continuously crosses all the mixed lines when the signal lamp state is the red lamp state, a signal that the pedestrian runs the red lamp is sent to the back-end server, and the step of sending the face target of the pedestrian to the back-end server is executed.

A pedestrian red light running monitoring device, comprising:

the acquisition module is used for acquiring a pedestrian scene image on the zebra stripes and acquiring a human body region image to be detected and a human face region image to be detected from the pedestrian scene image according to a preset human body region frame to be detected and a preset human face region frame to be detected; the human body region frame to be detected is positioned in the upper part region of the pedestrian scene image, the human face region frame to be detected is positioned in the lower part region of the pedestrian scene image, and an overlapping region exists between the human body region frame to be detected and the human face region frame to be detected;

the first determining module is used for determining the position of a human target in the human region image to be detected in the pedestrian scene image and determining the position of a human face target in the human face region image to be detected in the pedestrian scene image;

The second determining module is used for determining a human body target and a human face target belonging to the same pedestrian, acquiring the coordinate position of the pedestrian according to the human body target and the human face target belonging to the same pedestrian, and acquiring the running track of the pedestrian according to the coordinate position of the pedestrian in a plurality of frames of pedestrian scene images;

and the judging module is used for judging whether the pedestrian runs the red light or not according to the running track and the signal lamp state.

The utility model provides a pedestrian's monitoring system that makes a dash across red light, includes intelligent camera, with the back-end server that intelligent camera links to each other, wherein:

the intelligent camera is used for acquiring a pedestrian scene image on the zebra stripes and acquiring a human body region image to be detected and a human face region image to be detected from the pedestrian scene image according to a preset human body region frame to be detected and a preset human face region frame to be detected; the human body region frame to be detected is positioned in the upper part region of the pedestrian scene image, the human face region frame to be detected is positioned in the lower part region of the pedestrian scene image, and an overlapping region exists between the human body region frame to be detected and the human face region frame to be detected; determining the position of a human target in the human region image to be detected in the pedestrian scene image, and determining the position of a human face target in the human face region image to be detected in the pedestrian scene image; determining a human body target and a human face target belonging to the same pedestrian, acquiring the coordinate position of the pedestrian according to the human body target and the human face target belonging to the same pedestrian, and acquiring the running track of the pedestrian according to the coordinate position of the pedestrian in a plurality of frames of pedestrian scene images; judging whether the pedestrian runs a red light or not according to the running track and the signal lamp state;

And the back-end server is used for detecting the signal lamp state and sending the signal lamp state to the intelligent camera.

A computer readable storage medium having stored thereon a computer program which when executed by a processor implements the steps of the pedestrian red light running monitoring method of any one of the preceding claims.

The application provides a pedestrian red light running monitoring method, device and system and a computer readable storage medium, wherein the method comprises the following steps: acquiring a pedestrian scene image on a zebra crossing, and acquiring a human body region image to be detected and a human face region image to be detected from the pedestrian scene image according to a preset human body region frame to be detected and a preset human face region frame to be detected; the human body region frame to be detected is positioned in the upper part region of the pedestrian scene image, the human face region frame to be detected is positioned in the lower part region of the pedestrian scene image, and an overlapping region exists between the human body region frame to be detected and the human face region frame to be detected; determining the position of a human target in a human region image to be detected in a pedestrian scene image, and determining the position of a human face target in a human face region image to be detected in the pedestrian scene image; determining human body targets and human face targets belonging to the same pedestrian, acquiring coordinate positions of the pedestrian according to the human body targets and the human face targets belonging to the same pedestrian, and acquiring a running track of the pedestrian according to the coordinate positions of the pedestrian in a multi-frame pedestrian scene image; and judging whether the pedestrians run the red light according to the running track and the signal lamp state.

According to the technical scheme, the upper part area of the pedestrian scene image is provided with a human body to-be-detected area frame, the lower part area is provided with a human face to-be-detected area frame, an overlapping area exists between the two to-be-detected area frames, when the human body to-be-detected area image and the human face to-be-detected area image are correspondingly acquired from the pedestrian scene image according to the two to-be-detected area frames, the human body to be detected is focused far away and the human face to be focused near, then the human body target and the human face target belonging to the same pedestrian are determined through the positions of the human body target in the human body to-be-detected area image and the human face target in the human face to-be-detected area image in the pedestrian scene image, and according to the human body target and the human face target which are determined, the coordinate position of the pedestrian is obtained, the running track is obtained, and according to the running track and the signal lamp state, whether the pedestrian runs the red light is judged, compared with the problem that the evidence chain is interrupted only by monitoring the human face target at present, the method and the device have the advantages that the complete running track of the pedestrian is obtained by obtaining the human body target of the pedestrian at a distance and the human face target of the pedestrian at a near position and according to information fusion of the human body target and the human face target, so that the accuracy of the judgment of the pedestrian running the red light is improved.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present application, and that other drawings may be obtained according to the provided drawings without inventive effort to a person skilled in the art.

Fig. 1 is a flowchart of a method for monitoring pedestrian red light running provided in an embodiment of the present application;

fig. 2 is a flowchart of setting a human body region frame to be detected and a human face region frame to be detected according to an embodiment of the present application;

fig. 3 is a schematic structural diagram of a device for monitoring pedestrian red light running according to an embodiment of the present application;

fig. 4 is a schematic structural diagram of a pedestrian red light running monitoring system according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.

Referring to fig. 1, a flowchart of a method for monitoring pedestrian red light running provided by an embodiment of the present application is shown, where the method for monitoring pedestrian red light running provided by the embodiment of the present application may include:

s11: and acquiring a pedestrian scene image on the zebra stripes, and acquiring a human body region image to be detected and a human face region image to be detected from the pedestrian scene image according to a preset human body region frame to be detected and a preset human face region frame to be detected.

The human body region frame to be detected is located in the upper portion region of the pedestrian scene image, the human face region frame to be detected is located in the lower portion region of the pedestrian scene image, and an overlapping region exists between the human body region frame to be detected and the human face region frame to be detected.

When the pedestrians run the red light, the intelligent camera is arranged on one side of the zebra crossing in advance, and parameters such as pitch angle and focal length of the intelligent camera are adjusted, so that the scene worker survey meets the standard worker survey of the pedestrians running the red light. After the intelligent camera is installed, an effective detection area (i.e., a human to-be-detected area frame) of a human body and an effective detection area (i.e., a human face to-be-detected area frame) of a human face can be set in advance according to shooting parameters of the intelligent camera and a shot image, wherein, considering that the human face of a pedestrian located far in an image shot by the intelligent camera is smaller and the human face of a pedestrian located near is larger, the human to-be-detected area frame corresponds to a far picture of the image shot by the intelligent camera (i.e., the human to-be-detected area frame is located in an upper part area of a pedestrian scene image), the human face to-be-detected area frame corresponds to a near picture of the image shot by the intelligent camera (i.e., the human face to-be-detected area frame is located in a lower part area of the pedestrian scene image), and an overlapping area exists between the human to-be-detected area frame and the human face to-be-detected area frame.

When the pedestrians run the red light, the intelligent camera can be used for collecting and acquiring the images of the pedestrians on the zebra stripes in real time, and of course, the images of the pedestrians can be acquired in other modes. After the pedestrian scene images are acquired, the human body to-be-detected area images and the human face to-be-detected area images can be acquired from each pedestrian scene image according to the preset human body to-be-detected area frame and the human face to-be-detected area frame. The method comprises the steps that a pedestrian scene image is obtained, and meanwhile, the signal lamp state sent by a back-end server can be received in real time, so that whether the pedestrian runs a red light or not can be judged according to the running track of the pedestrian and the signal lamp state.

According to the method for acquiring the human body to-be-detected area image from the pedestrian scene image according to the preset human body to-be-detected area frame and acquiring the human face to-be-detected area image from the pedestrian scene image according to the preset human face to-be-detected area frame, the required effective areas can be contained, the processing area of the image can be reduced, the calculated amount of an intelligent camera is reduced, the operation efficiency of the system is improved, and the timeliness of judging the red light running of the pedestrian is improved conveniently.

S12: and determining the position of a human target in the human region image to be detected in the pedestrian scene image, and determining the position of a human face target in the human face region image to be detected in the pedestrian scene image.

After the human body region image to be detected and the human face region image to be detected are obtained, the position of the human body target in the human body region image to be detected in the pedestrian scene image can be determined, and the position of the human face target in the human face region image to be detected in the pedestrian scene image can be determined, so that whether the human body target and the human face target belonging to the same pedestrian exist or not can be determined according to the position of the human body target in the pedestrian scene image and the position of the human face target in the pedestrian scene image.

When the steps are executed, a human body sequence number can be allocated to each human body target contained in the human body to-be-detected area image, and a human face sequence number can be allocated to each human face target contained in the human face to-be-detected area image, so that pedestrians can be determined through the corresponding sequence numbers.

In addition, after the human body target and the human face target are acquired, both targets are tracked based on an MOT (Multiple Object Tracking, multi-target tracking) tracking frame, and only the human body target is much larger than the human face target, so that part of tracked parameters are adjusted in actual operation.

S13: the method comprises the steps of determining human body targets and human face targets belonging to the same pedestrian, acquiring coordinate positions of the pedestrian according to the human body targets and the human face targets belonging to the same pedestrian, and acquiring running tracks of the pedestrian according to the coordinate positions of the pedestrian in a multi-frame pedestrian scene image.

After step S12 is performed, after the human body target and the human face target (specifically, whether the human face target is in a preset area of the human body target or not) belonging to the same pedestrian may be determined according to the position of the human body target in the human body to-be-detected area image in the pedestrian scene image and the position of the human face target in the human face to-be-detected area image in the pedestrian scene image, the coordinate position of the pedestrian in the pedestrian scene image may be obtained according to the human body target and the human face target belonging to the same pedestrian, the running track of the pedestrian may be obtained through multi-target tracking and according to the coordinate position of the pedestrian in the multi-frame pedestrian scene image, and the running track of the pedestrian may be determined according to the coordinate position of the foot of the pedestrian in the pedestrian scene image, wherein the running track of the pedestrian may be determined along the human face target by moving down a preset length when the human face target is corresponding to obtain the foot of the pedestrian, and then the running track of the pedestrian is determined according to the foot of the pedestrian. Of course, in addition to the acquisition of the moving track by the foot, the moving track of the pedestrian may be acquired according to a line formed by a certain fixed point of the human body target of the pedestrian in the multi-frame pedestrian scene image.

When determining a human body target and a human face target belonging to the same pedestrian, if the human body target and the human face target are determined to belong to the same pedestrian in a certain frame of pedestrian scene image, in order to improve the accuracy of determination, the front and rear frames of pedestrian scene images of the frame of pedestrian scene image can be judged to determine whether the human body target and the human face target both belong to the same pedestrian in a plurality of frames of pedestrian scene images, if so, the human body target and the human face target can be determined to indeed belong to the pedestrian, that is, if the human face target is located in a preset area of the human body target in a plurality of frames of pedestrian scene images, the human body target and the human face target can be determined to belong to the pedestrian.

It should be noted that, no matter how many pedestrians exist in the pedestrian scene image, the positions of the human body targets and the human face targets in the pedestrian scene image can be judged and determined, after a plurality of pedestrians are determined, the pedestrian images corresponding to the pedestrians can be obtained according to the human body targets and the human face targets corresponding to the pedestrians in the above manner, and the running tracks can be obtained according to the pedestrian images corresponding to the pedestrians.

S14: and judging whether the pedestrians run the red light according to the running track and the signal lamp state.

After the running track of the pedestrian is obtained, whether the pedestrian runs the red light or not can be judged according to the running track and the signal lamp state, wherein the signal lamp state can be obtained by real-time detection and transmission by a signal lamp detector in the rear-end server, so that the accuracy of the judgment of running the red light by the pedestrian is improved according to the signal lamp state received in real time.

Compared with the prior art, the problem that a complete evidence chain cannot be formed due to the fact that the human face target detection and tracking are conducted only through the fact that a far human face is smaller and the middle shooting is unstable in snapshot is solved, the complete moving track of a pedestrian can be obtained through the acquisition, processing, judging and combination of the human body region image to be detected and the human face region image to be detected, the complete evidence chain is formed conveniently, and the accuracy of red light running judgment of the pedestrian can be improved. In addition, compared with the detection of the whole image in the prior art, the method and the device can reduce the target detection area by presetting the human body to-be-detected area frame and the human face to-be-detected area frame, so that the calculation cost of target detection is reduced, and the detection efficiency and the timeliness of pedestrian red light running judgment are improved.

The method for monitoring pedestrian red light running provided by the embodiment of the application determines the position of a human target in a human region image to be detected in a pedestrian scene image and determines the position of a human face target in a human face region image to be detected in the pedestrian scene image, and comprises the following steps:

inputting the human body to-be-detected area image into a human body detection model obtained by pre-training to obtain a human body target and the position of the human body target in the human body to-be-detected area image, and inputting the human face to-be-detected area image into a human face detection model obtained by pre-training to obtain a human face target and the position of the human face target in the human face to-be-detected area image;

the human body target and the position thereof in the image of the region to be detected of the human body are converted into the pedestrian scene image to determine the position of the human body target in the pedestrian scene image, and the human face target and the position thereof in the region to be detected of the human face are converted into the pedestrian scene image to determine the position of the human face target in the pedestrian scene image.

After the human body to be detected area image and the human face to be detected area image are obtained, the human body to be detected area image is input into a human body detection model obtained through pre-training, so that the human body target in the human body to be detected area image and the position of the human body target in the human body to be detected area image (specifically, the coordinates of the human body target in the human body to be detected area image) are obtained through the human body detection model obtained through pre-training, the human face to be detected area image is input into a human face detection model obtained through pre-training, so that the positions of the human face target in the human face to be detected area image and the position of the human face target in the human face to be detected area image (specifically, the coordinates of the human face target in the human face to be detected area image) are obtained through the human body detection model obtained through pre-training, then the position of the human body target in the human body to be detected area image and the position of the human face target in the human face to be detected area image are reversely returned to the original pedestrian scene image, so that the position of the human body target in the human face to be detected area image in the human body scene image is determined.

Specifically, a human body detection model and a human face detection model can be obtained by training based on RFCN (Object Detection via Region-based Fully ConvolutionalNetworks, a full convolution deep learning network architecture for target detection), then, an image of a region to be detected of a human body can be input into the human body detection model obtained by training based on RFCN in advance, and an image of a region to be detected of a human face can be input into the human body detection model obtained by training based on RFCN in advance, so that a human body target and coordinates thereof and a human face target and coordinates thereof can be determined.

The method for monitoring pedestrian red light running provided by the embodiment of the application, before inputting the image of the region to be detected of the human body into the human body detection model obtained by training in advance, further comprises the following steps:

the method comprises the steps of carrying out edge-repairing on an image of a region to be detected of a human body according to the requirement of a human body detection model on the aspect ratio of the input image, so that the aspect ratio of the image of the region to be detected of the human body after edge-repairing meets the requirement of the human body detection model on the aspect ratio of the input image and the content in the image of the region to be detected of the human body is not distorted;

before the image of the area to be detected of the human face is input into the human face detection model obtained by training in advance, the method can further comprise the following steps:

And carrying out edge-filling on the face to-be-detected region image according to the requirement of the face detection model on the aspect ratio of the input image, so that the aspect ratio of the edge-filled face to-be-detected region image meets the requirement of the face detection model on the aspect ratio of the input image and the content in the face to-be-detected region image is not distorted.

In order to avoid distortion of the target caused by the fact that the aspect ratio of the input image does not meet the requirement when the input image is detected by the detection model, the human body region image to be detected can be subjected to edge repair according to the requirement of the human body detection model on the aspect ratio of the input image before the human body region image to be detected is input into the human body detection model obtained in advance, specifically, the human body region image to be detected can be subjected to edge repair (specifically, black edge repair) along the length direction and/or the width direction of the human body region image according to the requirement of the human body detection model on the aspect ratio of the input image, so that the aspect ratio of the human body region image after edge repair meets the requirement of the human body detection model on the aspect ratio of the input image, and similarly, the human face region image to be detected can be subjected to edge repair according to the requirement of the aspect ratio of the human face detection model before the human face region image to be detected is input into the human face detection model obtained in advance, the aspect ratio of the human body region image to be detected can be subjected to edge repair according to the requirement of the aspect ratio of the human face detection model, the aspect ratio of the human body region image to be detected after edge repair can be reduced, and the problem of the human face region image to be detected after edge repair can be reduced, and the target can be detected due to the detected.

Referring to fig. 2, a flowchart of setting a human body region frame to be detected and a human face region frame to be detected provided in an embodiment of the present application is shown. The setting process of the human body region frame to be detected and the human face region frame to be detected can be as follows:

s21: determining an upper boundary line of a human face region frame to be detected and a lower boundary line of a human body region frame to be detected in advance according to shooting parameters of an intelligent camera; the intelligent camera is used for acquiring a pedestrian scene image on the zebra stripes, the shooting parameters comprise a lower limit value of the size of the human face which can be detected by the intelligent camera, and a lower boundary line of the human face region frame to be detected is positioned below an upper boundary line of the human face region frame to be detected and is a preset distance away from the upper boundary line of the human face region frame to be detected;

s22: determining a face to-be-detected area frame according to an upper boundary line of the face to-be-detected area frame, a lower boundary line of an image acquired by the intelligent camera and a pre-drawn zebra crossing boundary line;

s23: and determining the human body region frame to be detected according to the lower boundary line of the human body region frame to be detected, the upper boundary line of the image acquired by the intelligent camera and the zebra crossing boundary line.

Specifically, the human body region frame to be detected and the human face region frame to be detected can be preset in the following manner:

The method comprises the steps of determining an upper boundary line of a human face to be detected area frame and a lower boundary line of a human body to be detected area frame in advance according to shooting parameters of an intelligent camera, wherein the intelligent camera is used for acquiring a pedestrian scene image on a zebra crossing in real time, the shooting parameters comprise a lower limit value h of a human face size which can be stably detected by the intelligent camera under the current resolution (the lower limit value is different under different resolutions), namely, taking a position of an adult standing in a picture when the human face size reaches approximately h, taking a top of a human head as a horizontal line L1, taking the horizontal line L1 as the upper boundary line of the human face to be detected area frame, taking a chin of the human face as a horizontal line L2 (wherein the distance between L1 and L2 is h), taking a waist position of the human body (approximately moves downwards by 5 h or sets the size of other h according to actual conditions) as a horizontal line L3, and taking the horizontal line L3 as the lower boundary line of the human body to be detected area frame;

after determining the boundary line on the face to-be-detected area frame, considering that the near face is clearer, determining the face to-be-detected area frame according to the boundary line on the face to-be-detected area frame, the lower boundary line of the image acquired by the intelligent camera and zebra crossing boundary lines L4 and L5 (wherein L4 and L5 are respectively positioned at the left side and the right side of the zebra crossing) which are input by a client through a camera Web end or a camera configuration terminal in advance according to requirements, namely, the face to-be-detected area frame is an area frame surrounded by the boundary lines;

Considering that the far face is not clear, the far area can be set as a human body area frame to be detected, specifically, the human body area frame to be detected is determined according to the determined lower boundary line of the human body area frame to be detected, the upper boundary line of the image acquired in advance in an intelligent mode and the zebra crossing boundary line input by a client through a camera Web end or a camera configuration terminal in advance according to requirements, namely, the human body area frame to be detected is an area frame surrounded by the boundary lines.

By the setting method, pedestrian information on the zebra stripes can be conveniently contained, and areas outside the zebra stripes can be eliminated, so that the calculated amount is reduced, and the processing efficiency is improved.

The method for monitoring the pedestrian running the red light provided by the embodiment of the application can further comprise the following steps when determining that the pedestrian runs the red light:

the method comprises the steps of sending a face target of a pedestrian to a back-end server, displaying the face target by the back-end server, and identifying and storing identity information of the pedestrian according to the face target.

When the pedestrian runs the red light according to the running track and the signal lamp state, the face target of the pedestrian can be obtained, the face target with higher definition and quality can be obtained when the face target of the pedestrian is obtained, then the face target of the pedestrian can be sent to the back-end server, the back-end server displays the face target of the pedestrian so as to warn and remind the pedestrian, meanwhile, the back-end server can compare the face target of the pedestrian with the face comparison library so as to identify and obtain the identity information of the pedestrian and save the identity information of the pedestrian, and meanwhile, the corresponding pedestrian image can be saved when the pedestrian runs the red light so as to be convenient for carrying out punishment, education and the like on the pedestrian.

Through the process, whether the pedestrian runs the red light can be determined, and the pedestrian running the red light can be accurately positioned through the acquisition of the face target of the pedestrian running the red light and the identification of the identity information of the pedestrian running the red light.

The method for monitoring pedestrian red light running provided by the embodiment of the application can further comprise the following steps:

receiving a plurality of mixed lines set on a pedestrian scene image sent by a camera Web end or a camera configuration terminal in advance; wherein, a plurality of mixing lines are sequentially arranged from far to near in the pedestrian scene image;

correspondingly, judging whether the pedestrian runs the red light according to the running track and the signal lamp state can comprise:

and judging whether the pedestrians run the red light according to the state of the signal lamp when the running track is intersected with the plurality of mixing lines.

When the pedestrian runs the red light, a customer can set a plurality of mixing lines in advance through a camera Web end or a camera configuration terminal according to the actual conditions of the zebra stripes and the like, wherein the mixing lines are sequentially arranged from far to near in a scene image of the pedestrian and are approximately parallel to the zebra stripes, and accordingly, when judging whether the pedestrian runs the red light, the pedestrian can be judged whether to run the red light according to the condition that the running track of the pedestrian is intersected with the preset mixing lines and the signal lamp state when the pedestrian runs the red light, so that the accuracy of the pedestrian running the red light judgment is improved. For example: three mixing lines can be preset, and if the pedestrian spans the three mixing lines when the signal lamp state is the red lamp state according to the running track of the pedestrian, the pedestrian is determined to run the red lamp.

For the above process, when the pedestrian arrives at each mixing line, the identifier corresponding to the state of the signal lamp and the pedestrian at that time can be generated, for example: the method comprises the steps of generating the mark 1 corresponding to a pedestrian in a red light state, generating the mark 0 corresponding to the pedestrian in other signal light states, and subsequently extracting the marks corresponding to the set plurality of mixed lines and judging whether the pedestrian runs the red light according to the marks, so that convenience and accuracy of the judgment of running the red light by the pedestrian are improved.

if the pedestrian crosses the first mixed line when the signal lamp state is the red lamp state according to the pedestrian scene image, a signal that the pedestrian possibly runs the red lamp is sent to the back-end server, so that the back-end server sends out early warning, if the pedestrian continuously crosses all the set mixed lines when the signal lamp state is the red lamp state, a signal that the pedestrian runs the red lamp is sent to the back-end server, and the step of sending the face target of the pedestrian to the back-end server is executed.

In the monitoring of the pedestrian crossing the red light, if the pedestrian crosses the first mixed line when the signal lamp state is the red light state according to the pedestrian scene image, the intelligent camera can send a signal that the pedestrian possibly crosses the red light to the back-end server, at the moment, the back-end server can carry out early warning prompt through an early warning device such as a sound device and discourage the pedestrian from crossing the red light to return to the roadside, if the pedestrian continuously crosses all the set mixed lines when the signal lamp state is the red light state, the intelligent camera judges that the pedestrian has the red light-crossing behavior, at the moment, the intelligent camera can send the signal that the pedestrian crosses the red light to the back-end server and execute the step of sending the face target of the pedestrian to the back-end server so as to carry out processing such as display and the face target of the pedestrian by the back-end server.

The embodiment of the application also provides a pedestrian red light running monitoring device, see fig. 3, which shows a structural schematic diagram of the pedestrian red light running monitoring device provided by the embodiment of the application, and may include:

the acquiring module 31 is configured to acquire a pedestrian scene image on a zebra crossing, and acquire a human body region image to be detected and a human face region image to be detected from the pedestrian scene image according to a preset human body region frame to be detected and a preset human face region frame to be detected;

a first determining module 32, configured to determine a position of a human target in the image of the region to be detected of the human body in the image of the pedestrian scene, and determine a position of a human face target in the image of the region to be detected of the human face in the image of the pedestrian scene;

the second determining module 33 is configured to determine a human body target and a human face target that belong to the same pedestrian, obtain a coordinate position of the pedestrian according to the human body target and the human face target that belong to the same pedestrian, and obtain a running track of the pedestrian according to the coordinate position of the pedestrian in the multi-frame pedestrian scene image;

and the judging module 34 is used for judging whether the pedestrian runs the red light according to the running track and the signal lamp state.

The embodiment of the application provides a pedestrian red light running monitoring device, the first determining module 32 may include:

The acquisition unit is used for inputting the human body to-be-detected area image into a human body detection model obtained by pre-training so as to obtain a human body target and the position of the human body target in the human body to-be-detected area image, and inputting the human face to-be-detected area image into a human face detection model obtained by pre-training so as to obtain the human face target and the position of the human face target in the human face to-be-detected area image;

the conversion unit is used for converting the human body target and the position thereof in the image of the region to be detected of the human body into the pedestrian scene image to obtain the position of the human body target in the pedestrian scene image, and converting the human face target and the position thereof in the region to be detected of the human face into the pedestrian scene image to obtain the position of the human face target in the pedestrian scene image.

The embodiment of the application provides a pedestrian red light running monitoring device, the first determining module 32 may further include:

the first adjusting unit is used for carrying out edge-repairing on the human body to-be-detected region image according to the requirement of the human body detection model on the aspect ratio of the input image before inputting the human body to-be-detected region image into the human body detection model obtained by training in advance, so that the aspect ratio of the human body to-be-detected region image after edge-repairing meets the requirement of the human body detection model on the aspect ratio of the input image and the content in the human body to-be-detected region image is not distorted;

And the second adjusting unit is used for carrying out edge-repairing on the face to-be-detected area image according to the requirement of the face detection model on the aspect ratio of the input image before the face to-be-detected area image is input into the face detection model which is obtained through training in advance, so that the aspect ratio of the edge-repaired face to-be-detected area image meets the requirement of the face detection model on the aspect ratio of the input image and the content in the face to-be-detected area image is not distorted.

The embodiment of the application provides a pedestrian red light running monitoring device, including being used for predetermining human body and waiting to detect regional frame and the people's face and waiting to detect regional frame predetermine the module, predetermine the module and can include:

the first determining unit is used for determining an upper boundary line of a human face region frame to be detected and a lower boundary line of the human body region frame to be detected in advance according to shooting parameters of the intelligent camera; the intelligent camera is used for acquiring a pedestrian scene image on the zebra stripes, the shooting parameters comprise a lower limit value of the size of the human face which can be detected by the intelligent camera, and a lower boundary line of the human face region frame to be detected is positioned below an upper boundary line of the human face region frame to be detected and is a preset distance away from the upper boundary line of the human face region frame to be detected;

The second determining unit is used for determining the face to-be-detected area frame according to the upper boundary line of the face to-be-detected area frame, the lower boundary line of the image acquired by the intelligent camera and the pre-drawn zebra crossing boundary line;

and the third determining unit is used for determining the human body region frame to be detected according to the lower boundary line of the human body region frame to be detected, the upper boundary line of the image acquired by the intelligent camera and the zebra crossing boundary line.

The embodiment of the application provides a pedestrian red light running monitoring device can also include:

the first sending module is used for sending the face target of the pedestrian to the rear-end server when the pedestrian is determined to run the red light, displaying the face target by the rear-end server, and identifying and storing the identity information of the pedestrian according to the face target.

the receiving module is used for receiving a plurality of mixed lines which are arranged on the pedestrian image and sent by the camera Web end or the camera configuration terminal in advance; wherein, a plurality of mixing lines are sequentially arranged from far to near in the pedestrian scene image;

accordingly, the determination module 34 may include:

and the judging unit is used for judging whether the pedestrians run the red light according to the signal lamp state when the running track is intersected with the plurality of mixing lines.

the second sending module is used for sending a signal that a pedestrian possibly runs the red light to the back-end server if the pedestrian crosses the first mixed line when the signal lamp state is the red light state according to the pedestrian scene image, so that the back-end server sends an early warning, sending a signal that the pedestrian runs the red light to the back-end server if the pedestrian continuously crosses all the set mixed lines when the signal lamp state is the red light state, and executing the step of sending the face target of the pedestrian to the back-end server.

The embodiment of the application also provides a pedestrian red light running monitoring system, see fig. 4, which shows a structural schematic diagram of the pedestrian red light running monitoring system provided by the embodiment of the application, which may include an intelligent camera 41, a rear end server 42 connected with the intelligent camera 41, wherein:

the intelligent camera 41 is configured to obtain a pedestrian scene image on a zebra crossing, and obtain a human body region image to be detected and a human face region image to be detected from the pedestrian scene image according to a preset human body region frame to be detected and a preset human face region frame to be detected; the human body region frame to be detected is positioned in the upper part region of the pedestrian scene image, the human face region frame to be detected is positioned in the lower part region of the pedestrian scene image, and an overlapping region exists between the human body region frame to be detected and the human face region frame to be detected; determining the position of a human target in a human region image to be detected in a pedestrian scene image, and determining the position of a human face target in a human face region image to be detected in the pedestrian scene image; determining human body targets and human face targets belonging to the same pedestrian, acquiring coordinate positions of the pedestrian according to the human body targets and the human face targets belonging to the same pedestrian, and acquiring a running track of the pedestrian according to the coordinate positions of the pedestrian in a multi-frame pedestrian scene image; judging whether pedestrians run red light according to the running track and the signal lamp state;

The back-end server 42 is configured to detect the signal light status and send the signal light status to the smart camera.

In the pedestrian red light running monitoring system provided by the embodiment of the application, the rear end server 42 may include a processor, a signal lamp detector connected with the processor, an early warning device, a display, a memory, a face recognition device, and a human-computer interaction interface, where the processor is connected with an intelligent camera, and the following steps are included:

the signal lamp detector is used for detecting the signal lamp state, sending a high level to the intelligent camera when the signal lamp state is a red lamp, and sending a low level to the intelligent camera when the signal lamp state is a green lamp so as to help the intelligent camera to judge whether pedestrians violate rules;

the early warning device is used for carrying out early warning and reminding when the intelligent camera judges that the pedestrian crosses the first mixed line in the red light state; the early warning module can be specifically sound equipment and the like;

the display is used for displaying the face target of the pedestrian running the red light when the intelligent camera judges that the pedestrian runs the red light, and of course, the display can also be used for displaying an evidence chain image (namely a pedestrian image corresponding to the pedestrian running the red light) and identity information when the pedestrian runs the red light and used for warning illegal pedestrians and other pedestrians;

The storage is used for storing and archiving evidence chain images of pedestrians running red light, face targets of pedestrians running red light and identity information so as to facilitate relevant departments to review and take the images;

the human face recognition device is used for comparing the human face target of the pedestrian running the red light with the population comparison library to acquire the identity information of the pedestrian, acquire records of the pedestrian running the red light for the past, and the like;

and the man-machine interaction interface is used for enabling the client to change part of parameters at any time.

The embodiment of the application also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program can realize the following steps when being executed by a processor:

acquiring a pedestrian scene image on a zebra crossing, and acquiring a human body region image to be detected and a human face region image to be detected from the pedestrian scene image according to a preset human body region frame to be detected and a preset human face region frame to be detected; the human body region frame to be detected is positioned in the upper part region of the pedestrian scene image, the human face region frame to be detected is positioned in the lower part region of the pedestrian scene image, and an overlapping region exists between the human body region frame to be detected and the human face region frame to be detected; determining the position of a human target in a human region image to be detected in a pedestrian scene image, and determining the position of a human face target in a human face region image to be detected in the pedestrian scene image; determining human body targets and human face targets belonging to the same pedestrian, acquiring coordinate positions of the pedestrian according to the human body targets and the human face targets belonging to the same pedestrian, and acquiring a running track of the pedestrian according to the coordinate positions of the pedestrian in a multi-frame pedestrian scene image; and judging whether the pedestrians run the red light according to the running track and the signal lamp state.

The description of the relevant parts in the pedestrian red light running monitoring device, the pedestrian red light running monitoring system and the computer readable storage medium provided in the embodiments of the present application can be referred to the detailed description of the corresponding parts in the pedestrian red light running monitoring method provided in the embodiments of the present application, and will not be repeated here.

It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements is inherent to. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element. In addition, the parts of the above technical solutions provided in the embodiments of the present application, which are consistent with the implementation principles of the corresponding technical solutions in the prior art, are not described in detail, so that redundant descriptions are avoided.

The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. The pedestrian red light running monitoring method is characterized by comprising the following steps of:

acquiring a pedestrian scene image on a zebra line in real time, and acquiring a human body region image to be detected and a human face region image to be detected from the pedestrian scene image according to a preset human body region frame to be detected and a preset human face region frame to be detected; the human body region frame to be detected is positioned in the upper part region of the pedestrian scene image, the human face region frame to be detected is positioned in the lower part region of the pedestrian scene image, and an overlapping region exists between the human body region frame to be detected and the human face region frame to be detected;

judging whether the pedestrian runs a red light or not according to the running track and the signal lamp state;

the determining the human body target and the human face target belonging to the same pedestrian, and obtaining the coordinate position of the pedestrian according to the human body target and the human face target belonging to the same pedestrian, and obtaining the running track of the pedestrian according to the coordinate position of the pedestrian in the multi-frame pedestrian scene image comprises:

determining human body targets and human face targets belonging to the same pedestrian by whether the human face targets are in a preset area of the human body targets or not, acquiring coordinate positions of the pedestrian in a pedestrian scene image according to the human body targets and the human face targets belonging to the same pedestrian, and acquiring a running track of the pedestrian by multi-target tracking and according to the coordinate positions of the pedestrian in a multi-frame pedestrian scene image; and moving the pedestrian downwards along the human face target by a preset length when the human face target is corresponding to the human face target, so as to obtain feet of the pedestrian, and determining the running track of the pedestrian.

2. The pedestrian red light running monitoring method according to claim 1, wherein determining a position of a human target in the human region image to be detected in the pedestrian scene image and determining a position of a human face target in the human face region image to be detected in the pedestrian scene image includes:

3. The pedestrian red light running monitoring method according to claim 2, further comprising, before inputting the image of the region to be detected of the human body into a human body detection model trained in advance:

4. The pedestrian red light running monitoring method according to claim 1, wherein the setting process of the human body region frame to be detected and the human face region frame to be detected is as follows:

5. The pedestrian red light running monitoring method according to any one of claims 1 to 4, further comprising, when determining that the pedestrian runs a red light:

6. The pedestrian red light running monitoring method of claim 5, further comprising:

receiving a plurality of mixing lines which are sent by a camera Web end or a camera configuration terminal and are arranged on the pedestrian scene image in advance; wherein the plurality of mixing lines are sequentially arranged from far to near in the pedestrian scene image;

7. The pedestrian red light running monitoring method of claim 6, further comprising:

8. A pedestrian red light running monitoring device, comprising:

the acquisition module is used for acquiring the pedestrian scene image on the zebra stripes in real time, and acquiring the human body region image to be detected and the human face region image to be detected from the pedestrian scene image according to a preset human body region frame to be detected and a preset human face region frame to be detected; the human body region frame to be detected is positioned in the upper part region of the pedestrian scene image, the human face region frame to be detected is positioned in the lower part region of the pedestrian scene image, and an overlapping region exists between the human body region frame to be detected and the human face region frame to be detected;

the second determining module is used for determining a human body target and a human face target belonging to the same pedestrian, acquiring the coordinate position of the pedestrian according to the human body target and the human face target, and acquiring the running track of the pedestrian according to the coordinate position of the pedestrian in a plurality of frames of pedestrian scene images;

the judging module is used for judging whether the pedestrian runs the red light or not according to the running track and the signal lamp state;

the second determining module is specifically configured to:

9. The pedestrian red light running monitoring system is characterized by comprising an intelligent camera and a rear-end server connected with the intelligent camera:

the intelligent camera is used for acquiring a pedestrian scene image on the zebra crossing in real time, and acquiring a human body region image to be detected and a human face region image to be detected from the pedestrian scene image according to a preset human body region frame to be detected and a preset human face region frame to be detected; the human body region frame to be detected is positioned in the upper part region of the pedestrian scene image, the human face region frame to be detected is positioned in the lower part region of the pedestrian scene image, and an overlapping region exists between the human body region frame to be detected and the human face region frame to be detected; determining the position of a human target in the human region image to be detected in the pedestrian scene image, and determining the position of a human face target in the human face region image to be detected in the pedestrian scene image; determining a human body target and a human face target belonging to the same pedestrian, acquiring the coordinate position of the pedestrian according to the human body target and the human face target belonging to the same pedestrian, and acquiring the running track of the pedestrian according to the coordinate position of the pedestrian in a plurality of frames of pedestrian scene images; judging whether the pedestrian runs a red light or not according to the running track and the signal lamp state;

The intelligent camera is specifically used for:

determining human body targets and human face targets belonging to the same pedestrian by whether the human face targets are in a preset area of the human body targets or not, acquiring coordinate positions of the pedestrian in a pedestrian scene image according to the human body targets and the human face targets belonging to the same pedestrian, and acquiring a running track of the pedestrian by multi-target tracking and according to the coordinate positions of the pedestrian in a multi-frame pedestrian scene image; the method comprises the steps of moving down a preset length along a human face target when the human face target is corresponding to the human face target, so as to obtain feet of a pedestrian, and determining a running track of the pedestrian;

10. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the pedestrian red light running monitoring method as claimed in any one of claims 1 to 7.