CN112307936A

CN112307936A - Passenger flow volume analysis method, system and device based on head and shoulder detection

Info

Publication number: CN112307936A
Application number: CN202011169824.3A
Authority: CN
Inventors: 张愈其; 钟南昌; 於景膦
Original assignee: Jiangsu Yuncongxihe Artificial Intelligence Co ltd
Current assignee: Jiangsu Yuncongxihe Artificial Intelligence Co ltd
Priority date: 2020-10-28
Filing date: 2020-10-28
Publication date: 2021-02-02

Abstract

The invention relates to the technical field of image processing related to passenger flow volume analysis in safety monitoring, and particularly provides a passenger flow volume analysis method, system and device based on head and shoulder detection, aiming at solving the problem of insufficient accuracy of the existing passenger flow volume monitoring method. To this end, the method of the invention comprises the following steps: performing head and shoulder detection on each frame of image data in the passenger flow crowd video; determining a region of interest ROI in each frame of image data and judging whether a head and shoulder detection frame is in the ROI; analyzing the characteristic information of the head and shoulder detection frame in the ROI, matching the characteristic information of the head and shoulder detection frame in the ROI among the continuous frames, determining that the matched head and shoulder detection frame belongs to the same target if the matching is successful, and outputting the tracking result information; and carrying out passenger flow statistics according to the tracking result information. Under the condition of adopting the method, the invention improves the efficiency and the accuracy of monitoring image processing and analysis and greatly improves the efficiency and the accuracy of passenger flow statistics.

Description

Passenger flow volume analysis method, system and device based on head and shoulder detection

Technical Field

The invention relates to the technical field of image processing related to passenger flow volume analysis in security monitoring, in particular to a passenger flow volume analysis method, system and device based on head and shoulder detection.

Background

In application scenes such as security, aviation, stores, stations and the like, service analysis on crowd density and passenger flow can improve the experience of travel, shopping and the like of customers on the one hand, and can enhance the safe operation efficiency and reduce the occurrence of safety accidents on the other hand. However, in the prior art, the conventional manual monitoring video lacks suitable automatic processing of the monitoring image, and mostly completes analysis work such as statistical estimation and the like on the video image manually, which greatly increases the labor and financial cost, and in some scenes, pedestrians cannot be accurately statistically estimated and the like due to factors such as personnel shielding and dim light and the like in the monitoring video. The human face recognition technology is used as strategies such as statistical estimation of specific image data in the analysis graph image, the human and financial costs can be effectively relieved to a certain extent, however, when the camera cannot capture the front face features of pedestrians, the problems that the image data of the pedestrians cannot be accurately acquired, the analysis statistical estimation is carried out and the like are still easily caused.

Therefore, there is a need to improve the scheme for optimizing the image data of the surveillance video so as to more accurately and efficiently complete the analysis and processing of the surveillance image data, especially the analysis of passenger flow statistics and density estimation.

Disclosure of Invention

In order to overcome the defects, the invention is provided to solve or at least partially solve the technical problems that the analysis and statistics passenger flow is inaccurate and the estimation deviation of the crowd density is large, which are caused by the fact that the monitoring technology cannot snapshot the face, people have shielding, light is dim and the like, and how to more accurately and efficiently process and identify the object for the image data. The invention provides a method, a system and a device for analyzing passenger flow based on head and shoulder detection to solve the technical problems.

In a first aspect, a method for head-shoulder detection-based passenger flow analysis is provided, including:

performing head and shoulder detection on each frame of image data in the acquired passenger flow crowd video to obtain a head and shoulder detection frame of each target in each frame of image data;

determining a region of interest ROI in each frame of image data and judging whether the head and shoulder detection frame is in the region of interest ROI;

analyzing the characteristic information of the head and shoulder detection frame in the ROI, matching the characteristic information of the head and shoulder detection frame in the ROI among continuous frames, determining that the head and shoulder detection frame which is matched belongs to the same target if matching is successful, and outputting the tracking result information of the head and shoulder detection frame which is successfully matched;

and carrying out passenger flow statistics according to the output tracking result information.

The method for determining the region of interest ROI in each frame of image data and judging whether the head and shoulder detection frame is in the region of interest ROI specifically includes:

assigning a digital ID number to each ROI determined in each frame of image data to identify a different ROI in each frame of image data;

and determining whether the head and shoulder detection frame in the image data is in the ROI according to the position of the ROI, the position of the head and shoulder detection frame and a central point.

Determining whether the head and shoulder detection frame in the image data is in the region of interest ROI according to the position of the region of interest ROI, the position of the head and shoulder detection frame, and the center point, specifically including:

judging whether the central point of the head and shoulder detection frame is in the ROI through a position judgment algorithm;

if yes, then subsequently participating in matching of the head and shoulder detection box;

the position determination algorithm includes, but is not limited to, ray determination, angle determination, and area determination.

When the position determination algorithm is an area determination method, the method specifically includes:

calculating the sum of the area of n triangles formed by the central point of the head-shoulder detection frame and each vertex of the ROI polygon of the region of interest by the following formula:

wherein the half perimeter of the triangle is

x_i、y_i、z_iIs a side of a triangle, p_iRepresenting the half perimeter of the ith triangle in n triangles formed by the central point p of the head and shoulder detection frame and each fixed point of the ROI polygon of the interesting region, wherein the value of i ranges from 1 to n, i is a positive integer less than or equal to n, and n is the number of connected triangles;

if the sum S of the areas of the triangles is larger than the area of the polygon of the ROI, determining that the center point of the head-shoulder detection frame is outside the ROI, otherwise, determining that the center point of the head-shoulder detection frame is inside the ROI.

Analyzing feature information of the head and shoulder detection frame in the region of interest ROI, matching the feature information of the head and shoulder detection frame in the region of interest ROI among consecutive frames, and determining that the head and shoulder detection frame to be matched belongs to the same target if matching is successful, specifically comprising:

the characteristic information of the head and shoulder detection frame comprises: area, direction of travel and speed;

determining whether the head and shoulder detection boxes in the ROI between the continuous frames are matched or not by utilizing the characteristic information based on a tracking algorithm;

the tracking algorithm includes: a Kalman filtering algorithm and an area overlapping matching algorithm;

and if the matching is successful, the head and shoulder detection frames matched between the continuous frames can determine that the head and the shoulder belong to the same target, and a tracking ID number is given to each head and shoulder detection frame belonging to the head and the shoulder of the same target so as to indicate that each head and shoulder detection frame successfully matched is the same target.

Wherein, based on a tracking algorithm, determining whether the head-shoulder detection boxes within the region of interest ROI match between consecutive frames using the feature information, further comprises:

if the motion directions and the speeds of the adjacent frames in the continuous frames in the feature information of the head and shoulder detection frame in the ROI are similar (for example, the similarity degree reaches 90%), performing a tracking algorithm of area overlapping matching, and determining whether the matching is successful according to the area overlapping degree;

the calculation formula of the tracking algorithm of the area overlap is as follows:

iou＝(u1∩u2)/(u1∪u2)

wherein iou is the cross-over ratio of the head and shoulder detection frames for matching between the continuous frames, u1 is the area of the first head and shoulder detection frame 1 for matching, and u2 is the area of the second head and shoulder detection frame 2 for matching; u1 ≧ u2 is the overlapping area of the head-shoulder detection frame 1 and the head-shoulder detection frame 2, u1 ≦ u2 is the sum of the areas of the head-shoulder detection frame 1 and the head-shoulder detection frame 2;

and comparing the intersection ratio iou of the head and shoulder detection frame matched between the continuous frames obtained by calculation with a preset threshold, wherein if the intersection ratio iou is greater than the preset threshold, the matching is regarded as successful, and otherwise, the matching fails.

Outputting tracking result information of the head and shoulder detection frame successfully matched, specifically comprising:

the tracking result information includes: and tracking ID number, running direction and speed of the head and shoulder detection frame which are successfully matched.

Wherein, the passenger flow statistics is performed according to the output tracking result information, which specifically comprises:

if the tracking ID number indicates that the head and shoulder detection frame of the same target enters from one side of a current ROI between continuous frames and then exits from the other side of the current ROI, determining to form a crossing; (ii) a

When the current ROI is determined to go out of the current ROI according to the running direction, recording the number ID of the current ROI and the edge sequence number of the edge of the current ROI through which the running direction indication passes when the current ROI goes out, wherein the number ID is used as passenger flow information; counting the number of detection frames of a certain frame of picture to obtain passenger flow density estimation as passenger flow information;

and counting all the recorded passenger flow information to obtain a passenger flow analysis result.

Wherein, still include:

and outputting the passenger flow analysis result containing the passenger flow statistics by adopting an asynchronous callback mode.

The obtained passenger flow crowd video specifically comprises:

monitoring and collecting passenger flow crowd videos;

decoding the video through an engine decoder to obtain multi-frame initial image data;

carrying out coding format conversion on the decoded initial image data and carrying out size conversion on the converted image data to obtain the multi-frame image data;

the BRG coding format and the size of each frame of image data in the multi-frame image data all meet the input requirement during head and shoulder detection.

Wherein, carry out the head and shoulder and detect, specifically include:

a dynamic mode and a static mode;

under each mode, extracting the characteristics of head and shoulder detection of each frame of image data through a deep learning network model to obtain the coordinate value of the head and shoulder position information of the head and shoulder detection frame of each target and the height and width of the head and shoulder detection frame;

wherein, the head and shoulder detection frame is a rectangular frame.

In a second aspect, a system for head and shoulder detection based passenger flow analysis is provided, comprising:

the detection module is used for carrying out head and shoulder detection on each frame of image data in the acquired passenger flow crowd video so as to obtain a head and shoulder detection frame of each target in each frame of image data;

the ROI filtering module is used for determining a region of interest ROI in each frame of image data and judging whether the head and shoulder detection frame is in the region of interest ROI or not;

the tracking module is used for analyzing the characteristic information of the head and shoulder detection frame in the ROI, matching the characteristic information of the head and shoulder detection frame in the ROI among continuous frames, determining that the head and shoulder detection frame which is matched belongs to the same target if the matching is successful, and outputting the tracking result information of the head and shoulder detection frame which is successfully matched;

and the passenger flow analysis module is used for carrying out passenger flow statistics according to the output tracking result information.

The ROI filtering module is configured to determine whether the head and shoulder detection frame is in the ROI, and specifically includes:

The ROI filtering module determines whether the head and shoulder detection frame in the image data is in the ROI according to the position of the ROI, the position of the head and shoulder detection frame, and the center point, and specifically includes:

if yes, subsequently participating in the matching of the head and shoulder detection frame;

wherein the half perimeter of the triangle is

x_i、y_i、z_iIs a side of a triangle; p is a radical of_iRepresenting the half perimeter of the ith triangle in n triangles formed by the central point p of the head and shoulder detection frame and each fixed point of the ROI polygon of the region of interest; i takes values from 1 to n, i is an integer less than or equal to n and n is the number of triangles formed by connection;

Wherein, the tracking module specifically includes:

iou＝(u1∩u2)/(u1∪u2)

Wherein, passenger flow analysis module specifically includes:

if the tracking ID number indicates that the head and shoulder detection frame of the same target enters from one side of a current ROI between continuous frames and then exits from the other side of the current ROI, determining to form a crossing;

when the current ROI is determined to go out of the ROI according to the running direction, recording the number ID of the ROI and the edge sequence number of the edge of the current ROI through which the running direction indication passes when the ROI goes out, and taking the edge sequence number as passenger flow information; counting the number of detection frames of a certain frame of picture to obtain passenger flow density estimation as passenger flow information;

The passenger flow analysis module further comprises:

The obtained passenger flow crowd video specifically comprises:

monitoring and collecting passenger flow crowd videos;

Wherein, carry out the head and shoulder and detect, specifically include:

a dynamic mode and a static mode;

in each mode, extracting the characteristics of head and shoulder detection of each frame of image data through a deep learning network model to obtain the coordinate value of the head and shoulder position information of the head and shoulder detection frame of each target and the height and width of the head and shoulder detection frame;

wherein, the head and shoulder detection frame is a rectangular frame.

In a third aspect, a computer-readable storage medium is provided, comprising: the storage medium stores a plurality of program codes adapted to be loaded and run by a processor to perform any of the methods of head-shoulder detection based passenger flow analysis described in the preceding claims.

In a fourth aspect, a processing device is provided, comprising a processor and a memory, said memory device being adapted to store a plurality of program codes, said program codes being adapted to be loaded and run by said processor to perform the method of head-shoulder detection based passenger flow analysis according to any of the preceding claims.

In a fifth aspect, an apparatus for passenger flow volume analysis based on head and shoulder detection is provided, comprising:

one or more detectors: acquiring each frame of image data in an input passenger flow crowd video, and performing head-shoulder detection on each frame of image data to obtain a head-shoulder detection frame of each target in each frame of image data;

ROI filtering unit: connecting the detector, determining a region of interest (ROI) in each frame of image data and judging whether the head and shoulder detection frame is in the region of interest (ROI);

one or more trackers: connecting the ROI filtering unit, analyzing the feature information of the head and shoulder detection frame in the ROI, matching the feature information of the head and shoulder detection frame in the ROI among continuous frames, determining that the head and shoulder detection frame which is matched belongs to the same target if the matching is successful, and outputting the tracking result information of the head and shoulder detection frame which is successfully matched;

an analyzer: and receiving the output from the tracker, and carrying out passenger flow statistics according to the output tracking result information.

Wherein, ROI filter unit specifically includes:

assigning a digital ID number to each ROI rendered in each frame of image data to identify a different ROI in each frame of image data;

The ROI filtering unit determines whether the head and shoulder detection frame in the image data is in the ROI according to the position of the ROI, the position of the head and shoulder detection frame, and the center point, and specifically includes:

wherein the half perimeter of the triangle is

Wherein, the tracker specifically includes:

iou＝(u1∩u2)/(u1∪u2)

Wherein, the analyzer specifically includes:

Wherein, still include:

and the callback unit outputs the passenger flow analysis result containing the passenger flow statistics in an asynchronous callback mode.

Wherein, still include:

the monitoring equipment monitors and acquires the passenger flow crowd video;

inputting the passenger flow crowd video into a preprocessing device for preprocessing;

wherein the preprocessing device includes an engine decoder and a converter;

the engine decoder decodes the video to obtain a plurality of frames of initial image data;

the converter converts the coding format of the decoded initial image data and performs size conversion on the converted image data to obtain the multi-frame image data; the BRG coding format and the size of each frame of image data in the multi-frame image data meet the input requirement during head and shoulder detection;

the preprocessing device inputs the multi-frame image data into the detector after preprocessing the multi-frame image data into image data which meets the requirement of head and shoulder detection.

Wherein, the detector carries out head and shoulder detection, specifically includes:

a dynamic mode and a static mode;

wherein, the head and shoulder detection frame is a rectangular frame.

One or more technical schemes of the invention at least have one or more of the following beneficial effects:

according to the technical scheme of the passenger flow volume analysis based on the head and shoulder detection, the ROI is drawn through each frame of image data obtained after preprocessing collected videos and/or images, the head and shoulder detection frame of each person in the ROI is tracked, and passenger flow statistics and density estimation are carried out on tracking results. The scheme of the invention can effectively reduce the situations of inaccurate passenger flow statistics and large crowd density estimation deviation caused by the fact that the monitoring technology cannot snapshot the face, personnel shielding, dim light and the like, greatly improves the accuracy of image data processing analysis and object identification and the analysis processing efficiency, and particularly improves the image analysis processing efficiency and accuracy of passenger flow statistics, density estimation and the like through image processing.

Drawings

Embodiments of the invention are described below with reference to the accompanying drawings, in which:

FIG. 1 is a principal flow diagram of one embodiment of a method of head and shoulder detection based passenger flow analysis of the present invention;

FIG. 2 is a schematic diagram of one embodiment of an area judged position determination algorithm in accordance with aspects of the present invention;

FIG. 3 is a schematic diagram of one embodiment of an area overlap matched tracking algorithm in accordance with aspects of the present invention;

FIG. 4 is a schematic diagram of one embodiment of a real case of passenger flow statistics in accordance with the inventive arrangements;

FIG. 5 is a schematic diagram of one embodiment of a real case of passenger flow density estimation according to aspects of the present invention;

fig. 6 is a block diagram of an embodiment of a system for head-shoulder detection-based passenger flow analysis according to the present invention.

Fig. 7 is a schematic structural composition diagram of another embodiment of the head-shoulder detection-based passenger flow volume analysis device according to the present invention.

Detailed Description

For the purpose of facilitating understanding of the present invention, the present invention will be described more fully and in detail below with reference to the accompanying drawings and examples, but it will be understood by those skilled in the art that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of the present invention.

In the description of the present invention, a "module" or "processor" may include hardware, software, or a combination of both. A module may comprise hardware circuitry, various suitable sensors, communication ports, memory, may comprise software components such as program code, or may be a combination of software and hardware. The processor may be a central processing unit, microprocessor, image processor, digital signal processor, or any other suitable processor. The processor has data and/or signal processing functionality. The processor may be implemented in software, hardware, or a combination thereof. Non-transitory computer readable storage media include any suitable medium that can store program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random-access memory, and the like. The term "a and/or B" denotes all possible combinations of a and B, such as a alone, B alone or a and B. The term "at least one A or B" or "at least one of A and B" means similar to "A and/or B" and may include only A, only B, or both A and B. The singular forms "a", "an" and "the" may include the plural forms as well.

In application scenes such as security, aviation, stores, stations and the like, service analysis on crowd density and passenger flow can improve the experience of travel, shopping and the like of customers on the one hand, and can enhance the safe operation efficiency and reduce the occurrence of safety accidents on the other hand. However, in the prior art, the traditional manual surveillance video greatly increases the labor and financial costs, and meanwhile, in some scenes, due to factors such as personnel shielding and dim light, pedestrians cannot be accurately analyzed and counted. The face recognition technology is used as a statistical strategy, so that the manpower and financial cost can be effectively relieved to a certain extent, but when the camera cannot capture the face characteristics of the pedestrian, the problem that the pedestrian cannot be accurately counted is easily caused. Thereby causing the defects of inaccurate passenger flow statistics and large crowd density estimation deviation.

One embodiment of the passenger flow volume analysis scheme based on head and shoulder detection is as follows: preprocessing the collected passenger flow crowd video/image to obtain multi-frame image data, such as: decoding video frames/images of a monitoring video through a decoder, converting the coding format of the decoded data, performing size conversion and other operations on the converted data to meet the detection input requirement and obtain multi-frame image data to be processed; then, head-shoulder detection is performed on each frame of image data to obtain a head-shoulder detection frame for each person in each frame of image, such as: judging whether the frame image (a picture) has a head shoulder or not through a head shoulder detection algorithm (including but not limited to CenteNet, ResNet and the like), and determining the position of the head shoulder; furthermore, an interesting region ROI in each frame of image data is drawn, whether the central point of the head and shoulder detection frame is in the interesting region ROI is judged, and digital ID numbers are respectively given to the interesting region ROI so as to realize ROI filtering, such as: judging whether the center point of the head and the shoulder is in the polygonal area by using a position judgment algorithm (including but not limited to ray judgment, included angle judgment, area judgment and the like); then, analyzing and acquiring feature information of the head and shoulder detection frames of the image data of the consecutive frames in the multi-frame image data, matching the feature information of the head and shoulder detection frames between each frame of the image data of the consecutive frames, assigning a tracking ID number to the successfully matched head and shoulder detection frame belonging to the same person, and outputting tracking result information of the head and shoulder detection frame, thereby tracking the head and shoulder of the same person by using the detection frame matching, such as: analyzing the characteristic information of the detection frames in the image data of the continuous frames through a tracking algorithm (including but not limited to a Kalman filtering algorithm and the like), namely, matching whether the two detection frames belong to the head and the shoulder of the same person or not, determining that the two detection frames belong to the head and the shoulder of the same person if the two detection frames can be matched, namely, the two detection frames are successfully matched, giving a label of a tracking id to the matched detection frame, and meanwhile, outputting the direction information matched to the detection frames belonging to the same person by the tracking algorithm so as to facilitate passenger flow statistics, crowd density estimation/estimation and the like; furthermore, according to the tracking result information and the information of the ROI traversed by the same person, passenger flow analysis is performed, such as: when the same person passes through the ROI, the number ID of the current ROI and the side sequence number of the current ROI are recorded as passenger flow statistical data when the person goes out of the ROI, and the number of detection frames of a certain frame of picture can be counted to obtain passenger flow density estimation data; therefore, passenger flow statistics and density estimation are carried out, corresponding passenger flow analysis results are obtained, further, the results can be called back, for example, the analysis processing results of the passenger flow, the density and the like are returned in an asynchronous call-back mode, and the returned results are output through information including but not limited to voice, characters/texts/documents, graphic images and the like.

Therefore, through passenger flow volume analysis based on head and shoulder detection, such as passenger flow statistics and density estimation, the detection frame in the region of interest is tracked by mainly drawing the region of interest by using a tracking matching algorithm, and the tracking result is counted by using a passenger flow statistical algorithm, so that image data can be effectively and accurately analyzed, the problem of inaccurate statistics caused by shielding is solved, and the execution efficiency can be greatly improved by a multi-thread concurrent/parallel input detection processing mode for monitoring videos/images. Under the scenes of common stations, shopping malls and the like, the passenger flow statistics accuracy can be up to more than 90%, and further the crowd density estimation accuracy can be up to more than 90%.

The following are definitions and explanations of some terms involved in the present invention:

ROI: image data is, for example, a region of interest in a picture.

Edge number of current ROI: i.e. the edge number of the drawn ROI polygon, for performing relevant current passenger flow statistics.

The following describes an implementation of the present invention with reference to a main flowchart of an embodiment of a method for head-shoulder detection-based passenger flow analysis of the present invention shown in fig. 1.

Step S110, carrying out head and shoulder detection on each frame of image data in the acquired passenger flow crowd video to obtain a head and shoulder detection frame of each target in each frame of image data;

in one embodiment, in a place where a market, a square, a station, an airport, etc. can perform video/image monitoring, the video/image monitoring is performed and the video/image is extracted and collected in real time or at regular time or at random for preprocessing, so as to input one or more image data frames which can be processed subsequently according to requirements.

In one embodiment, the preprocessing may utilize a decoder to decode the video/image, convert the decoded image data to obtain image data, and further perform operations such as size transformation on the image data, so as to meet the requirements of subsequent processing such as transmission and detection. For example: decoding the passenger flow crowd video and/or image data through a gpu engine decoder and a cuda engine decoder; further, the decoded data is subjected to coding format conversion, and the converted data is subjected to size conversion operation so as to meet the input requirements of subsequent detection, such as BGR data coding format and image size meeting network input size, and at least one or more frames of image data can be obtained after the processing.

In one embodiment, the head and shoulder detection is performed on the image data (usually, multi-frame image data) obtained after the preprocessing. Preferably, the detection may be by a detector; further, the detector performs detection including at least two modes, a dynamic mode and a static mode. For example, the dynamic mode is mainly used for carrying out passenger flow statistics on the passenger flow crowd video, and the static mode is mainly used for carrying out density statistics on the crowd image.

In one embodiment, various head and shoulder detection algorithms may be used to perform head and shoulder detection on each frame of image data, such as a picture, determine whether there is a head or a shoulder in the frame of image data (the picture), and determine the positions of the head and the shoulder. Head and shoulder detection algorithms such as CenteNet, ResNet, etc. Thereby obtaining a head-shoulder detection frame for each person in one frame of image data.

Preferably, feature extraction of head and shoulder detection is performed on each frame of image data through a deep learning network model to obtain the head and shoulder position information coordinate values of the head and shoulder detection frame of each person, and the height and width of the head and shoulder detection frame, where the head and shoulder detection frame may be a rectangular frame, for example: can be mainly expressed by (x, y, w, h), and the rectangular box comprises x-upper left abscissa, y-upper left ordinate, w-width of the rectangular box and h-height of the rectangular box.

Step S120, drawing a region of interest (ROI) in each frame of image data and judging whether the head and shoulder detection frame is in the ROI;

in one embodiment, a region of interest ROI is drawn/painted for each frame of image data, and a digital ID number is given to a different region of interest ROI; for example: on the picture, passenger flow crowd areas, entrance and exit key areas and the like which are mainly required to be monitored and analyzed are marked, if a plurality of areas which need to be monitored exist on the picture, a plurality of interested areas can be marked out, and different digital ID numbers are given to the interested areas so as to identify and distinguish different ROIs. Furthermore, after the ROI is drawn, which head and shoulders (which people) in the image data are positioned in the ROI can be judged according to the position of the ROI, the position of the head and shoulder detection frame and the central point, and then the follow-up matching of the head and shoulder detection frame can be participated.

In one embodiment, the determination is performed by a position determination algorithm, which includes, but is not limited to, ray determination, angle determination, area determination, and the like, and mainly determines whether the center point of the head and shoulder detection frame is within the ROI. For example, on a picture, three ROI regions of interest are drawn, and the following digital labels are applied: 0. 1, 2; meanwhile, n (a plurality of) head and shoulder detection frames are also detected and positioned on the picture, wherein the central points of 6 head and shoulder detection frames are in the three ROI, and then the 6 head and shoulder detection frames can be matched subsequently.

In one embodiment, a position determination algorithm for area determination is taken as an example: determining whether the central point of the head and shoulder detection frame is in the region of interest ROI, as shown in fig. 2, the specific calculation formula method is as follows: by calculating the center point and the position of the head and shoulder detection frameThe sum of the areas of triangles formed by all the vertexes of the ROI polygon of the region of interest:

wherein the half perimeter of the triangle is

x_i、y_i、z_iIs a side of a triangle; p is a radical of_iRepresenting the half perimeter of the ith triangle in n triangles formed by the central point p of the head and shoulder detection frame and each fixed point of the ROI polygon of the region of interest; i takes values from 1 to n, i is an integer less than or equal to n and n is the number of triangles formed by connection; if the sum of the areas of the triangles is larger than the area of the ROI polygon, the center point of the head-shoulder detection frame is outside the ROI, otherwise, the center point of the head-shoulder detection frame is in the ROI.

For example, the area of the ROI polygon (square in FIG. 2) is 960 x 960, the sum of the areas of the triangles formed by the P point (the center point of one of the head and shoulder detection boxes) and the polygon vertices of the ROI is calculated to be 960 x 25 (triangle PAB PBC PCD PDA),

taking the calculation of the triangular PAB area as an example:

x₁＝y₁＝490.3，z₁＝960,p₁＝970.3

the sum of the triangle areas (i.e. determined by calculating the sum of the four triangles in FIG. 2) S₁960 x 1010, while 960 x 1010 is larger than 960 x 960, so P is outside the ROI region of interest polygon.

Similarly, if the sum of the areas of the triangles formed by the point P1 (the center point of the other head-shoulder detection box) and the polygon vertex of the ROI is calculated to be 960 × 960, the point P1 is in the ROI polygon.

Therefore, the head and shoulder detection boxes which can be matched in each ROI can be found through ROI filtering, and then the head and shoulder detection boxes are input into a tracker to be subjected to follow-up processing such as tracking matching.

Step S130, analyzing and acquiring feature information of the head and shoulder detection frame in the ROI, matching the feature information of the head and shoulder detection frame in the ROI between continuous frames, determining that the head and shoulder detection frame which is matched belongs to the same target if matching is successful, and outputting tracking result information of the head and shoulder detection frame which is successfully matched;

in one embodiment, multiple frames of image data, such as the picture A, B, C, D, E, are continuous, and each frame of image data has the digitally labeled ROI and the head-shoulder detection frame to be matched, where a is used as a basis (original continuous frame), a certain head-shoulder detection frame in a and another head-shoulder detection frame in an adjacent continuous frame, such as B, are tracked and matched, and if matching is successful, it is determined that the head-shoulder detection frames to be matched belong to the same target; and if the matching fails, matching the pictures A and C until the matching is successful. Further, if the picture a is not successfully matched with the remaining pictures E, it is determined that the head and shoulder detection frame is lost.

In one embodiment, matching and tracking are performed to determine whether the two head and shoulder detection boxes in the two frames of image data belong to the head and shoulder of the same person by executing a tracking algorithm or the like (for example, successful matching of the head and shoulder detection boxes indicates that the two head and shoulder detection boxes in the two frames of image data actually detect the same person).

In one embodiment, feature information (e.g., area, direction/trend of motion, speed, etc.) of the head and shoulder detection boxes of the image data of successive frames is analyzed. Specifically, the area, the running direction and the speed of the head and shoulder detection box in the image data of the continuous frames can be analyzed and acquired by utilizing a tracking algorithm. For example, the error of the running direction and the speed of the head and shoulder detection box in ROI No. 1 in five continuous frames is analyzed to be within 10%, namely that the running direction and the speed are similar. And then matching the conditions of overlapping area, similar running direction trend and/or similar speed and the like of the head and shoulder detection frames existing between the image data of the original frame and the image data of the adjacent frame, if the matching is successful, indicating that the head and shoulder detection frames matched in the image data of the two adjacent continuous frames belong to the same person, giving a tracking id to the head and shoulder detection frame matched successfully, if the matching is failed, matching the image data of the original continuous frame with the image data of the next continuous frame, and further outputting tracking result information of the tracking id, the running direction, the speed and the like of the head and shoulder detection frame belonging to the same person.

As an example, the matching tracking is performed by using a tracking algorithm of area overlapping matching:

the calculation formula is as follows: iou ═ (u1 ≈ u2)/(u1 ═ u2), where iou is an intersection-and-merge ratio of head-shoulder detection frames between image data of consecutive frames, u1 is the area of the head-shoulder detection frame 1, and u2 is the area of the head-shoulder detection frame 2; u1 ≧ u2 is the overlapping area of the head-shoulder detection frame 1 and the head-shoulder detection frame 2, u1 ≦ u2 is the sum of the areas of the head-shoulder detection frame 1 and the head-shoulder detection frame 2; and calculating the intersection ratio iou of two head and shoulder detection frames between the image data of two continuous adjacent frames, and if the intersection ratio iou is larger than a threshold value, determining that the matching is successful, otherwise, failing to match.

For example, as shown in fig. 3, when the matching threshold is set to 0.3, the area of u1 is 20 × 20, the area of u2 is 20 × 20, and if the intersection area of u1 and u2 is 10 × 10, and the union area of u1 and u2 is 20 × 20+20 × 20 to 10 × 10, iou is 1/7. 1/7<0.3, the detection box match fails. If the intersection area of u1 and u2 is 15 × 15, and the union area of u1 and u2 is 20 × 20+20 × 20-15 × 15, iou is 0.39. 0.39>0.3, the detection frame matching is successful.

For another example, referring to the above example of the pictures ABCDE of the consecutive frames, the moving direction of the head and shoulder detecting frame 1 in the picture a of the consecutive frame is the northeast direction and the speed is 2 m/s, the moving direction of the head and shoulder detecting frame 3 in the picture B of another consecutive frame adjacent to the picture a is the northeast direction and the speed is 2 m/s, and then the area overlapping matching is performed by using the tracking algorithm of the area overlapping matching, if the matching between the head and shoulder detecting frame 1 in the picture a and the head and shoulder detecting frame 3 in the adjacent picture B is successful, the same head and shoulder detecting frame tracking ID number 1 is assigned to the head and shoulder detecting frame 3; if the matching of the head and shoulder detection frame 1 in the A and the adjacent head and shoulder detection frame 3 in the B fails, the head and shoulder detection frame 1 in the A is matched with the head and shoulder detection frame in the C of the next continuous frame until the matching is successful, the tracking matching of the head and shoulder detection frames among the image data of all the continuous frames is completed, the head and shoulder matching of the same person is completed, and the tracking result information which is successfully matched is output.

And step S140, carrying out passenger flow statistics according to the output tracking result information of the head and shoulder detection frame.

In one embodiment, information of a tracking id number, a running direction, and a speed of a head-shoulder detection frame in the tracking result information may be combined, so that it may be determined that the same person corresponding to the head-shoulder detection frame is crossing and/or leaving the ROI. For example: if the tracking ID number indicates that the head and shoulder detection frame of the same target enters from one side of the ROI where the current ROI is located between the continuous frames and then exits from the other side of the ROI where the current ROI is located, determining that one-time crossing is formed; according to the crossing, recording the number ID of the current ROI and the edge sequence number of the edge of the current ROI which passes through the current ROI as passenger flow statistical data; counting the number of the head and shoulder detection frames of each frame of image data in the current ROI to obtain passenger flow density estimation data; the passenger flow statistical data and the passenger flow density estimation data are used as passenger flow analysis results

Further, the passenger flow statistics and the density estimation result (passenger flow analysis result) adopt an asynchronous callback mode to realize result output. The output result is returned to the application layer by a callback mode, for example, the information representation mode of the output result may include a graphic image, a word/text/document, a voice, and the like. Meanwhile, the detector also carries out the detection work of the next picture, instead of continuing to work until the result returns to the detector, and the passenger flow statistical result is returned, namely, the parallel input and processing are carried out, so that the detection efficiency is improved.

According to the invention, through a scheme of passenger flow and density analysis based on head and shoulder detection, a region of interest ROI is drawn in each frame of image data, a tracking matching algorithm is used for tracking a head and shoulder detection frame (head and shoulder of each person) in the region of interest ROI, a tracking result is counted by using a passenger flow statistical algorithm, image data can be effectively and accurately analyzed, the condition of inaccurate statistics caused by shielding is relieved, and the processing and execution efficiency of monitoring videos/images can be greatly improved by a multi-thread concurrent/parallel input detection processing mode.

It should be noted that, although the foregoing embodiments describe each step in a specific sequence, those skilled in the art will understand that, in order to achieve the effect of the present invention, different steps do not necessarily need to be executed in such a sequence, and they may be executed simultaneously (in parallel) or in other sequences, and these changes are all within the protection scope of the present invention.

The following describes an implementation of the present invention with reference to a block diagram of the system for head-shoulder detection-based passenger flow volume analysis of fig. 6. The system at least comprises:

the detection module 610 is configured to perform head and shoulder detection on each frame of image data in the acquired passenger flow crowd video to obtain a head and shoulder detection frame of each target in each frame of image data;

The ROI filtering module 620 is configured to draw a region of interest ROI in each frame of image data and determine whether the head and shoulder detection frame is in the region of interest ROI;

In one embodiment, the determination is performed by a position determination algorithm, which includes, but is not limited to, ray determination, angle determination, area determination, and the like, and mainly determines whether the center point of the head and shoulder detection frame is within the ROI. For example, on a picture, three ROI regions of interest are drawn, and the following digital labels are applied: 0. 1, 2; meanwhile, n (a plurality of) head and shoulder detection frames are also detected and positioned on the picture, wherein the central points of 6 head and shoulder detection frames are positioned in the three ROI (regions of interest), and then the 6 head and shoulder detection frames can be subsequently matched.

In one embodiment, a position determination algorithm for area determination is taken as an example: determining whether the central point of the head and shoulder detection frame is in the region of interest ROI, as shown in fig. 2, the specific calculation formula method is as follows: calculating the sum of the area of a triangle formed by the central point of the head and shoulder detection frame and each vertex of the ROI polygon of the region of interest:

wherein the half perimeter of the triangle is

taking the calculation of the triangular PAB area as an example:

x₁＝y₁＝490.3，z₁＝960,p₁＝970.3

The tracking module 630 analyzes and acquires feature information of the head and shoulder detection frame in the ROI, matches the feature information of the head and shoulder detection frame in the ROI among consecutive frames, determines that the head and shoulder detection frame to be matched belongs to the same target if matching is successful, and outputs tracking result information of the head and shoulder detection frame to be successfully matched;

the calculation formula is as follows: iou ═ (u1 ═ u2)/(u1 ═ u2), where iou is the intersection-and-merge ratio of head-shoulder detection frames between image data of consecutive frames, u1 is the area of the head-shoulder detection frame 1, and u2 is the area of the head-shoulder detection frame 2; u1 u2 is the overlapping area of the head-shoulder detection frame 1 and the head-shoulder detection frame 2, and u1 u2 is the sum of the areas of the head-shoulder detection frame 1 and the head-shoulder detection frame 2; and calculating the intersection ratio iou of two head and shoulder detection frames between the image data of two continuous adjacent frames, and if the intersection ratio iou is larger than a threshold value, determining that the matching is successful, otherwise, failing to match.

And the passenger flow analysis module 640 performs passenger flow statistics according to the output tracking result information of the head and shoulder detection box.

In one embodiment, information of a tracking ID number, a running direction, and a speed of a head-shoulder detection box in the tracking result information may be combined, so that it may be determined that the same person corresponding to the head-shoulder detection box is crossing and/or leaving the ROI. For example: if the tracking ID number indicates that the head and shoulder detection frame of the same target enters from one side of the ROI where the current ROI is located between the continuous frames and then exits from the other side of the ROI where the current ROI is located, determining that one-time crossing is formed; according to the crossing, recording the number ID of the current ROI and the edge sequence number of the edge of the current ROI which passes through the current ROI as passenger flow statistical data; counting the number of the head and shoulder detection frames of each frame of image data in the current ROI to obtain passenger flow density estimation data; the passenger flow statistical data and the passenger flow density estimation data are used as passenger flow analysis results

Further, in an embodiment of a computer storage device of the present invention, the computer storage device stores a plurality of program codes adapted to be loaded and run by a processor to perform the aforementioned head-shoulder detection-based passenger flow analysis method.

Further, in an embodiment of a processing device of the present invention, comprising a processor and a memory, the memory device is adapted to store a plurality of program codes, the program codes are adapted to be loaded and run by the processor to perform the aforementioned head-shoulder detection based passenger flow analysis method.

The following will further explain the implementation of the present invention with reference to the schematic structural diagram of an embodiment of the head-shoulder detection-based passenger flow analysis apparatus of the present invention shown in fig. 7. The device at least comprises:

a monitoring device 760 connected to the pre-processing device 770, the pre-processing device 770 being connected to the one or more detectors 710; an ROI filtering unit 720, one or more trackers 730, and an analyzer 740, said detector 710 being connected to said ROI filtering unit 720, said tracker being connected to said ROI filtering unit 720 and said analyzer 730.

Monitoring devices 760, including but not limited to various one or more video and/or image collectors, such as: cameras, infrared cameras, scanners, and the like. One or more videos and/or images of the monitored environment are captured. Such as: in places where video/image monitoring can be performed, such as shopping malls, squares, stations, airports and the like, video/image monitoring is performed, the videos/images are extracted and collected in real time or at regular time or at random, and the passenger flow crowd videos are input to a preprocessing device 770 for preprocessing.

The preprocessing device 770 is configured to perform preprocessing, and may utilize a decoder to decode the video/image, convert the decoded image data to obtain image data, and further perform operations such as size transformation on the image data, so as to meet requirements of subsequent processing such as transmission and detection. After such processing, at least one or more frames of image data can be obtained

Preferably, the preprocessing device 770 includes an engine decoder 7701 and a converter 7702.

Wherein, the engine decoder 7701 decodes the video to obtain multiple frames of initial image data; for example: decoding passenger flow crowd video and/or image data through gpu and cuda engine decoder

The converter 7702 performs encoding format conversion on the decoded initial image data and performs size conversion on the converted image data to obtain the multi-frame image data; wherein, the BRG coding format of each frame of image data in the multi-frame image data and the size of the image all meet the input requirement of the network input size (in head and shoulder detection).

The preprocessing device 770, after preprocessing the plurality of frames of image data into image data meeting the requirement of head and shoulder detection, inputs the image data into the detection 710 connected with the preprocessing device.

The detector 710 acquires each frame of image data in the passenger flow crowd video input from the monitoring device side, and performs head-shoulder detection on each frame of image data to obtain a head-shoulder detection frame of each target in each frame of image data.

Preferably, the detection may be by a detector; further, the detector performs detection including at least two modes, a dynamic mode and a static mode. For example, the dynamic mode is mainly used for carrying out passenger flow statistics on the passenger flow crowd video, and the static mode is mainly used for carrying out density statistics on the crowd image.

And an ROI filtering unit 720 for drawing a region of interest ROI in each frame of image data of which the detection is completed by the detector and determining whether the head and shoulder detection frame is within the region of interest ROI.

In one embodiment, a position determination algorithm for area determination is taken as an example: and determining whether the central point of the head and shoulder detection frame is in the region of interest ROI, as shown in fig. 2, and referring to the description of the foregoing method step S120 for specific calculation formulas and examples.

Therefore, the head and shoulder detection box in each ROI can be found through ROI filtering, the digital label is provided, and the digital label is further input into a tracker for follow-up processing such as tracking matching.

The tracker 730 analyzes the feature information of the head and shoulder detection frame judged to be in the ROI by the ROI filtering unit, matches the feature information of the head and shoulder detection frame in the same ROI among the continuous frames, determines that the head and shoulder detection frame which is matched belongs to the same target if the matching is successful, and outputs the tracking result information of the head and shoulder detection frame which is successfully matched.

In one embodiment, feature information (e.g., area, direction/trend of motion, speed, etc.) of the head and shoulder detection boxes of the image data of successive frames is analyzed. Specifically, the area, the running direction and the speed of the head and shoulder detection box in the image data of the continuous frames can be analyzed and acquired by utilizing a tracking algorithm. For example, the error of the running direction and the speed of the head and shoulder detection box in ROI No. 1 in five continuous frames is analyzed to be within 10%, namely that the running direction and the speed are similar. And then matching the conditions of overlapping area, similar running direction trend and/or similar speed and the like of the head and shoulder detection frames existing between the image data of the original frame and the image data of the adjacent frame, if the matching is successful, indicating that the head and shoulder detection frames matched in the image data of the two adjacent continuous frames belong to the same person, giving a tracking ID number to the head and shoulder detection frames matched successfully, if the matching is failed, matching the image data of the original continuous frame with the image data of the next continuous frame, and further outputting tracking result information of the tracking ID number, the running direction, the speed and the like of the head and shoulder detection frames belonging to the same person.

An example of the matching and tracking using the tracking algorithm of area overlapping matching is as described in step S130 of the foregoing method and in fig. 3.

And the analyzer 740 receives the tracking result information output by the tracker to perform passenger flow statistics.

In one embodiment, information of a tracking ID number, a running direction, and a speed of a head-shoulder detection box in the tracking result information may be combined, so that it may be determined that the same person corresponding to the head-shoulder detection box is crossing and/or leaving the ROI. For example: if the tracking ID number indicates that the head and shoulder detection frame of the same target enters from one side of the ROI where the current ROI is located between the continuous frames and then exits from the other side of the ROI where the current ROI is located, determining that one-time crossing is formed; according to the crossing, recording the number ID of the current ROI and the edge sequence number of the edge of the current ROI which passes through the current ROI as passenger flow statistical data; counting the number of the head and shoulder detection frames of each frame of image data in the current ROI to obtain passenger flow density estimation data; and taking the passenger flow statistical data and the passenger flow density estimation data as passenger flow analysis results.

Further, the callback unit 750 is connected to the analyzer 740, and outputs the passenger flow analysis result in an asynchronous callback manner. And the passenger flow statistics and the density estimation result (passenger flow analysis result) adopt an asynchronous callback mode to realize result output. The output result is returned to the application layer by a callback mode, for example, the information representation mode of the output result may include a graphic image, a word/text/document, a voice, and the like. Meanwhile, the detector also carries out the detection work of the next picture, instead of continuing to work until the result returns to the detector, and the passenger flow statistical result is returned, namely, the parallel input and processing are carried out, so that the detection efficiency is improved.

An example of an application scenario of the technical solution of the present invention is described below to further illustrate the implementation of the present invention:

the scheme of passenger flow volume analysis based on head and shoulder detection particularly needs to process image data and streaming media data when passenger flow statistics and passenger flow density estimation are carried out. The system is applied to developing passenger flow monitoring (cameras) in a certain airport and counting passenger flow. And decoding the passenger flow crowd video and image data obtained during monitoring through a gpu or cuda engine decoder, converting the coding format of the decoded data, and performing size conversion operation on the converted data so as to meet the detection input requirements of a data coding BGR format and an image size, and finally obtaining a plurality of frames of monitored image data. The detector comprises a dynamic mode and a static mode, wherein the dynamic mode is used for carrying out passenger flow analysis, particularly passenger flow statistics, on the passenger flow crowd video, and the static mode is used for carrying out density analysis, particularly density estimation, on the crowd image, and the scene is further described for the passenger flow statistics in the dynamic mode.

The plurality of detectors detect image data in parallel, detect the head and shoulder of each person in each image data by a head and shoulder detection algorithm, and are displayed with a head and shoulder detection box as shown in fig. 5.

And judging whether the central point of a head and shoulder detection frame in each image data is in the ROI1 by using a position judgment algorithm according to the drawn ROI area with the number ID of 1 in each image data, and if so, respectively tracking 6 head and shoulder detection frames in the ROI 1.

The plurality of trackers analyze and acquire characteristic information (area, running direction trend, speed and the like) of each head and shoulder detection frame of the image data ABCDE of the continuous frames in the plurality of frames of image data through a tracking algorithm. Such as: the running direction of one head and shoulder detection frame in the adjacent picture A, B is the northeast direction, the speed is 2 m/s, the area overlapping matching is performed on the two head and shoulder detection frames in A, B by using an area overlapping matching tracking algorithm, and if the two matching are successful, the same head and shoulder detection frame tracking ID number is given, for example, the tracking ID is 1. It shows that the two head and shoulder detection boxes detect the same person.

Combining the tracking result information, such as the tracking ID ═ 1, northeast direction of travel, and 2 m/s speed, with the same person crossing and leaving the current ROI such as: in the step A, the detection frame 1 is arranged on the left edge of the ROI1, and in the continuous BCDE frames, the detection frame 1 reaches the right edge, so that the detection frame is determined to pass through the ROI1, further leave the ROI1, record the number ID number 1 of the current ROI and the serial number of the right edge passed by the current ROI when the current ROI goes out, so that the passenger flow statistical data can be recorded as 1 person, other passenger flow statistics are calculated according to the number, and then asynchronous call-back is carried out and the statistical data are output as the passenger flow analysis result.

Of course, density analysis, for example, density estimation, may count the number of head and shoulder detection boxes in each image, such as a, to obtain passenger flow density estimation data, and output as a result of the passenger flow analysis, for example, detect the head and shoulder of each person in each image data in a direct static mode of a detector to obtain a head and shoulder detection box, estimate the passenger flow density, estimate the estimation using, for example, a multiple linear regression model, and the like.

It will be understood by those skilled in the art that all or part of the flow of the method according to the above-described embodiment may be implemented by a computer program, which may be stored in a computer-readable storage medium and used to implement the steps of the above-described embodiments of the method when the computer program is executed by a processor. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying said computer program code, media, usb disk, removable hard disk, magnetic diskette, optical disk, computer memory, read-only memory, random access memory, electrical carrier wave signals, telecommunication signals, software distribution media, etc. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.

Further, it should be understood that, since the modules are only configured to illustrate the functional units of the system of the present invention, the corresponding physical devices of the modules may be the processor itself, or a part of software, a part of hardware, or a part of a combination of software and hardware in the processor. Thus, the number of individual modules in the figures is merely illustrative.

Those skilled in the art will appreciate that the various modules in the system may be adaptively split or combined. Such splitting or combining of specific modules does not cause the technical solutions to deviate from the principle of the present invention, and therefore, the technical solutions after splitting or combining will fall within the protection scope of the present invention.

So far, the technical solution of the present invention has been described with reference to one embodiment shown in the drawings, but it is easily understood by those skilled in the art that the scope of the present invention is obviously not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.

Claims

1. A passenger flow analysis method based on head and shoulder detection is characterized by comprising the following steps:

analyzing the characteristic information of the head and shoulder detection frame in the ROI, matching the characteristic information of the head and shoulder detection frame in the ROI between continuous frames, if the matching is successful, determining that the head and shoulder detection frame which is matched belongs to the same target, and outputting the tracking result information of the head and shoulder detection frame which is successfully matched;

2. The method according to claim 1, wherein determining whether the head-shoulder detection frame is within the region of interest ROI specifically comprises:

assigning a digital ID number to each ROI determined in each frame of image data to identify different ROIs in each frame of image data;

3. The method according to claim 2, wherein determining whether the head-shoulder detection frame in the image data is within the region of interest ROI based on the position of the region of interest ROI, the position of the head-shoulder detection frame, and a center point comprises:

if so, participating in the matching of the subsequent head and shoulder detection frames;

the position judgment algorithm is at least one of ray judgment, included angle judgment and area judgment.

4. The method according to claim 3, wherein the position determination algorithm is an area determination method, and the determining whether the center point of the head-shoulder detection frame is within the region of interest ROI through the position determination algorithm specifically comprises:

wherein x is_i、y_i、z_iIs the side of the triangle, and the side of the triangle,

representing the half perimeter of the ith triangle in n triangles formed by the central point p of the head and shoulder detection frame and each vertex of the ROI polygon of the interesting region, wherein the value of i ranges from 1 to n, i is a positive integer less than or equal to n, and n is the number of connected triangles;

5. The method of claim 1,

matching the feature information of the head and shoulder detection frames in the same ROI between the continuous frames, and if the matching is successful, determining that the head and shoulder detection frames which are matched belong to the same target, specifically comprising:

determining whether the head and shoulder detection boxes in the same ROI between the continuous frames are matched or not by utilizing the characteristic information based on a tracking algorithm;

if the matching is successful, determining that the head and shoulder detection frames matched between the continuous frames belong to the head and the shoulder of the same target, assigning tracking ID numbers to the head and shoulder detection frames belonging to the head and the shoulder of the same target to indicate that the head and shoulder detection frames successfully matched are the same target,

wherein the tracking algorithm is a Kalman filtering algorithm and/or an area overlap matching algorithm.

6. The method of claim 5, wherein the tracking algorithm is an area overlap matching algorithm;

determining whether the head and shoulder detection boxes in the same ROI between the continuous frames are matched or not by utilizing the characteristic information based on a tracking algorithm, further comprising:

if the motion directions and the speeds of the adjacent frames in the continuous frames in the feature information of the head and shoulder detection frame in the same ROI are similar, executing a tracking algorithm of area overlapping matching, and determining whether the matching is successful according to the area overlapping degree;

iou＝(u1∩u2)/(u1∪u2)

7. The method of claim 5, wherein the tracking result information comprises: and tracking ID number, running direction and speed of the head and shoulder detection frame which are successfully matched.

8. The method according to claim 7, wherein performing passenger flow statistics based on the outputted tracking result information specifically comprises:

if the tracking ID number indicates that the head and shoulder detection frame of the same target enters from one side of the ROI where the current ROI is located between the continuous frames and then exits from the other side of the ROI where the current ROI is located, determining that one-time crossing is formed;

according to the crossing, recording the number ID of the current ROI and the edge sequence number of the edge of the current ROI which passes through the current ROI as passenger flow statistical data;

counting the number of the head and shoulder detection frames of each frame of image data in the current ROI to obtain passenger flow density estimation data;

and taking the passenger flow statistical data and the passenger flow density estimation data as passenger flow analysis results.

9. The method of any one of claims 1 to 8, further comprising:

and outputting the passenger flow analysis result in an asynchronous callback mode.

10. A passenger flow volume analysis system based on head and shoulder detection is characterized by comprising:

the ROI filtering module is used for determining a region of interest (ROI) in each frame of image data and judging whether the head and shoulder detection frame is in the region of interest (ROI);

the tracking module is used for analyzing the characteristic information of the head and shoulder detection frame in the ROI, matching the characteristic information of the head and shoulder detection frame in the ROI between continuous frames, determining that the head and shoulder detection frame which is matched belongs to the same target if the matching is successful, and outputting the tracking result information of the head and shoulder detection frame which is successfully matched;

11. The system of claim 10, wherein the ROI filtering module is configured to determine whether the head-shoulder detection box is within the ROI, and specifically includes:

12. The system of claim 11, wherein the ROI filtering module, when determining whether the head-shoulder detection frame in the image data is within the region of interest ROI based on the position of the region of interest ROI, the position of the head-shoulder detection frame, and a center point, specifically comprises:

13. The system according to claim 12, wherein the position determination algorithm is an area determination method, and the determining whether the center point of the head-shoulder detection frame is within the ROI by the position determination algorithm specifically includes:

wherein x is_i、y_i、z_iIs a side of a triangle;

representing the half perimeter of the ith triangle in n triangles formed by the central point p of the head and shoulder detection frame and each fixed point of the ROI polygon of the region of interest; i takes values from 1 to n, i is an integer less than or equal to n and n is the number of triangles formed by connection;

14. The system of claim 10,

the tracking module performs feature information matching of the head and shoulder detection frame, and specifically includes:

if the matching is successful, determining that the head and shoulder detection frames matched between the continuous frames are the head and the shoulder belonging to the same target, assigning a tracking ID number to each head and shoulder detection frame belonging to the head and the shoulder of the same target to indicate that each head and shoulder detection frame successfully matched is the same target,

15. The system of claim 14, wherein the tracking algorithm is an area overlap matching algorithm;

iou＝(u1∩u2)/(u1∪u2)

16. The system of claim 14, wherein the tracking result information comprises: and tracking ID number, running direction and speed of the head and shoulder detection frame which are successfully matched.

17. The system of claim 16, wherein the passenger flow analysis module performing passenger flow statistics specifically comprises:

18. A computer-readable storage medium, characterized in that a plurality of program codes are stored in the storage medium, which are adapted to be loaded and run by a processor to perform the head-shoulder detection-based passenger flow volume analysis method according to any one of claims 1 to 9.

19. A processing device comprising a processor and a memory, said memory device being adapted to store a plurality of program codes, characterized in that said program codes are adapted to be loaded and run by said processor to perform the method for head-shoulder detection based passenger flow analysis according to any of claims 1 to 9.