CN113065491A - Multi-shooting-based passenger flow statistical method and system - Google Patents
Multi-shooting-based passenger flow statistical method and system Download PDFInfo
- Publication number
- CN113065491A CN113065491A CN202110388818.5A CN202110388818A CN113065491A CN 113065491 A CN113065491 A CN 113065491A CN 202110388818 A CN202110388818 A CN 202110388818A CN 113065491 A CN113065491 A CN 113065491A
- Authority
- CN
- China
- Prior art keywords
- passenger flow
- human
- human body
- data
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007619 statistical method Methods 0.000 title claims abstract description 21
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 27
- 238000004458 analytical method Methods 0.000 claims abstract description 22
- 238000000034 method Methods 0.000 claims abstract description 21
- 238000012544 monitoring process Methods 0.000 claims abstract description 12
- 238000001514 detection method Methods 0.000 claims description 44
- 238000007405 data analysis Methods 0.000 claims description 32
- 230000004927 fusion Effects 0.000 claims description 28
- 238000012549 training Methods 0.000 claims description 18
- 238000012545 processing Methods 0.000 claims description 5
- 238000005516 engineering process Methods 0.000 description 7
- 230000008901 benefit Effects 0.000 description 5
- 238000013135 deep learning Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000010234 longitudinal analysis Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000011524 similarity measure Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/53—Recognition of crowd images, e.g. recognition of crowd congestion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a passenger flow statistical method and a system based on multi-camera, aiming at the problems that the passenger flow management and monitoring in the prior art are single and the identification method is poor in progress, a camera capable of clearly collecting the face, the human body and the head of a customer is deployed on site to realize an algorithm; through the multi-region distinguishing arrangement of the cameras and the analysis of the combination of the human face, the human body and the human head, the high-precision diversified passenger flow statistics is realized.
Description
Technical Field
The invention belongs to the technical field of computer image processing, and particularly relates to a passenger flow statistical method and system based on multi-shot.
Background
In recent years, with the technical progress of deep learning, the technology is more and more used for counting passenger flow, the infrared technology is used for counting and counting passenger flow in an early traditional mode, a large error exists, and in the process that the deep learning technology is gradually matured, a plurality of passenger flow counting technologies based on deep learning are started, so that the original infrared counting mode is gradually replaced.
The traditional passenger flow statistical method based on image recognition is mainly a passenger flow detection technology based on passenger head detection, the technology is easy to falsely detect spherical false targets such as basketball, and is easy to miss detection for confusing targets such as hats and headscarfs, so that the passenger head statistics cannot achieve an accurate effect, and the method cannot obtain face information and is not beneficial to subsequent intelligent promotion of a passenger flow system. With the development of computer vision, the passenger flow statistics problem can be solved by combining human face detection and human body detection.
In the prior art, a single camera is used for extracting character information, a third-party face recognition service interface is called to obtain identity characteristics of a customer, comparison and recognition are carried out, and then passenger flow information is counted.
The most similar prior art implementation of the present invention is application No. 202010463245.3 entitled: the human face passenger flow identification method, the device, the equipment and the medium suitable for a multi-person scene are mainly realized by the following scheme:
1. acquiring a current image of a person and extracting identity characteristics;
2. judging whether the celebrity is a new person or not according to the extracted identity characteristics;
3. if the person comes, storing the current image of the person comes into a preset image library and creating face data of the person comes into a new position;
4. if the person is not a new person, comparing the current image of the person with the image of the person pre-stored in the preset image library to select one with higher image quality to store in the preset image library and serve as a comparison object for next face recognition.
The above prior art has the following disadvantages:
1. the scheme only has the advantages that information extraction is carried out by a single camera at an entrance, image information is single, and the problems of missed acquisition or poor quality of acquired images exist;
2. because the scheme depends on a third-party interface, the scheme only has the clothing characteristics of the upper half body in the clothing characteristic extraction, compared with the human body characteristics of the whole body, the algorithm is single, the identification accuracy is not high, and the robustness is weak.
Disclosure of Invention
Aiming at the problems that the management and monitoring of passenger flow are single and the identification method is poor in progress in the prior art, the invention provides a passenger flow statistical method and a passenger flow statistical system based on multiple cameras, and the cameras capable of clearly collecting faces, human bodies and heads of customers are deployed on site to realize an algorithm; through the multi-region distinguishing arrangement of the cameras and the analysis of the combination of the human face, the human body and the human head, the high-precision diversified passenger flow statistics is realized.
The specific implementation content of the invention is as follows:
the invention provides a passenger flow statistical method based on multi-shot, which is based on a passenger flow statistical system and comprises the following steps:
step 1: the method comprises the following steps of performing real-time field image acquisition by using a data acquisition module, wherein the data acquisition module is connected with a plurality of face cameras, human body cameras and human head cameras which are respectively used for acquiring face, human body and human head images; transmitting the human face images, the human body images and the human head images which are acquired by the human face camera, the human body camera and the human head camera to a data fusion platform at a time interval of 30 frames per second by using an RTSP (real time streaming protocol);
step 2: the received human face image, human body image and human head image in the RTSP data format are decoded through a data fusion platform to obtain video data, the video data are decoded into continuous video frame data, and the continuous video frame data are sent to a passenger flow statistics and data analysis module;
and step 3: firstly, carrying out preliminary statistics on passenger flow by adopting a human face image in a passenger flow statistics and data analysis module, and storing human face information into a database; then, performing maximum passenger flow early warning management through the matching relation between the machine number and the area of the human body camera on site and timestamp information; finally, the corresponding human body shooting area is restrained through the human head image, and the accuracy of the human body counting number is optimized;
and 4, step 4: judging whether the passenger flow statistics and data analysis result has a condition that the area exceeds a preset passenger flow alarm threshold value, and performing passenger flow guiding processing through a passenger flow guiding module;
and 5: and receiving data of the database and the passenger flow statistics and data analysis module through the comprehensive analysis platform, and performing final statistical analysis and summarization.
In order to better implement the present invention, further, the step 3 processes data by adopting fusion of a human face algorithm, a human body algorithm and a human head algorithm, and the fusion of the three algorithms specifically comprises the following steps:
step S3.1: acquiring face image information of a customer, performing quality judgment on the acquired image information by using a face image quality algorithm, and entering a subsequent step only by using a face image with qualified quality;
step S3.2: judging whether the customer is a new customer or not through face detection, and carrying out passenger flow statistics;
step S3.3: and then, through target tracking, the database is maintained by matching the human face with the human body characteristics, and the walking track, movement and stay time information of the customer are obtained.
In order to better implement the present invention, further, in step 3.3, the specific operations of matching the human face and the human body features are as follows:
step S3.3.1: firstly, acquiring a human body image by using a human face camera, acquiring a human face image of a customer in the human body image from the human body image, storing the human face and human body characteristics of the customer obtained by the image into a database, and generating a corresponding faceID number and a PeopleID, wherein each human body image and the human face image are in one-to-one correspondence;
step S3.3.2: acquiring a human body image by using a human body camera, and storing human body characteristics into a database to generate peoples ID 2;
step S3.3.3: and (3) searching and comparing the human body characteristics obtained in the step (3.3.1) in the human body database generated in the step (3.3.2), and if similar peoples ID2 is found in the human body database through comparison, associating and matching the faceID with the peoples ID 2.
In order to better implement the present invention, further, in step 3, the specific steps of performing data analysis include:
first, the data set is trained:
acquiring image frames from a camera, marking position information of a human face and a human head, deleting image data which cannot be judged, and finally generating three training data sets of human face detection, human body detection and human head detection;
secondly, model training:
firstly, a trained basic detection model is used, then a training data set is imported into the model for retraining to generate a detection model, so that the detection of human faces, human bodies and human heads is realized, and then passenger flow statistics is carried out; on the basis that the camera continuously acquires images, the data set is continuously updated, so that the model is repeatedly trained in a specific scene, and the model is promoted to achieve higher accuracy than conventional training;
and finally, passenger flow statistics:
the method comprises the steps of counting and tracking customers in a shopping mall, monitoring each area or channel in the shopping mall in real time through a plurality of cameras, counting the number of the customers by using the three cameras in a combined mode, setting a detection time range t, setting a maximum passenger flow threshold value m of each area, triggering early warning prompt if the passenger flow number of a certain area exceeds m or is about to exceed m within the time t, and carrying out formal deployment of a multi-shooting passenger flow counting system when a passenger flow counting module can accurately count the passenger flow number of the shopping mall.
In order to better realize the invention, further, the face camera is arranged at the unique passage and the entrance of the passenger flow; the human body cameras are arranged on passenger flow channels on site and in multiple directions of all places; the human head camera is arranged at the crowded place of passenger flow on the spot.
The invention also provides a passenger flow statistical system based on multi-shot, which is used for a passenger flow statistical method based on multi-shot, wherein the passenger flow statistical system comprises a man-head camera, a face camera, a human body camera, a data acquisition module, a data fusion module, a database, a passenger flow guide module, a passenger flow statistical and data analysis module and a comprehensive analysis platform;
the data acquisition module is connected with a plurality of human head cameras, human face cameras and human body cameras and is connected with the data fusion module, the data fusion module is connected with the passenger flow statistics and data analysis module through a database, and the passenger flow statistics and data analysis module is also respectively connected with the comprehensive analysis platform and the passenger flow guide module; the database is also connected with the comprehensive analysis platform.
Compared with the prior art, the invention has the following advantages and beneficial effects:
the invention can overcome the defects of the traditional market passenger flow management and monitoring and provides an effective algorithm and system for the market passenger flow statistics and management. In the scene of related project deployment, the invention requires to count the passenger flow conditions of places such as various places, channels, entrances and exits of a market, which is difficult to achieve real-time, accurate and passenger flow prediction. Meanwhile, path analysis is carried out on the customers through person tracking, and potential customers are found.
Drawings
FIG. 1 is a schematic diagram of the structural components of the system of the present invention;
FIG. 2 is a schematic diagram of a data analysis procedure.
Detailed Description
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it should be understood that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments, and therefore should not be considered as a limitation to the scope of protection. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
In the description of the present invention, it is to be noted that, unless otherwise explicitly specified or limited, the terms "disposed," "connected," and "connected" are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
Example 1:
the invention provides a passenger flow statistical method based on multi-shot, which is based on a passenger flow statistical system and comprises the following steps as shown in figures 1 and 2:
step 1: the method comprises the following steps of performing real-time field image acquisition by using a data acquisition module, wherein the data acquisition module is connected with a plurality of face cameras, human body cameras and human head cameras which are respectively used for acquiring face, human body and human head images; transmitting the human face images, the human body images and the human head images which are acquired by the human face camera, the human body camera and the human head camera to a data fusion platform at a time interval of 30 frames per second by using an RTSP (real time streaming protocol);
step 2: the received human face image, human body image and human head image in the RTSP data format are decoded through a data fusion platform to obtain video data, the video data are decoded into continuous video frame data, and the continuous video frame data are sent to a passenger flow statistics and data analysis module;
and step 3: firstly, carrying out preliminary statistics on passenger flow by adopting a human face image in a passenger flow statistics and data analysis module, and storing human face information into a database; then, performing maximum passenger flow early warning management through the matching relation between the machine number and the area of the human body camera on site and timestamp information; finally, the corresponding human body shooting area is restrained through the human head image, and the accuracy of the human body counting number is optimized;
and 4, step 4: judging whether the passenger flow statistics and data analysis result has a condition that the area exceeds a preset passenger flow alarm threshold value, and performing passenger flow guiding processing through a passenger flow guiding module;
and 5: and receiving data of the database and the passenger flow statistics and data analysis module through the comprehensive analysis platform, and performing final statistical analysis and summarization.
In order to better implement the present invention, further, the step 3 processes data by adopting fusion of a human face algorithm, a human body algorithm and a human head algorithm, and the fusion of the three algorithms specifically comprises the following steps:
step S3.1: acquiring face image information of a customer, performing quality judgment on the acquired image information by using a face image quality algorithm, and entering a subsequent step only by using a face image with qualified quality;
step S3.2: judging whether the customer is a new customer or not through face detection, and carrying out passenger flow statistics;
step S3.3: and then, through target tracking, the database is maintained by matching the human face with the human body characteristics, and the walking track, movement and stay time information of the customer are obtained.
In order to better implement the present invention, further, in step 3.3, the specific operations of matching the human face and the human body features are as follows:
step S3.3.1: firstly, acquiring a human body image by using a human face camera, acquiring a human face image of a customer in the human body image from the human body image, storing the human face and human body characteristics of the customer obtained by the image into a database, and generating a corresponding faceID number and a PeopleID, wherein each human body image and the human face image are in one-to-one correspondence;
step S3.3.2: acquiring a human body image by using a human body camera, and storing human body characteristics into a database to generate peoples ID 2;
step S3.3.3: and (3) searching and comparing the human body characteristics obtained in the step (3.3.1) in the human body database generated in the step (3.3.2), and if similar peoples ID2 is found in the human body database through comparison, associating and matching the faceID with the peoples ID 2.
In order to better implement the present invention, further, in step 3, as shown in fig. 2, the specific steps of performing data analysis include:
first, the data set is trained:
acquiring image frames from a camera, marking position information of a human face and a human head, deleting image data which cannot be judged, and finally generating three training data sets of human face detection, human body detection and human head detection;
secondly, model training:
firstly, a trained basic detection model is used, then a training data set is imported into the model for retraining to generate a detection model, so that the detection of human faces, human bodies and human heads is realized, and then passenger flow statistics is carried out; on the basis that the camera continuously acquires images, the data set is continuously updated, so that the model is repeatedly trained in a specific scene, and the model is promoted to achieve higher accuracy than conventional training;
and finally, passenger flow statistics:
the method comprises the steps of counting and tracking customers in a shopping mall, monitoring each area or channel in the shopping mall in real time through a plurality of cameras, counting the number of the customers by using the three cameras in a combined mode, setting a detection time range t, setting a maximum passenger flow threshold value m of each area, triggering early warning prompt if the passenger flow number of a certain area exceeds m or is about to exceed m within the time t, and carrying out formal deployment of a multi-shooting passenger flow counting system when a passenger flow counting module can accurately count the passenger flow number of the shopping mall.
In order to better realize the invention, further, the face camera is arranged at the unique passage and the entrance of the passenger flow; the human body cameras are arranged on passenger flow channels on site and in multiple directions of all places; the human head camera is arranged at the crowded place of passenger flow on the spot.
Example 2:
the embodiment provides a passenger flow statistical system based on multiple shots, which is used for a passenger flow statistical method based on multiple shots, and as shown in fig. 1, the passenger flow statistical system comprises a human head camera, a human face camera, a human body camera, a data acquisition module, a data fusion module, a database, a passenger flow guide module, a passenger flow statistical and data analysis module and a comprehensive analysis platform;
the data acquisition module is connected with a plurality of human head cameras, human face cameras and human body cameras and is connected with the data fusion module, the data fusion module is connected with the passenger flow statistics and data analysis module through a database, and the passenger flow statistics and data analysis module is also respectively connected with the comprehensive analysis platform and the passenger flow guide module; the database is also connected with the comprehensive analysis platform.
Example 3:
this example gives a concrete example of implementation based on any of the above examples 1-2:
the object of the invention can be achieved by the following timely scheme: and (3) deploying a camera capable of clearly acquiring the face, the human body and the head of the customer on site to realize the algorithm. The algorithm can identify and track customers and carry out passenger flow statistics. The scheme mainly comprises five parts: a data acquisition module; a data fusion module; a passenger flow statistics and data analysis module; a passenger flow guidance module; and (4) a comprehensive analysis platform. The data acquisition is carried out on-site acquisition by using a plurality of cameras of human faces, human bodies and human heads, and is associated with the data fusion module through an RTSP (real time streaming protocol). The data fusion module and the data analysis module are processed in a queue mode, and performance can be maximized. The passenger flow guiding module comprises a field prompting medium, such as broadcasting, communication equipment and the like. The passenger flow guidance module can transmit guidance information to the on-site customers and staff according to correct standards. The comprehensive analysis platform can count information such as transverse and longitudinal passenger flow quantity, customer residence time and the like, and is convenient for post statistical analysis.
1. Data acquisition module
The data acquisition module is used for acquiring field image data in real time by using a plurality of human faces, human bodies and human head cameras, and after the data are acquired, the data are transmitted to the data fusion platform by using an RTSP protocol at a time interval of 30 frames per second.
The data acquisition in the part is divided into three steps:
step 1.1 face Camera Collection
The invention aims to perform preliminary passenger flow counting aiming at a shopping mall passenger flow environment. The mall environment generally includes a plurality of layers, a plurality of shopping channels and areas, and the entrance and exit of each area and the entrance and exit of each channel are different. The wide-angle camera is selected as the face camera, the face camera is deployed and installed at the unique passage and entrance and exit of people stream, and the erection height is about 2.0-3.5 meters, so that a positive and clear image can be shot. When a customer is at a long distance, the human body characteristics of the customer are collected, the face characteristic information of the corresponding customer is collected at a short distance, and then the preliminary statistics of the passenger flow is carried out by using face detection. The purpose of the acquired human body image information is to match with human body features acquired from a human body camera, and a mapping relation is formed between a human face database and a human body database through human body feature matching, so that subsequent analysis and tracking of value clients are facilitated.
The shopping mall passenger flow is large, the environment is complex, some customers can only acquire face information in the process of monitoring by the camera, some can only acquire human body information, independent face tracking or human body tracking is used, the problems of omission, incapability of identification and inaccuracy in identification exist, therefore, the image information detected by the cameras is matched by combining the cameras, rapid and accurate statistics and tracking positioning can be realized, and compared with the independent use of the face or the use of the human body tracking, the scheme has higher accuracy.
Step 1.2 human body camera acquisition
Because the face camera in step 1.1 has the problems of fuzzy image quality or face shielding, etc., a small number of customers cannot perform passenger flow statistics through face recognition, which is a problem existing in the prior art and is an important reason for not performing accurate passenger flow statistics. In order to solve the problem, the part is combined with a plurality of human body cameras to further collect the human body information of the customer, and if the human body information collection fails in 1.1, the supplementary counting can be carried out through the human body information. The human body camera selects each big stream channel in the mall and all public places to be deployed and installed in multiple directions, and real-time high-definition monitoring of the area is formed.
Step 1.3 human head Camera Collection
Based on the step 1.2, if the person is crowded and the person is blocked, the accuracy of the person detection and tracking can be reduced, and in order to solve the problem, a person head detection technology is provided to restrict the person detection. The human head camera is deployed at a position with crowded passenger flow, such as an activity site, a restaurant door or a recreation area in a shopping mall, the area photographed by the human head camera is consistent with the area photographed by the human body camera at the photographing point, and if the number of the passenger flow detected by the human body in the area is smaller than the number of the passenger flow detected in the area, the result of the human head detection is used as the final passenger flow statistical number.
The three cameras have the advantages and the disadvantages respectively, and the effects of complementing each other are achieved by combining the cameras, so that the system can accurately count passenger flow, locate the target and track the target in real time.
2. Data fusion module
After receiving the data, the data fusion module firstly unlocks the RTSP data and then decodes the video data into the data of the continuous video frames. And respectively transmitting the continuous video frames to a passenger flow statistics and data analysis module according to the setting of data fusion.
3. Passenger flow statistics and data analysis module
The passenger flow statistics of the part is firstly carried out by a human face camera at the entrance and exit of a market, human face information is stored in a database, and if the number of the counted customers exceeds a certain threshold value, early warning information prompt can be carried out on management personnel of the market, so that the management personnel can conveniently carry out early prevention work; then, the human body information is stored in a human body database through real-time monitoring of a human body camera in a market, maximum passenger flow early warning management is carried out by utilizing matching of a camera number and an area and a timestamp, namely, when the number of passenger flows in a certain area exceeds a maximum threshold value in a certain time period or a certain time point, a manager in the market can check real-time video information of all areas in the market in real time through a monitoring video and obtain alarm information, and field management and passenger flow evacuation can be carried out through broadcasting or communication equipment. The channel is guaranteed to be smooth in the aspect of safety, rapid response to unknown accident conditions is achieved, and fact basis is provided for personnel management and safety precaution of a market. And finally, the human body shooting area is restrained by utilizing the human head camera, so that the accuracy of the human body counting quantity is optimized.
In a single-channel human body and human head combined area, the passenger flow information of adjacent points can be predicted through the point positions, if a previous point is crowded, the situation that the next point is crowded can be necessarily caused, the passenger flow information can be early warned in advance, and the passenger flow information is returned to the passenger flow guiding module. The three methods are combined for use, so that the accuracy, the real-time performance and the predictability of the passenger flow statistics of the scheme are guaranteed.
In addition, the client is tracked in real time through the camera, the information of the walking path and the staying time of the client is analyzed and recorded and returned to the comprehensive analysis platform, so that the statistical analysis is carried out on the client by market management personnel and a shop owner subsequently.
In this module, three deep learning algorithms are used: face detection, target tracking and face image quality. The combination of the three algorithms is the basis of the accuracy of the scheme, in the prior art, only a face detection algorithm is used for carrying out passenger flow statistics, the same person can acquire a plurality of images, the quality difference of the face images acquired by a camera is large, the problems of fuzziness, nonuniformity and different face angles exist, the same person can be counted for a plurality of times, and the accuracy of the final result is reduced, and the combination of the three algorithms is mainly divided into the following three steps:
step 3.1: acquiring face image information of a customer, performing quality judgment on the acquired image information by using a face image quality algorithm, and entering a subsequent step only by using a face image with qualified quality;
step 3.2: judging whether the customer is a new customer or not through face detection, and carrying out passenger flow statistics;
step 3.3: then, through target tracking, the database is maintained by matching human faces with human body characteristics, and information such as walking tracks, movement, staying time and the like of customers is obtained;
the matching of the human face features and the human body features is mainly divided into 3 steps:
step 3.3.1: the method comprises the steps of firstly obtaining a human body image by using a human face camera, then obtaining the human face image of a customer from the human body image, then obtaining the human face and the human body characteristics of the customer through the image, storing the human face and the human body characteristics into a database, and generating corresponding faceID and peopleID, wherein each human body image and the human face image are in one-to-one correspondence.
Step 3.3.2: acquiring a human body image by using a human body camera, and storing human body characteristics into a database to generate peoples ID 2;
step 3.3.3: and (3) searching and comparing the human body characteristics obtained in the step (3.3.1) in the human body database generated in the step (3.3.2), and if similar peoples ID2 is found in the human body database through comparison, associating and matching the faceID with the peoples ID 2. If only one kind of customer information is used for comparison and positioning tracking, the problems that the information comparison has no result and the target positioning is inaccurate can be solved well, and the passenger flow statistics and the positioning of the customer position information can be carried out more accurately.
The feature matching algorithm uses cosine similarity as a judgment reference, which is defined as follows:
the cosine similarity measures the similarity between two vectors by measuring the cosine value of the included angle of the two vectors, and the specific calculation formula is as follows:
before the system is formally deployed, the data acquisition module is used for acquiring the image information of the on-site customer and researching and developing the data analysis module. The data analysis module comprises the following steps:
1) training data set:
the module is mainly used for preparing training data, acquiring image frames from a camera, marking the position information of human faces and human heads, deleting image data which cannot be judged, and finally generating three types of training data sets of human face detection, human body detection and human head detection.
2) Model training:
the module firstly uses a trained basic detection model, then leads a training data set into the model for retraining, generates a detection model, realizes the detection of human faces, human bodies and human heads, and then carries out passenger flow statistics. On the basis that the camera continuously acquires images, the data set is continuously updated, so that the model is repeatedly trained in a specific scene, and the model is promoted to achieve higher accuracy than conventional training.
3) Passenger flow statistics
The module has the main functions of counting and tracking customers in a shopping mall, monitoring each area or channel in the shopping mall in real time through a plurality of cameras, counting the number of the customers by using the combination of the three cameras, setting a detection time range t, setting a maximum passenger flow threshold value m of each area, triggering an early warning prompt if the passenger flow number of a certain area exceeds m or is about to exceed m in the time t, and carrying out formal deployment of the system when the passenger flow counting module can accurately count the passenger flow number of the shopping mall.
4. Passenger flow guiding module
The passenger flow guidance module depends mainly on the results of section 3. The management personnel can simultaneously check the real-time statistical information of a plurality of areas through the platform, if the passenger flow statistics of the third part exceeds a set threshold value, the early warning information can appear, and the market management personnel can arrange related working personnel to dredge the passenger flow through broadcasting or communication equipment, so that the management and control quality is immediately improved, the labor cost is greatly reduced in a centralized monitoring management mode, and the social benefit and the economic benefit are remarkable. High instantaneity, reducing cost and improving effect.
5. Comprehensive analysis platform
And the comprehensive analysis platform acquires real-time information from the passenger flow statistical data analysis module and displays the real-time data. And acquiring related data from the face database and the human body database, performing statistical analysis by taking days, months, seasons, years and the like as units, and then performing transverse analysis and longitudinal analysis to obtain an evaluation result. The manager can backtrack and count the result, is convenient for find out the management missing point, quickly positions the related personnel, effectively adjusts the management scheme and finally improves the management and control quality of the market.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention in any way, and all simple modifications and equivalent variations of the above embodiments according to the technical spirit of the present invention are included in the scope of the present invention.
Claims (6)
1. A passenger flow statistical method based on multi-shot is based on a passenger flow statistical system and is characterized by comprising the following steps:
step 1: the method comprises the following steps of performing real-time field image acquisition by using a data acquisition module, wherein the data acquisition module is connected with a plurality of face cameras, human body cameras and human head cameras which are respectively used for acquiring face, human body and human head images; transmitting the human face images, the human body images and the human head images which are acquired by the human face camera, the human body camera and the human head camera to a data fusion platform at a time interval of 30 frames per second by using an RTSP (real time streaming protocol);
step 2: the received human face image, human body image and human head image in the RTSP data format are decoded through a data fusion platform to obtain video data, the video data are decoded into continuous video frame data, and the continuous video frame data are sent to a passenger flow statistics and data analysis module;
and step 3: firstly, carrying out preliminary statistics on passenger flow by adopting a human face image in a passenger flow statistics and data analysis module, and storing human face information into a database; then, performing maximum passenger flow early warning management through the matching relation between the machine number and the area of the human body camera on site and timestamp information; finally, the corresponding human body shooting area is restrained through the human head image, and the accuracy of the human body counting number is optimized;
and 4, step 4: judging whether the passenger flow statistics and data analysis result has a condition that the area exceeds a preset passenger flow alarm threshold value, and performing passenger flow guiding processing through a passenger flow guiding module;
and 5: and receiving data of the database and the passenger flow statistics and data analysis module through the comprehensive analysis platform, and performing final statistical analysis and summarization.
2. The method for counting passenger flow based on multi-shot according to claim 1, wherein the step 3 is a fusion of a human face algorithm, a human body algorithm and a human head algorithm for data processing, and the fusion of the three algorithms comprises the following specific steps:
step S3.1: acquiring face image information of a customer, performing quality judgment on the acquired image information by using a face image quality algorithm, and entering a subsequent step only by using a face image with qualified quality;
step S3.2: judging whether the customer is a new customer or not through face detection, and carrying out passenger flow statistics;
step S3.3: and then, through target tracking, the database is maintained by matching the human face with the human body characteristics, and the walking track, movement and stay time information of the customer are obtained.
3. The method for statistics of passenger flow based on multiple shots as claimed in claim 2, wherein in the step 3.3, the specific operations of matching the human face and the human body features are as follows:
step S3.3.1: firstly, acquiring a human body image by using a human face camera, acquiring a human face image of a customer in the human body image from the human body image, storing the human face and human body characteristics of the customer obtained by the image into a database, and generating a corresponding faceID number and a PeopleID, wherein each human body image and the human face image are in one-to-one correspondence;
step S3.3.2: acquiring a human body image by using a human body camera, and storing human body characteristics into a database to generate peoples ID 2;
step S3.3.3: and (3) searching and comparing the human body characteristics obtained in the step (3.3.1) in the human body database generated in the step (3.3.2), and if similar peoples ID2 is found in the human body database through comparison, associating and matching the faceID with the peoples ID 2.
4. The method for multi-shot based passenger flow statistics as claimed in claim 1, wherein in the step 3, the specific steps for performing data analysis are as follows:
first, the data set is trained:
acquiring image frames from a camera, marking position information of a human face and a human head, deleting image data which cannot be judged, and finally generating three training data sets of human face detection, human body detection and human head detection;
secondly, model training:
firstly, a trained basic detection model is used, then a training data set is imported into the model for retraining to generate a detection model, so that the detection of human faces, human bodies and human heads is realized, and then passenger flow statistics is carried out; on the basis that the camera continuously acquires images, the data set is continuously updated, so that the model is repeatedly trained in a specific scene, and the model is promoted to achieve higher accuracy than conventional training;
and finally, passenger flow statistics:
the method comprises the steps of counting and tracking customers in a shopping mall, monitoring each area or channel in the shopping mall in real time through a plurality of cameras, counting the number of the customers by using the three cameras in a combined mode, setting a detection time range t, setting a maximum passenger flow threshold value m of each area, triggering early warning prompt if the passenger flow number of a certain area exceeds m or is about to exceed m within the time t, and carrying out formal deployment of a multi-shooting passenger flow counting system when a passenger flow counting module can accurately count the passenger flow number of the shopping mall.
5. The method of claim 1, wherein the face camera is installed at the only entrance and exit of the passenger flow; the human body cameras are arranged on passenger flow channels on site and in multiple directions of all places; the human head camera is arranged at the crowded place of passenger flow on the spot.
6. A passenger flow statistical system based on multi-shot is used for the passenger flow statistical method based on multi-shot according to any one of claims 1 to 5, and is characterized by comprising a human head camera, a human face camera, a human body camera, a data acquisition module, a data fusion module, a database, a passenger flow guiding module, a passenger flow statistical and data analysis module and a comprehensive analysis platform;
the data acquisition module is connected with a plurality of human head cameras, human face cameras and human body cameras and is connected with the data fusion module, the data fusion module is connected with the passenger flow statistics and data analysis module through a database, and the passenger flow statistics and data analysis module is also respectively connected with the comprehensive analysis platform and the passenger flow guide module; the database is also connected with the comprehensive analysis platform.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110388818.5A CN113065491A (en) | 2021-04-12 | 2021-04-12 | Multi-shooting-based passenger flow statistical method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110388818.5A CN113065491A (en) | 2021-04-12 | 2021-04-12 | Multi-shooting-based passenger flow statistical method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113065491A true CN113065491A (en) | 2021-07-02 |
Family
ID=76566359
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110388818.5A Pending CN113065491A (en) | 2021-04-12 | 2021-04-12 | Multi-shooting-based passenger flow statistical method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113065491A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114783037A (en) * | 2022-06-17 | 2022-07-22 | 浙江大华技术股份有限公司 | Object re-recognition method, object re-recognition apparatus, and computer-readable storage medium |
-
2021
- 2021-04-12 CN CN202110388818.5A patent/CN113065491A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114783037A (en) * | 2022-06-17 | 2022-07-22 | 浙江大华技术股份有限公司 | Object re-recognition method, object re-recognition apparatus, and computer-readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11288495B2 (en) | Object tracking and best shot detection system | |
| CN109934176B (en) | Pedestrian recognition system, recognition method, and computer-readable storage medium | |
| CN106998444B (en) | Big data face monitoring system | |
| CN110348348B (en) | Quick identification method and early warning system for entrance identities of participants | |
| CN107590439A (en) | Target person identification method for tracing and device based on monitor video | |
| CN110942545A (en) | Dense person entrance guard control system and method based on face recognition and video fence | |
| CN104361327A (en) | Pedestrian detection method and system | |
| US20130216107A1 (en) | Method of surveillance by face recognition | |
| CN108540751A (en) | Monitoring method, apparatus and system based on video and electronic device identification | |
| CN108540750A (en) | Based on monitor video and the associated method, apparatus of electronic device identification and system | |
| WO2019080669A1 (en) | Method for person re-identification in enclosed place, system, and terminal device | |
| CN111968152B (en) | Dynamic identity recognition method and device | |
| CN112396658A (en) | Indoor personnel positioning method and positioning system based on video | |
| CN111814510B (en) | Method and device for detecting legacy host | |
| CN111460985A (en) | On-site worker track statistical method and system based on cross-camera human body matching | |
| CN110516600A (en) | A kind of bus passenger flow detection method based on Face datection | |
| CN109344792A (en) | A kind of Motion parameters tracking | |
| CN116189052A (en) | Security method, system, intelligent terminal and storage medium based on video stream analysis | |
| CN113065491A (en) | Multi-shooting-based passenger flow statistical method and system | |
| CN114360013A (en) | High-precision face recognition camera | |
| CN117809351A (en) | Method and device for detecting wearing state of safety helmet | |
| CN114677608A (en) | Identity feature generation method, device and storage medium | |
| CN108540748A (en) | Monitor video and the associated method, apparatus of electronic device identification and system | |
| CN110930432A (en) | Video analysis method, device and system | |
| CN113743380B (en) | Active tracking method based on video image dynamic monitoring |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |