CN112347907A - 4S store visitor behavior analysis system based on Reid and face recognition technology - Google Patents
4S store visitor behavior analysis system based on Reid and face recognition technology Download PDFInfo
- Publication number
- CN112347907A CN112347907A CN202011219917.2A CN202011219917A CN112347907A CN 112347907 A CN112347907 A CN 112347907A CN 202011219917 A CN202011219917 A CN 202011219917A CN 112347907 A CN112347907 A CN 112347907A
- Authority
- CN
- China
- Prior art keywords
- human body
- face
- picture
- database
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/53—Querying
- G06F16/532—Query formulation, e.g. graphical querying
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30241—Trajectory
Abstract
The invention provides a 4S store visitor behavior analysis system based on Reid and face recognition technology, which comprises the following steps: step (1), decoding a video stream from video monitoring, and extracting a picture frame at regular time; step (2), carrying out pedestrian detection on the picture frame to obtain a pedestrian human body picture; step (3) calculating a pedestrian picture by using a Reid model, extracting a pedestrian feature vector, and step (4) storing the data in the step (3) into a database, wherein the data comprises picture shooting time and camera number; screening human body trajectory data associated with the human face to obtain trajectory behavior data of a corresponding customer; and calculating the pedestrian feature vector of each subsequent picture frame, and judging whether the pedestrian feature vector appears before. The passenger flow calculation method is higher in passenger flow calculation reliability and more complete in data, and repeated passenger flow can be effectively eliminated. Data is collected in a non-inductive mode, the behavior of a customer is not interfered, and experience is good.
Description
Technical Field
The invention relates to the field of data structuring processing, in particular to a 4S store visitor behavior analysis system based on Reid and face recognition technology.
Background
For e-commerce, data is very important, and the data can acquire the source, time and hobby of a user and even find the offline position of a consumer through clicking, browsing, purchasing and other behaviors of the user, so that a popularization channel and popularization content are found by analyzing the data, and accurate commodity recommendation is performed to really meet the requirements of the user to promote bargain.
The data sources of the entity 4S stores are very few, except for CRM, ERP, POS and other transaction data, entity merchants often ignore the importance of passenger flow or realize that the passenger flow is the leading person of all transactions, but have no good method for solving the problem. And the birth of passenger flow statistics and analysis just solves the problems of statistics, management and analysis of the passenger flow data of the entity store.
Through the structural processing of the 4S store monitoring video, the regions where the customers pay close attention to and stay are analyzed, on one hand, store optimization, exhibit display and store site selection can be supported, on the other hand, personalized services and personalized recommendations of the customers can be provided for different groups, and accurate marketing is achieved.
The traditional passenger flow statistics method is commonly provided with three types, namely infrared sensing passenger flow statistics, and the main principle is to detect resistance change generated by blocking infrared rays by a human body passing through an infrared sensing area or judge the number of the human body by detecting infrared rays with specific wavelengths emitted by the human body. However, this method has the disadvantages that when a plurality of people enter and exit simultaneously, the precision is low, and the passenger flow direction cannot be judged. And secondly, the passenger flow statistics of the three-roller gate mainly adopts a mechanical mode, and a customer enters a relevant place and needs to pass through a rolling gate opening, and the rolling gate rolls once, so that the number of the passengers entering and exiting the three-roller gate is recorded. This kind of mode degree of accuracy is higher, but the convenience is not enough, and experience is not good, can't be applied to market or 4S shop. Thirdly, head and shoulder detection passenger flow statistics based on an artificial intelligence deep learning technology, in the method, a camera is installed on a passageway or a ceiling, pictures of pedestrians passing right below are captured, and whether the pedestrians pass through is judged by detecting the head and shoulder. The method has high efficiency, the coverage area of a single camera is wide, but the problems that repeated passenger flow counting cannot be removed, more client characteristics except passenger flow cannot be obtained, and the client track cannot be obtained exist.
Disclosure of Invention
The technical problem to be solved by the invention is to provide a 4S store visitor behavior analysis system based on Reid and face recognition technology, so as to solve the problems proposed in the background technology.
The technical problem solved by the invention is realized by adopting the following technical scheme: A4S store visitor behavior analysis system based on Reid and face recognition technology comprises the following steps:
step (1), decoding a video stream from video monitoring, and extracting a picture frame at regular time;
step (2), carrying out pedestrian detection on the picture frame to obtain a pedestrian human body picture;
step (3), calculating a pedestrian picture by using a Reid model, extracting a pedestrian feature vector as a basis for comparing different people; meanwhile, a pedestrian picture is calculated by using a face detection and face recognition model, and if a clear face exists, the human body picture and the face picture are associated;
step (4) storing the data in the step (3) into a database, wherein the data comprise picture shooting time and camera number;
screening human body trajectory data associated with the human face to obtain trajectory behavior data of a corresponding customer; and calculating the pedestrian feature vector of each subsequent picture frame, and judging whether the pedestrian feature vector appears before.
The database in the step (4) comprises a human body feature database, a human face database and a track database, wherein the human body feature database uses a key-value redis database, is stored and updated according to the place and time, calculates the cosine distance between two human body feature vectors, and is considered as the same person if the cosine distance is smaller than a specified threshold value.
The face database manages the face data of all customers, and when searching for face pictures, if no corresponding client data exists, the face data is added into the database and used as a new client face record.
The track database stores the number and the position of a camera of each customer appearing in a store, and the time sequence data formed by the camera is track information of the customers.
The step (5) comprises analyzing a human body picture, detecting and identifying human faces, wherein the human body picture is analyzed by using a pedestrian to identify a neural network model, extracting a human body characteristic vector, comparing the human body characteristic vector to be identified with all human body characteristic records of a current store and the current day in a redis library, and calculating a cosine distance; if the minimum value of all the distance values is smaller than the distance threshold value, the two human body feature vectors are regarded as the same person in the pictures corresponding to the two human body feature vectors, otherwise, the two human body feature vectors are regarded as a new customer, and the new human body feature vector is stored in a redis database.
The human face detection, identification and analysis is that if a recognizable clear human face can be detected in the human body picture, the human body picture of the human face can be associated, the human face picture is searched and compared in a human face database, if a record larger than a similarity threshold exists, the faceid of the corresponding record is obtained, otherwise, the new human face record is added, and a new faceid is generated; and associating the faceid with the body id corresponding to the human body diagram, and maintaining the relationship between the faceid and the body id in a mysql relational database.
Compared with the prior art, the invention has the beneficial effects that: the passenger flow calculation method is higher in passenger flow calculation reliability and more complete in data, and repeated passenger flow can be effectively eliminated. Data is collected in a non-inductive mode, the behavior of a customer is not interfered, and experience is good.
Compared with a head-shoulder passenger flow or thermal camera mode, the invention can provide a finer-grained pedestrian track of customers, can acquire more customer characteristics to support business statistics and high-quality personalized content push, is high in speed, is provided with a processor of the Kurui i7, and can process a plurality of 1080P images within 1 second; based on real-time data acquisition and big data computing platform, customer's propelling movement information is more accurate.
Drawings
FIG. 1 is a flow chart of the system operation of the present invention.
FIG. 2 is a database schematic of the present invention.
Detailed Description
In the description of the present invention, it should be noted that unless otherwise specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly and may be, for example, fixedly connected, detachably connected, or integrally connected, mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements.
Example 1
As shown in fig. 1 and fig. 2, a 4S store hidden customer behavior analysis system based on Reid and face recognition technology includes the following steps:
and (1) decoding the monitoring video stream, extracting picture frames at intervals, detecting pedestrians in the picture frames by using a pedestrian detection deep neural network model, and capturing all human body pictures.
And (2) analyzing each human body picture in the previous step. And (3) identifying the neural network model by using the pedestrian, extracting the human body characteristic vector, comparing the human body characteristic vector to be identified with all human body characteristic records of the current store in the redis library on the same day, and calculating the cosine distance. If the minimum value of all the distance values is smaller than the distance threshold value, the two human body feature vectors are regarded as the same person in the pictures corresponding to the two human body feature vectors, otherwise, the two human body feature vectors are regarded as a new customer, and the new human body feature vector is stored in a redis database.
And (3) simultaneously carrying out face detection, identification and analysis on the human body picture. If the recognizable clear human face can be detected in the human body picture, the human body picture of the human face can be associated. And searching and comparing the face picture in a face database, if records larger than a similarity threshold exist, acquiring the faceid of the corresponding record, and if not, adding the new face record to generate a new faceid. And associating the faceid with the body id corresponding to the human body diagram, and maintaining the relationship between the faceid and the body id in a mysql relational database.
And (4) accumulating the analysis results of all the picture frames, and counting data with space-time continuity of all the body ids in different time and different camera monitoring areas, namely the trajectory behavior data of each customer entering the store.
And (5) the integrity of the human body image influences the extraction of the human body characteristic vector, so that the comparison results of different human body images are influenced. We draw the ROI in the field of view region to remove the possibly incomplete body map at the edge of the field of view and also remove the content with large distortion at the edge of the field of view.
And (6) the service data analysis platform can count the number of people entering the store at each time period, the residence time of the customer in the store, the attributes (portrait) of the customer, the judgment of new and old customers, the preference of the customer on the model of the exhibit and the like.
And (7) counting the final transaction rate from the residence time of the customer store and the visit frequency dimension of the customer store to predict the transaction possibility of the customer. The convenience is brought to the excavation and management of the store to the hidden passengers.
The invention discloses a system for monitoring and analyzing videos of 4S stores, collecting behavior track data of customers and analyzing the hidden customers based on Reid and face recognition technologies. Cross-mirror tracking: the (person Re-Identification, referred to as Reid) technology is the hot direction of computer vision research, and mainly solves the recognition and retrieval of pedestrians in a camera-crossing scene. The method has the advantages that the Reid technology is used for pedestrian detection, comparison and tracking, when clear human bodies and human face images are shot by the near-focus cameras in the store, the human faces and the human face images correspond to each other, and because human body targets are much larger than human face targets and the front or back and other angle problems do not exist, behavior track data in the customer 4S store collected by the system are complete and reliable.
The dimensionality of data statistics is that customer is in the store, and the length of stay of different monitoring points is long, and the customer of the same bank, the sales manager who receives waiters etc. can also detect customer age, sex, mood, and the customer secondary is gone into the shop and is reminded, is convenient for sell the better service customer of manager to and improve the rate of friendship. The store can reasonably distribute the number of shopping guides every day according to the number of customers, and reasonably arrange commercial operation activities, thereby realizing the efficient management of the store. The operator arranges matched shopping guides according to the picture (age/sex) of the customer, thereby achieving the effect of achieving twice the result with half the effort. And planning different promotion activities according to the group images of the customers.
The foregoing shows and describes the general principles and broad features of the present invention and advantages thereof. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are described in the specification and illustrated only to illustrate the principle of the present invention, but that various changes and modifications may be made therein without departing from the spirit and scope of the present invention, which fall within the scope of the invention as claimed. The scope of the invention is defined by the appended claims and equivalents thereof.
Claims (6)
1. A4S store visitor behavior analysis system based on Reid and face recognition technology is characterized in that: the method comprises the following steps:
step (1), decoding a video stream from video monitoring, and extracting a picture frame at regular time;
step (2), carrying out pedestrian detection on the picture frame to obtain a pedestrian human body picture;
step (3), calculating a pedestrian picture by using a Reid model, extracting a pedestrian feature vector as a basis for comparing different people; meanwhile, a pedestrian picture is calculated by using a face detection and face recognition model, and if a clear face exists, the human body picture and the face picture are associated;
step (4) storing the data in the step (3) into a database, wherein the data comprise picture shooting time and camera number;
screening human body trajectory data associated with the human face to obtain trajectory behavior data of a corresponding customer; and calculating the pedestrian feature vector of each subsequent picture frame, and judging whether the pedestrian feature vector appears before.
2. The 4S store hidden passenger behavior analysis system based on Reid and face recognition technology according to claim 1, characterized in that: the database in the step (4) comprises a human body feature database, a human face database and a track database, wherein the human body feature database uses a key-value redis database, is stored and updated according to the place and time, calculates the cosine distance between two human body feature vectors, and is considered as the same person if the cosine distance is smaller than a specified threshold value.
3. The 4S store hidden passenger behavior analysis system based on Reid and face recognition technology according to claim 2, characterized in that: the face database manages the face data of all customers, and when searching for face pictures, if no corresponding client data exists, the face data is added into the database and used as a new client face record.
4. The 4S store hidden passenger behavior analysis system based on Reid and face recognition technology according to claim 2, characterized in that: the track database stores the number and the position of a camera of each customer appearing in a store, and the time sequence data formed by the camera is track information of the customers.
5. The 4S store hidden passenger behavior analysis system based on Reid and face recognition technology according to claim 1, characterized in that: the step (5) comprises analyzing a human body picture, detecting and identifying human faces, wherein the human body picture is analyzed by using a pedestrian to identify a neural network model, extracting a human body characteristic vector, comparing the human body characteristic vector to be identified with all human body characteristic records of a current store and the current day in a redis library, and calculating a cosine distance; if the minimum value of all the distance values is smaller than the distance threshold value, the two human body feature vectors are regarded as the same person in the pictures corresponding to the two human body feature vectors, otherwise, the two human body feature vectors are regarded as a new customer, and the new human body feature vector is stored in a redis database.
6. The 4S store hidden passenger behavior analysis system based on Reid and face recognition technology according to claim 5, characterized in that: the human face detection, identification and analysis is that if a recognizable clear human face can be detected in the human body picture, the human body picture of the human face can be associated, the human face picture is searched and compared in a human face database, if a record larger than a similarity threshold exists, the faceid of the corresponding record is obtained, otherwise, the new human face record is added, and a new faceid is generated; and associating the faceid with the body id corresponding to the human body diagram, and maintaining the relationship between the faceid and the body id in a mysql relational database.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011219917.2A CN112347907A (en) | 2020-11-05 | 2020-11-05 | 4S store visitor behavior analysis system based on Reid and face recognition technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011219917.2A CN112347907A (en) | 2020-11-05 | 2020-11-05 | 4S store visitor behavior analysis system based on Reid and face recognition technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112347907A true CN112347907A (en) | 2021-02-09 |
Family
ID=74429249
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011219917.2A Withdrawn CN112347907A (en) | 2020-11-05 | 2020-11-05 | 4S store visitor behavior analysis system based on Reid and face recognition technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112347907A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113487357A (en) * | 2021-07-08 | 2021-10-08 | 上海叮铃铃信息技术有限公司 | Customer file management method and system based on face recognition |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109272347A (en) * | 2018-08-16 | 2019-01-25 | 苏宁易购集团股份有限公司 | A kind of statistical analysis technique and system of shops's volume of the flow of passengers |
| CN111241932A (en) * | 2019-12-30 | 2020-06-05 | 广州量视信息科技有限公司 | Automobile exhibition room passenger flow detection and analysis system, method and storage medium |
-
2020
- 2020-11-05 CN CN202011219917.2A patent/CN112347907A/en not_active Withdrawn
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109272347A (en) * | 2018-08-16 | 2019-01-25 | 苏宁易购集团股份有限公司 | A kind of statistical analysis technique and system of shops's volume of the flow of passengers |
| CN111241932A (en) * | 2019-12-30 | 2020-06-05 | 广州量视信息科技有限公司 | Automobile exhibition room passenger flow detection and analysis system, method and storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113487357A (en) * | 2021-07-08 | 2021-10-08 | 上海叮铃铃信息技术有限公司 | Customer file management method and system based on face recognition |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109784162B (en) | Pedestrian behavior recognition and trajectory tracking method | |
| US8351647B2 (en) | Automatic detection and aggregation of demographics and behavior of people | |
| Benezeth et al. | Abnormal events detection based on spatio-temporal co-occurences | |
| US8665333B1 (en) | Method and system for optimizing the observation and annotation of complex human behavior from video sources | |
| US8295597B1 (en) | Method and system for segmenting people in a physical space based on automatic behavior analysis | |
| JP6854881B2 (en) | Face image matching system and face image search system | |
| US20170169297A1 (en) | Computer-vision-based group identification | |
| US9740977B1 (en) | Method and system for recognizing the intentions of shoppers in retail aisles based on their trajectories | |
| Benezeth et al. | Abnormality detection using low-level co-occurring events | |
| US10713670B1 (en) | Method and system for finding correspondence between point-of-sale data and customer behavior data | |
| CN109272347A (en) | A kind of statistical analysis technique and system of shops's volume of the flow of passengers | |
| CN110399835B (en) | Analysis method, device and system for personnel residence time | |
| KR101779096B1 (en) | The object pursuit way in the integration store management system of the intelligent type image analysis technology-based | |
| CN112347909B (en) | Retail store entrance and exit passenger flow statistical method | |
| Merad et al. | Tracking multiple persons under partial and global occlusions: Application to customers’ behavior analysis | |
| CN109658194A (en) | A kind of lead referral method and system based on video frequency tracking | |
| Liu et al. | Customer behavior recognition in retail store from surveillance camera | |
| Xiang et al. | Autonomous Visual Events Detection and Classification without Explicit Object-Centred Segmentation and Tracking. | |
| AU2017231602A1 (en) | Method and system for visitor tracking at a POS area | |
| Hampapur et al. | Searching surveillance video | |
| WO2015003287A1 (en) | Behavior recognition and tracking system and operation method therefor | |
| CN112347907A (en) | 4S store visitor behavior analysis system based on Reid and face recognition technology | |
| Jiao et al. | Traffic behavior recognition from traffic videos under occlusion condition: a Kalman filter approach | |
| US11004093B1 (en) | Method and system for detecting shopping groups based on trajectory dynamics | |
| TW201502999A (en) | A behavior identification and follow up system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20210209 |
|
| WW01 | Invention patent application withdrawn after publication |