CN111340569A - Store people stream analysis method, device, system, terminal and medium based on cross-border tracking - Google Patents
Store people stream analysis method, device, system, terminal and medium based on cross-border tracking Download PDFInfo
- Publication number
- CN111340569A CN111340569A CN202010233594.6A CN202010233594A CN111340569A CN 111340569 A CN111340569 A CN 111340569A CN 202010233594 A CN202010233594 A CN 202010233594A CN 111340569 A CN111340569 A CN 111340569A
- Authority
- CN
- China
- Prior art keywords
- store
- tracking
- cross
- identity
- images
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 37
- 238000004422 calculation algorithm Methods 0.000 claims description 44
- 238000000034 method Methods 0.000 claims description 41
- 238000001514 detection method Methods 0.000 claims description 27
- 238000012545 processing Methods 0.000 claims description 27
- 238000005206 flow analysis Methods 0.000 claims description 19
- 238000004364 calculation method Methods 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 11
- 238000004891 communication Methods 0.000 claims description 10
- 238000010276 construction Methods 0.000 claims description 9
- 238000012216 screening Methods 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 7
- 238000013135 deep learning Methods 0.000 claims description 5
- 238000010801 machine learning Methods 0.000 claims description 4
- 230000006870 function Effects 0.000 abstract description 14
- 238000010586 diagram Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000007619 statistical method Methods 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000005021 gait Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 241000894007 species Species 0.000 description 2
- 235000013311 vegetables Nutrition 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- QVFWZNCVPCJQOP-UHFFFAOYSA-N chloralodol Chemical compound CC(O)(C)CC(C)OC(O)C(Cl)(Cl)Cl QVFWZNCVPCJQOP-UHFFFAOYSA-N 0.000 description 1
- 235000019800 disodium phosphate Nutrition 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 229940079593 drug Drugs 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 235000011888 snacks Nutrition 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0201—Market modelling; Market analysis; Collecting market data
- G06Q30/0203—Market surveys; Market polls
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/53—Recognition of crowd images, e.g. recognition of crowd congestion
Abstract
The invention provides a store people stream analysis method, a device, a system, a terminal and a medium based on cross-border tracking. In addition, the invention also analyzes the similarity of the extracted customer characteristics and the characteristics in the characteristic library, and simultaneously realizes the functions of the store people flow statistics and the functions of analyzing the shopping preference and the shopping path of customers in the store by depending on a single camera.
Description
Technical Field
The invention relates to the technical field of artificial intelligence and computer vision, in particular to a store people stream analysis method, device, system, terminal and medium based on cross-border tracking.
Background
At present, most of store people flow statistical methods in the market are based on face recognition or people detection tracking methods, and people detection tracking methods are generally applied to scenes that cameras are arranged right above entrances and exits of stores, and are only suitable for the situation that the entrances can only enter but not exit and the exits can only exit, if the entrances can enter and exit, the people repeatedly entering the lens can be repeatedly counted by the current people flow statistical methods, and therefore large statistical errors are caused.
On the other hand, in the above view angle, the camera is located right above the person, the camera can only collect the head features of the person, and cannot collect the overall features of the person, the identity of each person cannot be determined by the head features of the person, and the shopping path and shopping preference of the customer cannot be analyzed. Therefore, those skilled in the art have endeavored to develop a better solution.
Disclosure of Invention
In view of the above-mentioned defects of the prior art, the technical problem to be solved by the present invention is that the shopping path and shopping preferences of the customer cannot be accurately analyzed.
In order to achieve the above object, a first aspect of the present invention provides a store people stream analysis method based on cross-border tracking, including: constructing a plurality of feature libraries; each feature library prestores the identity features of pedestrians, pedestrian images and the position and time information corresponding to the images in the corresponding store; extracting the identity characteristics of the tracked target, and calculating the similarity between the identity characteristics of the tracked target and the identity characteristics in each feature library; and according to the similarity calculation result, screening out identity features meeting the similarity requirement from the feature library, and outputting corresponding pedestrian image, position and time information so as to perform cross-border tracking and preference analysis on the tracked target.
In a preferred embodiment of the present invention, the feature library is constructed in a manner including: acquiring in-store images of corresponding stores in a current statistical period; traversing pedestrian pictures in the in-store image and extracting pedestrian identity characteristics; clustering the identity characteristics of the pedestrians by using a clustering algorithm, and storing images of the pedestrians and position and time information corresponding to the images according to the categories; and the number of the clusters is the flow of people in the current statistical period of the store.
In another preferred embodiment of the present invention, the extracting the identity feature of the tracking target includes: inputting an image of the tracking target; and extracting the identity characteristic of the tracking target from the image of the tracking target by utilizing a pedestrian detection algorithm.
In another preferred embodiment of the present invention, the pedestrian detection algorithm includes a motion detection-based target tracking algorithm, a machine learning-based pedestrian detection algorithm, or a deep learning-based pedestrian detection algorithm.
In another preferred embodiment of the present invention, the cross-border tracking of the tracking target includes: and tracking the shopping paths of the target among a plurality of shops according to the time information in the image.
In another preferred embodiment of the present invention, the tracking target is subjected to a preference analysis, which includes analyzing any one or more combinations of store preferences, merchandise preferences, shopping period preferences and payment method preferences of the tracking target.
In order to achieve the above object, a second aspect of the present invention provides a store people flow analysis device based on cross-border tracking, including: the characteristic library construction module is used for constructing a plurality of characteristic libraries; each feature library prestores the identity features of pedestrians, pedestrian images and the position and time information corresponding to the images in the corresponding store; the characteristic extraction module is used for extracting the identity characteristic of the tracking target; the similarity calculation module is used for calculating the similarity between the identity characteristics of the tracking target and the identity characteristics in each feature library; and the tracking module is used for screening the identity characteristics meeting the similarity requirement from the characteristic library according to the similarity calculation result, outputting corresponding pedestrian images, positions and time information, and performing cross-border tracking and preference analysis on the tracked target.
In a preferred embodiment of the present invention, the feature library construction module performs the following sub-steps: acquiring in-store images of corresponding stores in a current statistical period; traversing pedestrian pictures in the in-store image and extracting pedestrian identity characteristics; clustering the identity characteristics of the pedestrians by using a clustering algorithm, and storing images of the pedestrians and position and time information corresponding to the images according to the categories; and the number of the clusters is the flow of people in the current statistical period of the store.
To achieve the above object, a third aspect of the present invention provides a store people stream analysis system based on cross-border tracking, including: the image acquisition units are respectively arranged at the inner sides of the corresponding stores and are used for shooting in-store images; the processing unit is in communication connection with the image acquisition unit so as to receive the in-store images shot by the processing unit; the processing unit is used for constructing a plurality of feature libraries; each feature library prestores the identity features of pedestrians, pedestrian images and the position and time information corresponding to the images in the corresponding store; extracting the identity characteristics of the tracked target, and calculating the similarity between the identity characteristics of the tracked target and the identity characteristics in each feature library; and according to the similarity calculation result, screening out identity characteristics meeting the similarity requirement from the characteristic library, and outputting corresponding pedestrian image, position and time information so as to perform cross-border tracking and preference analysis on the tracked target.
To achieve the above object, a fourth aspect of the present invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the cross-border tracking-based store people flow analysis method of the foregoing.
To achieve the above object, a fifth aspect of the present invention provides an electronic terminal, comprising a processor and a memory; the memory is used for storing a computer program; the processor is used for executing the computer program stored in the memory so as to enable the terminal to execute the store people flow analysis method based on cross-border tracking in the content.
The shop people flow analysis method, device, system, terminal and medium based on cross-border tracking provided by the invention have the following technical effects: the invention arranges the camera at the inner side of the store to shoot the whole image in the store, and realizes the people flow statistical function by a characteristic clustering mode after extracting the characteristics of the customers from the image in the store, the people flow statistical mode greatly improves the people counting accuracy in the store with unfixed entrance and exit, and well solves the problem that the camera arranged at the entrance and exit of the store repeatedly counts the entering and exiting customers in the prior art. In addition, the invention also analyzes the similarity of the extracted customer characteristics and the characteristics in the characteristic library, and simultaneously realizes the functions of the store people flow statistics and the functions of analyzing the shopping preference and the shopping path of customers in the store by depending on a single camera.
The conception, the specific structure and the technical effects of the present invention will be further described with reference to the accompanying drawings to fully understand the objects, the features and the effects of the present invention.
Drawings
Fig. 1 is a flowchart illustrating a store people flow analysis method based on cross-border tracking according to an embodiment of the present invention.
FIG. 2 is a schematic flow chart of constructing a feature library according to an embodiment of the present invention.
Fig. 3 is a schematic structural diagram of a store people flow analysis system based on cross-border tracking according to an embodiment of the present invention.
Fig. 4 is a schematic structural diagram of a store people flow analysis device based on cross-border tracking according to an embodiment of the present invention.
Fig. 5 is a schematic structural diagram of an electronic terminal according to an embodiment of the present invention.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict.
It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention, and the drawings only show the components related to the present invention rather than the number, shape and size of the components in actual implementation, and the type, quantity and proportion of the components in actual implementation may be changed freely, and the layout of the components may be more complicated.
Also, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context indicates otherwise. It will be further understood that the terms "comprises," "comprising," and/or "comprising," when used in this specification, specify the presence of stated features, operations, elements, components, items, species, and/or groups, but do not preclude the presence, or addition of one or more other features, operations, elements, components, items, species, and/or groups thereof. The terms "or" and/or "as used herein are to be construed as inclusive or meaning any one or any combination. Thus, "A, B or C" or "A, B and/or C" means "any of the following: a; b; c; a and B; a and C; b and C; A. b and C ". An exception to this definition will occur only when a combination of elements, functions or operations are inherently mutually exclusive in some way.
The invention aims to provide a store people flow analysis method, a device, a system, a terminal and a medium based on cross-border tracking, which can break through the limitation that a camera can only be arranged at the top of an entrance and an exit of a store, and can simultaneously realize the functions of accurate statistics of store people flow, shopping paths of customers in the store, shopping preference and the like by means of a single camera.
In order to make the objects, technical solutions and advantages of the present invention more apparent, the technical solutions in the embodiments of the present invention are further described in detail by the following embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The first embodiment is as follows:
fig. 1 is a flow chart of a store people flow analysis method based on cross-border tracking according to an embodiment of the present invention. The store people stream analysis method based on cross-border tracking mainly comprises the following steps of S11-S13.
It should be understood that the method of the present embodiment can be applied to controllers, such as arm (advanced RISC machines) controllers, fpga (field Programmable Gate array) controllers, soc (system on chip) controllers, dsp (digital Signal processing) controllers, or mcu (micro controller unit) controllers; the system can also be applied to personal computers such as desktop computers, notebook computers, tablet computers, smart phones, smart televisions, personal digital assistants (personal digital assistants, PDA for short) and the like; the present invention is also applicable to a server, where the server may be arranged on one or more entity servers according to various factors such as functions, loads, and the like, and may also be formed by a distributed or centralized server cluster, which is not limited in this embodiment.
In step S11, a plurality of feature libraries are constructed; each feature library prestores the identity features of pedestrians, images of the pedestrians and the corresponding position and time information of the images in the corresponding store.
In an optional implementation manner of this embodiment, a construction process of the feature library is as follows: acquiring in-store images of corresponding stores in a current statistical period; traversing pedestrian pictures in the in-store image and extracting pedestrian identity characteristics; clustering the identity characteristics of the pedestrians by using a clustering algorithm, and storing images of the pedestrians and position and time information corresponding to the images according to the categories; and the number of the clusters is the flow of people in the current statistical period of the store.
Generally speaking, a store corresponds to a feature library for storing pedestrian images and position and time information corresponding to the images in the store; in other embodiments, a plurality of stores may correspond to the same feature library, and the storage content of the feature library may be divided into a plurality of sub-feature libraries, where each sub-feature library corresponds to one store, so as to facilitate management and query.
It should be noted that the types of stores related to the present embodiment are various, including but not limited to, business places such as a store, a supermarket, a convenience store, a pharmacy or a vegetable market; the statistical period in this embodiment may be set to one day, one week, or one month according to actual requirements, which is not limited in this embodiment.
For the convenience of those skilled in the art to understand, the following takes any store as an example (specifically, refer to steps S200 to S207 in fig. 2), and further explains the construction principle of the feature library.
Step S200: and starting.
Step S201: store video is input.
The current statistical method for store people flow is generally realized by detecting, tracking and counting the head of a person by means of a camera vertically installed at the top of an entrance and an exit of a store, but is only suitable for the entrance which can not be entered and the exit which can not be entered, and the identity of a customer is difficult to identify and the shopping preference of the customer in the store can not be analyzed because the camera only shoots the entrance and the exit.
In view of the above, in the present embodiment, the camera is installed inside the store in order to enable the camera to capture the overall characteristics of the in-store customer and the overall situation of the store, and therefore the store video input in the present embodiment is the store video received from the camera installed in the store and captured by the camera, and the video can capture the overall characteristics of the in-store customer and the overall situation of the store.
Step S202: and detecting whether the current frame has a pedestrian by utilizing a pedestrian detection algorithm. The pedestrian detection algorithm includes, but is not limited to, the aforementioned target tracking algorithm based on motion detection, pedestrian detection algorithm based on machine learning, or pedestrian detection algorithm based on deep learning, etc.
Step S203: and if the pedestrian is detected in the current frame, extracting the identity characteristic of the pedestrian, storing the corresponding in-store image, the position information and the time information into a characteristic library, and then detecting the next frame. Specifically, after the pedestrians are detected in the current frame, a group of arrays containing the position information of all the pedestrians in the current frame is output, and each element in the array represents the position information of one of the pedestrians.
Step S204: and if the pedestrian is not detected in the current frame, detecting the next frame. That is, if no pedestrian is detected in the current frame, no array is output.
Step S205: it is determined whether all frames have been detected.
Specifically, after the previous frame is detected, the detection program can automatically acquire the next frame and continue to detect the next frame; if the detection program finds that the next frame is empty when the detection program acquires the next frame, the detection program indicates that all frames in the current statistical period are detected completely.
Step S206: if all frames are detected, clustering the identity characteristics of the pedestrians by using a clustering algorithm, and storing corresponding in-store images, position information and time information to construct a characteristic library; and the number of the clusters is the flow of people in the current statistical period of the store.
Specifically, an empty feature library may be initialized before clustering, the feature library does not contain any features, and each time a pedestrian is detected by the pedestrian detection neural network, the feature extraction neural network automatically extracts the identity features of the detected pedestrian and adds the features to the feature library until all frames are detected.
Step S207: if all frames have not been detected, the method returns to the step S202 to continue the detection.
As can be seen from the foregoing steps S200 to S207, after a statistical period is finished, clustering all overall features of the customers in the statistical period by using a clustering algorithm, and storing the position information and the time information of the corresponding customers while clustering, where the number of the clusters obtained is the number of the customers in the store in the statistical period; according to the invention, through the clustering mode, the people counting accuracy in the store with unfixed entrance and exit can be greatly improved, and the purchasing path and purchasing preference of each customer in the store can be accurately obtained through the stored position information and time information of the customer, which are functions that the existing people flow counting scheme does not have.
Step S12: and extracting the identity characteristics of the tracking target, and calculating the similarity between the identity characteristics of the tracking target and the identity characteristics in each feature library.
In an optional implementation manner of this embodiment, a process of extracting the identity feature of the tracking target is as follows: inputting an image of the tracking target; and extracting the identity characteristic of the tracking target from the image of the tracking target by utilizing a pedestrian detection algorithm. It should be noted that the identity feature referred to in the present invention refers to a feature that can identify the identity of a pedestrian, such as a face feature, a gait feature, a head feature, a hair style feature, a height feature, or a leg feature, alone or in combination with other features.
It should be understood that the pedestrian detection algorithm involved in the present embodiment is a computer vision algorithm for detecting all pedestrians from an image or video frame image. The pedestrian detection algorithm can adopt a target tracking algorithm based on motion detection, namely a moving foreground target is extracted by using a background modeling algorithm under the condition that a camera is still, then the moving target is classified by using a classifier, and whether the moving target contains a pedestrian is judged, wherein the pedestrian detection algorithm specifically comprises a Gaussian mixture model algorithm, a ViBe algorithm, a frame difference algorithm, a sample consistency modeling algorithm or a PBAS algorithm and the like; a pedestrian detection algorithm based on machine learning can be adopted, namely, the appearance characteristics (such as color, edge, texture and the like) of the human body are utilized to train a classifier and distinguish pedestrians and backgrounds, specifically, an algorithm based on HOG + SVM, an algorithm based on HOG + AdaBoost, an algorithm based on ICF + AdaBoost, an algorithm based on DPM + LatenstSVM and the like are adopted; the pedestrian detection algorithm based on deep learning can be adopted, namely, the classifier is trained and the pedestrian and the background are distinguished based on the human body characteristics learned by the deep learning, so that the classifier has strong hierarchical expression capability and good robustness, and specifically, the algorithms are the existing algorithms, so that the detailed description is omitted.
In an optional implementation manner of this embodiment, a similarity calculation method is used to calculate the similarity between the identity feature of the tracked target and the identity feature of the pedestrian in the feature library, that is, the distance between the identity feature of the tracked target and the identity feature of the pedestrian in the feature library is calculated, where the smaller the distance is, the higher the similarity is, and the larger the distance is, the lower the similarity is. The similarity calculation method may be, for example, euclidean distance method, cosine similarity method, pearson correlation coefficient method, minkowski distance method, or chebyshev distance method, and the embodiment is not limited thereto.
Step S13: and according to the similarity calculation result, screening out identity features meeting the similarity requirement from the feature library, and outputting corresponding pedestrian image, position and time information so as to perform cross-border tracking and preference analysis on the tracked target.
Specifically, when a shopping situation of a specific customer at another store is sought, the features are extracted from the image of the customer and then input into a feature library of another store or a plurality of stores, the similarity between the features of the customer and the features in the feature library is calculated, when the similarity is greater than a certain preset threshold value, the customer can be regarded as the same person, and finally, all images with the similarity greater than the preset threshold value and corresponding position and time information are output.
The method provided by the embodiment can track the shopping path of the customer in one store, and also track the shopping path of the customer among a plurality of stores. For example, according to the time information in the image, the shopping route of the customer in the same store can be analyzed, such as the customer enters the store, first to a snack area, then to a living goods area, and finally to a settlement area; or according to the time information in the image, cross-store analysis can be performed on the shopping paths of the customer in a plurality of different stores, for example, the customer enters the A1 store to purchase the B1 commodity, then enters the A2 store to purchase the B2 commodity, and the like.
The preference analysis related to the present embodiment includes not only the preference of purchasing goods but also stores, shopping periods, even payment methods, and the like preferred by customers. For example, the probability of a customer visiting a store may be calculated based on a shopping path analysis, and if the probability value is higher than a preset threshold, the customer may be considered to prefer the store; or the shopping path can be divided into a plurality of shopping path nodes based on the shopping path analysis, and the commodity corresponding to the shopping path node with high occurrence frequency can be regarded as the commodity preferred by the customer; or according to the time information in the image, searching the time law of shopping of the customer and determining the shopping time period preferred by the customer; the payment method (cash or electronic payment) of the customer may be determined based on the image of the payment area.
As can be seen from the above, the store people stream analysis method based on cross-border tracking provided by this embodiment not only takes the whole images in the stores through the cameras arranged at the inner sides of the stores, but also realizes people stream statistics of the stores through feature clustering, thereby greatly improving the people counting accuracy in the stores with unfixed entrances and exits; in addition, similarity analysis can be carried out on the extracted characteristics of the customers and the characteristics in the characteristic library, and cross-border tracking and preference analysis of the customers are achieved.
Example two:
fig. 3 is a schematic structural diagram illustrating a store people flow analysis system based on cross-border tracking according to an embodiment of the present invention. The system of the embodiment comprises a plurality of image acquisition units 31 and a processing unit 32, wherein the image acquisition units 31 are arranged at the inner sides of corresponding stores and are used for shooting in-store images; the processing unit 32 is communicatively connected to each image capturing unit 31 to receive the in-store images captured by the processing unit.
The image acquisition unit 31 may be a camera module, which includes a camera device, a storage device, and a processing device. The image capturing device includes but is not limited to: a camera, a video camera, an image pickup module integrated with an optical system or a CCD chip, an image pickup module integrated with an optical system and a CMOS chip, and the like, but the present embodiment is not limited thereto.
It should be noted that the specific location of the image capturing unit 31 inside the store is not limited in the present invention, and virtually any location capable of capturing the whole image inside the store may be used for installing the image capturing unit. In addition, the store according to the present embodiment has various types, including but not limited to, business places such as a store, a supermarket, a convenience store, a drug store, or a vegetable market, and the present embodiment is not limited thereto.
The processing unit 32 may be a controller, such as an arm (advanced RISC machines) controller, an fpga (field Programmable Gate array) controller, a soc (system on chip) controller, a dsp (digital signal processing) controller, or an mcu (micro controller unit) controller; the processing unit 32 may also be a Personal computer, such as a desktop computer, a notebook computer, a tablet computer, a smart phone, a smart television, a Personal Digital Assistant (PDA), etc.; the processing unit 32 may also be a server, and the server may be arranged on one or more physical servers according to various factors such as functions, loads, and the like, or may be formed by a distributed or centralized server cluster, which is not limited in this embodiment.
In particular, the processing unit 32 is used to build a plurality of feature libraries; each feature library prestores the identity features of pedestrians, pedestrian images and the position and time information corresponding to the images in the corresponding store; extracting the identity characteristics of the tracked target, and calculating the similarity between the identity characteristics of the tracked target and the identity characteristics in each feature library; and according to the similarity calculation result, screening out identity characteristics meeting the similarity requirement from the characteristic library, and outputting corresponding pedestrian image, position and time information so as to perform cross-border tracking and preference analysis on the tracked target.
The processing unit 32 and the image capturing unit 31 may be connected through WiFi communication, ZigBee communication, LoRa communication, NB-IoT communication, bluetooth communication, 3G/4G/5G cellular mobile communication, and the like, and the connection manner shown in the drawing is not limited.
It should be understood that, the embodiment of the store people stream analysis system based on cross-border tracking in this embodiment is similar to the embodiment of the store people stream analysis method based on cross-border tracking in the foregoing, and therefore, the detailed description is omitted. It should be noted that the store layout shown in fig. 3 of the present embodiment is for reference and is not a limitation of the present invention.
Example three:
fig. 4 is a schematic structural diagram of a store people flow analysis apparatus based on cross-border tracking according to an embodiment of the present invention. The apparatus of the present embodiment includes a feature library construction module 41, a feature extraction module 42, a similarity calculation module 43, and a tracking module 44.
The feature library construction module 41 is used for constructing a plurality of feature libraries; each feature library prestores the identity features of pedestrians, images of the pedestrians and the corresponding position and time information of the images in the corresponding store. The feature library construction module 41 performs the following sub-steps: acquiring in-store images of corresponding stores in a current statistical period; traversing pedestrian pictures in the in-store image and extracting pedestrian identity characteristics; clustering the identity characteristics of the pedestrians by using a clustering algorithm, and storing images of the pedestrians and position and time information corresponding to the images according to the categories; and the number of the clusters is the flow of people in the current statistical period of the store.
The feature extraction module 42 is configured to extract an identity feature of the tracking target; the process of extracting the identity features of the tracking target is as follows: inputting an image of the tracking target; and extracting the identity characteristic of the tracking target from the image of the tracking target by utilizing a pedestrian detection algorithm. It should be noted that the identity feature referred to in the present invention refers to a feature that can identify the identity of a pedestrian, such as a face feature, a gait feature, a head feature, a hair style feature, a height feature, or a leg feature, alone or in combination with other features.
The similarity calculation module 43 is configured to calculate similarities between the identity features of the tracking target and the identity features in the feature libraries; the similarity of the identity feature of the tracked target and the identity feature of the pedestrian in the feature library can be calculated by using a similarity calculation method, namely, the distance between the identity feature of the tracked target and the identity feature of the pedestrian in the feature library is calculated, the smaller the distance is, the higher the similarity is, and the larger the distance is, the lower the similarity is. The similarity calculation method may be, for example, euclidean distance method, cosine similarity method, pearson correlation coefficient method, minkowski distance method, or chebyshev distance method, and the embodiment is not limited thereto.
The tracking module 44 is configured to screen the identity features meeting the similarity requirement from the feature library according to the similarity calculation result, and output corresponding pedestrian images, positions, and time information, so as to perform cross-border tracking and preference analysis on the tracked target.
Specifically, when a shopping situation of a specific customer at another store is sought, the features are extracted from the image of the customer and then input into a feature library of another store or a plurality of stores, the similarity between the features of the customer and the features in the feature library is calculated, when the similarity is greater than a certain preset threshold value, the customer can be regarded as the same person, and finally, all images with the similarity greater than the preset threshold value and corresponding position and time information are output.
The tracking module 44 of the present embodiment can track the shopping path of the customer not only in one store, but also among a plurality of stores. The preference analysis performed by the tracking module 44 may include not only the preferences for purchasing items, but also the store, shopping period, and even payment method preferred by the customer, and so forth. For example, the probability of a customer visiting a store may be calculated based on a shopping path analysis, and if the probability value is higher than a preset threshold, the customer may be considered to prefer the store; or the shopping path can be divided into a plurality of shopping path nodes based on the shopping path analysis, and the commodity corresponding to the shopping path node with high occurrence frequency can be regarded as the commodity preferred by the customer; or according to the time information in the image, searching the time law of shopping of the customer and determining the shopping time period preferred by the customer; the payment method (cash or electronic payment) of the customer may be determined based on the image of the payment area.
It should be noted that the store people flow analysis apparatus provided in this embodiment is similar to the embodiment of the store people flow analysis method provided in the foregoing, and therefore, the description thereof is omitted. It should be noted that the division of the modules of the above apparatus is only a logical division, and the actual implementation may be wholly or partially integrated into one physical entity, or may be physically separated. And these modules can be realized in the form of software called by processing element; or may be implemented entirely in hardware; and part of the modules can be realized in the form of calling software by the processing element, and part of the modules can be realized in the form of hardware. For example, the feature extraction module may be a processing element separately set up, or may be implemented by being integrated in a chip of the apparatus, or may be stored in a memory of the apparatus in the form of program code, and a processing element of the apparatus calls and executes the functions of the feature extraction module. Other modules are implemented similarly. In addition, all or part of the modules can be integrated together or can be independently realized. The processing element described herein may be an integrated circuit having signal processing capabilities. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in the form of software.
For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or one or more microprocessors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs), among others. For another example, when one of the above modules is implemented in the form of a Processing element scheduler code, the Processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).
Example four:
fig. 5 is a schematic structural diagram of an electronic terminal according to an embodiment of the present invention. This example provides an electronic terminal, includes: a processor 51, a memory 52, a communicator 53; the memory 52 is connected with the processor 51 and the communicator 53 through a system bus and completes mutual communication, the memory 52 is used for storing computer programs, the communicator 53 is used for communicating with other devices, and the processor 51 is used for running the computer programs, so that the electronic terminal executes the steps of the store people flow analysis method based on cross-border tracking.
The above-mentioned system bus may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The system bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus. The communication interface is used for realizing communication between the database access device and other equipment (such as a client, a read-write library and a read-only library). The Memory may include a Random Access Memory (RAM), and may further include a non-volatile Memory (non-volatile Memory), such as at least one disk Memory.
The Processor may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the device can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component.
Example six:
the present embodiment provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the cross-border tracking-based store traffic analysis method.
Those of ordinary skill in the art will understand that: all or part of the steps for implementing the above method embodiments may be performed by hardware associated with a computer program. The aforementioned computer program may be stored in a computer readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
In summary, the present application provides a store people stream analysis method, apparatus, system, terminal and medium based on cross-border tracking, in which a camera is disposed inside a store to capture an overall image of the store, and a people stream statistical function is implemented by extracting characteristics of customers from the in-store image and then performing characteristic clustering. In addition, the invention also analyzes the similarity of the extracted customer characteristics and the characteristics in the characteristic library, and simultaneously realizes the functions of the store people flow statistics and the functions of analyzing the shopping preference and the shopping path of customers in the store by depending on a single camera. Therefore, the application effectively overcomes various defects in the prior art and has high industrial utilization value.
The foregoing embodiments are merely illustrative of the principles and utilities of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical spirit of the present invention be covered by the claims of the present invention.
Claims (11)
1. A store people stream analysis method based on cross-border tracking is characterized by comprising the following steps:
constructing a plurality of feature libraries; each feature library prestores the identity features of pedestrians, pedestrian images and the position and time information corresponding to the images in the corresponding store;
extracting the identity characteristics of the tracked target, and calculating the similarity between the identity characteristics of the tracked target and the identity characteristics in each feature library;
and according to the similarity calculation result, screening out identity features meeting the similarity requirement from the feature library, and outputting corresponding pedestrian image, position and time information so as to perform cross-border tracking and preference analysis on the tracked target.
2. The cross-border tracking-based store people flow analysis method as claimed in claim 1, wherein the feature library is constructed in a manner comprising:
acquiring in-store images of corresponding stores in a current statistical period;
traversing pedestrian pictures in the in-store image and extracting pedestrian identity characteristics;
clustering the identity characteristics of the pedestrians by using a clustering algorithm, and storing images of the pedestrians and position and time information corresponding to the images according to the categories; and the number of the clusters is the flow of people in the current statistical period of the store.
3. The cross-border tracking-based store people flow analysis method as claimed in claim 1, wherein the extracting the identity feature of the tracking target comprises:
inputting an image of the tracking target;
and extracting the identity characteristic of the tracking target from the image of the tracking target by utilizing a pedestrian detection algorithm.
4. The cross-border tracking-based store traffic analysis method according to claim 3, wherein the pedestrian detection algorithm comprises a motion detection-based target tracking algorithm, a machine learning-based pedestrian detection algorithm, or a deep learning-based pedestrian detection algorithm.
5. The cross-border tracking-based store people flow analysis method as claimed in claim 1, wherein cross-border tracking of the tracking target comprises: and tracking the shopping paths of the target among a plurality of shops according to the time information in the image.
6. The cross-border tracking-based store people flow analysis method as claimed in claim 1, wherein the tracking target is subjected to preference analysis, which includes analyzing any one or more combinations of store preferences, commodity preferences, shopping period preferences and payment method preferences of the tracking target.
7. An apparatus for analyzing store people flow based on cross-border tracking, comprising:
the characteristic library construction module is used for constructing a plurality of characteristic libraries; each feature library prestores the identity features of pedestrians, pedestrian images and the position and time information corresponding to the images in the corresponding store;
the characteristic extraction module is used for extracting the identity characteristic of the tracking target;
the similarity calculation module is used for calculating the similarity between the identity characteristics of the tracking target and the identity characteristics in each feature library;
and the tracking module is used for screening the identity characteristics meeting the similarity requirement from the characteristic library according to the similarity calculation result, outputting corresponding pedestrian images, positions and time information, and performing cross-border tracking and preference analysis on the tracked target.
8. The cross-border tracking-based store traffic analysis apparatus according to claim 7, wherein the feature library construction module performs the following sub-steps:
acquiring in-store images of corresponding stores in a current statistical period;
traversing pedestrian pictures in the in-store image and extracting pedestrian identity characteristics;
clustering the identity characteristics of the pedestrians by using a clustering algorithm, and storing images of the pedestrians and position and time information corresponding to the images according to the categories; and the number of the clusters is the flow of people in the current statistical period of the store.
9. A store people stream analysis system based on cross-border tracking, comprising:
the image acquisition units are respectively arranged at the inner sides of the corresponding stores and are used for shooting in-store images;
the processing unit is in communication connection with the image acquisition unit so as to receive the in-store images shot by the processing unit; the processing unit is used for constructing a plurality of feature libraries; each feature library prestores the identity features of pedestrians, pedestrian images and the position and time information corresponding to the images in the corresponding store; extracting the identity characteristics of the tracked target, and calculating the similarity between the identity characteristics of the tracked target and the identity characteristics in each feature library; and according to the similarity calculation result, screening out identity characteristics meeting the similarity requirement from the characteristic library, and outputting corresponding pedestrian image, position and time information so as to perform cross-border tracking and preference analysis on the tracked target.
10. A computer-readable storage medium on which a computer program is stored, the program, when executed by a processor, implementing the cross-border tracking-based store traffic analysis method of any one of claims 1 to 6.
11. An electronic terminal, comprising: a processor and a memory;
the memory is used for storing a computer program;
the processor is configured to execute the memory-stored computer program to cause the terminal to perform the store people flow analysis method based on cross-border tracking according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010233594.6A CN111340569A (en) | 2020-03-27 | 2020-03-27 | Store people stream analysis method, device, system, terminal and medium based on cross-border tracking |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010233594.6A CN111340569A (en) | 2020-03-27 | 2020-03-27 | Store people stream analysis method, device, system, terminal and medium based on cross-border tracking |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111340569A true CN111340569A (en) | 2020-06-26 |
Family
ID=71187397
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010233594.6A Pending CN111340569A (en) | 2020-03-27 | 2020-03-27 | Store people stream analysis method, device, system, terminal and medium based on cross-border tracking |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111340569A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111275479A (en) * | 2020-01-07 | 2020-06-12 | 北京爱笔科技有限公司 | People flow prediction method, device and system |
| CN112597879A (en) * | 2020-12-21 | 2021-04-02 | 上海商米科技集团股份有限公司 | Shop-passing passenger flow statistical method based on human head recognition |
| CN116320322A (en) * | 2023-05-12 | 2023-06-23 | 美宜佳控股有限公司 | Store cloud on-duty video push flow method, device, system, equipment and medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160259980A1 (en) * | 2015-03-03 | 2016-09-08 | Umm Al-Qura University | Systems and methodologies for performing intelligent perception based real-time counting |
| CN109190586A (en) * | 2018-09-18 | 2019-01-11 | 图普科技（广州）有限公司 | Customer's visiting analysis method, device and storage medium |
| CN109272347A (en) * | 2018-08-16 | 2019-01-25 | 苏宁易购集团股份有限公司 | A kind of statistical analysis technique and system of shops's volume of the flow of passengers |
| CN109344765A (en) * | 2018-09-28 | 2019-02-15 | 广州云从人工智能技术有限公司 | A kind of intelligent analysis method entering shop personnel analysis for chain shops |
| CN110880133A (en) * | 2018-09-05 | 2020-03-13 | 上海红星美凯龙悦家互联网科技有限公司 | Commodity information pushing method, system, storage medium and electronic equipment |
-
2020
- 2020-03-27 CN CN202010233594.6A patent/CN111340569A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160259980A1 (en) * | 2015-03-03 | 2016-09-08 | Umm Al-Qura University | Systems and methodologies for performing intelligent perception based real-time counting |
| CN109272347A (en) * | 2018-08-16 | 2019-01-25 | 苏宁易购集团股份有限公司 | A kind of statistical analysis technique and system of shops's volume of the flow of passengers |
| CN110880133A (en) * | 2018-09-05 | 2020-03-13 | 上海红星美凯龙悦家互联网科技有限公司 | Commodity information pushing method, system, storage medium and electronic equipment |
| CN109190586A (en) * | 2018-09-18 | 2019-01-11 | 图普科技（广州）有限公司 | Customer's visiting analysis method, device and storage medium |
| CN109344765A (en) * | 2018-09-28 | 2019-02-15 | 广州云从人工智能技术有限公司 | A kind of intelligent analysis method entering shop personnel analysis for chain shops |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111275479A (en) * | 2020-01-07 | 2020-06-12 | 北京爱笔科技有限公司 | People flow prediction method, device and system |
| CN111275479B (en) * | 2020-01-07 | 2023-11-10 | 北京爱笔科技有限公司 | People flow prediction method, device and system |
| CN112597879A (en) * | 2020-12-21 | 2021-04-02 | 上海商米科技集团股份有限公司 | Shop-passing passenger flow statistical method based on human head recognition |
| CN116320322A (en) * | 2023-05-12 | 2023-06-23 | 美宜佳控股有限公司 | Store cloud on-duty video push flow method, device, system, equipment and medium |
| CN116320322B (en) * | 2023-05-12 | 2023-08-22 | 美宜佳控股有限公司 | Store cloud on-duty video push flow method, device, system, equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zou et al. | Occupancy detection in the office by analyzing surveillance videos and its application to building energy conservation | |
| US9996737B2 (en) | Method and system for automatically recognizing facial expressions via algorithmic periocular localization | |
| US11335092B2 (en) | Item identification method, system and electronic device | |
| CN108629791B (en) | Pedestrian tracking method and device and cross-camera pedestrian tracking method and device | |
| Han et al. | Employing a RGB-D sensor for real-time tracking of humans across multiple re-entries in a smart environment | |
| Bedagkar-Gala et al. | A survey of approaches and trends in person re-identification | |
| CN109284733B (en) | Shopping guide negative behavior monitoring method based on yolo and multitask convolutional neural network | |
| Karasulu | Review and evaluation of well-known methods for moving object detection and tracking in videos | |
| CN111340569A (en) | Store people stream analysis method, device, system, terminal and medium based on cross-border tracking | |
| Ablavsky et al. | Layered graphical models for tracking partially occluded objects | |
| US9489566B2 (en) | Image recognition apparatus and image recognition method for identifying object | |
| Iguernaissi et al. | People tracking in multi-camera systems: a review | |
| Bouma et al. | Re-identification of persons in multi-camera surveillance under varying viewpoints and illumination | |
| Shi et al. | Smart cameras: Fundamentals and classification | |
| Linda et al. | Color-mapped contour gait image for cross-view gait recognition using deep convolutional neural network | |
| CN111738120B (en) | Character recognition method, character recognition device, electronic equipment and storage medium | |
| Allaert et al. | Optical flow techniques for facial expression analysis: Performance evaluation and improvements | |
| Xu et al. | A real-time, continuous pedestrian tracking and positioning method with multiple coordinated overhead-view cameras | |
| Oikonomopoulos et al. | Sparse B-spline polynomial descriptors for human activity recognition | |
| CN108875501B (en) | Human body attribute identification method, device, system and storage medium | |
| CN110880133A (en) | Commodity information pushing method, system, storage medium and electronic equipment | |
| Frontoni et al. | People counting in crowded environment and re-identification | |
| Miah et al. | An empirical analysis of visual features for multiple object tracking in urban scenes | |
| Luna et al. | People re-identification using depth and intensity information from an overhead camera | |
| Sicre et al. | Human behaviour analysis and event recognition at a point of sale |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200626 |