CN113723565A - Article monitoring method, device and system and storage medium - Google Patents
Article monitoring method, device and system and storage medium Download PDFInfo
- Publication number
- CN113723565A CN113723565A CN202111081822.3A CN202111081822A CN113723565A CN 113723565 A CN113723565 A CN 113723565A CN 202111081822 A CN202111081822 A CN 202111081822A CN 113723565 A CN113723565 A CN 113723565A
- Authority
- CN
- China
- Prior art keywords
- information data
- state information
- channel state
- article
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 65
- 238000012544 monitoring process Methods 0.000 title claims abstract description 60
- 238000003860 storage Methods 0.000 title claims abstract description 16
- 238000012806 monitoring device Methods 0.000 claims abstract description 30
- 238000003909 pattern recognition Methods 0.000 claims description 46
- 238000012360 testing method Methods 0.000 claims description 40
- 238000012549 training Methods 0.000 claims description 20
- 238000004140 cleaning Methods 0.000 claims description 14
- 230000003068 static effect Effects 0.000 claims description 8
- 230000009471 action Effects 0.000 claims description 7
- 238000010586 diagram Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 8
- 238000012545 processing Methods 0.000 description 8
- 238000004590 computer program Methods 0.000 description 7
- 238000000605 extraction Methods 0.000 description 7
- 238000000513 principal component analysis Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 230000006399 behavior Effects 0.000 description 5
- 238000005562 fading Methods 0.000 description 5
- 230000033001 locomotion Effects 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000009467 reduction Effects 0.000 description 3
- 238000007619 statistical method Methods 0.000 description 3
- 238000002474 experimental method Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000029058 respiratory gaseous exchange Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 230000001066 destructive effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000000556 factor analysis Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
- G06F18/2135—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on approximation criteria, e.g. principal component analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The disclosure relates to an article monitoring method, an article monitoring device, an article monitoring system and a storage medium. The article monitoring method comprises the following steps: acquiring actual channel state information data acquired by mobile hotspot equipment, wherein the mobile hotspot equipment is arranged on two sides of an article; and determining the frequency of the article taken up by the user according to the actual channel state information data. The present disclosure may conveniently monitor the frequency with which an item is picked up by a user.
Description
Technical Field
The present disclosure relates to the field of article monitoring, and in particular, to an article monitoring method, apparatus and system, and a storage medium.
Background
In a modern supermarket, for commodities on a commodity shelf of a user, the degree of attraction of the commodities to the user is difficult to count, and technical means of related technologies can only monitor or detect the commodities through equipment such as a camera and the like, but privacy is a problem, the privacy of the user is not paid more and more attention and is even invaded, and a plurality of current examples are provided, such as some house centers, people use the camera to capture faces to identify the faces and harass the user, and the behaviors are unscrupulous behaviors and illegal behaviors.
Disclosure of Invention
In view of at least one of the above technical problems, the present disclosure provides an item monitoring method, apparatus and system, and a storage medium, which can conveniently monitor how often an item is picked up by a user.
According to an aspect of the present disclosure, there is provided an item monitoring method, comprising:
acquiring actual channel state information data acquired by mobile hotspot equipment, wherein the mobile hotspot equipment is arranged on two sides of an article;
and determining the frequency of the article taken up by the user according to the actual channel state information data.
In some embodiments of the present disclosure, the item monitoring method further comprises:
acquiring a corresponding relation between test channel state information data and a state label in advance;
determining the frequency with which the item is picked up by the user based on the actual channel state information data comprises:
and determining the frequency of picking up the article by the user according to the corresponding relation between the actual channel state information data, the test channel state information data and the state label.
In some embodiments of the present disclosure, the determining, according to the correspondence between the actual channel state information data, the test channel state information data, and the state label, the frequency of picking up the article by the user includes:
training the first pattern recognition model according to the test channel state information data;
and inputting the actual channel state information data into the first mode recognition model, determining whether the current state is that the article is picked up by the user, and further determining the frequency of picking up the article by the user.
In some embodiments of the disclosure, the entering actual channel state information data into the first pattern recognition model, the determining whether the current state is that the item is picked up by the user, comprises:
inputting the actual channel state information data into the first pattern recognition model, and determining whether the current state is that the article is picked up by the user and the number of users picking up the article in the current state.
In some embodiments of the present disclosure, determining the frequency with which the article is picked up by the user according to the correspondence between the actual channel state information data, the test channel state information data, and the state label further includes:
training the second pattern recognition model according to the test channel state information data;
inputting actual channel state information data into a second mode identification model, and determining whether the current state is that the article is picked up by a user;
determining a final recognition result of the current state according to the recognition result of the first pattern recognition model and the recognition result of the second pattern recognition model;
and if the final identification result of the current state is that the article is picked up by the user, adding 1 to the counted number of times that the article is picked up by the user.
In some embodiments of the present disclosure, the item monitoring method further comprises:
carrying out data cleaning on the test channel state information data, and then training at least one model of the first pattern recognition model and the second pattern recognition model according to the test channel state information data;
and performing data cleaning on the actual channel state information data, and then inputting the actual channel state information data after the data cleaning into at least one of the first pattern recognition model and the second pattern recognition model.
In some embodiments of the disclosure, the entering actual channel state information data into the first pattern recognition model, the determining whether the current state is that the item is picked up by the user, comprises:
extracting an amplitude principal component and a phase principal component in actual channel state information data;
performing covariance operation according to the amplitude principal component, the phase principal component and the number of sliding windows, and determining a characteristic value corresponding to the amplitude and the phase;
and determining whether the current state is that the article is picked up by the user and the number of the users picking up the article in the current state according to the characteristic values of the amplitude corresponding to the phase and the maximum value and the minimum value of each sliding window.
In some embodiments of the disclosure, the entering actual channel state information data into the second pattern recognition model, the determining whether the current state is that the item is picked up by the user, comprises:
extracting an amplitude principal component and a phase principal component in training channel state information data;
extracting an amplitude principal component and a phase principal component in actual channel state information data;
taking an amplitude principal component and a phase principal component in actual channel state information data as target domain features, and taking the amplitude principal component and the phase principal component in training channel state information data of a static environment label as source domain features;
and performing heterogeneous feature enhancement based on the target domain feature and the source domain feature, and determining whether the current state is that the article is picked up by the user.
According to another aspect of the present disclosure, there is provided an item monitoring device comprising:
the data acquisition module is used for acquiring actual channel state information data acquired by the action hotspot equipment, wherein the action hotspot equipment is arranged at two sides of an article;
and the article monitoring module is used for determining the frequency of taking up the article by the user according to the actual channel state information data.
In some embodiments of the present disclosure, the item monitoring device is configured to perform operations for implementing the item monitoring method according to any of the embodiments described above.
According to another aspect of the present disclosure, there is provided an item monitoring device comprising:
a memory to store instructions;
a processor configured to execute the instructions to cause the item monitoring device to perform operations to implement the item monitoring method according to any of the embodiments described above.
According to another aspect of the present disclosure, there is provided an item monitoring system comprising a mobile hotspot device disposed on both sides of an item and an item monitoring apparatus as described in any one of the above embodiments.
According to another aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores computer instructions which, when executed by a processor, implement the item monitoring method according to any one of the above embodiments.
The present disclosure may conveniently monitor the frequency with which an item is picked up by a user.
Drawings
In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a schematic view of some embodiments of an item monitoring method of the present disclosure.
FIG. 2 is a schematic view of additional embodiments of the article monitoring methods of the present disclosure.
FIG. 3 is a schematic illustration of further embodiments of the article monitoring methods of the present disclosure.
FIG. 4 is a schematic illustration of still further embodiments of the item monitoring methods of the present disclosure.
Fig. 5 is a schematic view of some embodiments of an item monitoring device of the present disclosure.
Fig. 6 is a schematic structural view of other embodiments of the article monitoring device of the present disclosure.
Detailed Description
The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, and not all of the embodiments. The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the disclosure, its application, or uses. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.
The relative arrangement of the components and steps, the numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.
Meanwhile, it should be understood that the sizes of the respective portions shown in the drawings are not drawn in an actual proportional relationship for the convenience of description.
Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.
In all examples shown and discussed herein, any particular value should be construed as merely illustrative, and not limiting. Thus, other examples of the exemplary embodiments may have different values.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.
The inventor notices that: the attention of the related art for counting commodities is that a user buys a certain commodity or monitors the behavior of a customer in a computer vision mode, and with the improvement of laws, when face recognition even appears as an asset, a lot of legal problems occur in the monitoring mode. In the monitoring process of a supermarket, because the arrangement of the cameras is limited, the cameras cannot be arranged on each shelf, even if a customer is monitored, the attention degree of commodities in the supermarket cannot be counted, even if the customer stays for a long time, the behavior that the commodities are not bought after being picked up and watched by a user cannot be counted, the commodities are not sold, but the attention degree of the user exists, and therefore the attributes of the commodities need to be analyzed to judge whether to replace the commodities.
FIG. 1 is a schematic view of some embodiments of an item monitoring method of the present disclosure. Preferably, this embodiment may be performed by the article monitoring device of the present disclosure or the article monitoring system of the present disclosure. The method may comprise at least one of steps 11-12, wherein:
and 11, acquiring actual (Channel State Information) data acquired by Wi-Fi (network hotspot and action hotspot) equipment, wherein the action hotspot equipment is arranged on two sides of an article.
In some embodiments of the present disclosure, the items may be merchandise on a shelf.
In some embodiments of the present disclosure, Wi-Fi devices may be disposed on both sides of a shelf on which items to be monitored are to be placed.
In some embodiments of the present disclosure, the Wi-Fi device includes a transmitting end and a receiving end, and the transmitted signal is CSI.
In some embodiments of the present disclosure, Fresnel zones (Fresnel zones) are present at the transmitting and receiving ends, wherein a Fresnel Zone is a series of concentric elliptical areas of alternating enhanced and weakened intensity wave propagation due to the different phases of the wave entering and exiting different lengths of the path as it passes through an object and is partially diffracted/reflected due to the wave following multiple paths that cause constructive and destructive interference, and the propagation paths that are created when reflecting surfaces propagate along the radio waves from which the radio waves reflect may be out of phase or in phase with the signal to reach the receiver directly, depending on the relative position of the reflector with respect to the transmitter and receiver. The present disclosure can thereby capture the signal and record changes in the signal.
In some embodiments of the present disclosure, in the field of wireless communications, so-called CSI is the channel property of a communication link. It describes the fading factor of the signal on each transmission path, i.e. the value of each element in the channel gain matrix H, such as signal Scattering (Scattering), fading or fading, distance fading (power fading) and other information. The CSI may adapt the communication system to the current channel conditions, providing a guarantee for high reliability and high rate communication in a multi-antenna system.
And step 12, determining the frequency of picking up the article by the user according to the actual channel state information data.
In some embodiments of the present disclosure, the article monitoring method of the present disclosure may further comprise: and comparing the frequency of picking up the article by the user with the purchase rate of the article to track the popularity of the article.
Based on the item monitoring method of the above-mentioned embodiment of the present disclosure, the attention degree of the monitored item can be monitored by a non-invasive method. Wi-Fi based human activity awareness technology has the unique advantages of ubiquitous deployment, privacy protection, and non-invasiveness.
FIG. 2 is a schematic view of additional embodiments of the article monitoring methods of the present disclosure. Preferably, this embodiment may be performed by the article monitoring device of the present disclosure or the article monitoring system of the present disclosure. The method may comprise at least one of steps 10-12, wherein:
and step 10, acquiring the corresponding relation between the test channel state information data and the state label in advance.
And 11, acquiring actual channel state information data acquired by the mobile hotspot equipment, wherein the mobile hotspot equipment is arranged at two sides of the article.
And step 12, determining the frequency of picking up the article by the user according to the actual channel state information data.
In some embodiments of the present disclosure, step 12 may comprise: and determining the frequency of picking up the article by the user according to the corresponding relation between the actual channel state information data, the test channel state information data and the state label.
The article monitoring method based on the embodiment of the disclosure can count the attention of the articles in the supermarket and timely adjust the articles on the shelf according to the popularity of the articles.
FIG. 3 is a schematic illustration of further embodiments of the article monitoring methods of the present disclosure. Preferably, this embodiment may be performed by the article monitoring device of the present disclosure or the article monitoring system of the present disclosure. The method comprises at least one of the steps 30-37, wherein:
and step 30, acquiring test channel state information data, and acquiring the corresponding relation between the test channel state information data and the state label in advance.
In some embodiments of the present disclosure, the correspondence of the test channel state information data to the state label includes that the first test channel state information data corresponds to a signal state of the CSI in the static environment; the second test channel state information data corresponds to the state of the CSI in the case where someone should move in front of the shelf but not touch the goods; the third test channel state information data corresponds to the state of the CSI that someone is holding the article.
And step 31, performing data cleaning on the test channel state information data.
And step 32, training at least one of the first pattern recognition model and the second pattern recognition model according to the test channel state information data after data cleaning.
In some embodiments of the present disclosure, step 34 may comprise at least one of step 341-step 343, wherein:
in step 341, the amplitude principal component and the phase principal component in the actual channel state information data are extracted.
And 342, performing covariance operation according to the amplitude principal component, the phase principal component and the number of the sliding windows, and determining a characteristic value corresponding to the amplitude and the phase.
Step 343, determining whether the current state is that the article is picked up by the user and the number of users picking up the article in the current state according to the characteristic value corresponding to the amplitude and the phase, and the maximum value and the minimum value of each sliding window.
In some embodiments of the present disclosure, step 35 may comprise at least one of step 351-step 354, wherein:
step 351, extracting the amplitude principal component and the phase principal component in the training channel state information data.
In step 352, the amplitude principal component and the phase principal component in the actual channel state information data are extracted.
Step 353, using the amplitude principal component and the phase principal component in the actual channel state information data as target domain features, and using the amplitude principal component and the phase principal component in the training channel state information data of the static environment label as source domain features.
Step 354, performing heterogeneous feature enhancement based on the target domain feature and the source domain feature, and determining whether the current state is that the article is picked up by the user.
And step 36, determining the final recognition result of the current state according to the recognition result of the first pattern recognition model and the recognition result of the second pattern recognition model.
And step 37, if the final identification result of the current state is that the article is picked up by the user, adding 1 to the counted number of times that the article is picked up by the user.
According to the embodiment of the disclosure, when a customer purchases goods in a supermarket, the frequency of picking up the goods by the user can be counted in the process of staying in front of the shelf and picking up the goods, and the popularity of the goods can be tracked by comparing the purchase rate.
The embodiment of the disclosure provides a commercial Wi-Fi-based non-intrusive statistical method for commodity attention of supermarket customers, and the Wi-Fi signals are processed by using a series of signal processing and data mining technologies, so that the attention of customers to commodities can be counted by using conventional Wi-Fi equipment, and the commodities on a shelf are monitored.
FIG. 4 is a schematic illustration of still further embodiments of the item monitoring methods of the present disclosure. Preferably, this embodiment may be performed by the article monitoring device of the present disclosure or the article monitoring system of the present disclosure. The method comprises at least one of steps 41-45, wherein:
and step 41, acquiring CSI signals by utilizing commercial Wi-Fi.
The Wi-Fi equipment monitoring system is designed according to the size of a shelf and the environment, the approximate range of the monitoring range is about 1m, and the Wi-Fi equipment is installed on the shelf.
In some embodiments of the present disclosure, the Wi-Fi devices are one as the transmitting end and one as the receiving end, both devices being mounted on the shelf.
In some embodiments of the present disclosure, step 41 may comprise: and acquiring test channel state information data, and acquiring the corresponding relation between the test channel state information data and the state label in advance.
In some embodiments of the present disclosure, the step of obtaining the test channel state information data, and obtaining the corresponding relationship between the test channel state information data and the state label in advance may include: the CSI signal state under the static environment is tested, the CSI state under the condition that a person walks in front of a shelf but does not touch commodities and the CSI state under the condition that the person takes the commodities are tested, and under the general condition, the arm of the person and the Fresnel Zone form a vertical state.
In some embodiments of the present disclosure, volunteers were asked to walk through the monitoring area in the hallway of a supermarket in order to purchase goods and walk through the monitoring area within 10 s.
In some embodiments of the present disclosure, in the experiments, the data generated by volunteer walking was labeled with respective labels, mainly recording the labels of the walking volunteers, and the volunteers taking the goods beside the shelf.
In some embodiments of the present disclosure, step 42 may comprise: carrying out data cleaning on the test channel state information data; and carrying out data cleaning on the actual channel state information data.
The CSI data obtained from the experiment (test) contained: static analysis components, low-frequency interference and pulse noise are mainly caused by clock asynchronism, radio wave interference and emission power change of a transmitting terminal of Wi-Fi equipment; in addition, during data acquisition, low-frequency and high-frequency noises with different degrees can be mixed in CSI data due to small movements of surrounding people, weak actions of a subject such as breathing and the like, and changes of environmental temperature, so that difficulty in feature extraction is increased.
Therefore, the noise is effectively removed, the waveform is kept unchanged as much as possible after the noise is removed, and meanwhile, the noise cannot be effectively removed by a single filter due to the fact that the pulse noise in the CSI data is large in bandwidth and high in energy.
In some embodiments of the present disclosure, to solve these problems, the present disclosure performs data cleansing on each subcarrier in CSI data using a butterworth bandpass filter and a PCA, and may specifically include at least one of step 421 to step 423, where:
step 421, phase correction.
In some embodiments of the present disclosure, step 421 may include: since the collected phase information of the CSI data has some deviations such as CFO (Carrier Frequency Offset), SFO (Sampling Frequency Offset), and PDD (Packet detection delay), in order to reduce the influence of these errors on the final result, the present disclosure uses the method proposed in the Spotfi paper, and the least square method is used to fit the data to reduce these deviations.
Step 422, denoising each subcarrier in the CSI data by using a butterworth bandpass filter.
The speed of normal walking of the human is 1m/s, and according to the relation between CSI reflection frequency and the motion speed of the human:
wherein lambda is the wavelength of a wireless signal of 5.79cm, and the CSI fluctuation frequency generated by walking is 40 Hz. Considering the interference of other irrelevant information such as respiration and heartbeat, the cut-off frequency is set to be 5-40 Hz, and the order is set to be 5.
Step 423, using PCA (Principal Component Analysis) to perform dimensionality reduction and further denoising on the CSI data after denoising the band-pass filter.
In some embodiments of the present disclosure, since the subcarriers in the CSI data are in adjacent bands, the impact of walking on them is substantially the same, the data correlation between adjacent subcarriers is high, and the data table redundancy is large.
In order to remove data redundancy and further remove noise, the PCA is used for reducing the dimensions of all subcarriers, and the selected principal component quantities are different for different tasks (directions and people numbers).
In some embodiments of the present disclosure, step 43 may comprise: at least one of steps 431-432, wherein:
in step 431, the amplitude principal component and the phase principal component in the actual channel state information data are extracted.
In some embodiments of the present disclosure, the amplitude and phase principal components: for identification of the number of people, the extracted feature objects selected by the present disclosure are amplitude and phase information of CSI. It should be noted that after the PCA operation, the principal component is normalized, because it is obvious that the information of the magnitude of the amplitude and the phase cannot be used as the key feature of the number of discriminators, and the first principal component of the two is selected from the principal component pair.
In some embodiments of the present disclosure, the respective characteristics of phase and amplitude can be used to detect whether the human body moves, and there is also a complementary relationship between amplitude and phase.
In some embodiments of the present disclosure, step 432 may comprise: and performing covariance operation according to the amplitude principal component, the phase principal component and the number of sliding windows, and determining a characteristic value corresponding to the amplitude and the phase.
In some embodiments of the present disclosure, step 432 may comprise: when a human body successively enters and exits, the key points for extracting the features can be deviated to the amplitude waveform, and when the human body enters and exits in parallel, because signals need to penetrate through the drains of the two human bodies, the consumed time is longer, and the display on the phase is more obvious. The extraction features are shown in formula (1):
in equation (1), cov (·) is a covariance operation, and | | H | | ceiling1And
are respectively an amplitude and phase principal component, NslideThe number of sliding windows. In order to compress and extract information, the present disclosure adopts formula (2) to solve a characteristic value:
in equation (2), Eig is the function for calculating the eigenvalues and eigenvectors of matrix A is eig (A), μi,viThe characteristic values are corresponding to amplitude and phase respectively, the larger the characteristic value is, the closer the body trunk is to the LOS (straight line distance between the transmitting end and the receiving end), and the smaller the characteristic value is, the farther the body trunk is from the LOS. Then the maximum value and the minimum value of each sliding window are added as final characteristics and input into a training model, and the model training method adopts an algorithm in HFA to finally identify the number of people through the training model.
In some embodiments of the present disclosure, step 44 may include at least one of step 441-step 443, wherein:
In some embodiments of the present disclosure, step 441 may comprise: extracting an amplitude principal component and a phase principal component in training channel state information data; amplitude principal components and phase principal components in the actual channel state information data are extracted.
In some embodiments of the present disclosure, step 45 may include at least one of steps 451-452, wherein:
step 451, determining a final recognition result of the current state according to the recognition result of the first pattern recognition model and the recognition result of the second pattern recognition model.
In step 452, if the final recognition result of the current status is that the article is picked up by the user, the counted number of times that the article is picked up by the user is increased by 1.
The above embodiments of the present disclosure collect CSI data including human body movement information, and the device is mounted on a shelf and includes a transmitting terminal and a receiving terminal. Because the transmitting end and the receiving end have Fresnel zones, the signals can be captured and the changes of the signals can be recorded, but the phase of the received CSI data has deviation, and the phase information of the CSI data is corrected by adopting a Spotifi method. The embodiment of the disclosure utilizes a Butterworth band-pass filter to perform denoising processing on the data of the CSI; and by adopting principal component analysis, the dimension reduction algorithm removes data redundancy to further reduce noise, and can pick out appropriate data to provide an object for subsequent feature extraction. The embodiment of the disclosure normalizes the amplitude and phase information selected from the principal components, then performs feature extraction, fuses the features of the source domain and the target domain by using an HFA (Hidden Factor Analysis) method, and then trains the trained model to distinguish whether the signal fluctuation is caused by the walking of a customer on a shelf or the fluctuation of signal data caused by picking up a commodity, so as to count the attention of the commodity.
According to the embodiment of the disclosure, the attention degree of the goods on the shelf of the supermarket can be identified by using the conventional Wi-Fi device, the wireless device is installed on the shelf, and due to the fact that interference caused by walking exists in the corridor, the information of the part which causes signal fluctuation due to walking is required to be eliminated, and the part which is required by the disclosure is identified, so that the practicability is enhanced.
The embodiment of the present disclosure can monitor the use condition of the commodity on the goods shelf in real time, and can timely replenish the commodity, and also can protect the privacy of the customer, thereby avoiding the leakage of the customer information brought by the camera.
The characteristics used in the statistical process of the embodiment of the disclosure integrate the characteristics and the relation of the amplitude and the phase, and the relationship between the commodity comparison process of a quantitative customer and the walking process and the signal transformation is realized.
The above embodiments of the present disclosure are based on commercially available Wi-Fi equipment, which is inexpensive and does not require large investments in capital. The Wi-Fi equipment is widely deployed, is simple and easy to install, generates a data level much smaller than that of other modes, does not need to add any additional hardware, saves cost, and compared with wearable equipment, the Wi-Fi-based crowd counting method is non-invasive, does not require testers to wear any equipment, and compared with a computer-vision-direction-based statistical method, the Wi-Fi-based crowd counting method is not limited by light and shade, and is less prone to invading privacy of prisoners.
Fig. 5 is a schematic view of some embodiments of an item monitoring device of the present disclosure. As shown in fig. 5, the item monitoring device of the present disclosure may include a data acquisition module 51 and an item monitoring module 52, wherein:
the data obtaining module 51 is configured to obtain actual channel state information data collected by the mobile hotspot device, where the mobile hotspot device is disposed on two sides of an article.
And the article monitoring module 52 is configured to determine, according to the actual channel state information data, how often an article is picked up by the user.
In some embodiments of the present disclosure, the data obtaining module 51 may be configured to obtain a corresponding relationship between the test channel state information data and the state label in advance; the article monitoring module 52 may be configured to determine the frequency with which an article is picked up by a user according to the corresponding relationship between the actual channel state information data, the test channel state information data, and the state tag.
In some embodiments of the present disclosure, the item monitoring device may be configured to train the first pattern recognition model according to the test channel state information data; and inputting the actual channel state information data into the first mode recognition model, determining whether the current state is that the article is picked up by the user, and further determining the frequency of picking up the article by the user.
In some embodiments of the present disclosure, the item monitoring device may be configured to input actual channel state information data into the first pattern recognition model, in the event that the actual channel state information data is input into the first pattern recognition model to determine whether the current state is that the item is picked up by the user, and the number of users picking up the item in the current state.
In some embodiments of the present disclosure, the article monitoring device may be configured to train the second pattern recognition model according to the test channel state information data, when determining the frequency with which the article is picked up by the user according to the correspondence between the actual channel state information data, the test channel state information data, and the state label; inputting actual channel state information data into a second mode identification model, and determining whether the current state is that the article is picked up by a user; determining a final recognition result of the current state according to the recognition result of the first pattern recognition model and the recognition result of the second pattern recognition model; and if the final identification result of the current state is that the article is picked up by the user, adding 1 to the counted number of times that the article is picked up by the user.
In some embodiments of the present disclosure, the item monitoring device may be further configured to perform data cleaning on the test channel state information data, and then train at least one of the first pattern recognition model and the second pattern recognition model according to the test channel state information data; and performing data cleaning on the actual channel state information data, and then inputting the actual channel state information data after the data cleaning into at least one of the first pattern recognition model and the second pattern recognition model.
In some embodiments of the present disclosure, the article monitoring device may be configured to extract an amplitude principal component and a phase principal component from the actual channel state information data when the actual channel state information data is input into the first pattern recognition model to determine whether the current state is that the article is picked up by the user; performing covariance operation according to the amplitude principal component, the phase principal component and the number of sliding windows, and determining a characteristic value corresponding to the amplitude and the phase; and determining whether the current state is that the article is picked up by the user and the number of the users picking up the article in the current state according to the characteristic values of the amplitude corresponding to the phase and the maximum value and the minimum value of each sliding window.
In some embodiments of the present disclosure, the article monitoring device may be configured to extract the amplitude principal component and the phase principal component from the training channel state information data when inputting the actual channel state information data into the second pattern recognition model to determine whether the current state is that the article is picked up by the user; extracting an amplitude principal component and a phase principal component in actual channel state information data; taking an amplitude principal component and a phase principal component in actual channel state information data as target domain features, and taking the amplitude principal component and the phase principal component in training channel state information data of a static environment label as source domain features; and performing heterogeneous feature enhancement based on the target domain feature and the source domain feature, and determining whether the current state is that the article is picked up by the user.
In some embodiments of the present disclosure, the item monitoring device is configured to perform operations for implementing an item monitoring method as described in any of the above embodiments (e.g., any of fig. 1-4).
Fig. 6 is a schematic structural view of other embodiments of the article monitoring device of the present disclosure. As shown in fig. 6, the item monitoring device includes a memory 61 and a processor 62.
The memory 61 is used for storing instructions, the processor 62 is coupled to the memory 61, and the processor 62 is configured to execute the item monitoring method according to any of the embodiments (for example, any of fig. 1 to 4) based on the instructions stored in the memory.
As shown in fig. 6, the item monitoring device further includes a communication interface 63 for information interaction with other devices. Meanwhile, the article monitoring device further comprises a bus 64, and the processor 62, the communication interface 63 and the memory 61 are communicated with each other through the bus 64.
The memory 61 may comprise a high-speed RAM memory, and may further comprise a non-volatile memory (e.g., at least one disk memory). The memory 61 may also be a memory array. The storage 61 may also be partitioned and the blocks may be combined into virtual volumes according to certain rules.
Further, the processor 62 may be a central processing unit CPU, or may be an application specific integrated circuit ASIC, or one or more integrated circuits configured to implement embodiments of the present disclosure.
According to another aspect of the present disclosure, there is provided an item monitoring system comprising a mobile hotspot device disposed on both sides of an item and an item monitoring apparatus as described in any one of the embodiments above (e.g., the embodiments of fig. 5 or 6).
According to the embodiment of the disclosure, the attention degree of the goods on the shelf of the supermarket can be identified by using the conventional Wi-Fi device, the wireless device is installed on the shelf, and due to the fact that interference caused by walking exists in the corridor, the information of the part which causes signal fluctuation due to walking is required to be eliminated, and the part which is required by the disclosure is identified, so that the practicability is enhanced.
The embodiment of the present disclosure can monitor the use condition of the commodity on the goods shelf in real time, and can timely replenish the commodity, and also can protect the privacy of the customer, thereby avoiding the leakage of the customer information brought by the camera.
The characteristics used in the statistical process of the embodiment of the disclosure integrate the characteristics and the relation of the amplitude and the phase, and the relationship between the commodity comparison process of a quantitative customer and the walking process and the signal transformation is realized.
The above embodiments of the present disclosure are based on commercially available Wi-Fi equipment, which is inexpensive and does not require large investments in capital. The Wi-Fi equipment is widely deployed, is simple and easy to install, generates a data level much smaller than that of other modes, does not need to add any additional hardware, saves cost, and compared with wearable equipment, the Wi-Fi-based crowd counting method is non-invasive, does not require testers to wear any equipment, and compared with a computer-vision-direction-based statistical method, the Wi-Fi-based crowd counting method is not limited by light and shade, and is less prone to invading privacy of prisoners.
According to another aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores computer instructions, which when executed by a processor, implement the item monitoring method according to any one of the embodiments (for example, any one of fig. 1-4) above.
The above embodiments of the present disclosure collect CSI data including human body movement information, and the device is mounted on a shelf and includes a transmitting terminal and a receiving terminal. Because the transmitting end and the receiving end have Fresnel zones, the signals can be captured and the changes of the signals can be recorded, but the phase of the received CSI data has deviation, and the phase information of the CSI data is corrected by adopting a Spotifi method. The embodiment of the disclosure utilizes a Butterworth band-pass filter to perform denoising processing on the data of the CSI; and by adopting principal component analysis, the dimension reduction algorithm removes data redundancy to further reduce noise, and can pick out appropriate data to provide an object for subsequent feature extraction. The embodiment of the disclosure normalizes the amplitude and phase information selected from the principal components, then performs feature extraction, fuses the features of the source domain and the target domain by using an HFA method, and then trains the trained model to distinguish whether the signal fluctuation is caused by the walking of a customer on a shelf or the signal data fluctuation caused by picking up a commodity, so as to count the attention of the commodity.
As will be appreciated by one skilled in the art, embodiments of the present disclosure may be provided as a method, apparatus, or computer program product. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product embodied on one or more computer-usable non-transitory storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present disclosure is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The item monitoring device described above may be implemented as a general purpose processor, a Programmable Logic Controller (PLC), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any suitable combination thereof, for performing the functions described herein.
Thus far, the present disclosure has been described in detail. Some details that are well known in the art have not been described in order to avoid obscuring the concepts of the present disclosure. It will be fully apparent to those skilled in the art from the foregoing description how to practice the presently disclosed embodiments.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware to implement the above embodiments, where the program may be stored in a non-transitory computer readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic or optical disk, and the like.
The description of the present disclosure has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the disclosure in the form disclosed. Many modifications and variations will be apparent to practitioners skilled in this art. The embodiment was chosen and described in order to best explain the principles of the disclosure and the practical application, and to enable others of ordinary skill in the art to understand the disclosure for various embodiments with various modifications as are suited to the particular use contemplated.
Claims (13)
1. An item monitoring method, comprising:
acquiring actual channel state information data acquired by mobile hotspot equipment, wherein the mobile hotspot equipment is arranged on two sides of an article;
and determining the frequency of the article taken up by the user according to the actual channel state information data.
2. The item monitoring method of claim 1, further comprising:
acquiring a corresponding relation between test channel state information data and a state label in advance;
determining the frequency with which the item is picked up by the user based on the actual channel state information data comprises:
and determining the frequency of picking up the article by the user according to the corresponding relation between the actual channel state information data, the test channel state information data and the state label.
3. The item monitoring method of claim 2, wherein determining how often an item is picked up by a user based on the correspondence of actual channel state information data, test channel state information data, and status tags comprises:
training the first pattern recognition model according to the test channel state information data;
and inputting the actual channel state information data into the first mode recognition model, determining whether the current state is that the article is picked up by the user, and further determining the frequency of picking up the article by the user.
4. The item monitoring method of claim 3, wherein inputting actual channel state information data into the first pattern recognition model, and wherein determining whether the current state is an item picked up by the user comprises:
inputting the actual channel state information data into the first pattern recognition model, and determining whether the current state is that the article is picked up by the user and the number of users picking up the article in the current state.
5. The item monitoring method of claim 3, wherein determining how often an item is picked up by a user based on the correspondence of the actual channel state information data, the test channel state information data, and the status label further comprises:
training the second pattern recognition model according to the test channel state information data;
inputting actual channel state information data into a second mode identification model, and determining whether the current state is that the article is picked up by a user;
determining a final recognition result of the current state according to the recognition result of the first pattern recognition model and the recognition result of the second pattern recognition model;
and if the final identification result of the current state is that the article is picked up by the user, adding 1 to the counted number of times that the article is picked up by the user.
6. The item monitoring method of any one of claims 3-5, further comprising:
carrying out data cleaning on the test channel state information data, and then training at least one model of the first pattern recognition model and the second pattern recognition model according to the test channel state information data;
and performing data cleaning on the actual channel state information data, and then inputting the actual channel state information data after the data cleaning into at least one of the first pattern recognition model and the second pattern recognition model.
7. The item monitoring method of any of claims 3-5, said entering actual channel state information data into a first pattern recognition model, determining whether the current state is that the item is picked up by a user comprising:
extracting an amplitude principal component and a phase principal component in actual channel state information data;
performing covariance operation according to the amplitude principal component, the phase principal component and the number of sliding windows, and determining a characteristic value corresponding to the amplitude and the phase;
and determining whether the current state is that the article is picked up by the user and the number of the users picking up the article in the current state according to the characteristic values of the amplitude corresponding to the phase and the maximum value and the minimum value of each sliding window.
8. The item monitoring method of claim 5, said entering actual channel state information data into a second pattern recognition model, determining whether the current state is that the item is picked up by the user comprising:
extracting an amplitude principal component and a phase principal component in training channel state information data;
extracting an amplitude principal component and a phase principal component in actual channel state information data;
taking an amplitude principal component and a phase principal component in actual channel state information data as target domain features, and taking the amplitude principal component and the phase principal component in training channel state information data of a static environment label as source domain features;
and performing heterogeneous feature enhancement based on the target domain feature and the source domain feature, and determining whether the current state is that the article is picked up by the user.
9. An item monitoring device, comprising:
the data acquisition module is used for acquiring actual channel state information data acquired by the action hotspot equipment, wherein the action hotspot equipment is arranged at two sides of an article;
and the article monitoring module is used for determining the frequency of taking up the article by the user according to the actual channel state information data.
10. The item monitoring device according to claim 9, wherein the item monitoring device is configured to perform operations to implement the item monitoring method of any one of claims 1-8.
11. An item monitoring device, comprising:
a memory to store instructions;
a processor configured to execute the instructions to cause the item monitoring device to perform operations to implement the item monitoring method of any one of claims 1-8.
12. An item monitoring system comprising a mobile hotspot device disposed on both sides of an item and an item monitoring apparatus as claimed in any one of claims 9 to 11.
13. A non-transitory computer readable storage medium storing computer instructions which, when executed by a processor, implement the item monitoring method of any one of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111081822.3A CN113723565A (en) | 2021-09-15 | 2021-09-15 | Article monitoring method, device and system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111081822.3A CN113723565A (en) | 2021-09-15 | 2021-09-15 | Article monitoring method, device and system and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113723565A true CN113723565A (en) | 2021-11-30 |
Family
ID=78684239
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111081822.3A Pending CN113723565A (en) | 2021-09-15 | 2021-09-15 | Article monitoring method, device and system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113723565A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007088585A1 (en) * | 2006-01-31 | 2007-08-09 | Fujitsu Limited | Information collecting system |
| CN105512911A (en) * | 2015-12-08 | 2016-04-20 | 陶娜 | Store products sales monitoring device |
| CN107958397A (en) * | 2017-12-27 | 2018-04-24 | 成都信息工程大学 | The system that a kind of condition of merchandise using short-distance wireless communication technology detects |
| CN108734512A (en) * | 2018-05-03 | 2018-11-02 | 北京星网锐捷网络技术有限公司 | A kind of article attention rate processing method, equipment, system and storage medium |
| CN109065036A (en) * | 2018-08-30 | 2018-12-21 | 出门问问信息科技有限公司 | Method, apparatus, electronic equipment and the computer readable storage medium of speech recognition |
| CN112364769A (en) * | 2020-11-11 | 2021-02-12 | 天津大学 | Crowd counting method based on commercial Wi-Fi |
| CN112381063A (en) * | 2020-12-02 | 2021-02-19 | 苏州科技大学 | Channel state information-based people counting method |
-
2021
- 2021-09-15 CN CN202111081822.3A patent/CN113723565A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007088585A1 (en) * | 2006-01-31 | 2007-08-09 | Fujitsu Limited | Information collecting system |
| CN105512911A (en) * | 2015-12-08 | 2016-04-20 | 陶娜 | Store products sales monitoring device |
| CN107958397A (en) * | 2017-12-27 | 2018-04-24 | 成都信息工程大学 | The system that a kind of condition of merchandise using short-distance wireless communication technology detects |
| CN108734512A (en) * | 2018-05-03 | 2018-11-02 | 北京星网锐捷网络技术有限公司 | A kind of article attention rate processing method, equipment, system and storage medium |
| CN109065036A (en) * | 2018-08-30 | 2018-12-21 | 出门问问信息科技有限公司 | Method, apparatus, electronic equipment and the computer readable storage medium of speech recognition |
| CN112364769A (en) * | 2020-11-11 | 2021-02-12 | 天津大学 | Crowd counting method based on commercial Wi-Fi |
| CN112381063A (en) * | 2020-12-02 | 2021-02-19 | 苏州科技大学 | Channel state information-based people counting method |
Non-Patent Citations (1)
| Title |
|---|
| 张六;曾正;陈俊昌;杨晶晶;黄铭;: "基于信道状态信息的室内人数统计方法", 计算机工程与设计, no. 04, 16 April 2020 (2020-04-16) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20190175074A1 (en) | Fall detection method and system | |
| Palipana et al. | FallDeFi: Ubiquitous fall detection using commodity Wi-Fi devices | |
| Wang et al. | Respiration tracking for people counting and recognition | |
| Wang et al. | Gait recognition using wifi signals | |
| Li et al. | IDSense: A human object interaction detection system based on passive UHF RFID | |
| US10410048B2 (en) | System and method for detecting, tracking and counting human objects of interest using a counting system and a data capture device | |
| Zhang et al. | Anti-fall: A non-intrusive and real-time fall detector leveraging CSI from commodity WiFi devices | |
| US9805233B2 (en) | Systems and methods for following distinguishing members using tags in combination with a localizing device | |
| Yamato et al. | Security camera movie and ERP data matching system to prevent theft | |
| US10936859B2 (en) | Techniques for automatically identifying secondary objects in a stereo-optical counting system | |
| CA2849016C (en) | System and method for detecting, tracking and counting human objects of interest using a counting system and a data capture device | |
| CN112364769B (en) | Commercial Wi-Fi-based crowd counting method | |
| Feng et al. | Mais: Multiple activity identification system using channel state information of wifi signals | |
| Cao et al. | Contactless body movement recognition during sleep via WiFi signals | |
| CN102881100A (en) | Video-analysis-based antitheft monitoring method for physical store | |
| CN109658194A (en) | A kind of lead referral method and system based on video frequency tracking | |
| US11315294B2 (en) | Information processing device, display method, and program storage medium for monitoring object movement | |
| JP7075460B2 (en) | Information recognition system and its method | |
| Xin et al. | FreeSense: human-behavior understanding using Wi-Fi signals | |
| Bocanegra et al. | RFGo: a seamless self-checkout system for apparel stores using RFID | |
| Liu et al. | Physical-world attack towards WiFi-based behavior recognition | |
| CN110290584B (en) | Positioning method | |
| CN113723565A (en) | Article monitoring method, device and system and storage medium | |
| Xu et al. | PresSense: Passive respiration sensing via ambient WiFi signals in noisy environments | |
| Wan et al. | Capturing customer browsing insights through RFID tag motion detection in high tag density environments |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |