CN112509011B - Static commodity statistical method, terminal equipment and storage medium thereof - Google Patents

Static commodity statistical method, terminal equipment and storage medium thereof Download PDF

Info

Publication number
CN112509011B
CN112509011B CN202110169851.9A CN202110169851A CN112509011B CN 112509011 B CN112509011 B CN 112509011B CN 202110169851 A CN202110169851 A CN 202110169851A CN 112509011 B CN112509011 B CN 112509011B
Authority
CN
China
Prior art keywords
target
tracking
detection
list
tracking target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110169851.9A
Other languages
Chinese (zh)
Other versions
CN112509011A (en
Inventor
丁明
李海荣
陈永辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Xuanwu Wireless Technology Co Ltd
Original Assignee
Guangzhou Xuanwu Wireless Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Xuanwu Wireless Technology Co Ltd filed Critical Guangzhou Xuanwu Wireless Technology Co Ltd
Priority to CN202110169851.9A priority Critical patent/CN112509011B/en
Publication of CN112509011A publication Critical patent/CN112509011A/en
Application granted granted Critical
Publication of CN112509011B publication Critical patent/CN112509011B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/254—Fusion techniques of classification results, e.g. of results related to same input data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames

Abstract

The invention discloses a static commodity statistical method, which comprises the following steps: performing target detection and feature extraction on video data according to a target detector to obtain a detection target list and a tracking target list, and calculating a presumed position of a tracking target according to the tracking target list; carrying out weight optimal matching according to the detection target list and the tracking target list; if the detection target is matched with the tracking target, updating the tracking target list, if the detection target is not matched with the tracking target, establishing new tracking information for the detection target, adding the new tracking information into the tracking target list, and updating the current position for the tracking target; and deleting the tracking targets matched and identified for many times in the tracking list, and counting the types and the number of the tracking targets appearing in the tracking process after the video data is processed. Therefore, by dynamic tracking and identification of videos, the types and the number of commodities can be accurately checked in real time, and the positions of the commodities can be confirmed.

Description

Static commodity statistical method, terminal equipment and storage medium thereof
Technical Field
The invention relates to the technical field of image recognition, in particular to a static commodity statistical method, terminal equipment and a storage medium thereof.
Background
With the development of retail industry, more and more commodities are sold through off-line stores, supermarkets, markets, large stores and the like, and commodity shelves are often adopted for displaying and displaying the commodities for the convenience of customers to browse or purchase the commodities. Therefore, the detection, identification and number statistics of the static commodity position targets on the shelf display have wide application value, for example, a salesperson needs to count various commodities when laying the commodities so as to know the shortage situation and the commodity arrangement position situation of each commodity.
Most of the commonly used scene target detection and number statistics are carried out based on multi-picture splicing, the commodity display state cannot be tracked in real time, and then the number of commodities is effectively counted.
Disclosure of Invention
The invention aims to provide a static commodity counting method, which can accurately count the types and the number of commodities and confirm the positions of the commodities in real time.
In order to achieve the above object, an embodiment of the present invention provides a static commodity statistics method, including:
performing target detection and feature extraction on video data according to a target detector to obtain a detection target list and a tracking target list, wherein the detection target list comprises a detection frame, a classification category and feature expression;
calculating a speed vector of the movement of the tracking target according to the tracking target list, and calculating the presumed position of the tracking target according to the speed vector;
calculating the feature similarity between a detection target and the tracking target according to the detection target list and the tracking target list, and performing weight optimal matching according to the feature similarity;
if the detection target is matched with the tracking target, updating the tracking target list according to the detection frame and the feature expression of the current detection target, if the detection target is not matched with the tracking target, establishing new tracking information for the detection target, adding the new tracking information into the tracking target list, and updating the current position for the tracking target;
and deleting the tracking targets matched and identified for many times in the tracking list, and counting the types and the number of the tracking targets appearing in the tracking process after the video data is processed.
In one embodiment, the method further comprises training a target detection model according to a target detection algorithm and a data set of the commodity to obtain the target detector.
In one embodiment, the target detection algorithm includes Faster Rcnn, SSD, and RefineDet.
In one embodiment, the calculating a velocity vector of the movement of the tracking target according to the tracking target list and the calculating the estimated position of the tracking target according to the velocity vector specifically include:
calculating the velocity vector of the motion of the tracking target, wherein the calculation formula is as follows:
Figure 115206DEST_PATH_IMAGE001
wherein, S [0]]For the speed of movement of the tracked object in the X-axis direction, S1]For the velocity of the tracked target moving in the Y-axis direction, (x)1,y1) Previous target position, n, updated in real time for the tracked target1For its corresponding video frame number, (x)2,y2) First and second target positions, n, updated in real time for the tracked target2The corresponding video frame number is;
calculating the tracked target current inference position P (x)0,y0) The formula is as follows:
Figure 645544DEST_PATH_IMAGE002
the next frame position of each target in the tracking target list is marked as the current presumed position (x)0,y0)。
In one embodiment, the position of the tracking target is the central point of a target rectangular frame, and the width and height of the rectangular frame are kept unchanged.
In one embodiment, the calculating a feature similarity between the detection target and the tracking target according to the detection target list and the tracking target list, and performing weight optimal matching according to the feature similarity specifically includes:
calculating the characteristic cosine distance between the tracking target and the detection target, wherein the formula is as follows:
Figure DEST_PATH_IMAGE003
wherein d isnFor the nth detected target in the detected target list, tmFor the mth tracking target in the tracking target list, A is dnB is tmA, B are all one-dimensional vector features;
calculating the distance similarity between the tracking target and the detection target, wherein the formula is as follows:
Figure 523370DEST_PATH_IMAGE004
Figure DEST_PATH_IMAGE005
representing an object dnAnd tmValue of smaller area, p1Is dnTarget rectangular position, p2Is tmThe position of the rectangle is inferred and,
Figure 404739DEST_PATH_IMAGE006
representing a rectangle intersection area value;
calculating the similarity of fusion characteristics matching the tracking target and the detection target, wherein the formula is as follows:
Figure DEST_PATH_IMAGE007
wherein k is an empirical value of 0.4;
and calculating the optimal matching of the weight according to the fusion feature similarity and the optimal matching algorithm.
In one embodiment, the best match algorithm comprises a KM algorithm.
In a certain embodiment, the updating the current position for the tracking target specifically includes:
searching a target set in which the previous frame and the current frame in the tracking target list are successfully matched;
finding a target with the center of the target of the previous frame closest to the tracking target in the target set, and calculating the relative position of the target and the tracking target, wherein the formula is as follows:
Figure 26213DEST_PATH_IMAGE008
wherein M ismDetecting a frame position, P, for a previous frame of the tracked objectnDetecting the position of a frame in the last frame of the target;
updating the current position of the tracking target, wherein the formula is as follows:
Figure DEST_PATH_IMAGE009
wherein Q ismIs the current position of the target.
The embodiment of the invention also provides a static commodity counting device, which is applied to the static commodity counting method in any one of the embodiments. The method comprises the following steps:
the video data extraction module is used for carrying out target detection and feature extraction on video data according to a target detector to obtain a detection target list and a tracking target list, wherein the detection target list comprises a detection frame, classification categories and feature expressions;
the position presumption module is used for calculating a speed vector of the movement of the tracking target according to the tracking target list and calculating a presumed position of the tracking target according to the speed vector;
the optimal weight matching module is used for calculating the feature similarity between the detection target and the tracking target according to the detection target list and the tracking target list and performing optimal weight matching according to the feature similarity;
an information updating module, configured to update the tracking target list according to the detection frame and the feature expression of the current detection target if the detection target is matched with the tracking target, and establish new tracking information for the detection target and add the new tracking information to the tracking target list if the detection target is not matched with the tracking target, and update the current position for the tracking target;
and the counting module is used for deleting the tracking targets which are matched and identified for many times in the tracking list, and counting the types and the number of the tracking targets appearing in the tracking process after the video data is processed.
The embodiment of the invention also provides computer terminal equipment which comprises one or more processors and a memory. A memory coupled to the processor for storing one or more programs; when executed by the one or more processors, cause the one or more processors to implement a static merchandise statistics method as described in any of the embodiments above.
An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements a static commodity statistics method according to any one of the above embodiments.
In the static commodity statistical method provided by the embodiment of the invention, the types and the number of commodities are accurately counted in real time and the positions of the commodities are confirmed through video dynamic tracking and identification. Due to the fact that the texture feature and distance similarity fusion judgment rule is used, the target tracking effect is better for targets with similar or identical appearances, and the speed is higher than that of a common deep learning tracking method. Still can track to the static target leaving the video picture, judge the target classification with the current multiframe, fault-tolerant rate is high to the classifier classification; and the number of the statistical targets of the single picture in comparison shooting is superior in that the shot scene is large, the shooting is not influenced by a single angle, the comparison splicing recognition shooting freedom degree is high, and the recognition accuracy rate is high.
Drawings
In order to more clearly illustrate the technical solution of the present invention, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flow chart of a static merchandise statistics method according to an embodiment of the invention;
FIG. 2 is a flow chart illustrating a static merchandise statistics method according to an embodiment of the invention;
fig. 3 is a schematic structural diagram of a computer terminal device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be understood that the step numbers used herein are for convenience of description only and are not intended as limitations on the order in which the steps are performed.
It is to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification of the present invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
The terms "comprises" and "comprising" indicate the presence of the described features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
The term "and/or" refers to and includes any and all possible combinations of one or more of the associated listed items.
Referring to fig. 1, an embodiment of the invention provides a static merchandise statistics method, including:
s10, carrying out target detection and feature extraction on the video data according to a target detector to obtain a detection target list and a tracking target list, wherein the detection target list comprises a detection frame, a classification category and a feature expression;
s20, calculating a speed vector of the movement of the tracking target according to the tracking target list, and calculating the presumed position of the tracking target according to the speed vector;
s30, calculating the feature similarity between the detection target and the tracking target according to the detection target list and the tracking target list, and performing weight optimal matching according to the feature similarity;
s40, if the detection target is matched with the tracking target, updating the tracking target list according to the detection frame and the feature expression of the current detection target, if the detection target is not matched with the tracking target, establishing new tracking information for the detection target, adding the new tracking information into the tracking target list, and updating the current position for the tracking target;
and S50, deleting the tracking targets matched and identified for many times in the tracking list, and counting the types and the number of the tracking targets appearing in the tracking process after the video data is processed.
Referring to fig. 2, in the present embodiment, first, stationary object detection and feature extraction are performed by using a detector. The detector is typically implemented using a deep learning model, such as fast Rcnn, SSD, RefineDet, etc. detection models. In the tracking process, besides detecting the position and the identification category of the target, the detector also needs to extract the feature expression layer data of the detection model for matching the target. The preparation of a typical detector requires the following steps:
(1) data acquisition and detection model training of the detection and identification commodities;
(2) and carrying out target detection and feature extraction on each video frame.
Secondly, predicting a tracking target position, and calculating the target position of the current frame by using the tracking historical target position:
(1) calculating a velocity vector of the movement of the target by using the historical position of the tracking target;
(2) and calculating the presumed position of the current tracking target by using the calculated velocity vector and the position of the previous frame.
Then, calculating the feature similarity between the detection target and the tracking target:
(1) calculating the guessed position of the tracking target and the IOU of the current frame detection target pairwise to obtain distance similarity measurement;
(2) calculating the characteristics of a current frame detection target and the cosine distance of a tracking target to obtain appearance similarity measurement;
(3) linearly fusing the two similarity measurement values to obtain a final similarity matching score;
(4) and carrying out target matching by using an optimal weight matching algorithm.
Further, an update of the tracking list is performed:
(1) the location of the matching target in the tracking list is updated. Updating the matched tracking target by using the position and the characteristic value of the current detection target;
(2) and updating the positions of the commodities which are not matched and are included in the departure picture. And tracking on the unmatched position by using the relative position of the last frame and the position of the current existing target to follow the new position.
And finally, deleting the tracking target which is subjected to multiple matching identification, if the identification frequency of the tracking target which is not matched is more than 30 times, deleting the target, and if the identification frequency of the tracking target which is not matched is less than 30 times, updating the position of the tracking target which is not matched. And continuously tracking until the video processing is finished, and counting the types and the number of the tracking targets in the tracking process.
In one embodiment, the method further comprises training a target detection model according to a target detection algorithm and a data set of the commodity to obtain the target detector.
In the embodiment, training data of the target to be detected is collected, a target detector is trained, the detector generally uses a deep learning model, and the precision is high. And acquiring target data to be classified by using RefineDet, training a classifier, detecting by using SENet, and outputting the data F and the class data of the feature expression layer by using a classification network in the reasoning process. And firstly, detecting and classifying the target to be tracked by using a detector for each frame to obtain a detection target list D which comprises a detection frame, a classification category and feature expression F information of the target.
In one embodiment, the target detection algorithm includes Faster Rcnn, SSD, and RefineDet.
In one embodiment, the calculating a velocity vector of the movement of the tracking target according to the tracking target list and the calculating the estimated position of the tracking target according to the velocity vector specifically include:
calculating the velocity vector of the motion of the tracking target, wherein the calculation formula is as follows:
Figure 676637DEST_PATH_IMAGE001
wherein, S [0]]For the speed of movement of the tracked object in the X-axis direction, S1]For the velocity of the tracked target moving in the Y-axis direction, (x)1,y1) Previous target position, n, updated in real time for the tracked target1For its corresponding video frame number, (x)2,y2) First and second target positions, n, updated in real time for the tracked target2The corresponding video frame number is;
calculating the tracked target current inference position P (x)0,y0) The formula is as follows:
Figure 961250DEST_PATH_IMAGE002
the next frame position of each target in the tracking target list is marked as the current presumed position (x)0,y0). In this embodiment, the frame center position of the next frame of each target in the inference tracking list T is recorded as the current inference position (x)0,y0). Tracking the position information of the target 2 times before updating in real time, and setting the position of the target of the previous time as (x)1,y1) Number of frames n1The position of the previous secondary target is (x)2,y2) Number of frames n2Then, the velocity vector of the target motion is calculated as:
Figure 861073DEST_PATH_IMAGE001
s0 is the speed of the tracked target moving in the X-axis direction, i.e. the speed in the horizontal direction, and S1 is the speed of the tracked target moving in the Y-axis direction, i.e. the speed in the vertical direction.
Knowing the current frame number n0Calculating the tracked target current inference position P (x)0,y0)：
Figure 489501DEST_PATH_IMAGE002
The tracking algorithm uses a texture feature and distance similarity fusion judgment rule, and has a better tracking effect on targets with similar or identical appearances.
In one embodiment, the position of the tracking target is the central point of a target rectangular frame, and the width and height of the rectangular frame are kept unchanged.
In this embodiment, the positions of the tracked target expressions are all the center points of the target rectangular frame, and the width and height of the rectangle are kept unchanged in the inference process.
In one embodiment, the calculating a feature similarity between the detection target and the tracking target according to the detection target list and the tracking target list, and performing weight optimal matching according to the feature similarity specifically includes:
calculating the characteristic cosine distance between the tracking target and the detection target, wherein the formula is as follows:
Figure 791169DEST_PATH_IMAGE010
wherein d isnFor the nth detected target in the detected target list, tmFor the mth tracking target in the tracking target list, A is dnB is tmA, B are all one-dimensional vector features;
calculating the distance similarity between the tracking target and the detection target, wherein the formula is as follows:
Figure 620585DEST_PATH_IMAGE011
Figure 601179DEST_PATH_IMAGE012
representing an object dnAnd tmValue of smaller area, p1Is dnTarget rectangular position, p2Is tmThe position of the rectangle is inferred and,
Figure 970980DEST_PATH_IMAGE013
representing a rectangle intersection area value;
calculating the similarity of fusion characteristics matching the tracking target and the detection target, wherein the formula is as follows:
Figure 330418DEST_PATH_IMAGE014
wherein k is an empirical value of 0.4;
and calculating the optimal matching of the weight according to the fusion feature similarity and the optimal matching algorithm.
In the present embodiment, the detection target list D (D)nN target, N =0,1mTargets in the mth target, M =0,1.., M) match two by two.
（1）dnIs characterized by the expression of A, tmWherein A, B is a one-dimensional vector feature. Then the characteristic cosine distances FDist of the tracking target and the detection target can be calculated using the following formula:
Figure 127472DEST_PATH_IMAGE015
（2）dntarget rectangular position P1，tmIs deduced for the rectangular position P2Calculating the distance similarity DDist:
Figure 329783DEST_PATH_IMAGE016
wherein
Figure 503276DEST_PATH_IMAGE012
Representing an object dnAnd tmThe value of the smaller area is such that,
Figure 982799DEST_PATH_IMAGE017
representing the rectangle intersection area value.
(3) Calculating the similarity Like (d) of the matching fusion featuresn, tm):
Figure 950755DEST_PATH_IMAGE018
Where k is an empirical value taken to be 0.4.
The optimal matching of the weight is carried out by utilizing the similarity of pairwise matching,
in one embodiment, the best match algorithm comprises a KM algorithm.
In this embodiment, the KM algorithm is used to perform weight optimal matching on pairwise matching similarities,
in a certain embodiment, the updating the current position for the tracking target specifically includes:
searching a target set in which the previous frame and the current frame in the tracking target list are successfully matched;
finding a target with the center of the target of the previous frame closest to the tracking target in the target set, and calculating the relative position of the target and the tracking target, wherein the formula is as follows:
Figure 150616DEST_PATH_IMAGE019
wherein M ismDetecting a frame position, P, for a previous frame of the tracked objectnThe last frame of the target detects the frame positionPlacing;
updating the current position of the tracking target, wherein the formula is as follows:
Figure 862220DEST_PATH_IMAGE020
wherein Q ismIs the current position of the target.
In the present embodiment, for the tracking target T on the unmatchedm. Setting the position of the detection frame of the previous frame as MmThe location of the target may be updated using the following method:
(1) searching a target D set in which the previous frame and the current frame in the tracking target list are successfully matched;
(2) finding the distance between the center of the target and the target T in the previous frame in DmRecent target KnRecord KnThe position of the previous frame is PnThe current position is QmObtaining the relative position Diff = Mm-Pn；
(3) Update current Position = Qm+Diff；
As shown in table 1, by performing the method of the embodiment, a time-consuming statistic for static merchandise identification statistics is obtained,
TABLE 1 time-consuming statistics of the methods of the examples
Figure 196249DEST_PATH_IMAGE021
Therefore, the commodity statistical method is relatively quick.
The embodiment of the invention provides a static commodity counting device which is applied to the static commodity counting device in any one embodiment.
The video data extraction module is used for carrying out target detection and feature extraction on video data according to a target detector to obtain a detection target list and a tracking target list, wherein the detection target list comprises a detection frame, classification categories and feature expressions;
the position presumption module is used for calculating a speed vector of the movement of the tracking target according to the tracking target list and calculating a presumed position of the tracking target according to the speed vector;
the optimal weight matching module is used for calculating the feature similarity between the detection target and the tracking target according to the detection target list and the tracking target list and performing optimal weight matching according to the feature similarity;
an information updating module, configured to update the tracking target list according to the detection frame and the feature expression of the current detection target if the detection target is matched with the tracking target, and establish new tracking information for the detection target and add the new tracking information to the tracking target list if the detection target is not matched with the tracking target, and update the current position for the tracking target;
and the counting module is used for deleting the tracking targets which are matched and identified for many times in the tracking list, and counting the types and the number of the tracking targets appearing in the tracking process after the video data is processed.
For the specific definition of the static commodity counting device, reference may be made to the definition above, and details are not repeated here. The modules in the static commodity counting device can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
Referring to fig. 3, an embodiment of the invention provides a computer terminal device, which includes one or more processors and a memory. The memory is coupled to the processor for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the static merchandise statistics method as in any one of the embodiments above.
The processor is used for controlling the overall operation of the computer terminal equipment so as to complete all or part of the steps of the static commodity counting method. The memory is used to store various types of data to support the operation at the computer terminal device, which data may include, for example, instructions for any application or method operating on the computer terminal device, as well as application-related data. The Memory may be implemented by any type of volatile or non-volatile Memory device or combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic Memory, flash Memory, magnetic disk, or optical disk.
In an exemplary embodiment, the computer terminal Device may be implemented by one or more Application Specific 1 integrated circuits (AS 1C), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components, for performing the above static merchandise statistics method and achieving technical effects consistent with the above method.
In another exemplary embodiment, there is also provided a computer readable storage medium comprising program instructions which, when executed by a processor, implement the steps of the static merchandise statistics method in any one of the above embodiments. For example, the computer readable storage medium may be the above-mentioned memory including program instructions executable by the processor of the computer terminal device to perform the above-mentioned static commodity counting method, and achieve the technical effects consistent with the above-mentioned method.
In summary, in the static commodity statistical method provided by the invention, the types and the number of commodities are accurately counted in real time and the positions of the commodities are confirmed by video dynamic tracking and identification. Due to the fact that the texture feature and distance similarity fusion judgment rule is used, the target tracking effect is better for targets with similar or identical appearances, and the speed is higher than that of a common deep learning tracking method. Still can track to the static target leaving the video picture, judge the target classification with the current multiframe, fault-tolerant rate is high to the classifier classification; and the number of the statistical targets of the single picture in comparison shooting is superior in that the shot scene is large, the shooting is not influenced by a single angle, the comparison splicing recognition shooting freedom degree is high, and the recognition accuracy rate is high.
While the foregoing is directed to the preferred embodiment of the present invention, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention.

Claims (10)

1. A static commodity statistical method, comprising:
performing target detection and feature extraction on video data according to a target detector to obtain a detection target list and a tracking target list, wherein the detection target list comprises a detection frame, a classification category and feature expression;
calculating a speed vector of the movement of the tracking target according to the tracking target list, and calculating the presumed position of the tracking target according to the speed vector;
calculating the feature similarity between a detection target and the tracking target according to the detection target list and the tracking target list, and performing weight optimal matching according to the feature similarity;
if the detection target is matched with the tracking target, updating the tracking target list according to the detection frame and the feature expression of the current detection target, if the detection target is not matched with the tracking target, establishing new tracking information for the detection target, adding the new tracking information into the tracking target list, and updating the current position for the tracking target;
and deleting the tracking targets matched and identified for many times in the tracking list, and counting the types and the number of the tracking targets appearing in the tracking process after the video data is processed.
2. The static merchandise statistic method according to claim 1, further comprising training a target detection model according to a target detection algorithm and a data set of merchandise to obtain said target detector.
3. The static merchandise statistic method according to claim 2, wherein said object detection algorithm includes Faster Rcnn, SSD and RefineDet.
4. The static commodity counting method according to claim 1, wherein the calculating a velocity vector of the movement of the tracking target according to the tracking target list and the calculating the estimated position of the tracking target according to the velocity vector are specifically:
calculating the velocity vector of the movement of the tracking target, wherein the calculation formula is as follows:
Figure 487281DEST_PATH_IMAGE001
wherein, S [0]]For the speed of movement of the tracked object in the X-axis direction, S1]For the velocity of the tracked target moving in the Y-axis direction, (x)1,y1) Previous target position, n, updated in real time for the tracked target1(x) the number of video frames corresponding to the previous target position2,y2) First and second target positions, n, updated in real time for the tracked target2The video frame number corresponds to the previous and secondary target positions;
calculating the current estimated position P (x) of the tracked target0,y0) The formula is as follows:
Figure 247427DEST_PATH_IMAGE002
the next frame position of each target in the tracking target list is marked as the current presumed position (x)0,y0)，n0The number of video frames corresponding to the current presumed location.
5. The static commodity counting method according to claim 4, wherein the position of the tracking target is the center point of a target rectangular frame, and the width and the height of the rectangular frame are kept unchanged.
6. The static commodity statistical method according to claim 1, wherein the calculating of the feature similarity between the detection target and the tracking target according to the detection target list and the tracking target list and the performing of weight optimal matching according to the feature similarity specifically comprises:
calculating the characteristic cosine distance between the tracking target and the detection target, wherein the formula is as follows:
Figure 248881DEST_PATH_IMAGE003
wherein d isnFor the nth detected target in the detected target list, tmFor the mth tracking target in the tracking target list, A is dnB is tmA, B are all one-dimensional vector features;
calculating the distance similarity between the tracking target and the detection target, wherein the formula is as follows:
Figure 903241DEST_PATH_IMAGE004
Figure 680442DEST_PATH_IMAGE005
representing an object dnAnd tmValue of smaller area，p1Is dnTarget rectangular position, p2Is tmPresuming a rectangular position at the presumed position P (x)0，y0) The position of the rectangular frame as the center,
Figure 814751DEST_PATH_IMAGE006
representing a rectangle intersection area value;
calculating the similarity of fusion characteristics matching the tracking target and the detection target, wherein the formula is as follows:
Figure 631397DEST_PATH_IMAGE007
wherein k is an empirical value of 0.4;
and calculating the optimal matching of the weight according to the fusion feature similarity and the optimal matching algorithm.
7. The static merchandise statistics method of claim 6, wherein the best match algorithm comprises a KM algorithm.
8. The static commodity counting method according to claim 6, wherein the updating of the current position for the tracking target specifically comprises:
searching a target set in which the previous frame and the current frame in the tracking target list are successfully matched;
searching a target with the center of the target of the previous frame closest to the tracking target in the target set, taking the target as a nearby target, and calculating the relative position of the nearby target and the tracking target, wherein the formula is as follows:
Figure 102830DEST_PATH_IMAGE008
wherein M ismDetecting a frame position, P, for a previous frame of the tracked objectnDetecting a frame position for a previous frame of the adjacent target;
updating the current position of the tracking target, wherein the formula is as follows:
Figure 143991DEST_PATH_IMAGE009
wherein Q ismIs the current position of the nearby object.
9. A static merchandise statistic device, comprising:
the video data extraction module is used for carrying out target detection and feature extraction on video data according to a target detector to obtain a detection target list and a tracking target list, wherein the detection target list comprises a detection frame, classification categories and feature expressions;
the position presumption module is used for calculating a speed vector of the movement of the tracking target according to the tracking target list and calculating a presumed position of the tracking target according to the speed vector;
the optimal weight matching module is used for calculating the feature similarity between the detection target and the tracking target according to the detection target list and the tracking target list and performing optimal weight matching according to the feature similarity;
an information updating module, configured to update the tracking target list according to the detection frame and the feature expression of the current detection target if the detection target is matched with the tracking target, and establish new tracking information for the detection target and add the new tracking information to the tracking target list if the detection target is not matched with the tracking target, and update the current position for the tracking target;
and the counting module is used for deleting the tracking targets which are matched and identified for many times in the tracking list, and counting the types and the number of the tracking targets appearing in the tracking process after the video data is processed.
10. A computer terminal device, comprising:
one or more processors;
a memory coupled to the processor for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement the static merchandise statistics method of any of claims 1-8.
CN202110169851.9A 2021-02-08 2021-02-08 Static commodity statistical method, terminal equipment and storage medium thereof Active CN112509011B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110169851.9A CN112509011B (en) 2021-02-08 2021-02-08 Static commodity statistical method, terminal equipment and storage medium thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110169851.9A CN112509011B (en) 2021-02-08 2021-02-08 Static commodity statistical method, terminal equipment and storage medium thereof

Publications (2)

Publication Number Publication Date
CN112509011A CN112509011A (en) 2021-03-16
CN112509011B true CN112509011B (en) 2021-05-25

Family

ID=74953027

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110169851.9A Active CN112509011B (en) 2021-02-08 2021-02-08 Static commodity statistical method, terminal equipment and storage medium thereof

Country Status (1)

Country Link
CN (1) CN112509011B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113642406B (en) * 2021-07-14 2023-01-31 广州市玄武无线科技股份有限公司 System, method, device, equipment and storage medium for counting densely-suspended paper sheets
CN114445736A (en) * 2021-12-29 2022-05-06 广州市玄武无线科技股份有限公司 Video background restoration method and system for shielding of moving foreground target
CN116052062B (en) * 2023-03-07 2023-06-16 深圳爱莫科技有限公司 Robust tobacco display image processing method and device

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111667501A (en) * 2020-06-10 2020-09-15 杭州海康威视数字技术股份有限公司 Target tracking method and device, computing equipment and storage medium
CN111862147A (en) * 2020-06-03 2020-10-30 江西江铃集团新能源汽车有限公司 Method for tracking multiple vehicles and multiple human targets in video

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101976377B (en) * 2002-01-23 2016-12-07 泰科消防及安全有限公司 Inventory management system
US20040254759A1 (en) * 2003-06-13 2004-12-16 Uwe Kubach State tracking load storage system
TW201033908A (en) * 2009-03-12 2010-09-16 Micro Star Int Co Ltd System and method for counting people flow
CN103077398B (en) * 2013-01-08 2016-06-22 吉林大学 Based on Animal Group number monitoring method under Embedded natural environment
CN104463204B (en) * 2014-12-04 2018-10-26 四川九洲电器集团有限责任公司 Destination number statistical method
JP2018055607A (en) * 2016-09-30 2018-04-05 富士通株式会社 Event detection program, event detection device, and event detection method
CN109064457B (en) * 2018-07-25 2021-09-14 哈工大机器人（合肥）国际创新研究院 Quantity accumulation method and system based on vision
CN111209781B (en) * 2018-11-22 2023-05-23 珠海格力电器股份有限公司 Method and device for counting indoor people
CN111260628A (en) * 2020-01-15 2020-06-09 北京林业大学 Large nursery stock number counting method based on video image and electronic equipment
CN112037267B (en) * 2020-11-06 2021-02-02 广州市玄武无线科技股份有限公司 Method for generating panoramic graph of commodity placement position based on video target tracking

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111862147A (en) * 2020-06-03 2020-10-30 江西江铃集团新能源汽车有限公司 Method for tracking multiple vehicles and multiple human targets in video
CN111667501A (en) * 2020-06-10 2020-09-15 杭州海康威视数字技术股份有限公司 Target tracking method and device, computing equipment and storage medium

Also Published As

Publication number Publication date
CN112509011A (en) 2021-03-16

Similar Documents

Publication Publication Date Title
CN112509011B (en) Static commodity statistical method, terminal equipment and storage medium thereof
CN109829398B (en) Target detection method in video based on three-dimensional convolution network
CN104573614B (en) Apparatus and method for tracking human face
US20180204070A1 (en) Image processing apparatus and image processing method
US8345101B2 (en) Automatically calibrating regions of interest for video surveillance
JP2015210820A (en) Video tracking based method for automatic sequencing of vehicles in drive-thru applications
US11087271B1 (en) Identifying user-item interactions in an automated facility
US20070092110A1 (en) Object tracking within video images
WO2021036373A1 (en) Target tracking method and device, and computer readable storage medium
CN109858552B (en) Target detection method and device for fine-grained classification
CN111310706B (en) Commodity price tag identification method and device, electronic equipment and storage medium
CN109977824B (en) Article taking and placing identification method, device and equipment
CN111640089A (en) Defect detection method and device based on feature map center point
CN112464697A (en) Vision and gravity sensing based commodity and customer matching method and device
Shen et al. Real-time tracking and counting of grape clusters in the field based on channel pruning with YOLOv5s
US11238401B1 (en) Identifying user-item interactions in an automated facility
CN111429194B (en) User track determination system, method, device and server
CN111428743A (en) Commodity identification method, commodity processing device and electronic equipment
CN113298852A (en) Target tracking method and device, electronic equipment and computer readable storage medium
WO2005048196A2 (en) Object tracking within video images
Majdi et al. Product stock management using computer vision
Zhang et al. RKLT: 8 DOF real-time robust video tracking combing coarse RANSAC features and accurate fast template registration
Pham et al. Deepaco: A robust deep learning-based automatic checkout system
JP3680658B2 (en) Image recognition method and image recognition apparatus
CN111008210B (en) Commodity identification method, commodity identification device, codec and storage device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CP02 Change in the address of a patent holder

Address after: 510000 room 23bd, No. 109, TIYU West Road, Tianhe District, Guangzhou City, Guangdong Province

Patentee after: GUANGZHOU XUANWU WIRELESS TECHNOLOGY Co.,Ltd.

Address before: 32B, no.103b, TianLiHe Road, Guangzhou, 510000

Patentee before: GUANGZHOU XUANWU WIRELESS TECHNOLOGY Co.,Ltd.

CP02 Change in the address of a patent holder