CN112749638A - Error screening method for visual recognition track and visual recognition method for sales counter - Google Patents
Error screening method for visual recognition track and visual recognition method for sales counter Download PDFInfo
- Publication number
- CN112749638A CN112749638A CN202011597121.0A CN202011597121A CN112749638A CN 112749638 A CN112749638 A CN 112749638A CN 202011597121 A CN202011597121 A CN 202011597121A CN 112749638 A CN112749638 A CN 112749638A
- Authority
- CN
- China
- Prior art keywords
- track
- misrecognized
- features
- determining
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the application relates to the technical field of machine vision, and provides a method for screening errors of a visual identification track and a method for identifying the visual identification of a sales counter, wherein the method for screening errors of the visual identification track comprises the following steps: determining initial track characteristics corresponding to each camera; removing initial track features which have the same object attribute among the cameras and correspond to the same entity object from all the initial track features to obtain first misrecognized track features; determining a second misrecognized track feature group based on the similarity of the first misrecognized track features among the cameras; and judging that each second misrecognized track feature in the second misrecognized track feature group corresponds to the same entity object, and determining the second misrecognized track feature with the highest confidence level in the second misrecognized track feature group as the error track feature. According to the method and the device, the track characteristics of the object attribute error identification can be effectively eliminated by comparing the track characteristics of different cameras, so that the accuracy of follow-up identification is improved.
Description
Technical Field
The application relates to the technical field of machine vision, in particular to a method for screening errors of a visual identification track and a visual identification method of a sales counter.
Background
The sales counter is provided with a plurality of cameras for collecting images, track characteristics of articles taken/placed by a user can be obtained by analyzing the images, the articles actually taken/placed by the user are identified by analyzing the action attributes of the articles taken/placed in the track characteristics, and finally order information is generated according to the identification result.
The premise of identifying the article taking/placing result from the track characteristics is to obtain the article attribute with correct track characteristics. Due to the angle problem of each camera, the situation that the object attributes in the track characteristics of the same object identified by each camera are inconsistent often occurs, the subsequent object taking/placing result identification is directly influenced, and disputes between users and operators are easily caused.
Disclosure of Invention
The application provides a method for screening error tracks by visual recognition and a method for visually recognizing a sales counter, so as to eliminate track features of object attribute error recognition.
The application provides a method for screening errors of a visual identification track, which comprises the following steps: determining initial track characteristics corresponding to the cameras based on image information acquired by the cameras, wherein the initial track characteristics comprise article attributes; removing initial track features which have the same article attribute among the cameras and correspond to the same entity object from all the initial track features to obtain first misrecognized track features, wherein the article attributes of the first misrecognized track features corresponding to the cameras are different; determining a second misrecognized track feature group based on the similarity of the first misrecognized track features among the cameras; and judging that each second misrecognized track feature in the second misrecognized track feature group corresponds to the same entity object, and determining the second misrecognized track feature with the highest confidence level in the second misrecognized track feature group as the error track feature.
According to the method for screening errors of the visual recognition track, the initial track features which have the same article attribute and correspond to the same entity object among all the cameras are removed from all the initial track features to obtain a first error recognition track feature, and the method comprises the following steps of:
determining similarity between track features which have the same article attribute and correspond to different cameras;
and sequentially removing a group of track features with the highest similarity as the same entity object to obtain a first misrecognized track feature.
According to the method for screening errors of the visual recognition track, based on the similarity of the first error recognition track characteristics among the cameras, a second error recognition track characteristic group is determined, and the method comprises the following steps:
determining the similarity between each first misrecognition track characteristic corresponding to each camera and each first misrecognition track characteristic corresponding to other cameras;
and taking the first misrecognized track features with the similarity greater than the target similarity as a second misrecognized track feature group.
According to the error screening method for the visual recognition track, the similarity is determined by the following method:
extracting a characteristic value of the track characteristic;
and determining Euclidean distances among the track features based on the feature values, wherein the Euclidean distances are used for representing the similarity.
According to the method for screening errors of the visual recognition track, the step of judging that each second error recognition track feature in the second error recognition track feature group corresponds to the same entity object comprises the following steps:
determining the running direction characteristic and the time characteristic of each second misrecognized track characteristic in the second misrecognized track characteristic group;
and if the running direction characteristics are the same and the time characteristics have intersection, determining that each second misrecognized track characteristic corresponds to the same entity object.
The application also provides a visual identification method of the sales counter, which comprises the following steps:
the method for screening the error of the visual identification track as any one of the above;
and identifying the motion attributes in the other initial track features except the error track feature, and obtaining an identification result based on the identified motion attributes.
The application also provides a sieving mechanism of visual identification orbit, includes:
the track recognition module is used for determining initial track characteristics corresponding to the cameras based on image information acquired by the cameras, and the initial track characteristics comprise article attributes;
the first determining module is used for removing initial track features which have the same article attribute among the cameras and correspond to the same entity object from all the initial track features to obtain first misrecognized track features, wherein the article attributes of the first misrecognized track features corresponding to the cameras are different;
the second determining module is used for determining a second misrecognized track feature group based on the similarity of the first misrecognized track features among the cameras;
and the third determining module is used for judging that each second misrecognized track feature in the second misrecognized track feature group corresponds to the same entity object, and determining the second misrecognized track feature with the highest confidence level in the second misrecognized track feature group as an error track feature.
According to the screening device for the visual recognition track, the first determination module comprises:
the first determining unit is used for determining similarity between the track characteristics which have the same article attribute and correspond to different cameras;
and the first processing unit is used for removing the group of track features with the highest similarity as the same entity object in sequence to obtain a first misrecognized track feature.
According to the application, the second determination module comprises:
the second determining unit is used for determining the similarity between each first misrecognized track feature corresponding to each camera and each first misrecognized track feature corresponding to other cameras;
and the second processing unit is used for taking the first misrecognized track features with the similarity greater than the target similarity as a second misrecognized track feature group.
According to the application, the third determination module comprises:
the third determining unit is used for determining the running direction characteristic and the time characteristic of each second misrecognized track characteristic in the second misrecognized track characteristic group;
and the third processing unit is used for determining that each second misrecognized track feature corresponds to the same entity object if the running direction features are the same and the time features have intersection.
The application also provides a visual identification device of a sales counter, including:
the screening device is used for executing the screening method for visually recognizing the track;
and the identification module is used for identifying the action attributes in other initial track characteristics except the error track characteristics and obtaining an identification result based on the identified action attributes.
The application also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the computer program to realize the steps of any one of the method for screening errors of the visual identification track or the method for visually identifying the sales counter.
The present application further provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method for visually identifying a missing track or the method for visually identifying a sales counter as described in any of the above.
According to the method for screening the error tracks for visual recognition and the method for visual recognition of the sales counter, the track characteristics for error recognition of the attributes of the articles can be effectively eliminated by comparing the track characteristics among different cameras, so that the accuracy of subsequent recognition is improved.
Drawings
In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a schematic flow chart of a method for error screening of a visual recognition track provided by the present application;
FIG. 2 is a schematic flow chart diagram illustrating an embodiment of step 120 of the method for visually identifying a track error in a computer system provided in the present application;
FIG. 3 is a schematic flow chart diagram illustrating an embodiment of step 130 of the method for visually identifying a track error in a computer system provided in the present application;
FIG. 4 is a schematic flow chart diagram illustrating an embodiment of step 140 of the method for visually identifying a track error in a computer system provided herein;
FIG. 5 is a schematic structural diagram of a screening apparatus for visually recognizing a trajectory provided herein;
FIG. 6 is a schematic structural diagram of a first determining module of the screening apparatus for visually recognizing a trajectory provided in the present application;
FIG. 7 is a schematic structural diagram of a second determination module of the screening apparatus for visually recognizing a trajectory provided in the present application;
FIG. 8 is a schematic structural diagram of a third determining module of the screening apparatus for visually recognizing a trajectory provided in the present application;
FIG. 9 is a schematic flow chart of a visual identification method of a sales counter provided by the present application;
FIG. 10 is a schematic view of the visual identification apparatus of the sales counter provided in the present application;
fig. 11 is a schematic structural diagram of an electronic device provided in the present application.
Detailed Description
To make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be clearly and completely described below with reference to the drawings in the present application, and it is obvious that the described embodiments are some, but not all embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The error screening method for visually recognizing the trajectory of the present application is described below with reference to fig. 1 to 4.
As shown in fig. 1, the method for screening errors of a visual recognition track in an embodiment of the present application includes: step 110-step 140.
in the step, image recognition is carried out on each frame of image information collected by each camera to obtain the article attribute of each frame of image information, and then the results with the same article attribute in each camera are linked into a track by using a tracking algorithm to obtain the initial track characteristic.
When image recognition is carried out on each frame of image information collected by each camera, the recognition confidence coefficient is obtained.
Each initial trajectory feature has an item attribute, and from image information acquired by a camera, a plurality of initial trajectory features may be obtained.
For example, when a user takes 2 apples from the same shelf of a sales counter at a time, 2 initial trajectory features can be obtained from image information collected by one camera, and the article attributes of the 2 initial trajectory features are all apples.
Certainly, a user takes 2 apples from the same shelf of a sales counter at a time, if the identification is wrong in the image identification process, 2 initial track features may be obtained from the image information collected by one camera, and the article attributes of the 2 initial track features are apples and oranges respectively; from the image information collected by another camera, 2 initial track features may be obtained, and the article attributes of the 2 initial track features are all apples.
In actual implementation, an item attribute set corresponding to all cameras can be obtained based on the item attributes, and the item attribute set is a set of possible maximum results.
Specifically, a union set is taken for the article attribute sets corresponding to all the cameras, for example, 2 initial trajectory features of an apple and an orange are obtained in the image information acquired from one camera, and 2 initial trajectory features of an apple are obtained in the image information acquired from the other camera, so that the article attribute sets are (apple, orange).
Step 120, removing the initial track features which have the same article attribute among the cameras and correspond to the same entity object from all the initial track features to obtain first misrecognized track features, wherein the article attributes of the first misrecognized track features corresponding to the cameras are different;
in this step, it is necessary to determine which initial trajectory features with the same article attribute correspond to the same entity object in all the cameras, and the initial trajectory features corresponding to the same entity object may be determined to be the initial trajectory features with the correct recognition result and removed first.
The rest initial track features are first misrecognized track features, and certain article attributes do not exist, and exist in the first misrecognized track features of all the cameras.
In other words, the purpose of this step is to eliminate the method, and based on the same article attribute, the initial trajectory features agreed by all cameras are removed first.
akiThe item attribute set corresponding to the ith initial track characteristic of the kth camera is Ak，aki∈Ak。
Taking two cameras each corresponding to two initial trajectory features as an example,
A1＝(a11,a12)，a11═ apple, a12Becoming apple;
A2＝(a21,a22)，a21═ apple, a22Orange is prepared.
In this step, a needs to be determined11Corresponding initial trajectory feature and a12Which one of the corresponding initial trajectory features corresponds to a21The corresponding initial track features correspond to the same physical object, such as identifying a11Corresponding initial trajectory feature and a21Corresponding initial trajectory characteristicsIdentifying the same entity object, determining that the two objects are correctly identified, removing the two objects first to obtain a first misrecognized track characteristic a12Corresponding initial trajectory feature and a22Corresponding initial trajectory characteristics.
Taking two cameras each corresponding to three initial trajectory features as an example,
A1＝(a11,a12,a13)，a11═ apple, a12═ apple, a13Becoming apple;
A2＝(a21,a22,a23)，a21═ apple, a32(ii) orange, a23Apple.
In this step, a needs to be determined21And a23Corresponding initial track characteristics are respectively corresponding to a11、a12And a13Which of the corresponding initial trajectory features corresponds to the same physical object, e.g. to determine a11Corresponding initial trajectory feature and a21If the corresponding initial track characteristics correspond to the same entity object, judging that the two are correctly identified, removing the two, and confirming that a is12Corresponding initial trajectory feature and a23If the corresponding initial track characteristics correspond to the same entity object, judging that the two are correctly identified, removing the two, and obtaining a first misrecognized track characteristic a13Corresponding initial trajectory feature and a22Corresponding initial trajectory characteristics.
in step 120, in the obtained first misrecognized track features, there may be a case of misrecognization, and by comparing the similarity of the first misrecognized track features between different cameras, a second misrecognized track feature group may be obtained.
If the first misrecognized track characteristics of different cameras are not similar, the first misrecognized track characteristics are judged not to be misrecognized and are reserved.
And 140, judging that each second misrecognized track feature in the second misrecognized track feature group corresponds to the same entity object, and determining the second misrecognized track feature with the highest confidence level in the second misrecognized track feature group as the error track feature.
In the step, whether each second misrecognized track feature in the second misrecognized track feature group corresponds to the same entity object needs to be judged, if yes, the second misrecognized track feature with the highest confidence level in the second misrecognized track feature group is identified to the correct article attribute and is reserved; and the second misrecognized track features with the highest confidence level in the second misrecognized track feature group are error track features.
Therefore, by the error screening method, the error identification of the article attribute of the partial image information can be accurately found.
According to the error screening method for the visual identification track, track characteristics for error identification of the object attribute can be effectively eliminated through comparison of the track characteristics among different cameras, so that the accuracy of subsequent identification is improved.
In some embodiments, as shown in fig. 2, in step 120, removing the initial trajectory features, which have the same article attribute and correspond to the same entity object, from all the initial trajectory features to obtain a first misrecognized trajectory feature, includes: step 121 and step 122.
in an actual implementation, step 121 may include: extracting a characteristic value of the initial track characteristic; and determining Euclidean distance between the initial track features based on the feature values, wherein the Euclidean distance is used for representing similarity, and the smaller the Euclidean distance is, the higher the similarity is, and the larger the Euclidean distance is, the lower the similarity is.
And step 122, removing a group of initial track features with the highest similarity as the same entity object in sequence to obtain a first misrecognized track feature.
Taking two cameras as an example and each corresponding to three trajectory features,
A1＝(a11,a12,a13)，a11═ apple, a12═ apple, a13Becoming apple;
A2＝(a21,a22,a23)，a21═ apple, a22(ii) orange, a23Apple.
In this step, a needs to be determined21Corresponding initial track characteristics are respectively corresponding to a11、a12And a13Similarity of corresponding initial trajectory features, and a needs to be determined23Corresponding initial track characteristics are respectively corresponding to a11、a12And a13Similarity of corresponding initial trajectory features.
For example, a21Corresponding initial trajectory feature and a11The similarity of the corresponding initial track features is the highest, then a21Corresponding initial trajectory feature and a11If the corresponding initial track characteristics correspond to the same entity object, judging that the two are correctly identified, and removing the two; a is23Corresponding initial trajectory feature and the rest of a12And a13In (a)12The similarity of the corresponding initial track features is higher, then a23Corresponding initial trajectory feature and a12If the corresponding initial track characteristics correspond to the same entity object, judging that the two are correctly identified, and removing the two; the obtained first misrecognized track characteristic is a13Corresponding initial trajectory feature and a22Corresponding initial trajectory characteristics.
In some embodiments, as shown in fig. 3, the determining 130 a second misrecognized trajectory feature group based on the similarity of the first misrecognized trajectory features between the cameras includes: step 131 and step 132.
And 131, determining the similarity between each first misrecognized track feature corresponding to each camera and each first misrecognized track feature corresponding to other cameras.
In other words, the similarity between every two first misrecognized track features corresponding to different cameras is determined, and the similarity between every two first misrecognized track features between the cameras is determined.
Taking two cameras as an example and each corresponding to three trajectory features,
A1＝(a11,a12,a13)，a11═ apple, a12═ apple, a13Becoming apple;
A2＝(a21,a22,a23)，a21═ apple, a22(ii) orange, a23Peach.
a11And a21Corresponding to the same entity object, obtaining a first misrecognized track characteristic-a12,a13,a22,a23Then, in this step, the similarity of four sets of first misrecognized trajectory features as follows needs to be determined: (a)12,a22)、(a12,a23)、(a13,a22)、(a13,a23)。
In an actual implementation, step 131 may include: extracting a characteristic value of the first misrecognized track characteristic; based on the characteristic values, determining Euclidean distances between every two first misrecognized track characteristics corresponding to different cameras, wherein the Euclidean distances are used for representing similarity, and the smaller the Euclidean distances are, the higher the similarity is, and the larger the Euclidean distances are, the lower the similarity is.
And 132, taking the first misrecognized track features with the similarity greater than the target similarity as a second misrecognized track feature group.
Taking two cameras and each camera corresponding to three initial track features as an example, the finally obtained first misrecognized initial track feature is a13Corresponding initial trajectory feature and a22Corresponding initial trajectory feature, a13═ apple, a22Orange is prepared.
If a is13Corresponding initial trajectory feature and a22If the similarity of the corresponding initial track features is greater than the target similarity, an error may occur when one of the cameras identifies the object, and a is used13Corresponding initial trajectory feature and a22The corresponding initial trajectory features serve as a second misrecognized trajectory feature set.
If a is13Corresponding initial trajectory feature and a22If the similarity of the corresponding initial track features is not greater than the target similarity, a is determined13Corresponding initial trajectory feature and a22The corresponding initial trajectory characteristics are retained.
In this way, the screened first misrecognized track feature can be screened again from the features of the initial track feature itself.
In some embodiments, as shown in fig. 4, the step 140 of determining that each second misrecognized trajectory feature in the second misrecognized trajectory feature group corresponds to the same entity object includes: step 141 and step 142.
141, determining the running direction characteristic and the time characteristic of each second misrecognized track characteristic in the second misrecognized track characteristic group;
and 142, if the running direction features are the same and the time features have intersection, determining that each second misrecognized track feature corresponds to the same entity object.
In order to prevent some tracks from being deleted by mistake, for the second misrecognized track feature group, the actual information is required to be combined to judge whether the second misrecognized track feature group really corresponds to the same entity object, the important information of the track features is whether the article is taken out or put back and the time period of the action, and whether each second misrecognized track feature in the second misrecognized track feature group really corresponds to the same entity object can be accurately identified through the judgment of the two information.
The screening device for visual identification tracks provided by the present application is described below, and the screening device for visual identification tracks described below and the screening method for visual identification tracks described above may be referred to in correspondence with each other.
As shown in fig. 5, the present application further provides a screening apparatus for visually recognizing a trajectory.
The screening device of visual identification orbit of this application embodiment includes: a trajectory identification module 510, a first determination module 520, a second determination module 530, and a third determination module 540.
A track identification module 510, configured to determine, based on image information acquired by a plurality of cameras, an initial track feature corresponding to each camera, where the initial track feature includes an article attribute;
a first determining module 520, configured to remove initial track features that have the same article attribute among the cameras and correspond to the same entity object from all the initial track features to obtain first misrecognized track features, where the article attributes of the first misrecognized track features corresponding to the cameras are different;
a second determining module 530, configured to determine a second misrecognized trajectory feature group based on a similarity of the first misrecognized trajectory features between the cameras;
a third determining module 540, configured to determine that each second misrecognized trajectory feature in the second misrecognized trajectory feature group corresponds to the same entity object, and determine that a second misrecognized trajectory feature with non-highest confidence level in the second misrecognized trajectory feature group is an error trajectory feature.
According to the screening device for the visual identification tracks, track features which identify object attributes by mistake can be effectively eliminated by comparing the track features of different cameras, so that the accuracy of follow-up identification is improved.
In some embodiments, as shown in fig. 6, the first determination module 520 includes: a first determination unit 521 and a first processing unit 522.
The first determining unit 521 is configured to determine similarity between the track features that have the same article attribute and correspond to different cameras;
the first processing unit 522 is configured to sequentially remove a group of track features with the highest similarity as the same entity object, so as to obtain a first misrecognized track feature.
In some embodiments, as shown in fig. 7, the second determining module 530 includes: a second determination unit 531 and a second processing unit 532.
A second determining unit 531, configured to determine similarity between each first misrecognized trajectory feature corresponding to each camera and each first misrecognized trajectory feature corresponding to another camera;
the second processing unit 532 is configured to use the first misrecognized trajectory feature with the similarity greater than the target similarity as a second misrecognized trajectory feature group.
In some embodiments, as shown in fig. 8, the third determining module 540 includes: a third determination unit 541 and a third processing unit 542.
A third determining unit 541, configured to determine a running direction feature and a time feature of each second misrecognized track feature in the second misrecognized track feature group;
the third processing unit 542 is configured to determine that each second misrecognized track feature corresponds to the same entity object if the running direction features are the same and the time features have an intersection.
The screening device for the visual identification tracks is used for executing the error screening method for the visual identification tracks, the implementation mode of the screening device for the visual identification tracks is consistent with that of the error screening method for the visual identification tracks, the same beneficial effects can be achieved, and the screening device for the visual identification tracks is not repeated here.
The application also provides a visual identification method of the uniform sales counter.
The sales counter can be intelligent or unmanned.
The locker includes the cabinet body and the door body, and the cabinet body is used for placing article, and the cabinet body has the end of opening, and the door body is used for sealing this end of opening, and the door body passes through lock and cabinet body locking, locker and server communication connection. When a user needs to take articles from the storage cabinet, the door body is unlocked firstly, for example, the user can use the mobile terminal to scan an identification code on the storage cabinet, the mobile terminal logs in the server, the mobile terminal sends a door opening request to the server, the door opening request comprises an equipment identification of the storage cabinet, the server issues a door opening command, and the storage cabinet is unlocked.
Of course, the door may also be unlocked in other manners, for example, the storage cabinet is provided with a scanning device, and the door of the storage cabinet may also be unlocked in other manners such as scanning the identification code of the mobile terminal of the user, scanning the fingerprint of the user, scanning the palm print of the user, scanning the iris of the user, scanning the face of the user, and the like. Therefore, when the user unlocks the door body in any one mode, the door opening signal can be correspondingly sent out.
The cabinet body can be further provided with a shelf for placing articles (namely, commodities to be sold).
After the door body was opened, the user got the thing from the shelf by oneself, also can appear certainly and get the thing back, the situation of putting back again, the sales counter has a plurality of cameras, and the camera is used for gathering user's action image information, to every layer of shelf, can be equipped with a plurality of cameras in order to improve the accuracy of discernment, all be equipped with the camera about for example the shelf, perhaps all be equipped with the camera about the upper and lower of shelf.
The image collected by the camera can be sent to the server, and the goods taking information of the user or the goods supplementing information of the operator in the transaction process is determined through an image recognition algorithm.
The visual identification method of the sales counter of the present application is described below with reference to fig. 9. The execution subject of the method may be a server of the sales counter, or a hardware device of the server, or a processor local to the sales counter.
As shown in fig. 9, the visual identification method for a sales counter according to the embodiment of the present application includes:
a method of visually identifying a track error as in any of the embodiments above;
and 150, identifying the action attributes in the initial track features except the error track feature, and obtaining an identification result based on the identified action attributes.
The initial track feature is used for characterizing the position change feature of the target object when changing with time, and it is understood that a partial section of the initial track feature may indicate that the object is taken out, and a partial section of the initial track feature may indicate that the object is put back, so that the action attribute of the initial track feature can be obtained.
In this step, the identification result is obtained mainly by determining the corresponding action attribute of each initial trajectory feature. The action attribute is used to characterize whether the item is removed or replaced. After the action attribute of the initial track characteristic is obtained, an identification result is obtained by calculating the action attribute, and the identification result is used for representing how many articles are taken away from (or put back from) the sales counter by the user.
According to the visual identification method of the sales counter, the track characteristics of the object attribute error identification can be effectively eliminated by comparing the track characteristics of different cameras, and the accurate settlement of the sales counter is facilitated.
The following describes the visual recognition apparatus of the sales counter provided by the present application, and the visual recognition apparatus of the sales counter described below and the visual recognition method of the sales counter described above may be referred to in correspondence with each other.
As shown in fig. 10, the present application also provides a visual recognition apparatus for a sales counter.
The visual recognition device of the sales counter of the embodiment of the application comprises: a screening apparatus 500 and an identification module 600.
A screening device 500 for performing any one of the above-described screening methods for visually recognizing a trajectory; the screening apparatus 500 may include: a trajectory identification module 510, a first determination module 520, a second determination module 530, and a third determination module 540.
The identifying module 600 is configured to identify other track features except the error-correcting track feature and other initial track features to obtain an identification result.
The visual recognition device of the sales counter provided by the embodiment of the application is used for executing the visual recognition method of the sales counter, the implementation mode of the visual recognition device of the sales counter is consistent with the implementation mode of the visual recognition method of the sales counter provided by the application, the same beneficial effects can be achieved, and the detailed description is omitted here.
Fig. 11 illustrates a physical structure diagram of an electronic device, and as shown in fig. 11, the electronic device may include: a processor (processor)1110, a communication Interface (Communications Interface)1120, a memory (memory)1130, and a communication bus 1140, wherein the processor 1110, the communication Interface 1120, and the memory 1130 communicate with each other via the communication bus 1140. Processor 1110 may invoke logic instructions in memory 1130 to perform a method of error screening for visually recognizing a trajectory, the method comprising: determining initial track characteristics corresponding to the cameras based on image information acquired by the cameras, wherein the initial track characteristics comprise article attributes; removing initial track features which have the same article attribute among the cameras and correspond to the same entity object from all the initial track features to obtain first misrecognized track features, wherein the article attributes of the first misrecognized track features corresponding to the cameras are different; determining a second misrecognized track feature group based on the similarity of the first misrecognized track features among the cameras; judging that each second misrecognized track feature in the second misrecognized track feature group corresponds to the same entity object, and determining a second misrecognized track feature with non-highest confidence level in the second misrecognized track feature group as a wrong track feature; or performing a visual recognition method of a sales counter, the method comprising: a screening method of visually recognizing the trajectory as described above; and identifying the motion attributes in the other initial track features except the error track feature, and obtaining an identification result based on the identified motion attributes.
In addition, the logic instructions in the memory 1130 may be implemented in software functional units and stored in a computer readable storage medium when sold or used as a stand-alone product. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The processor 810 in the electronic device provided in the embodiment of the present application may call the logic instruction in the memory 830 to implement the error screening method for the visual identification track and the visual identification method for the sales counter, and an implementation manner of the error screening method for the visual identification track and the visual identification method for the sales counter provided in the present application is consistent with the implementation manner of the error screening method for the visual identification track and the visual identification method for the sales counter, and the same beneficial effects may be achieved, and details are not repeated here.
In another aspect, the present application further provides a computer program product, which is described below, and the computer program product described below and the method for visually identifying a track and the method for visually identifying a sales counter described above may be referred to in correspondence with each other.
The computer program product comprises a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform a method of error screening of visually identified tracks provided by the methods described above, the method comprising: determining initial track characteristics corresponding to the cameras based on image information acquired by the cameras, wherein the initial track characteristics comprise article attributes; removing initial track features which have the same article attribute among the cameras and correspond to the same entity object from all the initial track features to obtain first misrecognized track features, wherein the article attributes of the first misrecognized track features corresponding to the cameras are different; determining a second misrecognized track feature group based on the similarity of the first misrecognized track features among the cameras; judging that each second misrecognized track feature in the second misrecognized track feature group corresponds to the same entity object, and determining a second misrecognized track feature with non-highest confidence level in the second misrecognized track feature group as a wrong track feature; or performing a visual recognition method of a sales counter, the method comprising: a screening method of visually recognizing the trajectory as described above; and identifying the motion attributes in the other initial track features except the error track feature, and obtaining an identification result based on the identified motion attributes.
When the computer program product provided by the embodiment of the application is executed, the error screening method for the visual identification track and the visual identification method for the sales counter are implemented, the implementation modes of the error screening method for the visual identification track and the visual identification method for the sales counter are consistent with the implementation modes of the error screening method for the visual identification track and the visual identification method for the sales counter, the same beneficial effects can be achieved, and details are not repeated here.
In yet another aspect, the present application further provides a non-transitory computer readable storage medium, which is described below, and the non-transitory computer readable storage medium described below and the method for visually identifying a trajectory and the method for visually identifying a sales counter described above may be referred to in correspondence with each other.
The present application also provides a non-transitory computer readable storage medium having stored thereon a computer program that when executed by a processor performs the method of error screening for visually identifying trajectories provided above, the method comprising: determining initial track characteristics corresponding to the cameras based on image information acquired by the cameras, wherein the initial track characteristics comprise article attributes; removing initial track features which have the same article attribute among the cameras and correspond to the same entity object from all the initial track features to obtain first misrecognized track features, wherein the article attributes of the first misrecognized track features corresponding to the cameras are different; determining a second misrecognized track feature group based on the similarity of the first misrecognized track features among the cameras; judging that each second misrecognized track feature in the second misrecognized track feature group corresponds to the same entity object, and determining a second misrecognized track feature with non-highest confidence level in the second misrecognized track feature group as a wrong track feature; or performing a visual recognition method of a sales counter, the method comprising: a screening method of visually recognizing the trajectory as described above; and identifying the motion attributes in the other initial track features except the error track feature, and obtaining an identification result based on the identified motion attributes.
When the computer program stored on the non-transitory computer readable storage medium provided in the embodiment of the present application is executed, the error screening method for the visual identification track and the visual identification method for the sales counter are implemented, and an implementation manner of the error screening method for the visual identification track and the visual identification method for the sales counter provided in the present application is consistent with an implementation manner of the error screening method for the visual identification track and the visual identification method for the sales counter, and the same beneficial effects can be achieved, and details are not repeated here.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.
Claims (13)
1. A method for screening errors of a visual identification track is characterized by comprising the following steps:
determining initial track characteristics corresponding to the cameras based on image information acquired by the cameras, wherein the initial track characteristics comprise article attributes;
removing initial track features which have the same article attribute among the cameras and correspond to the same entity object from all the initial track features to obtain first misrecognized track features, wherein the article attributes of the first misrecognized track features corresponding to the cameras are different;
determining a second misrecognized track feature group based on the similarity of the first misrecognized track features among the cameras;
and judging that each second misrecognized track feature in the second misrecognized track feature group corresponds to the same entity object, and determining the second misrecognized track feature with the highest confidence level in the second misrecognized track feature group as the error track feature.
2. The method of claim 1, wherein the step of removing the initial track features, which have the same object attribute and correspond to the same physical object, from all the initial track features to obtain a first misrecognized track feature comprises:
determining similarity between track features which have the same article attribute and correspond to different cameras;
and sequentially removing a group of track features with the highest similarity as the same entity object to obtain a first misrecognized track feature.
3. The method of claim 1, wherein the determining a second misrecognized trajectory feature group based on the similarity of the first misrecognized trajectory features between the cameras comprises:
determining the similarity between each first misrecognition track characteristic corresponding to each camera and each first misrecognition track characteristic corresponding to other cameras;
and taking the first misrecognized track features with the similarity greater than the target similarity as a second misrecognized track feature group.
4. The method of claim 2 or 3, wherein the similarity is determined by:
extracting a characteristic value of the track characteristic;
and determining Euclidean distances among the track features based on the feature values, wherein the Euclidean distances are used for representing the similarity.
5. The method for screening errors of visual recognition tracks according to any one of claims 1 to 3, wherein the determining that each second misrecognized track feature in the second group of misrecognized track features corresponds to the same physical object comprises:
determining the running direction characteristic and the time characteristic of each second misrecognized track characteristic in the second misrecognized track characteristic group;
and if the running direction characteristics are the same and the time characteristics have intersection, determining that each second misrecognized track characteristic corresponds to the same entity object.
6. A visual identification method for a sales counter, comprising:
the method of visually identifying a track of any one of claims 1-5;
and identifying the motion attributes in the other initial track features except the error track feature, and obtaining an identification result based on the identified motion attributes.
7. A screening apparatus for visually recognizing a trajectory, comprising:
the track recognition module is used for determining initial track characteristics corresponding to the cameras based on image information acquired by the cameras, and the initial track characteristics comprise article attributes;
the first determining module is used for removing initial track features which have the same article attribute among the cameras and correspond to the same entity object from all the initial track features to obtain first misrecognized track features, wherein the article attributes of the first misrecognized track features corresponding to the cameras are different;
the second determining module is used for determining a second misrecognized track feature group based on the similarity of the first misrecognized track features among the cameras;
and the third determining module is used for judging that each second misrecognized track feature in the second misrecognized track feature group corresponds to the same entity object, and determining the second misrecognized track feature with the highest confidence level in the second misrecognized track feature group as an error track feature.
8. The apparatus for screening visual recognition tracks according to claim 7, wherein the first determining module comprises:
the first determining unit is used for determining similarity between the track characteristics which have the same article attribute and correspond to different cameras;
and the first processing unit is used for removing the group of track features with the highest similarity as the same entity object in sequence to obtain a first misrecognized track feature.
9. The apparatus for screening visual recognition trajectories of claim 7, wherein the second determining module comprises:
the second determining unit is used for determining the similarity between each first misrecognized track feature corresponding to each camera and each first misrecognized track feature corresponding to other cameras;
and the second processing unit is used for taking the first misrecognized track features with the similarity greater than the target similarity as a second misrecognized track feature group.
10. The apparatus for screening of visual recognition tracks according to any one of claims 7 to 9, wherein the third determining module comprises:
the third determining unit is used for determining the running direction characteristic and the time characteristic of each second misrecognized track characteristic in the second misrecognized track characteristic group;
and the third processing unit is used for determining that each second misrecognized track feature corresponds to the same entity object if the running direction features are the same and the time features have intersection.
11. A visual identification device for a sales counter, comprising:
a screening device for executing the error screening method for visually recognizing the trajectory according to any one of claims 1 to 5;
and the identification module is used for identifying the action attributes in other initial track characteristics except the error track characteristics and obtaining an identification result based on the identified action attributes.
12. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor when executing the program performs the steps of the method of visually identifying a missing track according to any of the claims 1-5 or the method of visually identifying a sales container according to claim 6.
13. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the steps of the method of visually identifying a missing track according to any one of claims 1 to 5 or the method of visually identifying a sales container according to claim 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011597121.0A CN112749638A (en) | 2020-12-28 | 2020-12-28 | Error screening method for visual recognition track and visual recognition method for sales counter |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011597121.0A CN112749638A (en) | 2020-12-28 | 2020-12-28 | Error screening method for visual recognition track and visual recognition method for sales counter |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112749638A true CN112749638A (en) | 2021-05-04 |
Family
ID=75646867
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011597121.0A Pending CN112749638A (en) | 2020-12-28 | 2020-12-28 | Error screening method for visual recognition track and visual recognition method for sales counter |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112749638A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115546703A (en) * | 2022-11-21 | 2022-12-30 | 浙江莲荷科技有限公司 | Risk identification method, device and equipment for self-service cash register and storage medium |
| CN115641359A (en) * | 2022-10-17 | 2023-01-24 | 北京百度网讯科技有限公司 | Method, apparatus, electronic device, and medium for determining motion trajectory of object |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20120062168A (en) * | 2010-12-06 | 2012-06-14 | 한국전자통신연구원 | Apparatus and method for recogniting sub-trajectory |
| US20140050455A1 (en) * | 2012-08-20 | 2014-02-20 | Gorilla Technology Inc. | Correction method for object linking across video sequences in a multiple camera video surveillance system |
| WO2017157119A1 (en) * | 2016-03-18 | 2017-09-21 | 中兴通讯股份有限公司 | Method and device for identifying abnormal behavior of vehicle |
| CN109816700A (en) * | 2019-01-11 | 2019-05-28 | 佰路得信息技术(上海)有限公司 | A kind of information statistical method based on target identification |
| WO2019109142A1 (en) * | 2017-12-06 | 2019-06-13 | University Of Technology Sydney | Monitoring systems, and computer implemented methods for processing data in monitoring systems, programmed to enable identification and tracking of human targets in crowded environments |
| CN111061822A (en) * | 2019-11-13 | 2020-04-24 | 北京旷视科技有限公司 | Track drawing method and device, electronic equipment and storage medium |
| US10679362B1 (en) * | 2018-05-14 | 2020-06-09 | Vulcan Inc. | Multi-camera homogeneous object trajectory alignment |
| CN111415461A (en) * | 2019-01-08 | 2020-07-14 | 虹软科技股份有限公司 | Article identification method and system and electronic equipment |
| CN111639968A (en) * | 2020-05-25 | 2020-09-08 | 腾讯科技（深圳）有限公司 | Trajectory data processing method and device, computer equipment and storage medium |
-
2020
- 2020-12-28 CN CN202011597121.0A patent/CN112749638A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20120062168A (en) * | 2010-12-06 | 2012-06-14 | 한국전자통신연구원 | Apparatus and method for recogniting sub-trajectory |
| US20140050455A1 (en) * | 2012-08-20 | 2014-02-20 | Gorilla Technology Inc. | Correction method for object linking across video sequences in a multiple camera video surveillance system |
| WO2017157119A1 (en) * | 2016-03-18 | 2017-09-21 | 中兴通讯股份有限公司 | Method and device for identifying abnormal behavior of vehicle |
| WO2019109142A1 (en) * | 2017-12-06 | 2019-06-13 | University Of Technology Sydney | Monitoring systems, and computer implemented methods for processing data in monitoring systems, programmed to enable identification and tracking of human targets in crowded environments |
| US10679362B1 (en) * | 2018-05-14 | 2020-06-09 | Vulcan Inc. | Multi-camera homogeneous object trajectory alignment |
| CN111415461A (en) * | 2019-01-08 | 2020-07-14 | 虹软科技股份有限公司 | Article identification method and system and electronic equipment |
| CN109816700A (en) * | 2019-01-11 | 2019-05-28 | 佰路得信息技术(上海)有限公司 | A kind of information statistical method based on target identification |
| CN111061822A (en) * | 2019-11-13 | 2020-04-24 | 北京旷视科技有限公司 | Track drawing method and device, electronic equipment and storage medium |
| CN111639968A (en) * | 2020-05-25 | 2020-09-08 | 腾讯科技（深圳）有限公司 | Trajectory data processing method and device, computer equipment and storage medium |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115641359A (en) * | 2022-10-17 | 2023-01-24 | 北京百度网讯科技有限公司 | Method, apparatus, electronic device, and medium for determining motion trajectory of object |
| CN115641359B (en) * | 2022-10-17 | 2023-10-31 | 北京百度网讯科技有限公司 | Method, device, electronic equipment and medium for determining movement track of object |
| CN115546703A (en) * | 2022-11-21 | 2022-12-30 | 浙江莲荷科技有限公司 | Risk identification method, device and equipment for self-service cash register and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108491799B (en) | Intelligent sales counter commodity management method and system based on image recognition | |
| US11188783B2 (en) | Reverse neural network for object re-identification | |
| CN111415461B (en) | Article identification method and system and electronic equipment | |
| US11288490B1 (en) | Utilizing sensor data for automated user identification | |
| CN106663203B (en) | Living body finger print recognition methods and device | |
| KR101005411B1 (en) | Method for managing authentication system | |
| CA3152812A1 (en) | Facial recognition method and apparatus | |
| CN108734092B (en) | Person authentication device | |
| CN112749638A (en) | Error screening method for visual recognition track and visual recognition method for sales counter | |
| CN109886080A (en) | Human face in-vivo detection method, device, electronic equipment and readable storage medium storing program for executing | |
| CN109377601B (en) | Intelligent office access control system based on fingerprint identification | |
| CN116129350B (en) | Intelligent monitoring method, device, equipment and medium for safety operation of data center | |
| CN111488798B (en) | Fingerprint identification method, fingerprint identification device, electronic equipment and storage medium | |
| JP7030092B2 (en) | Information generation method and equipment and equipment for human-computer interaction | |
| CN110175553B (en) | Method and device for establishing feature library based on gait recognition and face recognition | |
| US11663805B1 (en) | Utilizing sensor data for automated user identification | |
| WO2020205981A1 (en) | Adaptive learning and matching of face modalities | |
| CN113837006A (en) | Face recognition method and device, storage medium and electronic equipment | |
| CN112861742A (en) | Face recognition method and device, electronic equipment and storage medium | |
| CN111738059A (en) | Non-sensory scene-oriented face recognition method | |
| Cootes et al. | Locating objects of varying shape using statistical feature detectors | |
| CN114266267A (en) | Automatic identification method and device for collecting two-dimensional codes, documents, certificates and human faces and storage medium | |
| US20210166041A1 (en) | System and method for tracking customer movements in a customer service environment | |
| CN115497146B (en) | Model training method and device and identity verification method and device | |
| CN115375333B (en) | Property lease tracing system and method based on blockchain |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |