CN114612827A - Commodity identification method, commodity identification device and storage medium - Google Patents
Commodity identification method, commodity identification device and storage medium Download PDFInfo
- Publication number
- CN114612827A CN114612827A CN202210238271.5A CN202210238271A CN114612827A CN 114612827 A CN114612827 A CN 114612827A CN 202210238271 A CN202210238271 A CN 202210238271A CN 114612827 A CN114612827 A CN 114612827A
- Authority
- CN
- China
- Prior art keywords
- commodity
- image
- visible light
- light image
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 67
- 230000008447 perception Effects 0.000 claims description 20
- 238000001514 detection method Methods 0.000 claims description 18
- 230000006870 function Effects 0.000 claims description 10
- 238000013136 deep learning model Methods 0.000 claims description 9
- 230000002194 synthesizing effect Effects 0.000 claims description 3
- 238000012216 screening Methods 0.000 claims description 2
- 238000012545 processing Methods 0.000 abstract description 4
- 238000005303 weighing Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000004590 computer program Methods 0.000 description 5
- 230000006399 behavior Effects 0.000 description 4
- 230000003068 static effect Effects 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 235000013361 beverage Nutrition 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30241—Trajectory
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of image processing, in particular to a commodity identification method, a commodity identification device and a storage medium, and aims to solve the problem that the prior art cannot realize the identification of different types of commodities which are mixed together. To this end, the commodity identification method of the present invention comprises the steps of: acquiring video information and weight information of a commodity; determining a visible light image for each of the items based on the video information; the articles are identified based on the visible light image and the weight information of each article. Therefore, the commodity identification method realizes the commodity identification of different types of commodities which are mixed together, and improves the accuracy of the commodity identification.
Description
Technical Field
The invention relates to the technical field of image processing, and particularly provides a commodity identification method, a commodity identification device and a storage medium.
Background
With the development of artificial intelligence technology, the application of commodity detection and identification technology in intelligent retail is more and more extensive. The current solutions for detecting and identifying commodities are mainly classified into four categories: RFID, weighing, static vision, and dynamic vision recognition methods. However, these methods have certain disadvantages, and RFID has the disadvantages of requiring a lot of labor and being easy to fall off in some scenes. The weighing method cannot perform class discrimination on objects with the same weight. Static vision requires that objects be placed regularly, cannot be stacked, and the operation cost is higher. The dynamic vision is difficult to radically treat the condition that small objects are easily shielded, and has certain requirements on the size of an operated product. Meanwhile, when the four methods are applied to self-service vending cabinets to detect and identify commodities, the situation that different types of commodities are mixed and placed together cannot be realized, and the single situation of vending products cannot meet the user requirements.
Accordingly, there is a need in the art for a new article identification solution that addresses the above-mentioned problems.
Disclosure of Invention
The present invention has been developed in order to overcome the above-mentioned drawbacks, and to provide a solution or at least a partial solution to the problem of the prior art that does not allow identification of different types of goods placed together in a mixed manner. The invention provides a commodity identification method, a commodity identification device and a storage medium.
In a first aspect, the present invention provides a method of identifying an article, comprising the steps of: acquiring video information and weight information of a commodity; determining a visible light image for each commodity based on the video information; identifying the goods based on the visible light image and the weight information of each goods.
In one embodiment, determining a visible light image for each of the articles based on the video information comprises: determining the commodity position of each video frame image in the video information based on the video information; acquiring a commodity single picture of each commodity from the video frame image based on the commodity position; and adjusting the size of the commodity single picture to a preset size to obtain a visible light image of each commodity.
In one embodiment, the identifying includes identifying a quantity of the item and a type of the item; the weight information is weight information corresponding to each commodity; identifying the goods based on the visible light image and the weight information of each of the goods comprises: inputting the visible light image and the weight information of each commodity into a detection network model to obtain the commodity number of each commodity; inputting the visible light image and the weight information of each commodity into a feature recognition deep learning model to obtain the commodity type of each commodity; the detection network model is a fast-rcnn network; the feature recognition deep learning model is a VGG network or a lightweight network.
In one embodiment, further comprising: after video information and weight information of a commodity are obtained, aligning video frame images in the video information to obtain aligned images; determining a visible light image for each item based on the alignment image; identifying the goods based on the visible light image and the weight information of each goods.
In one embodiment, performing an alignment operation on a video frame image in the video information to obtain an aligned image includes: screening out a reference image from the video frame image, and taking other video frame images as perception images; carrying out feature detection on the reference image to obtain a reference feature, and carrying out feature detection on the perception image to obtain a perception feature; matching the reference features and the perception features to establish feature corresponding functions; aligning the perception images by adopting the feature corresponding function; and synthesizing an aligned image by using the aligned perception image and the reference image.
In one embodiment, further comprising: acquiring the motion track of each commodity from the video information; determining whether each commodity is purchased for the user based on the motion trail; and under the condition that the commodities are purchased by the user, judging whether the quantity of the commodities purchased by the user is larger than the preset quantity of the purchased commodities, and if so, sending early warning information to a third-party platform.
In one embodiment, further comprising: and respectively counting the number of sold commodities and the number of residual commodities, and displaying the number of sold commodities and the number of residual commodities.
In a second aspect, the present invention provides an article identification device comprising: an acquisition module configured to acquire video information and weight information of a commodity; a determination module configured to determine a visible light image for each item based on the video information; an identification module configured to identify the goods based on the visible light image and the weight information of each of the goods.
In a third aspect, an electronic device is provided, comprising a processor and a storage means adapted to store a plurality of program codes adapted to be loaded and run by the processor to perform the article identification method of any of the preceding claims.
In a fourth aspect, a computer readable storage medium is provided, having stored therein a plurality of program codes adapted to be loaded and run by a processor to perform the article identification method of any of the preceding claims.
One or more technical schemes of the invention at least have one or more of the following beneficial effects:
the invention provides a commodity identification method, which comprises the steps of firstly obtaining video information and weight information of commodities, then determining a visible light image of each commodity based on the video information, and finally identifying the quantity and types of the commodities based on the visible light image and the weight information of each commodity.
After the video information and the weight information of the commodity are obtained, the video frame images in the video information are aligned, so that the quality of the video frame images is improved, and noise interference in the process of determining the quantity and the type of the commodity by utilizing the video frame images at the later stage is reduced.
Each commodity is determined to be purchased by the user according to the movement track of the commodity, and the early warning information is sent to the third-party platform under the condition that the number of the commodities purchased by the user is larger than the preset number of the purchased commodities, so that manual intervention is realized, the loss caused when the purchasing behavior of the user is illegal or has greater risk is avoided, and the safety performance of the sales counter is improved.
Drawings
The disclosure of the present invention will become more readily understood with reference to the accompanying drawings. As is readily understood by those skilled in the art: these drawings are for illustrative purposes only and are not intended to constitute a limitation on the scope of the present invention. Moreover, in the drawings, like numerals are used to indicate like parts, and in which:
FIG. 1 is a flow chart illustrating the main steps of a merchandise identification method according to an embodiment of the present invention;
FIG. 2 is a side view of a sales counter according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of an overall purchase flow according to one embodiment of the present invention;
FIG. 4 is a schematic diagram of a model training flow according to one embodiment of the present invention;
fig. 5 is a block diagram illustrating a main structure of an article recognition apparatus according to an embodiment of the present invention.
List of reference numerals：
11: an acquisition module; 12: a determination module; 13: and identifying the module.
Detailed Description
Some embodiments of the invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only for explaining the technical principle of the present invention, and are not intended to limit the scope of the present invention.
In the description of the present invention, a "module" or "processor" may include hardware, software, or a combination of both. A module may comprise hardware circuitry, various suitable sensors, communication ports, memory, may comprise software components such as program code, or may be a combination of software and hardware. The processor may be a central processing unit, microprocessor, image processor, digital signal processor, or any other suitable processor. The processor has data and/or signal processing functionality. The processor may be implemented in software, hardware, or a combination thereof. Non-transitory computer readable storage media include any suitable medium that can store program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random-access memory, and the like. The term "a and/or B" denotes all possible combinations of a and B, such as a alone, B alone or a and B. The term "at least one A or B" or "at least one of A and B" means similar to "A and/or B" and may include only A, only B, or both A and B. The singular forms "a", "an" and "the" may include the plural forms as well.
At present, the traditional commodity detection and identification methods are mainly divided into four categories: RFID, weighing, static vision, and dynamic vision recognition methods. When the four methods are applied to a self-service vending machine to detect and identify commodities, the situation that different types of commodities are mixed and placed together cannot be realized, and the situation that the sold product is single cannot meet the user requirement. Therefore, the commodity identification method, the commodity identification device and the storage medium are mainly applied to unmanned sales cabinets, firstly video information and weight information of commodities are obtained, then the visible light image of each commodity is determined based on the video information, and finally the quantity and the types of the commodities are identified based on the visible light image and the weight information of each commodity.
Referring to fig. 1, fig. 1 is a flow chart illustrating main steps of a product identification method according to an embodiment of the present invention. As shown in fig. 1, the method for identifying a commodity in the embodiment of the present invention mainly includes the following steps S101 to S103.
Step S101: and acquiring video information and weight information of the commodity, wherein the video information is the video information of the commodity from opening the cabinet door of the sales counter to closing the cabinet door of the sales counter. In practical applications, the video information of the commodity may be obtained by at least one camera installed at a corresponding position of the sales counter, and specifically, as shown in the side view of the sales counter in fig. 2, the positions of the cameras installed in the sales counter may be six positions c1 to c6, but are not limited thereto, and only any one of c1 or c2 may be selected. Wherein c1 is the rear upper position of the sales counter, c2 is the position right above the sales counter, c3 is the front position of the first floor of the sales counter from top to bottom, c4 is the rear position of the first floor of the sales counter from top to bottom, c5 is the rear position of the last floor of the sales counter from top to bottom, and c6 is the front position of the last floor of the sales counter from top to bottom. The weight information is weight information corresponding to each commodity, the weight information of each layer of commodity is firstly obtained by measuring a weighing scale arranged on each layer of a sales counter, the measured weight information of each layer of commodity is compared with the original weight information of each layer, so that a commodity weight difference value of each layer is obtained, and finally the commodity weight difference value is matched with the weight information of each commodity stored in a database, so that the weight information corresponding to each commodity is obtained, wherein the positions of the weighing scales can be five positions d to h as shown in fig. 2. When the sales counter is a double-door counter, the installation position of the corresponding camera and the number of the weighing scales can be increased in proportion. Of course, the installation position of the camera may be other positions capable of collecting video information of the commodity, which is not described herein again. Based on the installed camera and the set weighing scale, the video of the loading process or the purchasing process of the sales counter during the whole door opening period and the weight information of each layer of the sales counter can be collected. In addition, the collected video information and the weight information can be preprocessed by adopting a conventional preprocessing method, for example, the abnormal condition of the color of the commodity caused by overexposure or excessively dark video image frames is optimized, and the weight information can be denoised to reduce the influence of abnormal data.
In order to bring good experience to the user, the sales counter in the application can also realize active voice reminding without being limited to the following situations, such as reminding the user of network disconnection, reminding the user of door lock opening failure, reminding the user of not forgetting articles in non-sales counter in the counter, reminding the user of not shielding the camera intentionally, reminding the user of how to contact with the operator, and the like. Meanwhile, the user can actively open the voice call with the background customer service through the APP or the application program to obtain help.
Step S102: a visible light image for each of the articles is determined based on the video information. Specifically, in the process of determining the visible light image of each commodity based on the video information, the commodity position of each video frame image in the video information may be determined based on the video information, and specifically, a commodity detection model may be used to obtain the commodity position in each video frame image, where the commodity detection model may be implemented by a neural network model. And then, acquiring a commodity single image of each commodity from the video frame image based on the commodity position, namely, extracting the commodity image at the corresponding position from each video frame image according to the commodity position in the video frame image, and finally, adjusting the size of the commodity single image to a preset size, wherein the preset size can be a preset fixed size, so as to obtain a visible light image of each commodity.
Step S103: the articles are identified based on the visible light image and the weight information of each article. Specifically, the weight information is weight information corresponding to each commodity, the step is mainly to identify the quantity and the type of the commodity, and in the process of identifying the quantity and the type of the commodity based on the visible light image and the weight information of each commodity, the visible light image and the weight information of each commodity can be sequentially detected by the network model to obtain the quantity of the commodity of each commodity, and the detection network model here can be a fast-rcnn network or a yolo series network model, but is not limited thereto. Similarly, the visible light image and the weight information of each commodity can be input into a feature recognition deep learning model to obtain the commodity type of each commodity, where the feature recognition deep learning model may be, but is not limited to, a VGG network or a lightweight (shufflenet) network.
Based on the steps S101 to S103, firstly, video information and weight information of the commodities are obtained, then, the visible light image of each commodity is determined based on the video information, and finally, the quantity and types of the commodities are identified based on the visible light image and the weight information of each commodity.
In one embodiment, the article identification method further comprises: after the video information and the weight information of the commodity are obtained, the video frame images in the video information can be aligned to obtain aligned images. Specifically, in the process of performing the alignment operation, a reference image may be screened from video frame images, and other video frame images are used as perceptual images, then feature detection is performed on the reference image to obtain a reference feature, feature detection is performed on the perceptual images to obtain a perceptual feature, then the reference feature and the perceptual feature are matched one by one to establish a feature correspondence function, the perceptual images are aligned by using the feature correspondence function, and finally the aligned perceptual images and the reference images are used to synthesize the aligned image. Data alignment refers to a process of overlapping multiple images of the same scene acquired at different time and different view angles as much as possible according to a certain rule. In this embodiment, data alignment may be achieved according to a matching algorithm of local features or an image edge-based registration method, etc.
Illustratively, 10 frames of video frame images are taken as an example for explanation, firstly, a clearest image in one frame is selected from the 10 frames of video frame images as a reference image, the rest other video frame images are taken as perception images, then feature detection is respectively carried out on the reference image and the perception images, specifically, salient objects of the images, such as edge contours, line intersections, angles and the like, are detected, the features can be represented by points, such as gravity centers, connection points, angles and the like, reference features can be obtained by carrying out feature detection on the reference image, and the perception features can be obtained by carrying out feature detection on the perception images. And secondly, enabling the reference features and the perception features to be in one-to-one correspondence to establish a feature correspondence function. After the feature corresponding function is obtained, the feature corresponding function is used for converting the perception image, specifically, a nearest neighbor method or bilinear interpolation is adopted to process the perception image, and finally, the aligned perception image and the reference image are used for synthesizing the aligned image. After the video information and the weight information of the commodity are obtained, the video frame images in the video information are aligned, so that the quality of the video frame images is improved, and noise interference in the process of determining the quantity and the type of the commodity by utilizing the video frame images at the later stage is reduced. After obtaining the alignment image, a visible light image of each article may be further determined based on the alignment image, and finally the number of articles and the article type of each article may be determined based on the visible light image and the weight information of each article. Specifically, the process of determining the visible light image of each commodity based on the alignment image is similar to the process of determining the visible light image of each commodity based on the video information, and the process of determining the visible light image of each commodity based on the alignment image may refer to the process of determining the visible light image of each commodity based on the video information, which is not described herein again.
In one embodiment, the article identification method further comprises: the motion trail of each commodity is obtained from the video information, and specifically, the motion trail of each commodity can be obtained from the video information through a Kalman filtering algorithm. Then, whether each commodity is a commodity purchased by the user can be determined based on the motion track, and for example, if the motion track of a certain commodity moves from a certain layer of corresponding position of a sales counter to the outside of the sales counter, the commodity can be determined as the commodity purchased by the user, and if the motion track of a certain commodity moves from the outside of the sales counter to a certain layer of corresponding position of the sales counter, the commodity can be determined as a replenishment commodity or a commodity returned by the user when the commodity is purchased. And under the condition that the commodities are purchased by the user, judging whether the quantity of the commodities purchased by the user is larger than the preset quantity of the purchased commodities, if so, sending early warning information to a third-party platform. For example, when a user purchases dozens of beverages at one time, the number of the beverages purchased by the user is larger than the preset number of purchased commodities, the system can judge that the purchasing behavior of the user is illegal or has a large risk, and manual intervention is realized by sending early warning information to a third party, wherein the third party can be a terminal where a person in charge of a sales counter is located. Meanwhile, when the number of the commodities purchased by the user is judged to be not more than the preset number of the purchased commodities, the current purchasing behavior of the user is reasonable purchasing, and then the user enters a normal settlement state. In addition, the collected video frame images in the purchase process of the user can be intelligently cut and uniformly coded so as to transmit the information to a settlement party or a manual customer service.
Each commodity is determined to be purchased by the user through the movement track of the commodity, and the early warning information is sent to the third-party platform under the condition that the number of the commodities purchased by the user is larger than the preset number of the purchased commodities, so that manual intervention is realized, the loss caused when the purchasing behavior of the user is illegal or has greater risk is avoided, and the safety performance of the sales counter is improved.
In one embodiment, the article identification method further comprises: and respectively counting the number of sold commodities and the number of residual commodities, and displaying the number of sold commodities and the number of residual commodities on a main interface of a sales counter. Generally, the conventional vending machine display case only displays the schematic diagram of the sold goods and whether the goods are available, and the actual sold goods and the remaining amount of the goods cannot be known by consumers. In addition, the commodity is mainly supplemented at regular time by manpower, the commodity is difficult to be supplemented in time by hot sales, and the commodity preference of consumers cannot be accurately known. Therefore, the number of sold commodities and the number of residual commodities are updated in real time, and the counted number of sold commodities and the counted number of residual commodities are displayed on the main interface of the sales counter so that a user can know the actual situation, and the user requirements are met. In addition, the method and the system can count the quantity of hot-sold and lost-sold commodities in real time and send the quantities to the third-party platform so that the goods supplier can know the preference of the consumer in real time and supply goods in time. In one embodiment, as shown in the overall purchase flow diagram of fig. 3, after the user completes the settlement, the background may count the number of sold goods, the number of remaining goods, the hot sales, the number of lost sales, and the like for displaying or enabling the supplier to know the consumer's preference in real time and supply goods in time.
In one embodiment, as shown in fig. 4, in the training process of the feature recognition deep learning model, the visible light image and the weight information for training are mainly input into the feature recognition deep learning model, the model determines the model parameters by comparing the input information with the preset features, and when the matching degree between the commodity type output by using the model and the label information thereof is high, the trained feature recognition deep learning model is obtained.
It should be noted that, although the foregoing embodiments describe each step in a specific sequence, those skilled in the art will understand that, in order to achieve the effect of the present invention, different steps do not necessarily need to be executed in such a sequence, and they may be executed simultaneously (in parallel) or in other sequences, and these changes are all within the protection scope of the present invention.
Furthermore, the invention also provides a commodity identification device. Referring to fig. 5, fig. 5 is a main structural block diagram of an article recognition apparatus according to an embodiment of the present invention. As shown in fig. 5, the product identification apparatus in the embodiment of the present invention mainly includes an obtaining module 11, a determining module 12, and an identifying module 13. In some embodiments, one or more of the acquisition module 11, the determination module 12, and the identification module 13 may be combined together into one module. In some embodiments, the obtaining module 11 may be configured to obtain video information and weight information of the goods. The determination module 12 may be configured to determine a visible light image for each of the articles based on the video information. The identification module 13 may be configured to identify the articles based on the visible light image and the weight information of each article. In one embodiment, the description of the specific implementation function may refer to the description of step S101 to step S103.
The above-mentioned commodity identification apparatus is used for implementing the embodiment of the commodity identification method shown in fig. 1, and the technical principles, the technical problems to be solved, and the technical effects produced by the two are similar, and it can be clearly understood by those skilled in the art that, for convenience and simplicity of description, the specific working process and related description of the commodity identification apparatus may refer to the content described in the embodiment of the commodity identification method, and are not repeated herein.
It will be understood by those skilled in the art that all or part of the flow of the method according to the above-described embodiment may be implemented by a computer program, which may be stored in a computer-readable storage medium and used to implement the steps of the above-described embodiments of the method when the computer program is executed by a processor. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable storage medium may include: any entity or device capable of carrying said computer program code, media, usb disk, removable hard disk, magnetic diskette, optical disk, computer memory, read-only memory, random access memory, electrical carrier wave signals, telecommunication signals, software distribution media, etc. It should be noted that the computer readable storage medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable storage media that does not include electrical carrier signals and telecommunications signals in accordance with legislation and patent practice.
Furthermore, the invention also provides electronic equipment. In an embodiment of the electronic device according to the invention, the electronic device comprises a processor and a storage, the storage may be configured to store a program for performing the article identification method of the above-mentioned method embodiment, and the processor may be configured to execute the program in the storage, the program including but not limited to the program for performing the article identification method of the above-mentioned method embodiment. For convenience of explanation, only the parts related to the embodiments of the present invention are shown, and details of the specific techniques are not disclosed.
Further, the invention also provides a computer readable storage medium. In one computer-readable storage medium embodiment according to the present invention, a computer-readable storage medium may be configured to store a program that executes the article identification method of the above-described method embodiment, and the program may be loaded and executed by a processor to implement the above-described article identification method. For convenience of explanation, only the parts related to the embodiments of the present invention are shown, and details of the specific techniques are not disclosed. The computer-readable storage medium may be a storage device formed by including various electronic devices, and optionally, the computer-readable storage medium is a non-transitory computer-readable storage medium in an embodiment of the present invention.
Further, it should be understood that, since the configuration of each module is only for explaining the functional units of the apparatus of the present invention, the corresponding physical devices of the modules may be the processor itself, or a part of software, a part of hardware, or a part of a combination of software and hardware in the processor. Thus, the number of individual modules in the figures is merely illustrative.
Those skilled in the art will appreciate that the various modules in the apparatus may be adaptively split or combined. Such splitting or combining of specific modules does not cause the technical solutions to deviate from the principle of the present invention, and therefore, the technical solutions after splitting or combining will fall within the protection scope of the present invention.
So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of the present invention is obviously not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.
Claims (10)
1. A method of identifying a commodity, comprising the steps of:
acquiring video information and weight information of a commodity;
determining a visible light image for each commodity based on the video information;
identifying the goods based on the visible light image and the weight information of each goods.
2. The article identification method of claim 1, wherein determining a visible light image for each article based on the video information comprises:
determining the commodity position of each video frame image in the video information based on the video information;
acquiring a commodity single picture of each commodity from the video frame image based on the commodity position;
and adjusting the size of the commodity single picture to a preset size to obtain a visible light image of each commodity.
3. The article identification method according to claim 1, wherein the identifying includes identifying an article number and an article type; the weight information is weight information corresponding to each commodity; identifying the goods based on the visible light image and the weight information of each of the goods comprises:
inputting the visible light image and the weight information of each commodity into a detection network model to obtain the commodity number of each commodity;
inputting the visible light image and the weight information of each commodity into a feature recognition deep learning model to obtain the commodity type of each commodity; the detection network model is a fast-rcnn network; the feature recognition deep learning model is a VGG network or a lightweight network.
4. The article identification method according to claim 1, further comprising:
after video information and weight information of a commodity are obtained, aligning video frame images in the video information to obtain aligned images;
determining a visible light image for each item based on the alignment image;
identifying the goods based on the visible light image and the weight information of each goods.
5. The product identification method according to claim 4, wherein performing an alignment operation on the video frame images in the video information to obtain an aligned image comprises:
screening out a reference image from the video frame image, and taking other video frame images as perception images;
carrying out feature detection on the reference image to obtain a reference feature, and carrying out feature detection on the perception image to obtain a perception feature;
matching the reference features and the perception features to establish feature corresponding functions;
aligning the perception images by adopting the feature corresponding function;
and synthesizing an aligned image by using the aligned perception image and the reference image.
6. The article identification method according to claim 1, further comprising:
acquiring the motion track of each commodity from the video information;
determining whether each commodity is purchased for the user based on the motion trail;
and under the condition that the commodities are purchased by the user, judging whether the quantity of the commodities purchased by the user is larger than the preset quantity of the purchased commodities, and if so, sending early warning information to a third-party platform.
7. The article identification method according to claim 1, further comprising: and respectively counting the number of sold commodities and the number of residual commodities, and displaying the number of sold commodities and the number of residual commodities.
8. An article identification device, comprising:
an acquisition module configured to acquire video information and weight information of a commodity;
a determination module configured to determine a visible light image for each item based on the video information;
an identification module configured to identify the goods based on the visible light image and the weight information of each of the goods.
9. An electronic device comprising a processor and a storage means adapted to store a plurality of program codes, wherein said program codes are adapted to be loaded and run by said processor to perform the method of article identification according to any of claims 1 to 7.
10. A computer-readable storage medium having stored therein a plurality of program codes, characterized in that the program codes are adapted to be loaded and executed by a processor to perform the article identification method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210238271.5A CN114612827A (en) | 2022-03-11 | 2022-03-11 | Commodity identification method, commodity identification device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210238271.5A CN114612827A (en) | 2022-03-11 | 2022-03-11 | Commodity identification method, commodity identification device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114612827A true CN114612827A (en) | 2022-06-10 |
Family
ID=81863745
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210238271.5A Pending CN114612827A (en) | 2022-03-11 | 2022-03-11 | Commodity identification method, commodity identification device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114612827A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107205119A (en) * | 2017-06-30 | 2017-09-26 | 维沃移动通信有限公司 | A kind for the treatment of method and apparatus of view data |
| CN108875664A (en) * | 2018-06-27 | 2018-11-23 | 北京京东尚科信息技术有限公司 | Recognition methods, device and the vending machine of selective purchase |
| CN110197561A (en) * | 2019-06-10 | 2019-09-03 | 北京华捷艾米科技有限公司 | A kind of commodity recognition method, apparatus and system |
| CN113743530A (en) * | 2021-09-16 | 2021-12-03 | 广东佩服科技有限公司 | Automatic vending identification method based on dynamic vision |
| CN113780248A (en) * | 2021-11-09 | 2021-12-10 | 武汉星巡智能科技有限公司 | Multi-view-angle identification commodity intelligent order generation method and device and intelligent vending machine |
-
2022
- 2022-03-11 CN CN202210238271.5A patent/CN114612827A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107205119A (en) * | 2017-06-30 | 2017-09-26 | 维沃移动通信有限公司 | A kind for the treatment of method and apparatus of view data |
| CN108875664A (en) * | 2018-06-27 | 2018-11-23 | 北京京东尚科信息技术有限公司 | Recognition methods, device and the vending machine of selective purchase |
| CN110197561A (en) * | 2019-06-10 | 2019-09-03 | 北京华捷艾米科技有限公司 | A kind of commodity recognition method, apparatus and system |
| CN113743530A (en) * | 2021-09-16 | 2021-12-03 | 广东佩服科技有限公司 | Automatic vending identification method based on dynamic vision |
| CN113780248A (en) * | 2021-11-09 | 2021-12-10 | 武汉星巡智能科技有限公司 | Multi-view-angle identification commodity intelligent order generation method and device and intelligent vending machine |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108922026B (en) | Replenishment management method and device for vending machine and user terminal | |
| CN110866429B (en) | Missing scanning identification method, device, self-service cashing terminal and system | |
| CN108335408B (en) | Article identification method, device and system for vending machine and storage medium | |
| CN108921645B (en) | Commodity purchase judgment method and device and user terminal | |
| US8746557B2 (en) | Secure self-checkout | |
| CN108320404A (en) | Commodity recognition method, device, self-service cashier based on neural network | |
| CN108549851A (en) | Method and device for identifying goods in intelligent container and intelligent container | |
| CN109726759B (en) | Unmanned vending method, device, system, electronic equipment and computer readable medium | |
| EP3355286A1 (en) | Remote weigh station with delayed fraud intervention | |
| KR101960900B1 (en) | Method for recognizing products | |
| CN112466035B (en) | Commodity identification method, device and system based on vision and gravity sensing | |
| JP6535775B2 (en) | Image data acquisition processing method and system | |
| CN108269371A (en) | Commodity automatic settlement method, device, self-service cashier | |
| CN105608808A (en) | Information processing apparatus, store system and method | |
| US20180039841A1 (en) | Object Recognition for Bottom of Basket Detection | |
| CN114898249B (en) | Method, system and storage medium for confirming number of articles in shopping cart | |
| CN111310733A (en) | Method, device and equipment for detecting personnel entering and exiting based on monitoring video | |
| CN109003403B (en) | Shopping cart commodity weighing and pricing method, device and system | |
| CN113888254A (en) | Shelf commodity management method and electronic equipment | |
| CN110837824B (en) | Commodity identification method for vending device, vending device and storage medium | |
| CN113468914A (en) | Method, device and equipment for determining purity of commodities | |
| CN110705620A (en) | Display image detection method and device based on copying recognition and storage medium | |
| CN109300265A (en) | Unmanned Supermarket Management System | |
| CN110781780B (en) | Vacancy detection method and related device | |
| CN114612827A (en) | Commodity identification method, commodity identification device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |