CN111797896A - Commodity identification method and device based on intelligent baking - Google Patents
Commodity identification method and device based on intelligent baking Download PDFInfo
- Publication number
- CN111797896A CN111797896A CN202010483443.6A CN202010483443A CN111797896A CN 111797896 A CN111797896 A CN 111797896A CN 202010483443 A CN202010483443 A CN 202010483443A CN 111797896 A CN111797896 A CN 111797896A
- Authority
- CN
- China
- Prior art keywords
- image
- preset
- commodity
- circumscribed rectangle
- area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30204—Marker
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Abstract
The invention discloses a commodity identification method and a device based on intelligent baking, wherein the method is applied to a server connected with at least one client through a cloud, and comprises the following steps: receiving a first image of a commodity to be identified uploaded by a client; determining whether the first image has commodity overlapping by using a preset classification model, wherein the classification model is obtained by deep learning through a convolutional neural network; detecting the first image by using a preset anomaly detection algorithm, and determining whether the first image meets an identification condition according to a detection result; and if the first image has commodity overlapping or does not meet the identification condition, sending an error prompt to the client. The embodiment of the invention can solve the problems of low commodity identification efficiency and high cost of baking shops in the prior art.
Description
Technical Field
The invention relates to the technical field of image recognition, in particular to a commodity recognition method and device based on intelligent baking.
Background
Conventional bakery goods such as bread, especially freshly baked bread, are not packaged and price tagged and therefore cannot be identified by scanning a bar code as is the case with conventional self-service cash. In the settlement and payment collection process, a cashier is required to calculate the price of the bread cake, but the premise is that the clerk is required to remember the price of each commodity, and the requirements and training cost for the clerk are high.
Particularly, when the customer selects the baked goods such as bread and the like and needs to settle accounts, the speed of manually calculating the price of the bread cake is low, mistakes are easy to occur, and the problem that the customer queues up for a long time easily occurs at the peak consumption time; the human cost is higher and efficient.
Disclosure of Invention
The embodiment of the invention provides a commodity identification method and device based on intelligent baking, which are used for solving the problems of low commodity identification efficiency and high cost of baking shops in the prior art.
The embodiment of the invention provides a commodity identification method based on intelligent baking, which is applied to a server connected with at least one client through a cloud, and comprises the following steps:
receiving a first image of a commodity to be identified uploaded by a client;
determining whether the first image has commodity overlapping by using a preset classification model, wherein the classification model is obtained by deep learning through a convolutional neural network; and
carrying out image detection on the first image by using a preset anomaly detection algorithm, and determining whether the first image meets an identification condition according to a detection result;
and if the first image has commodity overlapping or does not meet the identification condition, sending an error prompt to the client.
Wherein the determining whether the first image has the commodity overlap by using a preset classification model comprises:
the classification model classifies the first image according to a preset class library; the preset class library comprises an overlapped class library and a non-overlapped class library;
and when the first image belongs to the overlapping class library, determining that the first image has commodity overlapping.
Wherein, carry out deep learning through convolutional neural network and obtain the classification model, include:
performing two-layer convolution processing on each training image in the training set to obtain a convolution processing result;
performing maximum pooling on the convolution processing result to obtain a pooling processing result;
regularizing the pooling processing result and outputting a classification result;
comparing the classification result with the class of the training image to determine the classification accuracy of the training;
and when the classification accuracy is smaller than the preset accuracy threshold, performing the circular training again until the classification accuracy reaches the preset accuracy threshold.
The image detection of the first image by using a preset anomaly detection algorithm includes:
determining contour information of an object in the first image;
determining the minimum circumscribed rectangle of each outline according to the outline information;
deleting the minimum circumscribed rectangle with the area smaller than a preset first area threshold value to obtain a candidate minimum circumscribed rectangle;
traversing and calculating the area of the candidate minimum circumscribed rectangle, and determining the candidate minimum circumscribed rectangle with the largest area as a target circumscribed rectangle;
when the area of the target circumscribed rectangle is larger than the preset visible area, the detection result is abnormal;
when the area of the target circumscribed rectangle is not larger than a preset visible area, calculating the absolute value of the difference value between the area of the target circumscribed rectangle and the preset visible area;
when the absolute value of the difference value is larger than a preset second area threshold value, the detection result is abnormal;
correspondingly, the determining whether the first image meets the identification condition according to the detection result includes:
and when the detection result is abnormal, determining that the first image does not meet the identification condition.
Further, the method further comprises:
if the first image has no commodity overlapping and meets the identification condition, carrying out commodity identification on the first image through a preset image identification model, and sending an identification result to the client;
correspondingly, when the new supplier variety class is added, the method further comprises the following steps:
receiving a new commodity sample image uploaded by a client;
marking the new commodity sample image by using a preset automatic marking algorithm;
and training the image recognition model by using the marked new commodity sample image to complete the addition of the new commodity sample.
Wherein, the labeling of the new commodity sample image by using a preset automatic labeling algorithm comprises:
determining contour information of an object in the new commodity sample image;
determining the minimum circumscribed rectangle of each outline according to the outline information;
deleting the minimum circumscribed rectangle with the area smaller than the area threshold of the new commodity to obtain a candidate circumscribed rectangle;
and if the candidate external rectangles are multiple, merging the candidate external rectangles to obtain a sample labeling rectangle.
The embodiment of the invention also provides a commodity identification device based on intelligent baking, which is applied to a server connected with at least one client through a cloud, and comprises the following components: the device comprises a receiving unit, a first detection unit, a second detection unit and a feedback unit; wherein the content of the first and second substances,
the receiving unit is used for receiving a first image of a commodity to be identified uploaded by a client;
the first detection unit is used for determining whether the first image has commodity overlapping by using a preset classification model, and the classification model is obtained by deep learning through a convolutional neural network; and
the second detection unit is used for carrying out image detection on the first image by using a preset anomaly detection algorithm and determining whether the first image meets an identification condition according to a detection result;
and the feedback unit is used for sending an error prompt to the client if the first image has commodity overlapping or does not meet the identification condition.
The first detection unit determines whether the first image has commodity overlap by using a preset classification model, and is specifically configured to:
classifying the first image according to a preset class library by using the classification model; the preset class library comprises an overlapped class library and a non-overlapped class library; and when the first image belongs to the overlapping class library, determining that the first image has commodity overlapping.
The first detection unit performs deep learning through a convolutional neural network to obtain a classification model, and is specifically configured to:
performing two-layer convolution processing on each training image in the training set to obtain a convolution processing result; performing maximum pooling on the convolution processing result to obtain a pooling processing result; regularizing the pooling processing result and outputting a classification result; comparing the classification result with the class of the training image to determine the classification accuracy of the training; and when the classification accuracy is smaller than the preset accuracy threshold, performing the circular training again until the classification accuracy reaches the preset accuracy threshold.
The second detection unit is configured to perform image detection on the first image by using a preset anomaly detection algorithm, and is specifically configured to:
determining contour information of an object in the first image; determining the minimum circumscribed rectangle of each outline according to the outline information; deleting the minimum circumscribed rectangle with the area smaller than a preset first area threshold value to obtain a candidate minimum circumscribed rectangle; traversing and calculating the area of the candidate minimum circumscribed rectangle, and determining the candidate minimum circumscribed rectangle with the largest area as a target circumscribed rectangle; when the area of the target circumscribed rectangle is larger than the preset visible area, the detection result is abnormal; when the area of the target circumscribed rectangle is not larger than a preset visible area, calculating the absolute value of the difference value between the area of the target circumscribed rectangle and the preset visible area; when the absolute value of the difference value is larger than a preset second area threshold value, the detection result is abnormal;
correspondingly, the second detecting unit determines whether the first image meets the identification condition according to the detection result, and is specifically configured to:
and when the detection result is abnormal, determining that the first image does not meet the identification condition.
Further, the apparatus further comprises: the identification unit is used for identifying the commodities of the first image through a preset image identification model if the first image has no commodity overlapping and meets identification conditions, and sending an identification result to the client;
correspondingly, when the new supplier variety class is added, the identification unit is further configured to: receiving a new commodity sample image uploaded by a client; marking the new commodity sample image by using a preset automatic marking algorithm; and training the image recognition model by using the marked new commodity sample image to complete the addition of the new commodity sample.
The identification unit is used for marking the new commodity sample image by using a preset automatic marking algorithm, and is specifically used for:
determining contour information of an object in the new commodity sample image; determining the minimum circumscribed rectangle of each outline according to the outline information; deleting the minimum circumscribed rectangle with the area smaller than the area threshold of the new commodity to obtain a candidate circumscribed rectangle; and if the candidate external rectangles are multiple, merging the candidate external rectangles to obtain a sample labeling rectangle.
The invention has the following beneficial effects:
according to the commodity identification method and device based on intelligent baking, provided by the embodiment of the invention, a server side connected with at least one client side through a cloud side receives a first image of a commodity to be identified uploaded by the client side; determining whether the first image has commodity overlapping by using a preset classification model, wherein the classification model is obtained by deep learning through a convolutional neural network; detecting the first image by using a preset anomaly detection algorithm, and determining whether the first image meets an identification condition according to a detection result; and if the first image has commodity overlapping or does not meet the identification condition, sending an error prompt to the client. In the embodiment of the invention, the server can be connected with a plurality of clients through the cloud, the layout number of the server can be effectively reduced, the cost is effectively saved, the commodity overlapping is detected through the classification model obtained by the deep learning of the convolutional neural network, the abnormal condition of the image is detected through the abnormal detection algorithm of the image recognition, the problems of overlapping placing, placing and correcting of a placing plate, exceeding of the recognition range of the commodity to be recognized, blocking of obstacles and the like in the commodity recognition process can be effectively solved, the problems are prompted to be corrected, the model can be continuously trained, the commodity recognition efficiency of a baking shop is improved while the recognition accuracy is ensured, and the labor cost is saved.
Drawings
FIG. 1 is a flowchart illustrating a method for identifying smart bake-based merchandise in accordance with an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a product identification device based on smart baking according to an embodiment of the present invention.
Detailed Description
Aiming at the problems of low commodity identification efficiency and high cost of baking shops in the prior art, the commodity identification method based on intelligent baking provided by the embodiment of the invention has the advantages that the cost is saved by the way that a single server corresponds to multiple clients, the server detects the image of the commodity to be identified through the classification model of convolutional neural network deep learning and the anomaly detection algorithm, determines whether the image can correctly identify the commodity according to the detection result, and can send an error prompt to the client for correction when the image cannot be correctly identified. The flow of the method of the invention is shown in figure 1, and the execution steps are as follows:
the client side is provided with a camera and a polishing platform, when a user selects the baked goods and needs to settle accounts, the goods to be identified can be placed on the polishing platform, the client side obtains images of the goods to be identified through the camera, and the obtained images of the goods to be identified are marked as first images for convenient expression; the client is connected with the server through the cloud, and specifically, the first image can be sent to the server through the wireless communication module.
here, when the commodities to be recognized are placed in an overlapping manner, the accuracy of the commodity recognition cannot be guaranteed, and therefore, this situation is an abnormal situation, and the user needs to be prompted to correctly place the commodities to be recognized again.
103, carrying out image detection on the first image by using a preset anomaly detection algorithm, and determining whether the first image meets an identification condition according to a detection result;
this step is mainly to identify common abnormal situations, where the abnormal situations may include, but are not limited to, a camera being blocked, a tray not being correctly placed, baked items being beyond the tray, and so on.
And when the abnormal condition is detected, determining that the first image does not meet the identification condition.
It should be understood that step 102 and step 103 are not performed in strict order of succession.
And 104, if the first image has commodity overlapping or does not meet the identification condition, sending an error prompt to the client.
Optionally, the determining whether there is an overlap of the first image by using a preset classification model in step 102 includes:
the classification model classifies the first image according to a preset class library; the preset class library comprises an overlapped class library and a non-overlapped class library;
here, since the first image is a two-dimensional image and does not have three-dimensional depth information, it is necessary to classify the first image using a classification model obtained by deep learning using a convolutional neural network, and thus it is possible to effectively recognize the product overlap.
And when the first image belongs to the overlapping class library, determining that the first image has commodity overlapping.
Wherein, carry out deep learning through convolutional neural network and obtain the classification model, include:
performing two-layer convolution processing on each training image in the training set to obtain a convolution processing result; performing maximum pooling on the convolution processing result to obtain a pooling processing result; regularizing the pooling processing result and outputting a classification result; comparing the classification result with the class of the training image to determine the classification accuracy of the training; and when the classification accuracy is smaller than the preset accuracy threshold, performing the circular training again until the classification accuracy reaches the preset accuracy threshold. Preferably, a 13-layer deep learning network can be used for model learning, and the specific algorithm implementation operation is as follows:
1) 600 pieces of overlapped bread samples and 600 pieces of non-overlapped bread samples are collected, the resolution is 640 multiplied by 480, wherein 500 pieces are used as a training set, 1000 pieces are collected, the 1000 pieces are divided into two types, an Overlapped (OL) type and a non-overlapped (NoOL) type, pictures are preprocessed, and resize is 224 multiplied by 224 resolution and is used as network input.
2) Bringing the training set into the 13-layer deep learning network to perform overlapNet network training, wherein a shorter training time, such as 5min, can be set according to actual needs due to the fact that the number of layers of the deep learning network is less;
3) and testing after the training is finished until the accuracy of the OverLapNet network on whether the images are overlapped is as high as 95 percent, and finishing the training.
Optionally, the image detection on the first image by using a preset anomaly detection algorithm in step 103 includes:
determining contour information of an object in the first image; there are various ways to obtain the contour information, which is not limited in the embodiments of the present invention, and a simpler and more efficient way is preferred to exemplify: firstly, graying processing can be carried out on the first image, and the first image is converted into a single-channel gray image so as to reduce the calculation amount; the gray value of a pixel point in the image is set to be 0 or 255 through binarization, that is, the whole image presents an obvious black and white effect, and the binarized image only comprises two colors: black and white; and determining the contour information of the object according to the binarization result.
Determining the minimum circumscribed rectangle of each outline according to the outline information;
deleting the minimum circumscribed rectangle with the area smaller than a preset first area threshold value to obtain a candidate minimum circumscribed rectangle; specifically, the preset first area threshold is set according to the smallest commodity in the identifiable commodities, that is, the preset first area threshold is set as the area of the identifiable smallest commodity, and if the area of the smallest circumscribed rectangle is smaller than the area of the identifiable smallest commodity, the smallest circumscribed rectangle can be determined to be an invalid circumscribed rectangle, and a deletion operation can be performed to reduce the subsequent operation amount.
Traversing and calculating the area of the candidate minimum circumscribed rectangle, and determining the candidate minimum circumscribed rectangle with the largest area as a target circumscribed rectangle;
when the area of the target circumscribed rectangle is larger than the preset visible area, the detection result is abnormal; for example, the preset visual area may be set to be the area of a dinner plate, and so on. When the area of the target circumscribed rectangle is larger than the preset visible area, it indicates that the baked goods are beyond the dinner plate.
When the area of the target circumscribed rectangle is not larger than a preset visible area, calculating the absolute value of the difference value between the area of the target circumscribed rectangle and the preset visible area;
when the absolute value of the difference value is larger than a preset second area threshold value, the detection result is abnormal; here, the preset second area threshold may be set as 1/10, 1/15, etc. of the visible area as needed.
Correspondingly, the determining whether the first image meets the identification condition according to the detection result includes:
and when the detection result is abnormal, determining that the first image does not meet the identification condition.
Further, the method further comprises:
if the first image has no commodity overlapping and meets the identification condition, carrying out commodity identification on the first image through a preset image identification model, and sending an identification result to the client;
correspondingly, when the new supplier variety class is added, the method further comprises the following steps:
receiving a new commodity sample image uploaded by a client;
marking the new commodity sample image by using a preset automatic marking algorithm;
and training the image recognition model by using the marked new commodity sample image to complete the addition of the new commodity sample.
Here, the image recognition model selects a deep learning target detection network as a class recognition detection network, which is a YOLOv 3-based detection network, but modifies the backbone network Darknet53 of YOLOv3 to a shallower and effective MobileNet v1 network. The main contribution of MobileNet v1 is to propose a deep separable convolution instead of the normal convolution, the entire network of which is mostly based on deep separable convolution, only the first few layers of the network use normal convolution, the entire network parameters are 2.5 times smaller than google lenet, but the accuracy is higher than google lenet. The DarkNet53 network is composed of 53-layer neural network, the DarkNet53 is replaced by the MobileNet v1 network, the network layer is reduced from 53 layers to 21 layers, the size of the network parameter is greatly reduced, the network model parameter is reduced by more than 50%, and the operation time of a single frame on the server can be reduced to 32ms from 100 ms. In order to shorten the consumed time as better, the MobileNet network can be further subjected to network optimization, the deep learning network model optimization generally adopts network model sparsification and other operations, and structural pruning/sparsification operations can be utilized, namely, a channel is pruned when an image recognition model is trained, and then the precision is recovered through fine tuning; sparsification is introduced by randomly abandoning the connection of the channel-wise, so that a smaller network can be obtained; in the training process, channel-wise sparsization is imposed on an optimized objective function, so that a smoother channel pruning process and smaller precision loss are obtained. Therefore, a simplified and efficient neural network model Sparse-mobilene-YOLOv 3 can be obtained as an image recognition model for commodity recognition, for example, the running time on 1050GPU is reduced from 32ms to 20ms, the recognition rate is not greatly reduced while the network model is reduced, and experiments prove that the recognition rate can be basically kept at about 95%.
Preferably, the labeling the new commodity sample image by using a preset automatic labeling algorithm includes:
determining contour information of an object in the new commodity sample image; there are various ways to obtain the contour information, which is not limited in the embodiments of the present invention, and a simpler and more efficient way is preferred to exemplify: firstly, graying processing can be carried out on the first image, and the first image is converted into a single-channel gray image so as to reduce the calculation amount; the gray value of a pixel point in the image is set to be 0 or 255 through binarization, that is, the whole image presents an obvious black and white effect, and the binarized image only comprises two colors: black and white; and determining the contour information of the object according to the binarization result.
Determining the minimum circumscribed rectangle of each outline according to the outline information;
deleting the minimum circumscribed rectangle with the area smaller than the area threshold of the new commodity to obtain a candidate circumscribed rectangle;
if the candidate external rectangles are multiple, the candidate external rectangles are merged to obtain a sample labeling rectangle, because the types of the newly added commodities are performed one by one, one sample labeling rectangle is needed to be obtained when the sample labeling is performed, if multiple candidate external rectangles exist, the multiple candidate external rectangles still need to be merged to obtain a final sample labeling rectangle, specifically, the non-maximum suppression algorithm can be used for merging, other merging processing modes in the prior art can be used, and the embodiment of the invention does not limit the merging processing modes.
Based on the same inventive concept, an embodiment of the present invention provides a product identification device based on smart baking, which can be applied to a server connected to at least one client through a cloud, and the structure of the device is shown in fig. 2, and includes: a receiving unit 21, a first detecting unit 22, a second detecting unit 23, and a feedback unit 24; wherein the content of the first and second substances,
the receiving unit 21 is configured to receive a first image of a commodity to be identified, which is uploaded by a client;
the first detection unit 22 is configured to determine whether the first image has commodity overlap by using a preset classification model, where the classification model is obtained by performing deep learning through a convolutional neural network; and
the second detecting unit 23 is configured to perform image detection on the first image by using a preset anomaly detection algorithm, and determine whether the first image meets an identification condition according to a detection result;
the feedback unit 24 is configured to send an error prompt to the client if the first image has a product overlap or does not satisfy the identification condition.
The first detecting unit 22 determines whether there is a product overlap in the first image by using a preset classification model, and is specifically configured to:
classifying the first image according to a preset class library by using the classification model; the preset class library comprises an overlapped class library and a non-overlapped class library; and when the first image belongs to the overlapping class library, determining that the first image has commodity overlapping.
The first detection unit 22 performs deep learning through a convolutional neural network to obtain a classification model, and is specifically configured to:
performing two-layer convolution processing on each training image in the training set to obtain a convolution processing result; performing maximum pooling on the convolution processing result to obtain a pooling processing result; regularizing the pooling processing result and outputting a classification result; comparing the classification result with the class of the training image to determine the classification accuracy of the training; and when the classification accuracy is smaller than the preset accuracy threshold, performing the circular training again until the classification accuracy reaches the preset accuracy threshold.
The second detecting unit 23 performs image detection on the first image by using a preset anomaly detection algorithm, and is specifically configured to:
determining contour information of an object in the first image; determining the minimum circumscribed rectangle of each outline according to the outline information; deleting the minimum circumscribed rectangle with the area smaller than a preset first area threshold value to obtain a candidate minimum circumscribed rectangle; traversing and calculating the area of the candidate minimum circumscribed rectangle, and determining the candidate minimum circumscribed rectangle with the largest area as a target circumscribed rectangle; when the area of the target circumscribed rectangle is larger than the preset visible area, the detection result is abnormal; when the area of the target circumscribed rectangle is not larger than a preset visible area, calculating the absolute value of the difference value between the area of the target circumscribed rectangle and the preset visible area; when the absolute value of the difference value is larger than a preset second area threshold value, the detection result is abnormal;
correspondingly, the second detecting unit 23 determines whether the first image satisfies the identification condition according to the detection result, and is specifically configured to:
and when the detection result is abnormal, determining that the first image does not meet the identification condition.
Further, the apparatus further comprises: the identification unit is used for identifying the commodities of the first image through a preset image identification model if the first image has no commodity overlapping and meets identification conditions, and sending an identification result to the client;
correspondingly, when the new supplier variety class is added, the identification unit is further configured to: receiving a new commodity sample image uploaded by a client; marking the new commodity sample image by using a preset automatic marking algorithm; and training the image recognition model by using the marked new commodity sample image to complete the addition of the new commodity sample.
The identification unit is used for marking the new commodity sample image by using a preset automatic marking algorithm, and is specifically used for:
determining contour information of an object in the new commodity sample image; determining the minimum circumscribed rectangle of each outline according to the outline information; deleting the minimum circumscribed rectangle with the area smaller than the area threshold of the new commodity to obtain a candidate circumscribed rectangle; and if the candidate external rectangles are multiple, merging the candidate external rectangles to obtain a sample labeling rectangle.
It should be understood that the implementation principle and process of the product identification device based on intelligent baking according to the embodiment of the present invention are similar to those of the above-mentioned embodiment shown in fig. 1, and will not be described herein again.
According to the commodity identification method and device based on intelligent baking, provided by the embodiment of the invention, a server side connected with at least one client side through a cloud side receives a first image of a commodity to be identified uploaded by the client side; determining whether the first image has commodity overlapping by using a preset classification model, wherein the classification model is obtained by deep learning through a convolutional neural network; detecting the first image by using a preset anomaly detection algorithm, and determining whether the first image meets an identification condition according to a detection result; and if the first image has commodity overlapping or does not meet the identification condition, sending an error prompt to the client. In the embodiment of the invention, the server can be connected with a plurality of clients through the cloud, the layout number of the server can be effectively reduced, the cost is effectively saved, the commodity overlapping is detected through the classification model obtained by the deep learning of the convolutional neural network, the abnormal condition of the image is detected through the abnormal detection algorithm of the image recognition, the problems of overlapping placing, placing and correcting of a placing plate, exceeding of the recognition range of the commodity to be recognized, blocking of obstacles and the like in the commodity recognition process can be effectively solved, the problems are prompted to be corrected, the model can be continuously trained, the commodity recognition efficiency of a baking shop is improved while the recognition accuracy is ensured, and the labor cost is saved.
Those of ordinary skill in the art will understand that: the figures are merely schematic representations of one embodiment, and the blocks or flow diagrams in the figures are not necessarily required to practice the present invention.
From the above description of the embodiments, it is clear to those skilled in the art that the present invention can be implemented by software plus necessary general hardware platform. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which may be stored in a storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for apparatus or system embodiments, since they are substantially similar to method embodiments, they are described in relative terms, as long as they are described in partial descriptions of method embodiments. The above-described embodiments of the apparatus and system are merely illustrative, and the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
In addition, in some of the flows described in the above embodiments and the drawings, a plurality of operations are included in a specific order, but it should be clearly understood that the operations may be executed out of the order presented herein or in parallel, and the sequence numbers of the operations, such as 201, 202, 203, etc., are merely used for distinguishing different operations, and the sequence numbers themselves do not represent any execution order. Additionally, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first", "second", etc. in this document are used for distinguishing different messages, devices, modules, etc., and do not represent a sequential order, nor limit the types of "first" and "second" to be different.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While alternative embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following appended claims be interpreted as including alternative embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various modifications and variations can be made in the embodiments of the present invention without departing from the spirit or scope of the embodiments of the invention. Thus, if such modifications and variations of the embodiments of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to encompass such modifications and variations.
Claims (12)
1. A commodity identification method based on intelligent baking is applied to a server side connected with at least one client side through a cloud side, and comprises the following steps:
receiving a first image of a commodity to be identified uploaded by a client;
determining whether the first image has commodity overlapping by using a preset classification model, wherein the classification model is obtained by deep learning through a convolutional neural network; and
carrying out image detection on the first image by using a preset anomaly detection algorithm, and determining whether the first image meets an identification condition according to a detection result;
and if the first image has commodity overlapping or does not meet the identification condition, sending an error prompt to the client.
2. The method of claim 1, wherein the determining whether the first image has the overlap of the products by using a preset classification model comprises:
the classification model classifies the first image according to a preset class library; the preset class library comprises an overlapped class library and a non-overlapped class library;
and when the first image belongs to the overlapping class library, determining that the first image has commodity overlapping.
3. The method of claim 1 or 2, wherein deep learning through a convolutional neural network results in a classification model, comprising:
performing two-layer convolution processing on each training image in the training set to obtain a convolution processing result;
performing maximum pooling on the convolution processing result to obtain a pooling processing result;
regularizing the pooling processing result and outputting a classification result;
comparing the classification result with the class of the training image to determine the classification accuracy of the training;
and when the classification accuracy is smaller than the preset accuracy threshold, performing the circular training again until the classification accuracy reaches the preset accuracy threshold.
4. The method according to claim 1, wherein the image detecting the first image by using a preset anomaly detection algorithm comprises:
determining contour information of an object in the first image;
determining the minimum circumscribed rectangle of each outline according to the outline information;
deleting the minimum circumscribed rectangle with the area smaller than a preset first area threshold value to obtain a candidate minimum circumscribed rectangle;
traversing and calculating the area of the candidate minimum circumscribed rectangle, and determining the candidate minimum circumscribed rectangle with the largest area as a target circumscribed rectangle;
when the area of the target circumscribed rectangle is larger than the preset visible area, the detection result is abnormal;
when the area of the target circumscribed rectangle is not larger than a preset visible area, calculating the absolute value of the difference value between the area of the target circumscribed rectangle and the preset visible area;
when the absolute value of the difference value is larger than a preset second area threshold value, the detection result is abnormal;
correspondingly, the determining whether the first image meets the identification condition according to the detection result includes:
and when the detection result is abnormal, determining that the first image does not meet the identification condition.
5. The method of claim 1, further comprising:
if the first image has no commodity overlapping and meets the identification condition, carrying out commodity identification on the first image through a preset image identification model, and sending an identification result to the client;
correspondingly, when the new supplier variety class is added, the method further comprises the following steps:
receiving a new commodity sample image uploaded by a client;
marking the new commodity sample image by using a preset automatic marking algorithm;
and training the image recognition model by using the marked new commodity sample image to complete the addition of the new commodity sample.
6. The method according to claim 5, wherein the labeling of the new commodity sample image by using a preset automatic labeling algorithm comprises:
determining contour information of an object in the new commodity sample image;
determining the minimum circumscribed rectangle of each outline according to the outline information;
deleting the minimum circumscribed rectangle with the area smaller than the area threshold of the new commodity to obtain a candidate circumscribed rectangle;
and if the candidate external rectangles are multiple, merging the candidate external rectangles to obtain a sample labeling rectangle.
7. The utility model provides a commodity recognition device based on wisdom is baked, a serial communication port, the device is applied to the server side of being connected through high in the clouds and at least one customer end, includes: the device comprises a receiving unit, a first detection unit, a second detection unit and a feedback unit; wherein the content of the first and second substances,
the receiving unit is used for receiving a first image of a commodity to be identified uploaded by a client;
the first detection unit is used for determining whether the first image has commodity overlapping by using a preset classification model, and the classification model is obtained by deep learning through a convolutional neural network; and
the second detection unit is used for carrying out image detection on the first image by using a preset anomaly detection algorithm and determining whether the first image meets an identification condition according to a detection result;
and the feedback unit is used for sending an error prompt to the client if the first image has commodity overlapping or does not meet the identification condition.
8. The apparatus according to claim 7, wherein the first detecting unit determines whether there is a product overlap in the first image by using a preset classification model, and is specifically configured to:
classifying the first image according to a preset class library by using the classification model; the preset class library comprises an overlapped class library and a non-overlapped class library; and when the first image belongs to the overlapping class library, determining that the first image has commodity overlapping.
9. The apparatus according to claim 7 or 8, wherein the first detecting unit is configured to perform deep learning through a convolutional neural network to obtain a classification model, and is specifically configured to:
performing two-layer convolution processing on each training image in the training set to obtain a convolution processing result; performing maximum pooling on the convolution processing result to obtain a pooling processing result; regularizing the pooling processing result and outputting a classification result; comparing the classification result with the class of the training image to determine the classification accuracy of the training; and when the classification accuracy is smaller than the preset accuracy threshold, performing the circular training again until the classification accuracy reaches the preset accuracy threshold.
10. The apparatus according to claim 7, wherein the second detecting unit performs image detection on the first image by using a preset anomaly detection algorithm, and is specifically configured to:
determining contour information of an object in the first image; determining the minimum circumscribed rectangle of each outline according to the outline information; deleting the minimum circumscribed rectangle with the area smaller than a preset first area threshold value to obtain a candidate minimum circumscribed rectangle; traversing and calculating the area of the candidate minimum circumscribed rectangle, and determining the candidate minimum circumscribed rectangle with the largest area as a target circumscribed rectangle; when the area of the target circumscribed rectangle is larger than the preset visible area, the detection result is abnormal; when the area of the target circumscribed rectangle is not larger than a preset visible area, calculating the absolute value of the difference value between the area of the target circumscribed rectangle and the preset visible area; when the absolute value of the difference value is larger than a preset second area threshold value, the detection result is abnormal;
correspondingly, the second detecting unit determines whether the first image meets the identification condition according to the detection result, and is specifically configured to:
and when the detection result is abnormal, determining that the first image does not meet the identification condition.
11. The apparatus of claim 7, further comprising: the identification unit is used for identifying the commodities of the first image through a preset image identification model if the first image has no commodity overlapping and meets identification conditions, and sending an identification result to the client;
correspondingly, when the new supplier variety class is added, the identification unit is further configured to: receiving a new commodity sample image uploaded by a client; marking the new commodity sample image by using a preset automatic marking algorithm; and training the image recognition model by using the marked new commodity sample image to complete the addition of the new commodity sample.
12. The apparatus according to claim 11, wherein the identification unit labels the new product sample image by using a preset automatic labeling algorithm, and is specifically configured to:
determining contour information of an object in the new commodity sample image; determining the minimum circumscribed rectangle of each outline according to the outline information; deleting the minimum circumscribed rectangle with the area smaller than the area threshold of the new commodity to obtain a candidate circumscribed rectangle; and if the candidate external rectangles are multiple, merging the candidate external rectangles to obtain a sample labeling rectangle.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010483443.6A CN111797896B (en) | 2020-06-01 | 2020-06-01 | Commodity identification method and device based on intelligent baking |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010483443.6A CN111797896B (en) | 2020-06-01 | 2020-06-01 | Commodity identification method and device based on intelligent baking |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111797896A true CN111797896A (en) | 2020-10-20 |
| CN111797896B CN111797896B (en) | 2023-06-27 |
Family
ID=72806625
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010483443.6A Active CN111797896B (en) | 2020-06-01 | 2020-06-01 | Commodity identification method and device based on intelligent baking |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111797896B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114627315A (en) * | 2022-01-31 | 2022-06-14 | 南通爱米食品有限公司 | Method for identifying baking stage of mass pastry |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3217324A1 (en) * | 2016-03-07 | 2017-09-13 | Ricoh Company, Ltd. | Hybrid detection recognition system |
| CN108647588A (en) * | 2018-04-24 | 2018-10-12 | 广州绿怡信息科技有限公司 | Goods categories recognition methods, device, computer equipment and storage medium |
| CN109816045A (en) * | 2019-02-11 | 2019-05-28 | 青岛海信智能商用系统股份有限公司 | A kind of commodity recognition method and device |
| CN109919211A (en) * | 2019-02-26 | 2019-06-21 | 南京旷云科技有限公司 | Commodity recognition method, device, system and computer-readable medium |
| CN110222587A (en) * | 2019-05-13 | 2019-09-10 | 杭州电子科技大学 | A kind of commodity attribute detection recognition methods again based on characteristic pattern |
| CN110458096A (en) * | 2019-08-09 | 2019-11-15 | 广州众聚智能科技有限公司 | A kind of extensive commodity recognition method based on deep learning |
| CN110807808A (en) * | 2019-10-14 | 2020-02-18 | 浙江理工大学 | Commodity identification method based on physical engine and deep full convolution network |
| CN111079699A (en) * | 2019-12-30 | 2020-04-28 | 北京每日优鲜电子商务有限公司 | Commodity identification method and device |
| CN111178379A (en) * | 2018-11-12 | 2020-05-19 | 株式会社石田 | Teaching data generation method and generation device, and commodity identification device |
-
2020
- 2020-06-01 CN CN202010483443.6A patent/CN111797896B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3217324A1 (en) * | 2016-03-07 | 2017-09-13 | Ricoh Company, Ltd. | Hybrid detection recognition system |
| CN108647588A (en) * | 2018-04-24 | 2018-10-12 | 广州绿怡信息科技有限公司 | Goods categories recognition methods, device, computer equipment and storage medium |
| CN111178379A (en) * | 2018-11-12 | 2020-05-19 | 株式会社石田 | Teaching data generation method and generation device, and commodity identification device |
| CN109816045A (en) * | 2019-02-11 | 2019-05-28 | 青岛海信智能商用系统股份有限公司 | A kind of commodity recognition method and device |
| CN109919211A (en) * | 2019-02-26 | 2019-06-21 | 南京旷云科技有限公司 | Commodity recognition method, device, system and computer-readable medium |
| CN110222587A (en) * | 2019-05-13 | 2019-09-10 | 杭州电子科技大学 | A kind of commodity attribute detection recognition methods again based on characteristic pattern |
| CN110458096A (en) * | 2019-08-09 | 2019-11-15 | 广州众聚智能科技有限公司 | A kind of extensive commodity recognition method based on deep learning |
| CN110807808A (en) * | 2019-10-14 | 2020-02-18 | 浙江理工大学 | Commodity identification method based on physical engine and deep full convolution network |
| CN111079699A (en) * | 2019-12-30 | 2020-04-28 | 北京每日优鲜电子商务有限公司 | Commodity identification method and device |
Non-Patent Citations (2)
| Title |
|---|
| 匡青: "基于卷积神经网络的商品图像分类研究", 《软件导刊》 * |
| 胡正委等: "基于改进Faster RCNN与Grabcut的商品图像检测", 《计算机系统应用》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114627315A (en) * | 2022-01-31 | 2022-06-14 | 南通爱米食品有限公司 | Method for identifying baking stage of mass pastry |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111797896B (en) | 2023-06-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11151427B2 (en) | Method and apparatus for checkout based on image identification technique of convolutional neural network | |
| CN108320404B (en) | Commodity identification method and device based on neural network and self-service cash register | |
| US11861584B2 (en) | Self-service settlement method, apparatus and storage medium | |
| US11481751B1 (en) | Automatic deep learning computer vision based retail store checkout system | |
| WO2019228063A1 (en) | Product inspection terminal, method and system, computer apparatus and readable medium | |
| CN107767590A (en) | Automatic identification commercialization bar code electronic scale and Automatic identification method | |
| CN110705424B (en) | Method and device for positioning commodity display position and storage medium | |
| US10503961B2 (en) | Object recognition for bottom of basket detection using neural network | |
| US10372998B2 (en) | Object recognition for bottom of basket detection | |
| CN108960132B (en) | Method and device for purchasing commodities in open type vending machine | |
| US20220067568A1 (en) | Computer vision transaction monitoring | |
| CN113627411A (en) | Super-resolution-based commodity identification and price matching method and system | |
| CN103714498A (en) | Quick payment system based on image recognition technology and method thereof | |
| CN106023159A (en) | Disease spot image segmentation method and system for greenhouse vegetable leaf | |
| CN110909698A (en) | Electronic scale recognition result output method, system, device and readable storage medium | |
| CN114898249A (en) | Method, system and storage medium for confirming number of articles in shopping cart | |
| CN111797896B (en) | Commodity identification method and device based on intelligent baking | |
| US11562338B2 (en) | Automated point of sale systems and methods | |
| RU2724797C1 (en) | Cash register system and method for identification of courses on tray | |
| CN109146834A (en) | Car damage identification method and device, computer readable storage medium, terminal | |
| CN114332058A (en) | Serum quality identification method, device, equipment and medium based on neural network | |
| CN110110594B (en) | Product distribution identification method and device | |
| CN111325084A (en) | Dish information identification method and terminal based on YOLO neural network | |
| CN109858446A (en) | Article register method and device under a kind of new public safety | |
| CN210573826U (en) | Intelligent commodity identification equipment and intelligent settlement table |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |