CN114155453A - Training method for ice chest commodity image recognition, model and occupancy calculation method - Google Patents
Training method for ice chest commodity image recognition, model and occupancy calculation method Download PDFInfo
- Publication number
- CN114155453A CN114155453A CN202210124985.3A CN202210124985A CN114155453A CN 114155453 A CN114155453 A CN 114155453A CN 202210124985 A CN202210124985 A CN 202210124985A CN 114155453 A CN114155453 A CN 114155453A
- Authority
- CN
- China
- Prior art keywords
- commodity
- image
- grid
- cold drink
- commodities
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a training method, a model and an occupancy calculation method for freezer commodity image recognition, relates to the technical field of image recognition, and solves the technical problems that artificial intelligence cannot accurately and quickly recognize different types of commodities in a freezer and cannot quickly calculate the market occupancy of specific types of commodities. The training method comprises the following steps: acquiring an image of a commodity in the freezer, inputting the image into a first neural network model, and labeling clear cold drink commodities to obtain a plurality of labeled polygonal frames; analyzing the polygonal frame to obtain the mapping of the cold drink commodity on a plurality of characteristic layers, an initial training image and a corresponding mask label; substituting the image data of the cold drink commodity and the mask label data into a loss function for calculation, and updating the parameters of the initial neural network model; and repeatedly executing to finally obtain the trained second neural network model. The invention can calculate the number of each type of goods in the refrigerator and the market share of the goods.
Description
Technical Field
The invention relates to the technical field of image recognition, in particular to a training method, a model and an occupancy calculation method for freezer commodity image recognition.
Background
With the development of social economy, the consumption level of people is improved year by year, the market scale of cold drinks represented by ice cream, ice cream and the like is rapidly increased, the cold drinks are steadily increased every year, and the ice drink has great market prospect. Information collection of the cold drink market is an important basic work for establishing a marketing system. The acquisition, analysis and utilization of market information drive the development of marketing key businesses such as cold drink market demand prediction, source organization, source supply, brand cultivation and the like, and are important means for the ice cream industry to research and develop new products to master market dynamics, understand the conditions of competitors and formulate a sales scheme.
The existing cold drink market inspection task (namely market information acquisition) is used for realizing statistics on the market sales condition of specific cold drink products, is mainly completed manually, and has very low efficiency. Meanwhile, in recent years, with the continuous development of artificial intelligence, automatic detection technology is increasingly applied to various aspects such as industrial production, social security, life consumption and the like. The machine has the advantages of high speed, no fatigue, adaptability to severe environment and the like, and the popularization of the automatic detection and identification technology greatly improves the detection and identification efficiency, thereby improving the industrial production level and improving the life quality of people.
In the process of implementing the invention, the inventor finds that at least the following problems exist in the prior art:
in most scenes, the existing artificial intelligence technology cannot accurately and quickly identify different types of commodities in the freezer, so that the market share of specific types of commodities cannot be quickly calculated.
Disclosure of Invention
The invention aims to provide a training method, a model and an occupancy calculation method for freezer commodity image recognition, and aims to solve the technical problem that the artificial intelligence technology in the prior art cannot accurately and quickly recognize different types of commodities in a freezer, so that the market occupancy of specific types of commodities cannot be quickly calculated. The technical effects that can be produced by the preferred technical scheme in the technical schemes provided by the invention are described in detail in the following.
In order to achieve the purpose, the invention provides the following technical scheme:
the invention provides a training method for refrigerator commodity image recognition, which is used for recognizing cold drink commodities in a refrigerator and comprises the following steps: s10: acquiring an ice chest commodity image containing a plurality of cold drink commodities, and inputting a first neural network model, wherein the first neural network model labels clear cold drink commodities to obtain a plurality of labeled polygonal borders; s20: analyzing the polygonal frame to obtain the mapping of the cold drink commodity on a plurality of characteristic layers, an initial training image and a corresponding mask label; s30: inputting the initial training image into an initial neural network model to obtain cold drink commodity image data with the same format as the mask label; s40: substituting the image data of the cold drink commodity and the mask label data into a loss function for calculation, returning the loss obtained by calculating the loss function through a back propagation algorithm, and updating the parameters of the initial neural network model; s50: and repeatedly executing the step S40 to finally obtain a trained parameter matrix, and inputting the parameter matrix into the initial neural network model to obtain a trained second neural network model.
Preferably, in the first neural network model, training is performed according to four labeling categories of clear ice cream, unclear ice cream, advertising paper and commodities except ice cream, so as to form category information of the commodities of the freezer, and the polygonal frame is obtained after the clear ice cream is labeled.
Preferably, in the step S10, the polygonal frame is a closed region formed by uniformly connecting 20 points in sequence.
Preferably, in step S20, the number of the feature layers is three, and the sizes of the three feature layers are 19 × 19 pixels, 38 × 38 pixels, and 76 × 76 pixels, respectively.
Preferably, in the step S20, the mapped parameters are the confidence of the cold drink product, and the x offset and the y offset of each point on the polygonal border; the x offset and the y offset are respectively the offset of an x coordinate and a y coordinate of each point relative to the upper left corner of the grid, and the grid is a corresponding grid of the position of each point on the feature layer.
Preferably, in the step S40, the loss function includes:
wherein L isobjRepresents a target loss, L, of the cold beverage productoffIndicating loss of center point coordinate offset, LpolyRepresenting a loss of boundary coordinates of the polygon frame with respect to a center point, LclsIndicating a loss of classification of said cold drink product, LlossRepresents the total loss; lambda [ alpha ]obj、λnobj、λoff、λpoly、λclassRespectively, a weight coefficient corresponding to the loss, S a side length of the feature map, P20 coordinate points on the polygon border,
indicating whether the target exists in the ith grid on the feature map,
representing whether the ith grid on the feature map has no target; c. Ci、
Respectively representing the real value and the predicted value, x, of the ith grid target of the characteristic diagrami、yi、
、
Respectively representing the values of the actual offset and the predicted offset of the target in the ith grid to the upper left corner of the grid in the x and y directions; smoothl1Representing the regression loss function, xi,j、yi,j、
、
Respectively representing the actual offset and the predicted offset of the actual and predicted 20 object boundary point coordinates relative to the upper left corner of the grid in the x and y directions, i represents the ith grid in the S x S grids, j represents the jth coordinate in the ith grid, B represents the number of categories of the ice chest commodities, d represents the categories of the cold drink commodities,
and information entropy representing that the cold drink product belongs to the d category.
An image recognition model of a freezer commodity, wherein the model is an improved model with an example segmentation function based on yolov5, and an acquired freezer commodity image containing a plurality of cold drink commodities is trained by the training method, and the operation flow of the model is as follows: s100: inputting the collected commodity image of the freezer, adjusting the size of the image, standardizing the image, and sending the image into a backbone convolutional neural network to extract features to obtain a first feature layer; s200: obtaining a second characteristic layer and a third characteristic layer with different sizes through the first characteristic layer; s300: taking each pixel in the first feature layer, the second feature layer and the third feature layer as a grid, and generating a first offset relative to the upper left corner of the grid in an x coordinate direction and a y coordinate direction and a second offset relative to the upper left corner of the grid for each point in a polygonal frame for each grid; s400: and taking the first offset, the second offset, the target confidence coefficient and the commodity category information as output parameters of the first characteristic layer, the second characteristic layer and the third characteristic layer together.
Preferably, in the step S100, the adjusted image size is 608 × 608 pixels, and the normalization process has a mean value of 0.45 and a variance of 0.3.
A freezer commodity occupancy calculation method performed by any one of the freezer commodity image recognition models comprises the following steps: s1000: acquiring images of the refrigerator comprising commodities to obtain images of the commodities of the refrigerator, judging whether the quality of the images of the commodities of the refrigerator is qualified or not, if so, executing the step S2000, otherwise, executing the step S7000; s2000: adjusting the size of the freezer commodity image to 608 x 608 pixels, and performing data enhancement processing to obtain a preprocessed image; s3000: inputting the preprocessed image into any one of the second neural network models to obtain an initial polygonal frame of the cold drink commodity in the preprocessed image, and outputting the probability that the target contained in each grid is the target center; s4000: judging whether the probability is greater than a set threshold, if so, executing a step S5000, otherwise, executing a step S7000; s5000: reserving the grid coordinates, determining a polygonal frame of the cold drink commodity according to the grid coordinates and the point coordinates corresponding to the grid coordinates, and determining a mask label of the cold drink commodity according to the polygonal frame; s6000: calculating the area occupation ratio of the cold drink commodity through the mask label to obtain the market occupation ratio of the cold drink commodity; s7000: and returning to the step S1000, and executing the steps S1000-S7000 on the new ice chest commodity image.
Preferably, in the step S2000, the data enhancement processing operation includes one or more of up-down flipping, left-right flipping, gaussian blurring, and rotation.
The implementation of one of the technical schemes of the invention has the following advantages or beneficial effects:
the invention realizes the calculation of the display areas of different types of commodities in the freezer through an image recognition technology, can accurately recognize the quantity of the commodities with different specifications and calculate the area ratio, knows the quantity of each type of commodities displayed in the freezer and the area ratio of the commodities in all the commodities, is used for predicting the market share of each type of commodities and mastering the market dynamics in real time to formulate a marketing strategy.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without inventive efforts, wherein:
FIG. 1 is a flow chart of an embodiment of a training method for ice chest merchandise image recognition in the present invention;
FIG. 2 is a flow chart of an embodiment of an image recognition model for freezer goods in the present invention;
FIG. 3 is a flow chart of an embodiment of a bin occupancy calculation method of the present invention.
Detailed Description
In order that the objects, aspects and advantages of the present invention will become more apparent, various exemplary embodiments will be described below with reference to the accompanying drawings, which form a part hereof, and in which are shown by way of illustration various exemplary embodiments in which the invention may be practiced. The same numbers in different drawings identify the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. It is to be understood that they are merely examples of processes, methods, apparatus, etc. consistent with certain aspects of the present disclosure as detailed in the appended claims, and that other embodiments may be used or structural and functional modifications may be made to the embodiments set forth herein without departing from the scope and spirit of the present disclosure.
In the description of the present invention, it is to be understood that the terms "central," "longitudinal," "lateral," and the like are used in the orientations and positional relationships illustrated in the accompanying drawings for the purpose of facilitating the description of the present invention and simplifying the description, and do not indicate or imply that the elements so referred to must have a particular orientation, be constructed in a particular orientation, and be operated. The terms "first", "second", etc. are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. The term "plurality" means two or more. The terms "coupled" and "connected" are to be construed broadly and may include, for example, a fixed connection, a removable connection, a unitary connection, a mechanical connection, an electrical connection, a communicative connection, a direct connection, an indirect connection via intermediate media, and may include, but are not limited to, a connection between two elements or an interactive relationship between two elements. The term "and/or" includes any and all combinations of one or more of the associated listed items. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
In order to explain the technical solution of the present invention, the following description is made by way of specific examples, which only show the relevant portions of the embodiments of the present invention.
The first embodiment is as follows:
as shown in FIG. 1, the invention provides a training method for image recognition of ice chest commodities, which is used for recognizing cold drink commodities in an ice chest and comprises the following steps. S10: the method comprises the steps of obtaining an image of ice chest commodities comprising a plurality of cold drink commodities, inputting a first neural network model, marking clear cold drink commodities by the first neural network model to obtain a plurality of marked polygonal frames (polygon in image processing), and enabling the cold drink commodities to be surrounded into a closed polygon by the polygonal frames. S20: analyzing the polygonal frame to obtain the mapping of the cold drink commodity on a plurality of characteristic layers, and initially training images and corresponding mask labels; s30: inputting the initial training image into an initial neural network model to obtain cold drink commodity image data with the same format as the mask label; s40: and substituting the image data of the cold drink commodity and the mask label data into the loss function for calculation, returning the loss obtained by calculating the loss function through a back propagation algorithm, and updating the parameters of the initial neural network model. Through the iteration of the loss function calculation, new neural network model parameters can be obtained, and therefore the initial neural network model is updated and optimized. S50: and (6) repeatedly executing the step S40 to finally obtain a trained parameter matrix, and inputting the parameter matrix into the initial neural network model to obtain a trained second neural network model. The second neural network model obtained by the training method can quickly identify the target cold drink product.
As an optional implementation manner, in the first neural network model, training is performed according to four labeling categories of clear ice cream, unclear ice cream, advertising paper and goods other than ice cream to form category information of goods in the freezer, and the clear ice cream is labeled to obtain a polygonal frame. At the moment, the aim is to identify the ice cream in the cold drink commodity, if only the ice cream is labeled, the model is difficult to distinguish which ice cream is but not because other commodities and advertising paper are too similar to the ice cream, so that the model is divided into 4 categories for training, and the implicit difficulty of the training is greatly reduced. Four categories of clear ice cream, unclear ice cream, advertisement paper and commodities except the ice cream are trained by directly adopting a target recognition training method in the prior art.
As an optional implementation manner, in the step S10, the polygonal frame is a closed area formed by sequentially and uniformly connecting 20 points, and the 20 points facilitate accurate contouring of the contour of the cold drink product, so as to obtain a more accurate neural network model.
As an alternative embodiment, in step S20, the number of feature layers is three, the sizes of the three feature layers are 19 × 19 pixels, 38 × 38 pixels and 76 × 76 pixels, respectively, the first feature layer, the second feature layer and the third feature layer, and the pixels are also grids, that is, the sizes of the three feature layers are 19 × 19 grids, 38 × 38 grids and 76 × 76 grids, respectively, and the sizes of the three feature layers are adapted to yolov5, so as to facilitate fast processing.
As an alternative embodiment, in step S20, the mapped parameters are the confidence of the cold drink product, and the x offset and the y offset of each point on the polygonal border, and since there are 20 points on the polygonal border of the cold drink product, 20 sets of x offset and y offset are obtained. The x offset and the y offset are respectively the offset of an x coordinate and a y coordinate of the pixel relative to the upper left corner of the grid, and the grid is a corresponding grid of each point on the feature layer. The mesh is the mesh corresponding to the position of the feature layer where one point on the polygonal frame is located. For the confidence coefficient, the value of the center of each cold drink commodity mapped to the corresponding coordinate position on the feature layer is set to be 1, and if not, the value is set to be 0.
As an alternative embodiment, in step S40, the loss function includes:
wherein L isobjIndicating the target loss of the cold drink product, LoffLoss of coordinate offset representing center pointLose, LpolyRepresenting the loss of boundary coordinates of the polygon frame relative to the center point, LclsIndicating a loss of classification of the cold drink product, LlossRepresents the total loss; lambda [ alpha ]obj、λnobj、λoff、λpoly、λclassRespectively representing the weight coefficients corresponding to the losses, wherein S represents the side length of a characteristic graph, and the characteristic graph is the number of the characteristic layers in each row or each column of grids and is the side length of the characteristic layers; p denotes 20 coordinate points on the polygon frame,
indicating whether the target exists in the ith grid on the feature map,
representing whether the ith grid on the feature map has no target; c. Ci、
Respectively representing the real value and the predicted value of the ith grid (the grids are ordered from left to right, from top to bottom, the 0 th grid at the leftmost upper corner and the last grid at the rightmost lower corner) of the characteristic diagram, and xi、yi、
、
Respectively representing the values of the actual offset and the predicted offset of the target in the ith grid to the upper left corner of the grid in the x and y directions; smoothl1Representing the regression loss function, xi,j、yi,j、
、
Respectively representing the actual and predicted 20 object boundary point coordinates (i.e. on polygon frame polygon of ice cream)20 points) the actual offset from the upper left corner of the grid and the predicted offset are plotted in the x, y direction, i represents the ith grid in the S x S grids, j represents the jth coordinate in the ith grid, and a total of S x S20 coordinate offsets are predicted. B represents the number of categories of ice chest products (four categories in this embodiment), d represents the category to which the cold drink products belong (four categories in this embodiment, d is 0, 1, 2, 3),
and the information entropy of the cold drink product belonging to the d-th category is represented.
The embodiment is only a specific example and does not indicate such an implementation of the invention.
Example two:
a freezer commodity image recognition model is an improved model with an example segmentation function based on yolov5, and is trained by the freezer commodity image acquisition method provided by the invention, wherein the operation flow of the model is as follows. S100: inputting the collected commodity image of the freezer, adjusting the size of the image, standardizing the image, and sending the image into a backbone convolutional neural network to extract characteristics to obtain a first characteristic layer; s200: obtaining a second characteristic layer and a third characteristic layer with different sizes through the first characteristic layer; s300: and taking each pixel in the first characteristic layer, the second characteristic layer and the third characteristic layer as a grid, and generating a first offset relative to the upper left corner of the grid in the x coordinate direction and the y coordinate direction and a second offset relative to the upper left corner of the grid of each point in the polygonal frame for each grid, wherein the second offsets also comprise the x coordinate direction and the y coordinate direction. S400: and taking the first offset, the second offset, the target confidence coefficient and the commodity category information as output parameters of the first characteristic layer, the second characteristic layer and the third characteristic layer together. The first offset corresponding to each point has 2 values in the x coordinate direction and the y coordinate direction, 20 points in the polygonal frame have 40 values in the x coordinate direction and the y coordinate direction, the confidence coefficient of the target is 1 value, the probability values of 4 categories are 4, 47 values are counted, the size of the feature layer output by each layer is S, and the output of each layer is S47. Through the image recognition model, target cold drink products such as ice cream can be quickly and accurately recognized.
As an alternative embodiment, in step S100, the adjusted image size is 608 × 608 pixels, and the parameters of the normalization process are a mean value of 0.45 and a variance of 0.3. 608 pixels 608 are the input size required by the backbone network of the original yolov5, the adjusted image size is matched with the requirement of yolov5, and the yolov5 is easier to extract the characteristics of the input image size after the average value is 0.45 and the variance is 0.3.
Example three:
a method for calculating the occupancy of a freezer, as shown in FIG. 3, comprises the following steps: s1000: the method comprises the steps of collecting images of the freezer comprising the commodities to obtain images of the commodities of the freezer, judging whether the images of the commodities of the freezer are qualified or not, wherein the image quality of the images of the commodities of the freezer is judged to be qualified by adopting the prior art or manually. If qualified, execute step S2000, otherwise execute step S7000. S2000: and adjusting the size of the freezer commodity image to 608 x 608 pixels, performing data enhancement processing to obtain a preprocessed image, and processing the freezer commodity image in the second neural network model conveniently after preprocessing. S3000: inputting the preprocessed image into a second neural network model provided by the invention to obtain an initial polygonal frame of the cold drink commodity in the preprocessed image, directly not labeling the other three commodity categories (unclear commodities except ice cream, advertisement paper and ice cream) in the second neural network model, and outputting the probability that the target contained in each grid is the target center. S4000: and judging whether the probability is greater than a set threshold, if so, setting the threshold according to experience, such as 0.3, 0.5 and the like, executing the step S5000, and otherwise, executing the step S7000. S5000: and reserving the grid coordinates, determining a polygonal frame of the cold drink commodity according to the grid coordinates and the corresponding point coordinates, and determining a mask label of the cold drink commodity according to the polygonal frame, wherein the mask label is the outline seen by the cold drink commodity. S6000: and calculating the area occupation ratio of the cold drink commodity through the mask label to obtain the market occupation ratio of the cold drink commodity. Specifically, when market occupancy rate data of a certain specific type or brand of cold drink commodity (such as lovely ice cream) in the ice cream market and the whole cold drink market is needed, the corresponding market occupancy rate can be obtained by combining a mask label, all ice cream labels (which can be labeled by adopting the prior art) and all cold drink commodity labels (which can be labeled by adopting the prior art) and calculating the occupation rate of each type of label. S7000: returning to the step S1000, and executing the steps S1000-S7000 on the new ice chest commodity image. 268 actually shot pictures are tested through actual experimental tests, masks are marked on each type of ice cream in the pictures, the area ratio of the masks of each type of ice cream in all the commodity masks is calculated, the area ratio is compared with the area ratio calculated by the model, and the average accuracy rate of the method is calculated to be more than 95%. The invention realizes the calculation method of the display areas of different types of commodities in the freezer through the image recognition technology, can accurately recognize the quantity of the commodities with different specifications and calculate the area ratio, knows the quantity of each type of commodities displayed in the freezer and the area ratio of the commodities in all the commodities, is used for knowing the market share of each type of commodities and mastering the market dynamic in real time to formulate the marketing strategy.
As an alternative embodiment, in step S2000, the data enhancement processing operation includes one or more of flip-up and flip-down, flip-left and flip-right, gaussian blur, and rotation. Because the actual scene is complex and changeable, and the acquired image samples can not cover all the situations, the situations in the actual scene are simulated by the data enhancement mode, and the actual scene can be better identified.
While the invention has been described with reference to a preferred embodiment, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted for elements thereof without departing from the spirit and scope of the invention. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the invention without departing from the essential scope thereof. Therefore, it is intended that the invention not be limited to the particular embodiment disclosed, but that the invention will include all embodiments falling within the scope of the appended claims.
Claims (10)
1. A training method for ice chest commodity image recognition is used for recognizing cold drink commodities in an ice chest, and is characterized by comprising the following steps:
s10: acquiring an ice chest commodity image containing a plurality of cold drink commodities, and inputting a first neural network model, wherein the first neural network model labels clear cold drink commodities to obtain a plurality of labeled polygonal borders;
s20: analyzing the polygonal frame to obtain the mapping of the cold drink commodity on a plurality of characteristic layers, an initial training image and a corresponding mask label;
s30: inputting the initial training image into an initial neural network model to obtain cold drink commodity image data with the same format as the mask label;
s40: substituting the image data of the cold drink commodity and the mask label data into a loss function for calculation, returning the loss obtained by calculating the loss function through a back propagation algorithm, and updating the parameters of the initial neural network model;
s50: and repeatedly executing the step S40 to finally obtain a trained parameter matrix, and inputting the parameter matrix into the initial neural network model to obtain a trained second neural network model.
2. The training method for ice chest commodity image recognition according to claim 1, wherein in the first neural network model, training is performed according to four labeling categories of commodities except clear ice cream, unclear ice cream, advertising paper and ice cream to form category information of ice chest commodities, and the polygonal border is obtained after the clear ice cream is labeled.
3. A training method for image recognition of refrigerator goods according to claim 2, wherein in said step S10, said polygonal frame is a closed area formed by 20 points which are connected uniformly in sequence.
4. A training method for ice chest commodity image recognition according to claim 2, wherein in said step S20, said number of said feature layers is three, and the size of said three feature layers is 19 x 19 pixels, 38 x 38 pixels, 76 x 76 pixels.
5. A training method for image recognition of ice chest merchandise according to claim 2, wherein in said step S20, said mapped parameters are confidence of said cold drink merchandise and x offset and y offset of each point on said polygonal border; the x offset and the y offset are respectively the offset of an x coordinate and a y coordinate of each point relative to the upper left corner of the grid, and the grid is a corresponding grid of the position of each point on the feature layer.
6. A training method for ice chest merchandise image recognition according to claim 2, wherein in said step S40, said loss function comprises:
wherein L isobjRepresents a target loss, L, of the cold beverage productoffIndicating loss of center point coordinate offset, LpolyRepresenting a loss of boundary coordinates of the polygon frame with respect to a center point, LclsIndicating a loss of classification of said cold drink product, LlossRepresents the total loss;
λobj、λnobj 、λoff 、λpoly 、λclassrespectively, a weight coefficient corresponding to the loss, S a side length of the feature map, P20 coordinate points on the polygon border,
indicating whether the target exists in the ith grid on the feature map,
representing whether the ith grid on the feature map has no target;
ci、
respectively representing the real value and the predicted value, x, of the ith grid target of the characteristic diagrami、yi、
、
Respectively representing the values of the actual offset and the predicted offset of the target in the ith grid to the upper left corner of the grid in the x and y directions; smoothl1Representing the regression loss function, xi,j、yi,j、
、
Respectively representing the actual offset and the predicted offset of the actual and predicted 20 object boundary point coordinates relative to the upper left corner of the grid in the x and y directions, i represents the ith grid in the S x S grids, j represents the jth coordinate in the ith grid, B represents the number of categories of the ice chest commodities, d represents the categories of the cold drink commodities,
and information entropy representing that the cold drink product belongs to the d category.
7. An image recognition model of ice chest commodities, which is an improved model based on yolov5 and has an example segmentation function, wherein an acquired ice chest commodity image containing a plurality of cold drink commodities is trained by the training method of any one of claims 2 to 6, and the operation flow of the model is as follows:
s100: inputting the collected commodity image of the freezer, adjusting the size of the image, standardizing the image, and sending the image into a backbone convolutional neural network to extract features to obtain a first feature layer;
s200: obtaining a second characteristic layer and a third characteristic layer with different sizes through the first characteristic layer;
s300: taking each pixel in the first feature layer, the second feature layer and the third feature layer as a grid, and generating a first offset relative to the upper left corner of the grid in an x coordinate direction and a y coordinate direction and a second offset relative to the upper left corner of the grid for each point in a polygonal frame for each grid;
s400: and taking the first offset, the second offset, the target confidence coefficient and the commodity category information as output parameters of the first characteristic layer, the second characteristic layer and the third characteristic layer together.
8. An image recognition model for ice chest merchandise according to claim 7, wherein in step S100, the adjusted image size is 608 x 608 pixels, and the parameters of the normalization process are mean 0.45 and variance 0.3.
9. A refrigerator commodity occupancy calculation method, wherein the calculation method is performed by the refrigerator commodity image recognition model according to any one of claims 7 to 8, and comprises the following steps:
s1000: acquiring images of the refrigerator comprising commodities to obtain images of the commodities of the refrigerator, judging whether the quality of the images of the commodities of the refrigerator is qualified or not, if so, executing the step S2000, otherwise, executing the step S7000;
s2000: adjusting the size of the freezer commodity image to 608 x 608 pixels, and performing data enhancement processing to obtain a preprocessed image;
s3000: inputting the preprocessed image into a second neural network model as claimed in any one of claims 2-6, obtaining an initial polygonal frame of cold drink goods in the preprocessed image, and outputting the probability that a target included in each grid is a target center;
s4000: judging whether the probability is greater than a set threshold, if so, executing a step S5000, otherwise, executing a step S7000;
s5000: reserving the grid coordinates, determining a polygonal frame of the cold drink commodity according to the grid coordinates and the point coordinates corresponding to the grid coordinates, and determining a mask label of the cold drink commodity according to the polygonal frame;
s6000: calculating the area occupation ratio of the cold drink commodity through the mask label to obtain the market occupation ratio of the cold drink commodity;
s7000: and returning to the step S1000, and executing the steps S1000-S7000 on the new ice chest commodity image.
10. A refrigerator merchandising occupancy calculation method according to claim 9, wherein in said S2000 step said data enhancement processing operation comprises one or more of flip up and down, flip left and right, gaussian blur and rotation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210124985.3A CN114155453B (en) | 2022-02-10 | 2022-02-10 | Image recognition training method, recognition method and occupancy calculation method for refrigerator commodities |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210124985.3A CN114155453B (en) | 2022-02-10 | 2022-02-10 | Image recognition training method, recognition method and occupancy calculation method for refrigerator commodities |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114155453A true CN114155453A (en) | 2022-03-08 |
| CN114155453B CN114155453B (en) | 2022-05-10 |
Family
ID=80450356
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210124985.3A Active CN114155453B (en) | 2022-02-10 | 2022-02-10 | Image recognition training method, recognition method and occupancy calculation method for refrigerator commodities |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114155453B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109409365A (en) * | 2018-10-25 | 2019-03-01 | 江苏德劭信息科技有限公司 | It is a kind of that method is identified and positioned to fruit-picking based on depth targets detection |
| US20200012904A1 (en) * | 2018-07-03 | 2020-01-09 | General Electric Company | Classification based on annotation information |
| WO2020124247A1 (en) * | 2018-12-21 | 2020-06-25 | Canscan Softwares And Technologies Inc. | Automated inspection system and associated method for assessing the condition of shipping containers |
| CN111582345A (en) * | 2020-04-29 | 2020-08-25 | 中国科学院重庆绿色智能技术研究院 | Target identification method for complex environment under small sample |
| US20200394413A1 (en) * | 2019-06-17 | 2020-12-17 | The Regents of the University of California, Oakland, CA | Athlete style recognition system and method |
| CN112446437A (en) * | 2020-12-11 | 2021-03-05 | 上海品览数据科技有限公司 | Goods shelf commodity specification identification method based on machine vision |
| CN112949572A (en) * | 2021-03-26 | 2021-06-11 | 重庆邮电大学 | Slim-YOLOv 3-based mask wearing condition detection method |
| CN113269051A (en) * | 2021-04-30 | 2021-08-17 | 广州图匠数据科技有限公司 | Commodity identification method, intelligent terminal and storage device |
| US20210390419A1 (en) * | 2020-06-12 | 2021-12-16 | Robert Bosch Gmbh | Device and Method for Training and Testing a Classifier |
| CN113920443A (en) * | 2021-09-29 | 2022-01-11 | 海南大学 | Yoov 5-based remote sensing directed target detection method |
-
2022
- 2022-02-10 CN CN202210124985.3A patent/CN114155453B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200012904A1 (en) * | 2018-07-03 | 2020-01-09 | General Electric Company | Classification based on annotation information |
| CN109409365A (en) * | 2018-10-25 | 2019-03-01 | 江苏德劭信息科技有限公司 | It is a kind of that method is identified and positioned to fruit-picking based on depth targets detection |
| WO2020124247A1 (en) * | 2018-12-21 | 2020-06-25 | Canscan Softwares And Technologies Inc. | Automated inspection system and associated method for assessing the condition of shipping containers |
| US20200394413A1 (en) * | 2019-06-17 | 2020-12-17 | The Regents of the University of California, Oakland, CA | Athlete style recognition system and method |
| CN111582345A (en) * | 2020-04-29 | 2020-08-25 | 中国科学院重庆绿色智能技术研究院 | Target identification method for complex environment under small sample |
| US20210390419A1 (en) * | 2020-06-12 | 2021-12-16 | Robert Bosch Gmbh | Device and Method for Training and Testing a Classifier |
| CN112446437A (en) * | 2020-12-11 | 2021-03-05 | 上海品览数据科技有限公司 | Goods shelf commodity specification identification method based on machine vision |
| CN112949572A (en) * | 2021-03-26 | 2021-06-11 | 重庆邮电大学 | Slim-YOLOv 3-based mask wearing condition detection method |
| CN113269051A (en) * | 2021-04-30 | 2021-08-17 | 广州图匠数据科技有限公司 | Commodity identification method, intelligent terminal and storage device |
| CN113920443A (en) * | 2021-09-29 | 2022-01-11 | 海南大学 | Yoov 5-based remote sensing directed target detection method |
Non-Patent Citations (2)
| Title |
|---|
| 单宇翔 等: "基于深度学习的复杂场景中卷烟烟盒检测与识别方法", 《中国烟草学报》 * |
| 张烽: "基于倾斜目标检测技术的快消品识别研究与应用", 《中国优秀硕士学位论文全文数据库（硕士） 信息科技辑》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114155453B (en) | 2022-05-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113160192B (en) | Visual sense-based snow pressing vehicle appearance defect detection method and device under complex background | |
| CN107977671B (en) | Tongue picture classification method based on multitask convolutional neural network | |
| CN109523520B (en) | Chromosome automatic counting method based on deep learning | |
| CN103593670B (en) | A kind of copper plate/strip detection method of surface flaw based on online limit of sequence learning machine | |
| CN111080622B (en) | Neural network training method, workpiece surface defect classification and detection method and device | |
| CN108345912A (en) | Commodity rapid settlement system based on RGBD information and deep learning | |
| CN111982910B (en) | Weak supervision machine vision detection method and system based on artificial defect simulation | |
| CN110135430A (en) | A kind of aluminium mold ID automatic recognition system based on deep neural network | |
| CN114092389A (en) | Glass panel surface defect detection method based on small sample learning | |
| CN112926652B (en) | Fish fine granularity image recognition method based on deep learning | |
| CN111612747B (en) | Rapid detection method and detection system for product surface cracks | |
| CN116863274A (en) | Semi-supervised learning-based steel plate surface defect detection method and system | |
| CN111476084A (en) | Deep learning-based parking lot dynamic parking space condition identification method | |
| CN111754502A (en) | Method for detecting surface defects of magnetic core based on fast-RCNN algorithm of multi-scale feature fusion | |
| CN115752683A (en) | Weight estimation method, system and terminal based on depth camera | |
| CN112232334B (en) | Intelligent commodity selling identification and detection method | |
| CN114155453B (en) | Image recognition training method, recognition method and occupancy calculation method for refrigerator commodities | |
| Kusanti et al. | Combination of otsu and canny method to identify the characteristics of solo batik as Surakarta traditional batik | |
| CN117372332A (en) | Fabric flaw detection method based on improved YOLOv7 model | |
| CN117079125A (en) | Kiwi fruit pollination flower identification method based on improved YOLOv5 | |
| CN114155374B (en) | Ice cream image training method, detection method and processing equipment | |
| CN115641575A (en) | Leafhopper agricultural pest detection method based on sparse candidate frame | |
| CN115205626A (en) | Data enhancement method applied to field of coating defect detection | |
| CN112365324A (en) | Commodity picture detection method suitable for E-commerce platform | |
| CN113469984A (en) | Display panel appearance detection method based on YOLO structure |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |