CN109472205B - Commodity identification method, commodity identification device, and storage medium - Google Patents
Commodity identification method, commodity identification device, and storage medium Download PDFInfo
- Publication number
- CN109472205B CN109472205B CN201811171626.3A CN201811171626A CN109472205B CN 109472205 B CN109472205 B CN 109472205B CN 201811171626 A CN201811171626 A CN 201811171626A CN 109472205 B CN109472205 B CN 109472205B
- Authority
- CN
- China
- Prior art keywords
- information
- commodity
- data
- key point
- calculation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/35—Categorising the entire scene, e.g. birthday party or wedding scene
- G06V20/36—Indoor scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention relates to the field of image recognition, and discloses a commodity recognition method, a commodity recognition device and a storage medium. The invention discloses a commodity identification method, which comprises the following steps: acquiring an image containing a commodity to be detected; detecting the image through the trained convolutional neural network model to obtain a detection result, wherein the detection result is the key point position information and the key point connection relation information of the commodity to be detected; and acquiring information of the to-be-detected commodity according to the detection result. By adopting the embodiment of the invention, the detection rate and the identification accuracy of the commodities are improved under various conditions.
Description
Technical Field
The present invention relates to the field of image recognition, and in particular, to a commodity recognition method, a commodity recognition apparatus, and a storage medium.
Background
With the rapid development of science and technology, novel retail stores such as unmanned convenience stores, unmanned coffee stores and the like come into the market one by one, and commodities taken away by a customer during shopping are automatically detected and subjected to online settlement, so that a salesperson does not need to wait for commodity settlement; currently, usually, a fast-rcnn, ssd and other detection methods based on anchor points and frames are used to search for corresponding ROIs on an image of a commodity to be detected, so as to complete the identification of the commodity.
However, the inventors found that the following problems exist in the prior art: when the commodities in the detection area are densely arranged and mutually shielded, when the commodities are located at corner positions, or when the commodities fall down or are obliquely placed, a plurality of anchor points and frames are easily detected at the same position, or the anchor points and the frames cannot be detected, so that the condition that the commodities are missed or cannot be detected easily occurs, and the detection rate and the recognition rate of the commodities are low.
Disclosure of Invention
An object of embodiments of the present invention is to provide a commodity identification method, a commodity identification device, and a storage medium, so as to solve the problem that the detection rate and the identification rate of a commodity are not high, and improve the detection rate and the identification accuracy rate of the commodity under various conditions.
In order to solve the above technical problem, an embodiment of the present invention provides a method for identifying a commodity, including the following steps: acquiring an image containing a commodity to be detected; detecting the image through the trained convolutional neural network model to obtain a detection result, wherein the detection result is the key point position information and the key point connection relation information of the commodity to be detected; and acquiring information of the to-be-detected commodity according to the detection result.
An embodiment of the present invention further provides a commodity identification device, including: at least one processor; and a memory communicatively coupled to the at least one processor; the storage stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the file storage method or the commodity identification method.
Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which when executed by a processor, performs the above-mentioned article identification method.
Compared with the prior art, the method and the device have the advantages that the image containing the commodity to be detected is obtained, the image is detected through the trained convolutional neural network model, and the detection result is obtained, wherein the detection result is the key point position information and the key point connection relation information of the commodity to be detected, so that the key points of the commodity to be detected are extracted through the trained convolutional neural network model, specific characteristics influencing commodity information can be effectively extracted, and the method and the device are suitable for commodity identification under various specific complex scenes; according to the detection result, the information of the commodity to be detected is obtained, so that the information of the commodity is identified through the key point information, the identification process is stable in operation, the detection rate and the identification accuracy of the commodity are effectively improved, and the commodity identification effect is good.
In addition, the key points at least comprise a starting point, a middle point and an end point; according to the detection result, acquiring the information of the commodity to be detected, which specifically comprises the following steps: searching commodity information corresponding to the detection result according to the corresponding relation between preset commodity key point information and commodity information; the key point information comprises key point position information and key point connection relation information.
In addition, still include: the convolutional neural network model specifically comprises the following steps: convolutional neural networks of multi-branch architecture.
In addition, the convolutional neural network model is trained by: constructing a commodity data set, wherein the commodity data set comprises commodity pictures with labeling information; inputting the commodity data set into a convolutional neural network model, and performing convolution calculation for multiple times to obtain output data; performing at least one calculation of a continuous convolution on the output data; the calculation of continuous convolution is continuous convolution calculation for L times, wherein L is a natural number more than 1; and comparing the output data after the calculation of the continuous convolution with the true value of the key point information, and performing model convergence according to the comparison result. Through training the convolutional neural network model with the multi-branch structure, the method can be used for learning aiming at different attributes at the same time, so that the attribute features of the commodity on different attribute dimensions can be extracted, the specific features influencing commodity information can be better identified and extracted, and the identification accuracy of the commodity is improved.
In addition, the true value of the key point information is obtained by the following method: acquiring a commodity picture set with labeling information, wherein the labeling information at least comprises key point information and category information of a commodity; reading the labeling information, carrying out Gaussian transformation on the labeling information to generate L2 data with commodity key point position information, and carrying out transformation processing on the labeling information to generate L1 data with commodity key point connection relation information; the L2 data and the L1 data are true values of the keypoint information. Therefore, the labeling information subjected to Gaussian transformation becomes a response area with Gaussian distribution, the detection rate of the commodities under the condition of being shielded can be effectively improved, and the occurrence of false detection can be effectively inhibited.
In addition, the calculation of at least one continuous convolution of the output data specifically includes: performing continuous convolution calculation on the output data of the first branch and the second branch to obtain data output by the first branch and data output by the second branch; the calculation of the first branch is used for calculating the connection relation information of the key points, and the calculation of the second branch is used for calculating the position information of the key points; comparing the calculated output data after continuous convolution with the true value of the key point information, specifically: and calculating the square loss of the true value of the data output by the first branch and the key point connection relation information, and calculating the square loss of the true value of the data output by the second branch and the key point position information.
In addition, the calculation of continuous convolution for at least once is performed on the output data, specifically: if the current calculation of the continuous convolution is not the calculation of the first continuous convolution, the output result of the previous calculation of the continuous convolution is superposed with the output data of the multiple convolution calculations, and the superposed data is used as the input data of the current calculation of the continuous convolution. Therefore, at least one time of calculation of continuous convolution is carried out, the receptive field is increased, and the calculation of the key point connection relation information and the key point position information is more accurate.
In addition, model convergence is performed according to the comparison result, specifically: and performing back propagation and gradient updating according to the square loss of the true value of the data output by the first branch and the key point connection relation information and the square loss of the true value of the data output by the second branch and the key point position information. In this way, model parameters are updated according to the calculated square loss until the algorithm finds the model parameters with the lowest possible square loss; the square loss is calculated for the output data obtained by each continuous convolution calculation and the true value, and model parameter updating, namely relay supervision, is performed, so that the problems of gradient loss and overfitting are effectively solved, the gradient distribution condition of each layer is good, and the convolution neural network model can be normally updated along with training.
Drawings
One or more embodiments are illustrated by the corresponding figures in the drawings, which are not meant to be limiting.
Fig. 1 is a flowchart of a product identification method according to a first embodiment of the present invention;
FIG. 2 is a flow chart of training a convolutional neural network model according to a first embodiment of the present invention;
FIG. 3 is a flow chart of training a convolutional neural network model according to a second embodiment of the present invention;
fig. 4 is a schematic configuration diagram of an article recognition apparatus according to a third embodiment of the present invention;
FIG. 5 is an algorithm flow diagram of a convolutional neural network model in accordance with a first embodiment of the present invention;
fig. 6 is an algorithm flowchart of a convolutional neural network model according to a second embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, it will be appreciated by those of ordinary skill in the art that in various embodiments of the invention, numerous technical details are set forth in order to provide a better understanding to the reader, but that the invention is not limited to the specific implementation. The technical solution claimed in the present application can be implemented without these technical details and with various changes and modifications based on the following embodiments.
A first embodiment of the present invention relates to a method for identifying a commodity, and a specific flow is shown in fig. 1. In the embodiment, the image containing the commodity to be detected is obtained, the image is detected through the trained convolutional neural network model to obtain a detection result, specific characteristics influencing commodity information can be effectively extracted, and the method is suitable for commodity identification under various specific complex scenes; according to the detection result, the information of the commodity to be detected is obtained, the identification process is stable in operation, the detection rate and the identification accuracy of the commodity are effectively improved, and the commodity identification effect is good. The flow shown in FIG. 1 is described in detail below:
Specifically, a camera is arranged in a space where the commodity to be detected is located, such as a freezer or a storage compartment for placing the commodity, to shoot the commodity to be detected, and the shot image is an image containing the commodity to be detected.
And 102, detecting the image through the trained convolutional neural network model to obtain a detection result.
Specifically, the detection result is the key point position information and the key point connection relation information of the to-be-detected commodity, wherein the key points at least comprise a starting point, a middle point and an end point. Since the key point is important for describing commodity information, the trained convolutional neural network model is used for extracting the key point information from the image, so that specific characteristics influencing the commodity information can be effectively extracted, and the method is suitable for commodity identification under various specific complex scenes.
And 103, acquiring information of the to-be-detected commodity according to the detection result.
Specifically, commodity information corresponding to the detection result is searched according to the detection result and the corresponding relation between preset key point information and the commodity information; the key point information comprises key point position information and key point connection relation information. Due to the fact that the corresponding relation between the key point information and the commodity information is preset, the corresponding commodity information can be found through the detection result, the commodity is identified through the method, the identification process is stable, the detection rate and the identification accuracy of the commodity are effectively improved, and the commodity identification effect is good.
The convolutional neural network model in this embodiment is specifically: convolutional neural networks of multi-branch architecture. The convolutional neural network model is trained through the process shown in fig. 2: constructing a commodity data set; inputting the commodity data set into a convolutional neural network model, and performing convolution calculation for multiple times to obtain output data; performing at least one calculation of a continuous convolution on the output data; and comparing the output data after the calculation of the continuous convolution with the true value of the key point information, and performing model convergence according to the comparison result. Through training the convolutional neural network model with the multi-branch structure, the method can be used for learning aiming at different attributes at the same time, so that the attribute features of the commodity on different attribute dimensions can be extracted, the specific features influencing commodity information can be better identified and extracted, and the identification accuracy of the commodity is improved. The flow shown in FIG. 2 will be described in detail
Specifically, the commodity data set comprises a commodity picture with label information. In the commodity data set, the labeling information comprises position information and category information of the commodity, namely comprises key point information of the commodity; in practical application, the visible dominant key points can be represented by green points, the invisible key points which are not visible can be represented by red points, and the invisible key points which are possessed by the commodities can be deduced through the operation of the dominant key points and the invisible key points. In addition, the number of commodity pictures in the commodity data set is at least 2000.
And 202, inputting the commodity data set into a convolutional neural network model, and performing convolution calculation for multiple times to obtain output data. As shown in fig. 5, the picture with the key point information is convolved a plurality of times to obtain output data f (conv4_ 4).
Specifically, the pictures in the commodity data set have labeling information, that is, key point information of the commodity, that is, key point position information and key point connection relation information. And performing convolution calculation on the picture with the key point information for multiple times to obtain output data, wherein the convolution calculation can be realized based on a convolution neural network such as vgg or resnet.
Specifically, the calculation of the continuous convolution is a convolution calculation for L times in succession, where L is a natural number greater than 1. The calculation of continuous convolution is realized through the multi-branch structure model, so that the convolution neural network model can learn aiming at different attributes at the same time, the attribute features of the commodity on different attribute dimensions can be extracted, the specific features influencing commodity information can be better identified and extracted, and the identification accuracy of the commodity is improved.
And step 204, comparing the output data after the calculation of the continuous convolution with the true value of the key point information, and performing model convergence according to the comparison result.
Specifically, the calculated output data after continuous convolution is compared with the true value of the key point information, when the error between the calculated output data after continuous convolution and the true value of the key point information is within a preset threshold, convergence of the convolutional neural network model is carried out, namely the parameters of the convolutional neural network model are adjusted, so that the output result of the convolutional neural network model is closer to the true value, and the reliability and the accuracy of the convolutional neural network model are improved. The preset threshold is a certain error range set according to user requirements.
More specifically, the true value of the key point information is obtained by: acquiring a commodity picture set with label information; reading the labeling information, carrying out Gaussian transformation on the labeling information to generate L2 data with commodity key point position information, and carrying out transformation processing on the labeling information to generate L1 data with commodity key point connection relation information; the L2 data and the L1 data are true values of the keypoint information. The labeling information at least comprises key point information and category information of the commodity.
More specifically, the labeling information is subjected to gaussian transformation to generate L2 data having a product key point, specifically: obtaining the position of a key point in the labeling information and the pixel value of the key point, carrying out Gaussian transformation on the pixel value in a certain range by taking the key point as the center, and changing the position information of the key point into a response area with Gaussian distribution after the Gaussian transformation, wherein the transformation formula involved in the Gaussian transformation is
The size of the gaussian radius varies with the size of the item being labeled. The method comprises the following steps of performing conversion processing on the labeling information to generate L1 data with the commodity key point connection relation information, specifically: presetting a connection relation between two key points; acquiring coordinate information of key points in the labeling information, taking the coordinate difference value in the x direction as the pixel value in the x direction of the connection between the two key points, and taking the y direction as the pixel value in the y directionThe coordinate difference value of the key point is used as a pixel value of the y direction of the connection between the two key points, so that the connection relation between each key point generates connection relation information of 2 directions, namely the connection relation information in the x direction and the connection relation information in the y direction; all the connection relation information is combined to constitute L1 data.
Compared with the prior art, the method and the device have the advantages that the image containing the commodity to be detected is obtained, the image is detected through the trained convolutional neural network model, the detection result is obtained, the detection result is the key point position information and the key point connection relation information of the commodity to be detected, so that the key points of the commodity to be detected are extracted through the trained convolutional neural network model, specific characteristics influencing commodity information can be effectively extracted, and the method and the device are suitable for commodity identification under various specific complex scenes; according to the detection result, the information of the commodity to be detected is obtained, the commodity is identified by the method, the identification process is stable in operation, the detection rate and the identification accuracy of the commodity are effectively improved, and the commodity identification effect is good. Training a convolutional neural network model to construct a commodity data set, wherein the commodity data set comprises commodity pictures with labeling information; inputting the commodity data set into a convolutional neural network model, performing convolutional calculation for multiple times to obtain output data, and performing at least one continuous convolutional calculation on the output data, so that the convolutional neural network model can learn different attributes at the same time, namely, the attribute features of the commodity on different attribute dimensions can be extracted, the identification and the extraction of specific features influencing commodity information are facilitated, and the identification accuracy of the commodity is improved; and comparing the output data after the calculation of the continuous convolution with the true value of the key point information, and performing model convergence according to the comparison result, so that the output result of the model is closer to the true data, and the reliability and the accuracy of the model are improved.
The second embodiment of the present invention relates to a training method for a convolutional neural network model, and is specifically shown in fig. 3. The second embodiment is substantially the same as the training method for the convolutional neural network model according to the first embodiment, and the main differences are that: in the second embodiment of the present invention, a way of performing at least one continuous convolution calculation on the output data is specified, and this way, it is helpful to better identify and extract specific features that affect the information of the commodity, so as to improve the identification accuracy of the commodity; the calculation of continuous convolution can be carried out for multiple times, so that the receptive field is increased, and the precision of the calculation result is improved; the output result of the model is closer to the real data, the reliability and the accuracy of the model are improved, the problems of gradient loss and overfitting are effectively relieved, the gradient distribution condition of each layer is good, and the convolutional neural network model can be normally updated along with training. The flow shown in fig. 3 is explained in detail below:
Specifically, in which the calculation of the first branch is used to calculate the key point connection relation information, and the calculation of the second branch is used to calculate the key point location information, as shown in fig. 5, the network structure to conv4_4_ CPM is divided into two branches, the first branch (i.e. the left branch in fig. 5): the con5_1_ CPM _ L1, con5_2_ CPM _ L1, con5_3_ CPM _ L1, con5_4_ CPM _ L1, and con5_5_ CPM _ L1 perform convolution calculation of the connection relationship of commodities, and finally take the output data f (con5_5_ CPM _ L1) of the con5_5_ CPM _ L1.
Second branch (i.e. the right branch in fig. 5): the con5_1_ CPM _ L2, con5_2_ CPM _ L2, con5_3_ CPM _ L2, con5_4_ CPM _ L, and con5_5_ CPM _ L2 perform convolution calculation of the positions of the key points of the commodity, and finally take the output data f of con5_5_ CPM _ L2 (con5_5_ CPM _ L2).
By the mode, the convolutional neural network model can be learned according to different attributes at the same time, so that the attribute features of the commodity on different attribute dimensions can be extracted, the specific features influencing commodity information can be better identified and extracted, and the identification accuracy of the commodity is improved.
And step 304, calculating the square loss of the true value of the data output by the first branch and the key point connection relation information, and calculating the square loss of the true value of the data output by the second branch and the key point position information.
Specifically, the square Loss of the true value of the data and the key point connection relationship information output by the first branch is calculated by calculating the square Loss through a matrix Y1 of output data f (con5_5_ CPM _ L1) and L1 data, and the square Loss is Loss _ stage1_ direction _ Loss1(Y1, f (con5_5_ CPM _ L1))) (Y1-f (con5_5_ CPM _ L1))2(ii) a The square Loss of the true value of the data output by the second branch and the key point position information is calculated by a matrix Y2 of output data f (con5_5_ CPM _ L2) and L2 data, wherein the square Loss is calculated by a matrix Y2 of the output data f (con5_5_ CPM _ L2), and the square Loss is Loss _ stage1_ direction _ Loss1(Y2, f (con5_5_ CPM _ L2)), (Y2-f (con5_5_ CPM _ L2))2(ii) a The squared loss is used to represent the amount of loss on the calculated output of the current continuous convolution.
And 305, performing back propagation and gradient updating according to the square loss of the true value of the data output by the first branch and the key point connection relation information and the square loss of the true value of the data output by the second branch and the key point position information.
Specifically, updating model parameters according to the calculated square loss until the algorithm finds the model parameters with the lowest possible square loss; calculating the square loss of the output data obtained by each continuous convolution calculation and the real data, and updating the model parameters, thereby effectively relieving the problems of gradient loss and overfitting; if the method is called relay supervision, the top layer gradient is uniformly distributed, and the bottom layer gradient is concentrated near 0, so that the bottom layer convolutional neural network model can hardly be updated normally; when the relay supervision exists, the gradient distribution condition of each layer is good, and the convolutional neural network model can be updated normally along with training.
It should be noted that, in practical applications, multiple calculations of consecutive convolutions may also be performed, as shown in fig. 6: if the calculation of the current continuous convolution is not the calculation of the first continuous convolution, overlapping the output results of the calculation of the previous continuous convolution (such as output data f (con5_5_ CPM _ L1) and output data f (con5_5_ CPM _ L2) in fig. 6) with the output data f (conv4_4) of the multiple convolution calculation, and taking the overlapped data as the input data of the calculation of the current continuous convolution (such as f (concat _ stage2) in fig. 6); performing continuous convolution calculation of the first branch and the second branch on the input data of the current continuous convolution calculation to obtain data output by the first branch and data output by the second branch, as in step 303; calculating the square loss of the real value of the data output by the first branch and the key point connection relation information and the square loss of the real value of the data output by the second branch and the key point position information (step 304); by the method, the calculation of continuous convolution is carried out for multiple times, so that the receptive field is increased, and the precision of the calculation result is improved; and the input data for each calculation comprises output data for carrying out convolution calculation on the picture with the key point information for multiple times, so that the calculation of the key point connection relation information and the key point position information is more accurate.
Compared with the prior art, the embodiment performs continuous convolution calculation of the first branch and the second branch on the output data, so that the convolution neural network model can learn different attributes at the same time, namely, the attribute features of the commodity on different attribute dimensions can be extracted, the specific features influencing commodity information can be better identified and extracted, and the identification accuracy of the commodity is improved; if the current continuous convolution calculation is the calculation of non-first continuous convolution, the output result of the previous continuous convolution calculation is superposed with the output data of the multiple convolution calculations, and the superposed data is used as the input data of the current continuous convolution calculation, so that the receptive field is increased, the precision of the calculation result is improved, and the calculation of the key point connection relation information and the key point position information is more accurate; and calculating the square loss of the true value of the data output by the first branch and the key point connection relation information and the square loss of the true value of the data output by the second branch and the key point position information, and performing back propagation and gradient updating to effectively solve the problems of gradient loss and overfitting, so that the gradient distribution condition of each layer is good, and the convolutional neural network model can be normally updated along with training.
The steps of the above methods are divided for clarity, and the implementation may be combined into one step or split some steps, and the steps are divided into multiple steps, so long as the same logical relationship is included, which are all within the protection scope of the present patent; it is within the scope of the patent to add insubstantial modifications to the methods or processes or to introduce design implications that do not alter the core design of the methods or processes.
A third embodiment of the present invention relates to an article identification device, as shown in fig. 4, comprising at least one processor 402; and a memory 401 communicatively coupled to the at least one processor 402; the memory 401 stores instructions executable by the at least one processor 402, and the instructions are executed by the at least one processor 402 to enable the at least one processor 402 to execute the above-mentioned article identification method.
Where memory 401 and processor 402 are coupled by a bus, which may include any number of interconnected buses and bridges that couple one or more of the various circuits of memory 401 to each other. The bus may also connect various other circuits such as peripherals, voltage regulators, power management circuits, and the like, which are well known in the art, and therefore, will not be described any further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver may be one element or a plurality of elements, such as a plurality of receivers and transmitters, providing a means for communicating with various other apparatus over a transmission medium. The data processed by the processor 402 is transmitted over a wireless medium through an antenna, which further receives the data and transmits the data to the processor 402.
The processor 402 is responsible for managing the bus and general processing and may also provide various functions including timing, peripheral interfaces, voltage regulation, power management, and other control functions. And memory 401 may be used to store data used by processor 402 in performing operations.
It should be noted that each module referred to in this embodiment is a logical module, and in practical applications, one logical unit may be one physical unit, may be a part of one physical unit, and may be implemented by a combination of multiple physical units. In addition, in order to highlight the innovative part of the present invention, elements that are not so closely related to solving the technical problems proposed by the present invention are not introduced in the present embodiment, but this does not indicate that other elements are not present in the present embodiment.
A fifth embodiment of the present invention relates to a computer-readable storage medium storing a computer program. The computer program implements the above-described file storage method embodiments, or implements the above-described file deletion method embodiments, when executed by the processor 402.
That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiments of the file storage method or the file deletion method can be completed by instructing the relevant hardware through a program, where the program is stored in a storage medium and includes several instructions to enable a device (which may be a single chip, a chip, etc.) or a processor (processor) to execute all or part of the steps of the methods according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
It will be understood by those of ordinary skill in the art that the foregoing embodiments are specific examples for carrying out the invention, and that various changes in form and details may be made therein without departing from the spirit and scope of the invention in practice.
Claims (7)
1. A method for identifying an article, comprising:
acquiring an image containing a commodity to be detected;
detecting the image through the trained convolutional neural network model to obtain a detection result,
the detection result is the position information of the key points of the to-be-detected commodity and the connection relation information of the key points;
acquiring information of the to-be-detected commodity according to the detection result;
the convolutional neural network model specifically comprises the following steps: a convolutional neural network of a multi-branch structure;
the convolutional neural network model is trained in the following way:
constructing a commodity data set, wherein the commodity data set comprises commodity pictures with labeling information; inputting the commodity data set into the convolutional neural network model, and performing convolution calculation for multiple times to obtain output data;
performing at least one calculation of a continuous convolution on the output data; the calculation of the continuous convolution is continuous convolution calculation for L times, wherein L is a natural number more than 1;
comparing the calculated output data subjected to the continuous convolution with the true value of the key point information, and performing model convergence according to a comparison result;
the real value of the key point information is obtained through the following method: acquiring a commodity picture set with labeling information, wherein the labeling information at least comprises key point information and category information of the commodity; reading the annotation information, performing Gaussian transformation on the annotation information to generate L2 data with the commodity key point position information, and performing transformation processing on the annotation information to generate L1 data with the commodity key point connection relation information; the L2 data and the L1 data are true values of the keypoint information.
2. The article identification method according to claim 1,
the key points at least comprise a starting point, a middle point and an end point;
the acquiring of the information of the to-be-detected commodity according to the detection result specifically comprises:
searching commodity information corresponding to the detection result according to the corresponding relation between preset commodity key point information and commodity information;
the key point information comprises key point position information and key point connection relation information.
3. The product identification method according to claim 1, wherein the calculating of the at least one continuous convolution of the output data specifically includes:
performing continuous convolution calculation on the first branch and the second branch on the output data to obtain data output by the first branch and data output by the second branch;
the calculation of the first branch is used for calculating the connection relation information of the key points, and the calculation of the second branch is used for calculating the position information of the key points;
comparing the calculated output data after the continuous convolution with the true value of the key point information, specifically: and calculating the square loss of the true value of the data output by the first branch and the key point connection relation information, and calculating the square loss of the true value of the data output by the second branch and the key point position information.
4. The product identification method according to claim 1, wherein the calculation of the at least one continuous convolution of the output data is specifically:
and if the current calculation of the continuous convolution is not the calculation of the first continuous convolution, overlapping the output result of the previous calculation of the continuous convolution and the output data of the multiple convolution calculations, and taking the overlapped data as the input data of the current calculation of the continuous convolution.
5. The product identification method according to claim 3, wherein the model convergence is performed according to the comparison result, specifically:
and performing back propagation and gradient updating according to the square loss of the true value of the data output by the first branch and the key point connection relation information and the square loss of the true value of the data output by the second branch and the key point position information.
6. An article identification device, comprising:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the article identification method of any one of claims 1 to 5.
7. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the article identification method of any one of claims 1 to 5.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811171626.3A CN109472205B (en) | 2018-10-09 | 2018-10-09 | Commodity identification method, commodity identification device, and storage medium |
| PCT/CN2019/078162 WO2020073601A1 (en) | 2018-10-09 | 2019-03-14 | Goods recognition method, goods recognition apparatus, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811171626.3A CN109472205B (en) | 2018-10-09 | 2018-10-09 | Commodity identification method, commodity identification device, and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109472205A CN109472205A (en) | 2019-03-15 |
| CN109472205B true CN109472205B (en) | 2021-07-30 |
Family
ID=65664796
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811171626.3A Active CN109472205B (en) | 2018-10-09 | 2018-10-09 | Commodity identification method, commodity identification device, and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN109472205B (en) |
| WO (1) | WO2020073601A1 (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110163315A (en) * | 2019-05-06 | 2019-08-23 | 南京倍便利智能科技有限公司 | A kind of commodity visual identity method for chain unmanned supermarket |
| CN110245580B (en) * | 2019-05-24 | 2022-09-23 | 北京百度网讯科技有限公司 | Method, device and equipment for detecting image and computer storage medium |
| CN111259822A (en) * | 2020-01-19 | 2020-06-09 | 杭州微洱网络科技有限公司 | Method for detecting key point of special neck in E-commerce image |
| CN111783653B (en) * | 2020-07-01 | 2024-04-26 | 创新奇智(西安)科技有限公司 | Trademark label detection method, device, equipment and storage medium |
| CN111862031A (en) * | 2020-07-15 | 2020-10-30 | 北京百度网讯科技有限公司 | Face synthetic image detection method and device, electronic equipment and storage medium |
| CN112132131B (en) * | 2020-09-22 | 2024-05-03 | 深兰科技（上海）有限公司 | Measuring cylinder liquid level identification method and device |
| CN112257646B (en) * | 2020-11-02 | 2023-09-12 | 创新奇智(南京)科技有限公司 | Commodity detection method and device, electronic equipment and storage medium |
| CN113537234A (en) * | 2021-06-10 | 2021-10-22 | 浙江大华技术股份有限公司 | Quantity counting method and device, electronic device and computer equipment |
| CN113780441B (en) * | 2021-09-16 | 2022-08-30 | 广东佩服科技有限公司 | Method for constructing commodity identification model |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105138610A (en) * | 2015-08-07 | 2015-12-09 | 深圳码隆科技有限公司 | Image element based image feature value prediction method and apparatus |
| CN106874296B (en) * | 2015-12-14 | 2021-06-04 | 阿里巴巴集团控股有限公司 | Method and device for identifying style of commodity |
| CN106126579B (en) * | 2016-06-17 | 2020-04-28 | 北京市商汤科技开发有限公司 | Object identification method and device, data processing device and terminal equipment |
| CN106295567B (en) * | 2016-08-10 | 2019-04-12 | 腾讯科技（深圳）有限公司 | A kind of localization method and terminal of key point |
| CN107798277A (en) * | 2016-09-05 | 2018-03-13 | 合肥美的智能科技有限公司 | Food materials identifying system and method, food materials model training method, refrigerator and server |
| KR102610030B1 (en) * | 2016-11-15 | 2023-12-04 | 매직 립, 인코포레이티드 | Deep learning system for cuboid detection |
| CN108229287B (en) * | 2017-05-31 | 2020-05-22 | 北京市商汤科技开发有限公司 | Image recognition method and device, electronic equipment and computer storage medium |
| CN209132890U (en) * | 2017-09-27 | 2019-07-19 | 中山市宾哥网络科技有限公司 | Settle accounts case |
| CN108447061B (en) * | 2018-01-31 | 2020-12-08 | 深圳市阿西莫夫科技有限公司 | Commodity information processing method and device, computer equipment and storage medium |
| CN108521589A (en) * | 2018-04-25 | 2018-09-11 | 北京比特智学科技有限公司 | Method for processing video frequency and device |
-
2018
- 2018-10-09 CN CN201811171626.3A patent/CN109472205B/en active Active
-
2019
- 2019-03-14 WO PCT/CN2019/078162 patent/WO2020073601A1/en active Application Filing
Also Published As
| Publication number | Publication date |
|---|---|
| WO2020073601A1 (en) | 2020-04-16 |
| CN109472205A (en) | 2019-03-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109472205B (en) | Commodity identification method, commodity identification device, and storage medium | |
| CN109447078B (en) | Detection and identification method for natural scene image sensitive characters | |
| US10885365B2 (en) | Method and apparatus for detecting object keypoint, and electronic device | |
| CN108416902B (en) | Real-time object identification method and device based on difference identification | |
| CN108073902B (en) | Video summarizing method and device based on deep learning and terminal equipment | |
| CN111061890B (en) | Method for verifying labeling information, method and device for determining category | |
| Barroso-Laguna et al. | Key. net: Keypoint detection by handcrafted and learned cnn filters revisited | |
| CN110555399B (en) | Finger vein identification method and device, computer equipment and readable storage medium | |
| CN108416258B (en) | Multi-human body tracking method based on human body part model | |
| CN112612913A (en) | Image searching method and system | |
| CN115953665B (en) | Target detection method, device, equipment and storage medium | |
| CN110019895B (en) | Image retrieval method and device and electronic equipment | |
| US11475500B2 (en) | Device and method for item recommendation based on visual elements | |
| WO2019007253A1 (en) | Image recognition method, apparatus and device, and readable medium | |
| CN113627508B (en) | Display scene recognition method, device, equipment and storage medium | |
| CN114861842B (en) | Few-sample target detection method and device and electronic equipment | |
| CN115115825B (en) | Method, device, computer equipment and storage medium for detecting object in image | |
| CN113935774A (en) | Image processing method, image processing device, electronic equipment and computer storage medium | |
| CN115471681A (en) | Image recognition method, device and storage medium | |
| CN117115571A (en) | Fine-grained intelligent commodity identification method, device, equipment and medium | |
| CN111222546A (en) | Multi-scale fusion food image classification model training and image classification method | |
| CN111126457A (en) | Information acquisition method and device, storage medium and electronic device | |
| CN115641449A (en) | Target tracking method for robot vision | |
| CN113901175A (en) | Article relation judging method and device | |
| Guo et al. | Image saliency detection based on geodesic‐like and boundary contrast maps |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20221108 Address after: 315000 No. 138-1, Zhongshan West Road, Fenghua District, Ningbo City, Zhejiang Province (self declaration) Patentee after: Shenlan Intelligent Technology Research Institute (Ningbo) Co.,Ltd. Address before: 200050 room 6113, 6th floor, 999 Changning Road, Changning District, Shanghai Patentee before: DEEPBLUE TECHNOLOGY (SHANGHAI) Co.,Ltd. |
|
| TR01 | Transfer of patent right |