CN113705594B - Image identification method and device - Google Patents

Image identification method and device Download PDF

Info

Publication number
CN113705594B
CN113705594B CN202010436720.8A CN202010436720A CN113705594B CN 113705594 B CN113705594 B CN 113705594B CN 202010436720 A CN202010436720 A CN 202010436720A CN 113705594 B CN113705594 B CN 113705594B
Authority
CN
China
Prior art keywords
model
probability
full
connection
picture
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010436720.8A
Other languages
Chinese (zh)
Other versions
CN113705594A (en
Inventor
周彬
黄明星
李银锋
赖晨东
李晓敏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jingdong Century Trading Co Ltd
Beijing Wodong Tianjun Information Technology Co Ltd
Original Assignee
Beijing Jingdong Century Trading Co Ltd
Beijing Wodong Tianjun Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jingdong Century Trading Co Ltd, Beijing Wodong Tianjun Information Technology Co Ltd filed Critical Beijing Jingdong Century Trading Co Ltd
Priority to CN202010436720.8A priority Critical patent/CN113705594B/en
Publication of CN113705594A publication Critical patent/CN113705594A/en
Application granted granted Critical
Publication of CN113705594B publication Critical patent/CN113705594B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • G06F18/241Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/047Probabilistic or stochastic networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a method and a device for identifying images, and relates to the technical field of computers. One embodiment of the method comprises the following steps: constructing a plurality of recognition probability models based on a plurality of convolutional neural network models, respectively calculating sub-image recognition probabilities, acquiring the recognition probability of a target sub-image in a picture to be recognized by utilizing a prediction classification model through fusing each recognition probability model, and further determining whether the picture contains the target sub-image according to the recognition probability; the technical means solves the problem that the accuracy of the identification image is lower in a part of the technology under the condition of limited data samples, improves the accuracy of the identification image, and reduces the workload of manually checking and verifying the identification image.

Description

Image identification method and device
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method and apparatus for identifying an image.
Background
In the outer package of the commodity, an authentication mark which reflects commodity characteristics such as commodity quality, functions, environmental protection and the like is usually contained, the authentication mark usually appears in an outer package picture in a small picture form, and a consumer can judge the commodity characteristics through the authentication mark of the outer package of the commodity; for example, with the improvement of living standard, health products are increasingly favored by consumers, but some vendors sell general foods as health products, so that the national issues health product authentication marks for standardizing the behaviors of the vendors and guaranteeing the benefits of the consumers, and only the health products which are authenticated by the national can be identified in the commodity outer package;
In the process of implementing the present invention, the inventor finds that at least the following problems exist in the prior art:
At present, the problem of low accuracy rate caused by limited data samples exists in an identification model of a commodity authentication mark in a commodity outer package, manual auditing is often needed to be further clarified, and the problems of low working efficiency, dependence on subjective judgment, possible auditing cheating and the like exist in the manual auditing.
Disclosure of Invention
In view of the above, the embodiments of the present invention provide a method and an apparatus for identifying an image, which can construct multiple recognition probability models based on multiple convolutional neural network models, respectively calculate sub-image recognition probabilities, and obtain the recognition probability of a target sub-image in a picture to be identified by fusing each recognition probability model and using a prediction classification model, and further determine whether the picture contains the target sub-image according to the recognition probability; the technical means solves the problem that the accuracy of the identification target subgraph is lower in part of the technology under the condition of limited data samples, improves the accuracy of the identification image, and reduces the workload of manual verification and authentication of the identification image.
To achieve the above object, according to one aspect of the embodiments of the present invention, there is provided a method of recognizing an image, including: constructing a first full-connection model based on the first convolutional neural network model to form a first recognition probability model; constructing a second full-connection model based on the second convolutional neural network model to form a second recognition probability model; calculating the class probability of the target subgraph existing in the picture by using the first recognition probability model, the second recognition probability model and the classification prediction model; and determining whether the target subgraph is contained in the picture according to a predefined probability threshold.
Optionally, the method for identifying an image is characterized in that,
Based on a first convolutional neural network model, a first fully connected model is constructed to form a first recognition probability model, comprising:
And constructing a first full-connection model, replacing a classifier of the first convolutional neural network model, and forming the first recognition probability model according to a convolutional layer, a pooling layer, a full-connection layer and the first full-connection model of the first convolutional neural network model.
Optionally, the method for identifying an image is characterized in that,
Based on a second convolutional neural network model, constructing a second fully connected model to form a second recognition probability model, comprising:
And constructing a second full-connection model, replacing a full-connection layer of the second convolutional neural network model, and forming the second recognition probability model according to the convolutional layer, the pooling layer, the data block and the second full-connection model of the second convolutional neural network model.
Optionally, the method for identifying an image is characterized in that,
Building the first fully connected model or the second fully connected model comprises the following steps:
and constructing at least two linear layers, respectively setting the quantity of neurons contained in the linear layers, and performing full-connection calculation on the values of the neurons contained in the linear layers by using a full-connection neural network model.
Optionally, the method for identifying an image is characterized in that,
Based on a second convolutional neural network model, constructing a second fully connected model to form a second recognition probability model, comprising:
constructing at least two second linear full-connection models, and respectively replacing full-connection layers of the corresponding second convolutional neural network models; and respectively forming the second recognition probability model according to a convolution layer, a pooling layer, a data block and the corresponding second full-connection model of the second convolution neural network model.
Optionally, the method for identifying an image is characterized in that,
Calculating the class probability of the target subgraph existing in the picture by using the first recognition probability model, the second recognition probability model and the classification prediction model, wherein the method comprises the following steps:
Calculating the class probability of the target subgraph existing in the picture by using the first recognition probability model to obtain a first probability;
calculating the class probability of the target subgraph existing in the picture by using one or more second recognition probability models to obtain second probabilities respectively;
and calculating the class probability of the target subgraph existing in the picture based on the class prediction model by using the first probability and the one or more second probabilities.
Optionally, the method for identifying an image is characterized in that,
The classification prediction model comprises: and calculating the class probability of the target subgraph existing in the picture based on the first probability, one or more second probabilities, the weight value of the first probability and the weight value corresponding to one or more second probabilities.
In order to achieve the above object, according to a second aspect of the embodiments of the present invention, there is provided an apparatus for recognizing an image, comprising: constructing an identification model module, a calculation identification probability module and an identification sub-graph module; wherein,
The recognition model constructing module is used for constructing a first full-connection model based on the first convolutional neural network model to form a first recognition probability model; constructing a second full-connection model based on the second convolutional neural network model to form a second recognition probability model;
The recognition probability calculating module is used for calculating the class probability of the target subgraph existing in the picture by using the first recognition probability model, the second recognition probability model and the classification prediction model;
And the sub-picture identifying module is used for determining whether the picture contains the target sub-picture or not according to a predefined probability threshold.
Optionally, the device for recognizing an image is characterized in that,
The recognition model constructing module is further used for constructing a first full-connection model, replacing a classifier of the first convolutional neural network model and forming the first recognition probability model according to a convolutional layer, a pooling layer, a full-connection layer and the first full-connection model of the first convolutional neural network model.
Optionally, the device for recognizing an image is characterized in that,
The recognition model constructing module is further used for constructing a second full-connection model to replace a full-connection layer of the second convolutional neural network model, and the second recognition probability model is formed according to the convolutional layer, the pooling layer, the data block and the second full-connection model of the second convolutional neural network model.
Optionally, the device for recognizing an image is characterized in that,
The recognition model constructing module is further used for constructing at least two linear layers, setting the number of neurons contained in the linear layers respectively, and carrying out full-connection calculation on the values of the neurons contained in the linear layers by utilizing a full-connection neural network model.
Optionally, the device for recognizing an image is characterized in that,
The recognition model constructing module is further used for constructing at least two second linear full-connection models, and the full-connection layers of the corresponding second convolutional neural network models are replaced respectively; and respectively forming the second recognition probability model according to a convolution layer, a pooling layer, a data block and the corresponding second full-connection model of the second convolution neural network model.
Optionally, the device for recognizing an image is characterized in that,
The recognition probability calculation module is further used for calculating the class probability of the target subgraph existing in the picture by using the first recognition probability model to obtain a first probability; calculating the class probability of the target subgraph existing in the picture by using one or more second recognition probability models to obtain second probabilities respectively; and calculating the class probability of the target subgraph existing in the picture based on the class prediction model by using the first probability and the one or more second probabilities.
Optionally, the device for recognizing an image is characterized in that,
The identified subgraph module also includes the classification prediction model, which includes: and calculating the class probability of the target subgraph existing in the picture based on the first probability, one or more second probabilities, the weight value of the first probability and the weight value corresponding to one or more second probabilities.
In order to achieve the above object, according to a third aspect of an embodiment of the present invention, there is provided an electronic apparatus for recognizing an image, comprising: one or more processors; and a storage means for storing one or more programs that, when executed by the one or more processors, cause the one or more processors to implement the method of any of the methods of identifying images described above.
To achieve the above object, according to a fourth aspect of embodiments of the present invention, there is provided a computer-readable medium having stored thereon a computer program, characterized in that the program, when executed by a processor, implements a method as described in any one of the above methods of recognizing an image.
The embodiment of the invention has the following advantages or beneficial effects that the embodiment can respectively calculate the sub-graph recognition probabilities by constructing a plurality of recognition probability models based on a plurality of convolutional neural network models, and acquire the recognition probability of the target sub-graph in the picture to be recognized by fusing each recognition probability model and utilizing the prediction classification model, and further, determine whether the picture contains the target sub-graph according to the recognition probability; the technical means solves the problem that the accuracy of the identification target subgraph is lower in part of the technology under the condition of limited data samples, improves the accuracy of the identification image, and reduces the workload of manual verification and authentication of the identification image.
Further effects of the above-described non-conventional alternatives are described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a flow chart of a method for recognizing an image according to a first embodiment of the present invention;
FIG. 2 is a schematic flow chart of a first recognition probability model according to one embodiment of the present invention;
FIG. 3 is a flow chart of a second recognition probability model provided by one embodiment of the present invention;
FIG. 4A is an example of a picture to be identified provided by one embodiment of the present invention;
FIG. 4B is an example of a target subgraph provided by one embodiment of the present invention;
FIG. 5 is a flowchart of a method for recognizing an image according to a second embodiment of the present invention;
FIG. 6A is an exemplary diagram of a first full connection model provided by one embodiment of the present invention;
FIG. 6B is an exemplary diagram of a second full connection model provided by one embodiment of the present invention;
FIG. 6C is an exemplary diagram of another second full connection model provided by one embodiment of the present invention;
FIG. 7 is a schematic diagram of an apparatus for recognizing an image according to an embodiment of the present invention;
FIG. 8 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;
Fig. 9 is a schematic diagram of a computer system suitable for use in implementing an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
As shown in fig. 1, an embodiment of the present invention provides a method for identifying an image, which may include the steps of:
Step S101: constructing a first full-connection model based on the first convolutional neural network model to form a first recognition probability model; constructing a second full-connection model based on the second convolutional neural network model to form a second recognition probability model;
Specifically, a first full-connection model is built based on a first convolutional neural network model, and a first recognition probability model is formed, wherein the first convolutional neural network model is a known model, such as a VGG16 model; removing a classifier in the VGG16 model based on the pre-trained VGG16 model with target sub-graph recognition completed, and replacing the classifier with the constructed first full-connection model; further, the data output of a convolution layer, a pooling layer and a full-connection layer which are included in the VGG16 model are utilized as the data input of the first full-connection model, and the recognition probability of the subgraph in the large graph is output through calculation of the first full-connection model; the process of forming the first recognition probability model based on the VGG16 model is shown as step S201 to step S203 in fig. 2;
That is, based on the first convolutional neural network model, a first fully connected model is constructed, forming a first recognition probability model, comprising: and constructing a first full-connection model, replacing a classifier of the first convolutional neural network model, and forming the first recognition probability model according to a convolutional layer, a pooling layer, a full-connection layer and the first full-connection model of the first convolutional neural network model.
Constructing a second fully connected model based on a second convolutional neural network model, forming a second recognition probability model, wherein the second convolutional neural network model is another known model, such as DenseNet model; removing the fully connected layer in the DenseNet model based on the DenseNet model which has completed the target subgraph recognition pre-training, and replacing the fully connected layer with the constructed second fully connected model; further, the data output of the convolutional layer, the pooling layer and the data block included by the DenseNet model is utilized, and in DenseNet, the data block is a dense block; the data input of the second full-connection model is used for outputting the recognition probability of the subgraph in the large graph through the calculation of the second full-connection model; the process of forming the first recognition probability model based on DenseNet models is shown as step S301-step S303 in fig. 3;
that is, constructing a second fully connected model based on the second convolutional neural network model to form a second recognition probability model, comprising: and constructing a second full-connection model, replacing a full-connection layer of the second convolutional neural network model, and forming the second recognition probability model according to the convolutional layer, the pooling layer, the data block and the second full-connection model of the second convolutional neural network model.
Further, constructing the first fully connected model or the second fully connected model includes: and constructing at least two linear layers, respectively setting the quantity of neurons contained in the linear layers, and performing full-connection calculation on the values of the neurons contained in the linear layers by using a full-connection neural network model.
The above steps are described below in connection with the examples of fig. 6A and 6B, and as shown in fig. 6A, four linear layers are included, each of which includes a different number of neurons, for example 1024 neurons in the linear layer 1 in fig. 6A, and each circle in the linear layer represents the value of one neuron; similarly, the linear layer 2 contains 512 neurons, with each circle in the linear layer representing the value of one neuron; the linear layer 3 contains 256 neurons, and each circle in the linear layer represents the value of one neuron; the linear layer 4 contains 1 neuron, and each circle in the linear layer represents the value of one neuron; further, as shown in fig. 6A, the value of each neuron of the linear layer 1 to the value of each neuron of the linear layer 4 is calculated based on the fully connected neural network model, wherein the linear layer 1 is an input layer of the fully connected neural network model; the linear layer 2 and the linear layer 3 are hidden layers of the fully-connected network neural model; the linear layer 4 is an output layer of the fully-connected neural network model;
It can be understood that the value of each neuron represents a characteristic value related to a picture, and when calculating based on the fully connected neural network model, a corresponding weight value is set for each operation relation of the value of each neuron of the linear layer 1 and the value of each neuron of the linear layer 2; similarly, the same is true between the linear layers 2 and 3, and between the linear layers 3 and 4; further, calculating and classifying characteristics of the linear layer 1, the linear layer 2, the linear layer 3 and the linear layer 4 to obtain a class probability value of the subgraph in the large graph; it can be understood that the number of the linear layers and the number of the neurons contained in the linear layers are set according to application scenes, experimental data or experience;
An example of a first fully connected model as shown in fig. 6A is for forming a first recognition probability model based on the VGG16 model; similarly, as shown in fig. 6B, the linear layer 1, the linear layer 2, the linear layer 3 and the linear layer 4 are included, and the number of neurons included in each of the linear layers is 512, 256, 128 and 1; an example of a second fully connected model as shown in FIG. 6B is to form a second recognition probability model based on DenseNet models; similarly, as shown in fig. 6C, the linear layer 1, the linear layer 2, the linear layer 3 and the linear layer 4 are included, and the number of neurons included in each of the layers is 256, 128, 64 and 1; an example of a second fully connected model as shown in fig. 6C is to form another second recognition probability model based on the DenseNet model.
The number of the linear layers and the number of the neurons contained in the linear layers contained in the first full-connection model or the second full-connection model, and the numerical value of the neurons are not limited.
Step S102: and calculating the class probability of the target subgraph existing in the picture by using the first recognition probability model, the second recognition probability model and the classification prediction model.
Specifically, based on the description of step S101, a first probability of a target subgraph in a picture to be identified is obtained through a first identification probability model; acquiring a second probability of a target subgraph in the picture to be identified through a second identification probability model;
Further, based on the classification prediction model, a classification probability of the target subgraph existing in the picture is calculated. Specifically, classifying the prediction model includes calculating a class probability of the target sub-graph existing in the picture based on the first probability, one or more of the second probabilities, a weight value of the first probability, and one or more weight values corresponding to the second probabilities.
For example, a first probability of a target sub-graph in a picture to be identified is 0.9 through a first identification probability model, and a second probability of the target sub-graph in the picture to be identified is 0.8 through a second identification probability model; taking a formula (1) as an example to calculate through a classification prediction model, wherein, C represents a class probability, C1 represents a first probability value, C2 represents a second probability value, w1 represents a weight value corresponding to the first probability, w2 represents a weight value corresponding to the second probability, for example, w1 is 0.3, w2 is 0.7, and C is equal to 0.83 through calculation;
C=w1*c1+w2*c2 (1)
that is, calculating the class probability of the target subgraph existing in the picture using the first and second recognition probability models and the classification prediction model includes:
Calculating the class probability of the target subgraph existing in the picture by using the first recognition probability model to obtain a first probability;
calculating the class probability of the target subgraph existing in the picture by using one or more second recognition probability models to obtain second probabilities respectively;
and calculating the class probability of the target subgraph existing in the picture based on the class prediction model by using the first probability and the one or more second probabilities.
The specific formulas and the specific contents in the formulas of the classification prediction model are not limited in the invention.
Step S103: and determining whether the target subgraph is contained in the picture according to a predefined probability threshold.
Specifically, whether the pre-defined probability threshold of the target subgraph is contained in the picture is determined according to experimental data or experience, for example, the pre-defined probability threshold is set to be 0.78; taking the calculation result of step S102 as an example, the class probability obtained in step S102 is 0.83;0.83>0.78, the picture is considered to contain a target subgraph, for example, the picture of the outer package of the health product is considered to contain a target subgraph of 'health food', the outer package picture of the health product is shown in fig. 4A, and the upper left corner of the picture contains a small picture as shown in fig. 4B, namely the target subgraph;
That is, it is determined whether the target subgraph is contained in the picture according to a predefined probability threshold.
It can be understood that the class probabilities are obtained by bringing the pictures to be identified into different pre-trained convolutional neural network models respectively, and the probability of the target subgraph contained in the pictures to be identified is further calculated through the weighted value, so that the accuracy of identification is improved.
As shown in fig. 2, an embodiment of the present invention provides a flowchart of a first recognition probability model, and the method may include the following steps:
in the embodiment, a VGG16 model is taken as a first convolutional neural network model as an example, and a flow of a first recognition probability model is described;
Step S201: and acquiring a picture to be identified, and taking the data of the picture as input data of a VGG16 model.
Specifically, a picture to be identified, for example, a commodity outer package picture to be identified is obtained; since the VGG16 model requires the picture size to be 224×224, performing a preprocessing operation on the picture, redefining the picture size to be 224×224; further, inputting data such as pixels and colors of the picture into a VGG16 model;
Step S202: and obtaining output data of the VGG16 model, and taking the output data as input data of the first full connection model.
Specifically, output data is obtained through calculation of a convolution layer, a pooling layer and a full connection layer contained in the VGG16 model, and the output data is used as input data of a first full connection model; further, replacing the classifier in the VGG16 model with the first fully connected model;
That is, based on the first convolutional neural network model, a first fully connected model is constructed, forming a first recognition probability model, comprising: and constructing a first full-connection model, replacing a classifier of the first convolutional neural network model, and forming the first recognition probability model according to a convolutional layer, a pooling layer, a full-connection layer and the first full-connection model of the first convolutional neural network model.
Step S203: obtaining output data of a first full-connection model, and obtaining first class probability;
Specifically, constructing the first full connection model includes: constructing at least two linear layers, respectively setting the quantity of neurons contained in the linear layers, and performing full-connection calculation on the values of the neurons contained in the linear layers by utilizing a full-connection neural network model;
The specific description of the first full connection model is consistent with step S101, and will not be repeated here;
further, calculating the class probability of the target subgraph existing in the picture by using the first recognition probability model to obtain a first probability;
step S201 to step S203 take the VGG16 model as an example, illustrate:
constructing a first full-connection model based on the first convolutional neural network model to form a first recognition probability model;
Comprising the following steps: constructing a first full-connection model, replacing a classifier of the first convolutional neural network model, and forming the first recognition probability model according to a convolutional layer, a pooling layer, a full-connection layer and the first full-connection model of the first convolutional neural network model;
further, calculating the class probability of the target subgraph existing in the picture by using the first recognition probability model to obtain a first probability.
As shown in fig. 3, an embodiment of the present invention provides a flowchart of a second recognition probability model, and the method may include the following steps:
In the embodiment, denseNet models are taken as second convolutional neural network models as examples, and the flow of the second recognition probability models is described;
Step S301: and acquiring a picture to be identified, and taking the data of the picture as input data of DenseNet models.
Specifically, a picture to be identified, for example, a commodity outer package picture to be identified is obtained, and since the DenseNet model requires that the picture size is 224×224, the picture is subjected to preprocessing operation, and the picture size is redefined as 224×224; further, inputting data such as pixels and colors of the picture into DenseNet models;
step S302: and obtaining the output data of the DenseNet model, and taking the output data as the input data of the first full-connection model.
Specifically, output data is obtained through calculation of a convolution layer, a pooling layer and a dense block contained in DenseNet models, wherein the dense block is a data block, and the output data is used as input data of a second full-connection model; further, replacing DenseNet the fully connected layers in the model with a second fully connected model;
That is, constructing a second fully connected model based on the second convolutional neural network model to form a second recognition probability model, comprising:
And constructing a second full-connection model, replacing a full-connection layer of the second convolutional neural network model, and forming the second recognition probability model according to the convolutional layer, the pooling layer, the data block and the second full-connection model of the second convolutional neural network model.
Step S303: obtaining output data of a second full-connection model, and obtaining second class probability;
specifically, constructing the second full connection model includes: constructing at least two linear layers, respectively setting the quantity of neurons contained in the linear layers, and performing full-connection calculation on the values of the neurons contained in the linear layers by utilizing a full-connection neural network model;
the specific description of the second full connection model is consistent with step S101, and will not be repeated here;
further, calculating the class probability of the target subgraph existing in the picture by using the second recognition probability model to obtain a second probability;
Step S301-step S303 illustrate the step of constructing a second fully connected model based on a second convolutional neural network model to form a second recognition probability model, comprising: constructing a second full-connection model, replacing a full-connection layer of the second convolutional neural network model, and forming the second recognition probability model according to the convolutional layer, the pooling layer, the data block and the second full-connection model of the second convolutional neural network model;
further, calculating the class probability of the target subgraph existing in the picture by using the first recognition probability model to obtain a first probability.
As shown in fig. 5, another embodiment of the present invention provides a flowchart of a method for identifying an image, which may include the following steps:
step S501: acquiring a picture to be identified;
Specifically, a picture to be identified, for example, a commodity outer package picture to be identified is obtained; taking the VGG16 model and DenseNet model in the convolutional neural network model as an example, since the VGG16 model and DenseNet model require the picture size to be 224×224, the picture is subjected to preprocessing operation, and the picture size is redefined to be 224×224; further, inputting data such as pixels and colors of the pictures into a VGG16 model and a DenseNet model;
step S502: constructing a first full-connection model based on the first convolutional neural network model to form a first recognition probability model;
Specifically, a first full-connection model is built, a classifier of the first convolutional neural network model is replaced, and the first recognition probability model is formed according to a convolutional layer, a pooling layer, a full-connection layer and the first full-connection model of the first convolutional neural network model.
Taking the VGG16 model as a first convolutional neural network model as an illustration, constructing a first full-connection model based on the first convolutional neural network model, and forming a description of the first recognition probability model consistent with the step S101, which is not repeated here;
Step S503: based on the second convolutional neural network model, a second full-connection model is built to form a second recognition probability model
Step S504: constructing another second full-connection model based on the second convolutional neural network model to form another second recognition probability model;
steps S503-S504 describe a process of forming a plurality of second recognition probability models;
Specifically, a second full-connection model is built, a full-connection layer of the second convolutional neural network model is replaced, and the second recognition probability model is formed according to the convolutional layer, the pooling layer, the data block and the second full-connection model of the second convolutional neural network model;
Further, at least two second linear full-connection models are constructed, and full-connection layers of the corresponding second convolutional neural network models are replaced respectively; and respectively forming the second recognition probability model according to a convolution layer, a pooling layer, a data block and the corresponding second full-connection model of the second convolution neural network model.
Taking DenseNet model as second convolution neural network model as illustration, constructing a second full-connection model based on the second convolution neural network model, and forming the description of the second recognition probability model consistent with the description of the step S101, which is not repeated here;
In this embodiment, based on the second convolutional neural network model, taking DenseNet models as examples, two different second full-connection models are constructed, so that two second recognition probability models are formed; as shown in fig. 6B and 6C, the specific description of the construction of the second full connection model is consistent with step S101, and will not be repeated here;
It can be appreciated that at least two different convolutional neural network models can be employed to form a plurality of different second recognition probability models, respectively; the method is beneficial to synthesizing the recognition probability results of different convolutional neural network models by using the weights, and improves the recognition accuracy under the same sample set.
Step S505: calculating and identifying class probability of target subgraph based on class prediction model
Specifically, using the first recognition probability model and the second recognition probability model, and a classification prediction model, calculating a class probability that a target subgraph exists in the picture,
Still referring to the example model of steps S501-S504, for example, the first probability is calculated to be 0.9, the first second probability is calculated to be 0.8, and the second probability is calculated to be 0.7; the calculation is performed through a classification prediction model, namely a formula (2), wherein C represents the classification probability, C1 represents the first probability value, C2 represents the first second probability value, and C3 represents the second probability value
W1 represents a weight value corresponding to the first probability, w2 represents a weight value corresponding to the first second probability, w3 represents a weight value corresponding to the second probability, for example, w1 is 0.2, w2 is 0.5, and w3 is 0.3; then C is calculated to be equal to 0.79;
C=w1*c1+w2*c2+w3*c3+…wi*ci (2)
Calculating the class probability of the target subgraph existing in the picture by using the first recognition probability model to obtain a first probability;
calculating the class probability of the target subgraph existing in the picture by using one or more second recognition probability models to obtain second probabilities respectively;
and calculating the class probability of the target subgraph existing in the picture based on the class prediction model by using the first probability and the one or more second probabilities.
The classification prediction model comprises the steps of calculating the classification probability of the target subgraph existing in the picture based on the first probability, one or more second probabilities, the weight value of the first probability and the weight value corresponding to the one or more second probabilities.
Further, it is determined whether the target subgraph is included in the picture according to a predefined probability threshold. Specifically, whether the pre-defined probability threshold of the target subgraph is contained in the picture is determined according to experimental data or experience, for example, the pre-defined probability threshold is set to be 0.78; the class probability value calculated in step S505 is 0.79;0.79>0.78, the target subgraph is considered to be included in the picture, for example, the "health food" target subgraph shown in fig. 4B is included in an example picture of the commodity outer package of the health product in fig. 4A.
As shown in fig. 7, an embodiment of the present invention provides an apparatus 700 for identifying a subgraph, including: a recognition model module 701, a calculation recognition probability module 702 and a recognition sub-graph module 703 are constructed,
The recognition model building module 701 is configured to build a first fully connected model based on a first convolutional neural network model to form a first recognition probability model; constructing a second full-connection model based on the second convolutional neural network model to form a second recognition probability model;
The recognition probability calculating module 702 is configured to calculate a class probability of the target sub-graph existing in the picture by using the first recognition probability model, the second recognition probability model, and the classification prediction model;
the identifying sub-picture module 703 is configured to determine whether the picture contains the target sub-picture according to a predefined probability threshold.
Optionally, the recognition model building module 701 is further configured to build a first fully connected model, replace a classifier of the first convolutional neural network model, and form the first recognition probability model according to a convolutional layer, a pooling layer, a fully connected layer, and the first fully connected model of the first convolutional neural network model.
Optionally, the recognition model building module 701 is further configured to build a second fully connected model, replace a fully connected layer of the second convolutional neural network model, and form the second recognition probability model according to the convolutional layer, the pooling layer, the data block, and the second fully connected model of the second convolutional neural network model.
Optionally, the recognition model building module 701 is further configured to build at least two linear layers, set the number of neurons included in the linear layers respectively, and perform full-connection calculation on the number of neurons included in the linear layers by using a full-connection neural network model.
Optionally, the recognition model building module 701 is further configured to build at least two second linear full-connection models, and replace full-connection layers of the corresponding second convolutional neural network models respectively; and respectively forming the second recognition probability model according to a convolution layer, a pooling layer, a data block and the corresponding second full-connection model of the second convolution neural network model.
Optionally, the recognition probability calculating module 702 is further configured to calculate, using the first recognition probability model, a class probability of the target sub-graph existing in the picture, so as to obtain a first probability; calculating the class probability of the target subgraph existing in the picture by using one or more second recognition probability models to obtain second probabilities respectively; and calculating the class probability of the target subgraph existing in the picture based on the class prediction model by using the first probability and the one or more second probabilities.
Optionally, the sub-graph identifying module 703 is further configured to calculate a class probability of the target sub-graph existing in the picture based on the first probability, one or more second probabilities, a weight value of the first probability, and a weight value corresponding to the one or more second probabilities.
The embodiment of the invention also provides an electronic device for identifying images, which comprises: one or more processors; and a storage device for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the method provided by any of the embodiments described above.
The embodiment of the invention also provides a computer readable medium, on which a computer program is stored, which when executed by a processor implements the method provided by any of the above embodiments.
Fig. 8 illustrates an exemplary system architecture 800 to which the method of recognizing an image or the apparatus of recognizing an image of the embodiment of the present invention may be applied.
As shown in fig. 8, a system architecture 800 may include terminal devices 801, 802, 803, a network 804, and a server 805. The network 804 serves as a medium for providing communication links between the terminal devices 801, 802, 803 and the server 805. The network 804 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
A user may interact with the server 805 through the network 804 using the terminal devices 801, 802, 803 to receive or send messages or the like. Various client applications such as a web browser application, a search class application, an instant messaging tool, a mailbox client, and the like may be installed on the terminal devices 801, 802, 803.
The terminal devices 801, 802, 803 may be a variety of electronic devices having a display screen and supporting a variety of client applications including, but not limited to, smartphones, tablet computers, laptop and desktop computers, and the like.
The server 805 may be a server providing various services, such as a background management server providing support for pictures to be identified provided by the user using the terminal devices 801, 802, 803. The background management server can analyze and process the received picture to be identified and feed back the identification result of the target subgraph to the terminal device.
It should be noted that, the method for identifying an image provided by the embodiment of the present invention is generally performed by the server 805, and accordingly, the device for identifying an image is generally disposed in the server 805.
It should be understood that the number of terminal devices, networks and servers in fig. 8 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 9, there is illustrated a schematic diagram of a computer system 900 suitable for use in implementing an embodiment of the present invention. The terminal device shown in fig. 9 is only an example, and should not impose any limitation on the functions and the scope of use of the embodiment of the present invention.
As shown in fig. 9, the computer system 900 includes a Central Processing Unit (CPU) 901, which can execute various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 902 or a program loaded from a storage section 908 into a Random Access Memory (RAM) 903. In the RAM 903, various programs and data necessary for the operation of the system 900 are also stored. The CPU 901, ROM 902, and RAM 903 are connected to each other through a bus 904. An input/output (I/O) interface 905 is also connected to the bus 904.
The following components are connected to the I/O interface 905: an input section 906 including a keyboard, a mouse, and the like; an output portion 907 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and a speaker; a storage portion 908 including a hard disk or the like; and a communication section 909 including a network interface card such as a LAN card, a modem, or the like. The communication section 909 performs communication processing via a network such as the internet. The drive 910 is also connected to the I/O interface 905 as needed. A removable medium 911 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed as needed on the drive 910 so that a computer program read out therefrom is installed into the storage section 908 as needed.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from the network via the communication portion 909 and/or installed from the removable medium 911. The above-described functions defined in the system of the present invention are performed when the computer program is executed by a Central Processing Unit (CPU) 901.
The computer readable medium shown in the present invention may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules and/or units involved in the embodiments of the present invention may be implemented in software, or may be implemented in hardware. The described modules and/or units may also be provided in a processor, e.g., may be described as: a processor comprising: and constructing an identification model module, a calculation identification probability module and an identification sub-graph module. The names of these modules do not in any way limit the module itself, and for example, the building identification model module may also be described as "a module that builds a fully connected model from a convolutional neural network model and forms an identification model".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be present alone without being fitted into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to include: constructing a first full-connection model based on the first convolutional neural network model to form a first recognition probability model; constructing a second full-connection model based on the second convolutional neural network model to form a second recognition probability model; calculating the class probability of the target subgraph existing in the picture by using the first recognition probability model, the second recognition probability model and the classification prediction model; and determining whether the target subgraph is contained in the picture according to a predefined probability threshold.
According to the technical scheme provided by the embodiment of the invention, the problem that the accuracy of the identification target subgraph is lower in part of the technology under the condition of limited data samples is solved, the accuracy of the identification image is improved, and the workload of manually checking and authenticating the identification image is reduced.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives can occur depending upon design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.

Claims (9)

1. A method of identifying an image, comprising:
constructing a first full-connection model, replacing a classifier of a first convolutional neural network model, and forming a first recognition probability model according to a convolutional layer, a pooling layer, a full-connection layer and the first full-connection model of the first convolutional neural network model;
Constructing a second full-connection model, replacing a full-connection layer of a second convolutional neural network model, and forming a second recognition probability model according to the convolutional layer, the pooling layer, the data block and the second full-connection model of the second convolutional neural network model;
The first fully connected model or the second fully connected model is composed of at least two linear layers; building the first fully connected model or the second fully connected model comprises the following steps: constructing at least two linear layers, respectively setting the quantity of neurons contained in the linear layers, and performing full-connection calculation on the values of the neurons contained in the linear layers by utilizing a full-connection neural network model;
Calculating the class probability of the target subgraph existing in the picture by using the first recognition probability model, the second recognition probability model and the classification prediction model;
and determining whether the target subgraph is contained in the picture according to a predefined probability threshold.
2. The method of claim 1, wherein the step of determining the position of the substrate comprises,
Constructing at least two second linear full-connection models, and respectively replacing full-connection layers of the corresponding second convolutional neural network models; and respectively forming the second recognition probability model according to a convolution layer, a pooling layer, a data block and the corresponding second full-connection model of the second convolution neural network model.
3. The method of claim 2, wherein the step of determining the position of the substrate comprises,
Calculating the class probability of the target subgraph existing in the picture by using the first recognition probability model, the second recognition probability model and the classification prediction model, wherein the method comprises the following steps:
Calculating the class probability of the target subgraph existing in the picture by using the first recognition probability model to obtain a first probability;
calculating the class probability of the target subgraph existing in the picture by using one or more second recognition probability models to obtain second probabilities respectively;
and calculating the class probability of the target subgraph existing in the picture based on the class prediction model by using the first probability and the one or more second probabilities.
4. The method of claim 3, wherein the step of,
The classification prediction model comprises: and calculating the class probability of the target subgraph existing in the picture based on the first probability, one or more second probabilities, the weight value of the first probability and the weight value corresponding to one or more second probabilities.
5. An apparatus for recognizing an image, comprising: constructing an identification model module, a calculation identification probability module and an identification sub-graph module; wherein,
The build identification model module is used for building a first full-connection model, replacing a classifier of a first convolutional neural network model, and forming a first identification probability model according to a convolutional layer, a pooling layer, a full-connection layer and the first full-connection model of the first convolutional neural network model;
The recognition model constructing module is further used for constructing a second full-connection model to replace a full-connection layer of a second convolutional neural network model, and a second recognition probability model is formed according to the convolutional layer, the pooling layer, the data block and the second full-connection model of the second convolutional neural network model;
The first fully connected model or the second fully connected model is composed of at least two linear layers; building the first fully connected model or the second fully connected model comprises the following steps: constructing at least two linear layers, respectively setting the quantity of neurons contained in the linear layers, and performing full-connection calculation on the values of the neurons contained in the linear layers by utilizing a full-connection neural network model;
The recognition probability calculating module is used for calculating the class probability of the target subgraph existing in the picture by using the first recognition probability model, the second recognition probability model and the classification prediction model;
And the sub-picture identifying module is used for determining whether the picture contains the target sub-picture or not according to a predefined probability threshold.
6. The apparatus of claim 5, wherein the device comprises a plurality of sensors,
The recognition model constructing module is further used for constructing at least two second linear full-connection models, and the full-connection layers of the corresponding second convolutional neural network models are replaced respectively; and respectively forming the second recognition probability model according to a convolution layer, a pooling layer, a data block and the corresponding second full-connection model of the second convolution neural network model.
7. The apparatus of claim 5, wherein the device comprises a plurality of sensors,
The recognition probability calculation module is further used for calculating the class probability of the target subgraph existing in the picture by using the first recognition probability model to obtain a first probability; calculating the class probability of the target subgraph existing in the picture by using one or more second recognition probability models to obtain second probabilities respectively; and calculating the class probability of the target subgraph existing in the picture based on the class prediction model by using the first probability and the one or more second probabilities.
8. An electronic device, comprising:
one or more processors;
storage means for storing one or more programs,
When executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-4.
9. A computer readable medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any of claims 1-4.
CN202010436720.8A 2020-05-21 2020-05-21 Image identification method and device Active CN113705594B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010436720.8A CN113705594B (en) 2020-05-21 2020-05-21 Image identification method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010436720.8A CN113705594B (en) 2020-05-21 2020-05-21 Image identification method and device

Publications (2)

Publication Number Publication Date
CN113705594A CN113705594A (en) 2021-11-26
CN113705594B true CN113705594B (en) 2024-05-21

Family

ID=78646102

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010436720.8A Active CN113705594B (en) 2020-05-21 2020-05-21 Image identification method and device

Country Status (1)

Country Link
CN (1) CN113705594B (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107145904A (en) * 2017-04-28 2017-09-08 北京小米移动软件有限公司 Determination method, device and the storage medium of image category
CN109034034A (en) * 2018-07-12 2018-12-18 广州麦仑信息科技有限公司 A kind of vein identification method based on nitrification enhancement optimization convolutional neural networks
WO2019007524A1 (en) * 2017-07-06 2019-01-10 Toyota Motor Europe Tracking objects in sequences of digital images
CN109740679A (en) * 2019-01-13 2019-05-10 胡燕祝 A kind of target identification method based on convolutional neural networks and naive Bayesian
CN109858533A (en) * 2019-01-18 2019-06-07 莆田学院 A kind of RBF-MLP mixed structure neural network classifier of prepositus nucleus
CN110347857A (en) * 2019-06-06 2019-10-18 武汉理工大学 The semanteme marking method of remote sensing image based on intensified learning
CN111178341A (en) * 2020-04-10 2020-05-19 支付宝(杭州)信息技术有限公司 Living body detection method, device and equipment

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107145904A (en) * 2017-04-28 2017-09-08 北京小米移动软件有限公司 Determination method, device and the storage medium of image category
WO2019007524A1 (en) * 2017-07-06 2019-01-10 Toyota Motor Europe Tracking objects in sequences of digital images
CN109034034A (en) * 2018-07-12 2018-12-18 广州麦仑信息科技有限公司 A kind of vein identification method based on nitrification enhancement optimization convolutional neural networks
CN109740679A (en) * 2019-01-13 2019-05-10 胡燕祝 A kind of target identification method based on convolutional neural networks and naive Bayesian
CN109858533A (en) * 2019-01-18 2019-06-07 莆田学院 A kind of RBF-MLP mixed structure neural network classifier of prepositus nucleus
CN110347857A (en) * 2019-06-06 2019-10-18 武汉理工大学 The semanteme marking method of remote sensing image based on intensified learning
CN111178341A (en) * 2020-04-10 2020-05-19 支付宝(杭州)信息技术有限公司 Living body detection method, device and equipment

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于CenterNet-GYolov3的车辆检测方法;徐仲谋;杨亦乐;董飞;沈梦蓝;;软件;20200515(05);全文 *

Also Published As

Publication number Publication date
CN113705594A (en) 2021-11-26

Similar Documents

Publication Publication Date Title
CN108898186B (en) Method and device for extracting image
CN109816039B (en) Cross-modal information retrieval method and device and storage medium
CN108805091B (en) Method and apparatus for generating a model
CN108197652B (en) Method and apparatus for generating information
CN108520470B (en) Method and apparatus for generating user attribute information
CN107679466B (en) Information output method and device
CN109389072B (en) Data processing method and device
KR102002024B1 (en) Method for processing labeling of object and object management server
CN109377508B (en) Image processing method and device
CN113128419B (en) Obstacle recognition method and device, electronic equipment and storage medium
US20210064919A1 (en) Method and apparatus for processing image
CN110659657B (en) Method and device for training model
CN110084317B (en) Method and device for recognizing images
CN114494784A (en) Deep learning model training method, image processing method and object recognition method
CN114385869A (en) Method and device for detecting data abnormity, storage medium and computer equipment
CN110633717A (en) Training method and device for target detection model
CN113705362A (en) Training method and device of image detection model, electronic equipment and storage medium
CN115311469A (en) Image labeling method, training method, image processing method and electronic equipment
CN113140012B (en) Image processing method, device, medium and electronic equipment
CN114360027A (en) Training method and device for feature extraction network and electronic equipment
CN110895699B (en) Method and apparatus for processing feature points of image
CN113705594B (en) Image identification method and device
CN116363641A (en) Image processing method and device and electronic equipment
CN115861255A (en) Model training method, device, equipment, medium and product for image processing
CN115546554A (en) Sensitive image identification method, device, equipment and computer readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant