CN113610832A - Logo defect detection method, device, equipment and storage medium - Google Patents
Logo defect detection method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN113610832A CN113610832A CN202110956354.3A CN202110956354A CN113610832A CN 113610832 A CN113610832 A CN 113610832A CN 202110956354 A CN202110956354 A CN 202110956354A CN 113610832 A CN113610832 A CN 113610832A
- Authority
- CN
- China
- Prior art keywords
- logo
- defect detection
- detected
- neural network
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a logo defect detection method, a logo defect detection device, logo defect detection equipment and a logo defect detection storage medium, wherein the method comprises the following steps: collecting images containing logos in stores as logo images to be detected; inputting the logo image to be detected into a logo defect detection model, enabling the logo defect detection model to extract the features of the logo image to be detected, and identifying whether the logo image to be detected has defects or not based on the extracted features to obtain a defect detection result of the logo image to be detected; the method saves the defect detection result of the logo image to be detected to a preset database, and solves the technical problems that the existing store logos are usually checked manually at regular time, and are time-consuming, labor-consuming and low in efficiency.
Description
Technical Field
The present application relates to the field of image processing technologies, and in particular, to a logo defect detection method, apparatus, device, and storage medium.
Background
Existing retail establishments are typically designed with a specific logo that represents the image of the entire establishment, and thus, logos in physical stores are typically required to remain clean, with intact designs, and without defects. The traditional store logo is usually checked at a mobile phone end or a computer end in a timing mode through manual work, whether the logo has defects or not is checked manually, the method is time-consuming and labor-consuming, especially when the number of stores is large, a spot check mode is usually adopted for detection, and the efficiency is low.
Disclosure of Invention
The application provides a logo defect detection method, a logo defect detection device, logo defect detection equipment and a logo storage medium, which are used for solving the technical problems that the existing store logos are usually checked in a manual timing mode, time and labor are wasted, and the efficiency is low.
In view of the above, a first aspect of the present application provides a logo defect detection method, including:
collecting images containing logos in stores as logo images to be detected;
inputting the logo image to be detected into a logo defect detection model, enabling the logo defect detection model to perform feature extraction on the logo image to be detected, and identifying whether the logo image to be detected has defects or not based on the extracted features to obtain a defect detection result of the logo image to be detected;
and storing the defect detection result of the logo image to be detected into a preset database.
Optionally, the training process of the logo defect detection model is as follows:
acquiring an original image containing a logo;
constructing a training sample according to the completeness of the logo in the original image;
training the capability of a pre-constructed neural network for detecting the defects of the logo through the training sample until the neural network meets the preset training condition, and taking the trained neural network as a logo defect detection model.
Optionally, the training samples include positive samples and negative samples; the constructing of the training sample according to the completeness of the logo in the original image comprises:
detecting the completeness of the logo in the original image to take the original image with complete logo as a positive sample and the original image with incomplete logo as a negative sample;
respectively executing data enhancement operation on the positive sample and the negative sample to obtain a training sample consisting of the positive sample subjected to data enhancement and the negative sample subjected to data enhancement;
wherein the data enhancement operation includes, but is not limited to, random cropping, center cropping, random rotation, random horizontal flipping.
Optionally, the detecting the completeness of the logo in the original image to use the original image with complete logo as a positive sample and the original image with incomplete logo as a negative sample includes:
detecting feature information of the logo in the original image, wherein the feature information comprises character information, pattern information, texture information and brightness information;
if the characteristic information does not meet the preset logo design requirement, determining that the currently detected logo is an incomplete logo, and taking the original image with the incomplete logo as a negative sample;
and if the characteristic information meets the logo design requirement, determining that the currently detected logo is a complete logo, and taking the original image with the complete logo as a positive sample.
Optionally, the text information includes text content and a typesetting sequence of the text, the pattern information includes pattern content and a color of the pattern, the texture information includes background texture and background color, and the brightness information includes a brightness value of the logo; the logo design requirement comprises target character content, a typesetting sequence of the target characters, target pattern content, target pattern color, target background texture, target background color and target brightness value of the logo;
after the detecting the feature information of the logo in the original image, the method further includes:
when the character content is determined to be not in accordance with the target character content and/or the typesetting sequence of the characters is determined to be not in accordance with the typesetting sequence of the target characters, determining that the characteristic information does not meet the preset logo design requirement;
and/or the presence of a gas in the gas,
when the pattern content is determined not to accord with the target pattern content and/or the color of the pattern is determined not to accord with the color of the target pattern, determining that the characteristic information does not meet the preset logo design requirement;
and/or the presence of a gas in the gas,
when the background texture is determined not to accord with the target background texture and/or the background color is determined not to accord with the target background color, determining that the characteristic information does not meet the preset logo design requirement;
and/or the presence of a gas in the gas,
and when the brightness value of the logo is determined not to meet the target brightness value of the logo, determining that the characteristic information does not meet the preset logo design requirement.
Optionally, the training of the capability of the pre-constructed neural network to detect the logo defect through the training sample until the neural network meets the preset training condition, and using the trained neural network as a logo defect detection model includes:
inputting the training sample into a pre-constructed neural network, extracting the characteristics of the logo in the training sample under multiple dimensions by using the neural network, and outputting to obtain logo classification characteristics fused by multiple characteristics;
calculating a loss value of the logo classification feature;
if the loss value is larger than or equal to a preset threshold value, updating parameters in the neural network by using the loss value, and returning to execute the step of inputting the training sample into a pre-constructed neural network;
if the loss value is smaller than a preset threshold value, determining that the neural network training is finished;
and taking the trained neural network as a logo defect detection model.
Optionally, the pre-constructed neural network includes a residual error network, an attention mechanism module, and a deformable convolution module; inputting the training sample into a pre-constructed neural network, extracting features of the logo in the training sample under multiple dimensions by using the neural network, and outputting to obtain logo classification features fused by multiple features, wherein the method comprises the following steps:
performing convolution operation on the training sample by utilizing a plurality of residual blocks in the residual error network to obtain convolution characteristics;
feature selection is carried out on the convolution features according to the types of logo defects through the attention mechanism module, a plurality of convolution features after feature selection are fused, and logo classification features are output; the category of the logo defects comprises defects of the logo in any dimension of characters, patterns, textures and brightness;
the deformable convolution module is used for adjusting the characteristic receptive field of the training sample in the forward propagation process of the whole neural network.
The present application provides in a second aspect a logo defect detecting device, comprising:
the system comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring images containing logos in stores as to-be-detected logo images;
the defect detection unit is used for inputting the logo image to be detected into a logo defect detection model, so that the logo defect detection model performs feature extraction on the logo image to be detected, and identifies whether the logo image to be detected has defects or not based on the extracted features to obtain a defect detection result of the logo image to be detected;
and the storage unit is used for storing the defect detection result of the logo image to be detected to a preset database.
A third aspect of the present application provides a logo defect detecting apparatus, the apparatus comprising a processor and a memory;
the memory is used for storing program codes and transmitting the program codes to the processor;
the processor is configured to execute the logo defect detection method according to any one of the first aspect according to instructions in the program code.
A fourth aspect of the present application provides a computer-readable storage medium for storing program code, which when executed by a processor implements the logo defect detection method of any one of the first aspects.
According to the technical scheme, the method has the following advantages:
the application provides a logo defect detection method, which comprises the following steps: collecting images containing logos in stores as logo images to be detected; inputting the logo image to be detected into a logo defect detection model, enabling the logo defect detection model to extract the features of the logo image to be detected, and identifying whether the logo image to be detected has defects or not based on the extracted features to obtain a defect detection result of the logo image to be detected; and storing the defect detection result of the logo image to be detected in a preset database.
In this application, after the badge image to be detected is obtained, treat through the badge defect detection model and detect the badge image and carry out feature extraction and defect detection, and save the defect detection result to preset database and supply the user to look up, carry out the badge defect detection through the badge defect detection model, detection speed is fast, do not need the manual work to carry out the badge defect detection, detection efficiency has been improved, thereby it usually is through the manual work timing inspection to have improved current shop's badge, the technical problem that exists and wastes time and energy, inefficiency.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.
Fig. 1 is a schematic flowchart of a logo defect detection method according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of a residual block after SE-Net is added according to an embodiment of the present application;
FIG. 3 is a schematic flow chart illustrating a logo defect detection method according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a logo defect detecting apparatus according to an embodiment of the present application.
Detailed Description
The application provides a logo defect detection method, a logo defect detection device, logo defect detection equipment and a logo storage medium, which are used for solving the technical problems that the existing store logos are usually checked in a manual timing mode, time and labor are wasted, and the efficiency is low.
In order to make the technical solutions of the present application better understood, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
For easy understanding, please refer to fig. 1, the present application provides an embodiment of a Logo defect detection method, which may be applied to a scene of detecting a Logo of a merchant (Logo) in an offline physical store and locating a defect on the Logo, and the Logo defect detection method may include:
The timing photographing method can be used for photographing the scenes at the doorways of all the shops in batches at regular time, including the conditions of opening and closing the doors in the morning and evening. The camera installed in the store or the mobile terminal device of the store can be used for regularly acquiring the logo image placed in the store as the logo image to be detected, for example, the camera of the store can be controlled by a background system to regularly execute the shooting task of the appointed scene so as to acquire the logo image of the store as the logo image to be detected. Store logo images can also be shot at regular time through the clerk mobile terminal (such as a mobile phone and an iPad) of each store, and after the logo images to be detected are collected, the logo images can be uploaded to a shop patrol APP.
After the logo images to be detected are obtained, a program can be set to obtain the logo images to be detected in batches from the patrol shop APP at regular time, then the trained logo defect detection model is called to extract the features of the logo images to be detected, and the logo images to be detected are identified whether to be complete or not according to the extracted features through the logo defect detection model, so that whether the logo images to be detected have defects or not is determined, and the defect detection result of the logo images to be detected is obtained.
Further, the training process of the logo defect detection model comprises the following steps:
and S1021, acquiring an original image containing the logo.
A large number of original images containing logos may be acquired through an API (application program Interface) provided by the shop patrol APP.
And S1022, constructing a training sample according to the completeness of the logo in the original image.
In the embodiment of the application, after the original image is acquired, the completeness of the logo in the original image is detected, the original image with the complete logo is used as a positive sample, and the original image with the incomplete logo is used as a negative sample. Specifically, the constructing of the training sample according to the completeness of the logo in the original image includes:
and detecting the completeness of the logo in the original image to take the original image with complete logo as a positive sample and the original image with incomplete logo as a negative sample.
And respectively executing data enhancement operation on the positive sample and the negative sample to obtain a training sample consisting of the positive sample subjected to data enhancement and the negative sample subjected to data enhancement, wherein the data enhancement operation comprises but is not limited to random clipping, center clipping, random rotation and random horizontal inversion.
In this embodiment of the application, detecting the completeness of the logo in the original image to regard the original image with complete logo as a positive sample, and regard the original image with incomplete logo as a negative sample includes:
detecting the characteristic information of a logo in an original image, wherein the characteristic information comprises character information, pattern information, texture information and brightness information;
if the characteristic information does not meet the preset logo design requirement, determining that the currently detected logo is an incomplete logo, and taking the original image with the incomplete logo as a negative sample;
and if the characteristic information meets the logo design requirement, determining that the currently detected logo is a complete logo, and taking the original image with the complete logo as a positive sample.
The text information in the embodiment of the present application may include text content, a typesetting order of the text, and the like, the pattern information may include pattern content, a color of the pattern, a size of the pattern, and the like, the texture information may include a background texture (i.e., a texture of a background in the logo except for the pattern and the text), a background color (i.e., a color of the background in the logo except for the pattern and the text), and the like, and the brightness information may include a brightness value, a color contrast, and the like of the logo; the logo design requirement in the embodiment of the present application may include a specific text content (target text content), a specific text typesetting order (target text typesetting order), a specific pattern composition (target pattern content), a specific pattern size (size specification of target pattern), a specific pattern color (color of target pattern), a specific background texture (target background color), a specific background color (target background color), a target brightness value of the logo, and the like.
In a preferred embodiment of the present invention, the text information includes text content and a typesetting order of the text, the pattern information includes pattern content and a color of the pattern, the texture information includes background texture and background color, and the brightness information includes a brightness value of the logo; the logo design requirement comprises target character content, target character typesetting sequence, target pattern content, target pattern color, target background texture, target background color and target brightness value of the logo; after detecting the feature information of the logo in the original image, the method further comprises the following steps:
when the character content is determined not to be in accordance with the target character content and/or the typesetting sequence of the characters is determined not to be in accordance with the typesetting sequence of the target characters, determining that the characteristic information does not meet the preset logo design requirement;
and/or the presence of a gas in the gas,
when the pattern content is determined not to accord with the target pattern content and/or the color of the pattern does not accord with the color of the target pattern, determining that the characteristic information does not meet the preset logo design requirement;
and/or the presence of a gas in the gas,
when the background texture is determined not to accord with the target background texture and/or the background color is determined not to accord with the target background color, determining that the characteristic information does not meet the preset logo design requirement;
and/or the presence of a gas in the gas,
and when the brightness value of the logo is determined not to be in accordance with the target brightness value of the logo, determining that the characteristic information does not meet the preset logo design requirement.
After detecting the feature information of the logo in the original image, the embodiment of the application judges whether the feature information of the logo meets the preset logo design requirement, and distinguishes the positive sample and the negative sample according to the obtained judgment result.
Specifically, in an example, when the logo design requirement includes the target text content and/or the typesetting sequence of the target text, the feature information of the logo in the original image is detected as text information, and the text information includes the text content and the typesetting sequence of the text; when the character content is determined not to be in accordance with the target character content and/or the typesetting order of the characters is determined not to be in accordance with the typesetting order of the target characters, determining that the feature information of the logo does not meet the preset logo design requirement, determining that the currently detected logo is an incomplete logo, and taking the original image with the incomplete logo as a negative sample; when the character content is determined to accord with the target character content and/or the typesetting sequence of the characters is determined to accord with the typesetting sequence of the target characters, the feature information of the logo is determined to meet the logo design requirement, the currently detected logo is determined to be a complete logo, and the complete original image of the logo is used as a positive sample.
In another example, when the logo design requirement includes the target pattern content and/or the color of the target pattern, the feature information of the logo in the original image is detected as pattern information, and the pattern information includes the pattern content and the color of the pattern; when the pattern content is determined to be not in accordance with the target pattern content and/or the color of the pattern is determined to be not in accordance with the color of the target pattern, determining that the feature information of the logo does not meet the preset logo design requirement, determining that the currently detected logo is an incomplete logo, and taking the original image with the incomplete logo as a negative sample; when the pattern content is determined to meet the target pattern content and/or the color of the pattern meets the color of the target pattern, determining that the feature information of the logo meets the design requirement of the logo, determining that the currently detected logo is a complete logo, and taking the complete original image of the logo as a positive sample.
In yet another example, when the logo design requirement includes a target background texture and/or a target background color, the feature information of the logo in the original image is detected as texture information, and the texture information includes a background texture and a background color; when the background texture is determined not to accord with the target background texture and/or the background color is determined not to accord with the target background color, determining that the feature information of the logo does not meet the preset logo design requirement, determining that the currently detected logo is an incomplete logo, and taking the original image with the incomplete logo as a negative sample; when the background texture is determined to be in accordance with the target background texture and/or the background color is determined to be in accordance with the target background color, the feature information of the logo is determined to meet the logo design requirement, the currently detected logo is determined to be a complete logo, and the complete original image of the logo is used as a positive sample.
In yet another example, when the logo design requirement includes a target brightness value of the logo, then detecting the feature information of the logo in the original image as brightness information, the brightness information including the brightness value of the logo; when the brightness value of the logo is determined not to be in accordance with the target brightness value of the logo, determining that the characteristic information of the logo does not meet the preset logo design requirement, determining that the currently detected logo is an incomplete logo, and taking the original image with the incomplete logo as a negative sample; when the brightness value of the logo is determined to be in accordance with the target brightness value of the logo, and the feature information of the logo is determined to meet the preset logo design requirement, the currently detected logo is determined to be a complete logo, and the complete original image of the logo is used as a positive sample.
In yet another example, when the logo design requirement includes target text content of the logo, a typesetting sequence of the target text, target pattern content, a color of the target pattern, a target background texture, a target background color, and a target brightness value, then detecting that the feature information of the logo in the original image includes text information, pattern information, texture information, and brightness information, if any one of the text information, the pattern information, the texture information, and the brightness information in the feature information of the logo does not meet the design requirement of the logo, determining that the logo in the current original image is an incomplete logo, and taking the incomplete original image of the logo as a negative sample; if the fact that character information, pattern information, texture information and brightness information in the feature information of the logo meet the design requirements of the logo is detected, the logo in the current original image is determined to be a complete logo, and the original image with the complete logo is used as a positive sample.
It is understood that the logo design requirement may further include the target text content of the logo, the typesetting order of the target text, the target pattern content, and the color of the target pattern; or, including a target background texture, a target background color, a target brightness value; or, the Logo design requirements may be different for different enterprises including various combination conditions such as the content of the target characters, the typesetting order of the target characters, the target brightness values, and the like, and the specific Logo design requirements may be specifically set according to the actual conditions, which is not described in the embodiments of the present application one by one.
Further, in the embodiment of the present application, when the convolutional neural network is trained, if the number of the training samples is too small, the convolutional neural network cannot be trained sufficiently, which results in poor network defect detection capability. Based on the method, after the positive and negative samples are obtained through construction, data enhancement operation can be carried out on the positive sample and the negative sample, and a training sample formed by the positive sample after data enhancement and the negative sample after data enhancement is obtained, so that the number of the training samples is increased, and the training effect is improved. Specifically, data enhancement can be achieved by performing random clipping on positive and negative samples (e.g., randomly clipping the positive and negative samples to 256 × 256), center clipping (e.g., center clipping the positive and negative samples to 224 × 224), random rotation, random horizontal flipping, and the like.
And S1023, training the capability of the pre-constructed neural network for detecting the defects of the logo through the training samples until the neural network meets the preset training conditions, and taking the trained neural network as a logo defect detection model.
After the training sample is constructed and obtained, training the capability of defect detection on the logo by the neural network constructed in advance through the training sample until the neural network meets the preset training condition, and taking the trained neural network as a logo defect detection model, wherein the capability comprises the following steps:
and S10231, inputting the training sample into a pre-constructed neural network, extracting the characteristics of the logo in the training sample under multiple dimensions by using the neural network, and outputting to obtain the logo classification characteristics fused by multiple characteristics.
The neural network pre-constructed in the embodiment of the application comprises a residual error network, an attention mechanism module and a deformable convolution module. Specifically, a training sample is input into a pre-constructed neural network, and a plurality of residual blocks in a residual network are utilized to perform convolution operation on the training sample to obtain a convolution characteristic; feature selection is carried out on the convolution features according to the types of logo defects through an attention mechanism module, a plurality of convolution features after feature selection are fused, and logo classification features are output; the deformable convolution module is used for adjusting the characteristic receptive field of the training sample in the forward propagation process of the whole neural network.
The Residual network in the embodiment of the present application preferably adopts a ResNet50 network, and a ResNet50 network is composed of a convolutional layer, 4 Residual blocks (Residual blocks), and a full link layer. Wherein, the first residual Block1 in the ResNet50 network comprises 9 two-dimensional convolutional layers; the second residual Block2 includes 12 two-dimensional convolutional layers; the third residual Block3 includes 18 two-dimensional convolutional layers; the fourth residual Block4 includes 9 two-dimensional convolutional layers. During training, the ResNet50 network performs convolution operation on input images (i.e., training samples) in forward propagation, then processes features extracted from convolution layers by 4 residual blocks, classifies the features by full-link layers, or improves the last full-link layer to optimize output, for example, the input of the last full-link layer may be fed to a linear layer having a plurality of output units (e.g., 256), then connects a ReLU layer and a Dropout layer, and finally connects a Softmax layer (also called an active layer), normalizes the classified output features by the Softmax layer, outputs confidence degrees of the attribution categories of the features, and obtains a final classification result. In the training process of the deep learning network, Dropout refers to that a neural network unit is temporarily discarded from the network according to a certain probability, namely is randomly discarded, so as to prevent overfitting.
The attention mechanism module in the embodiment of the present application preferably adopts SE-Net(s) -and-importance networks), which may be respectively added to a plurality of residual blocks of a residual network, where the SE-Net is composed of a global pooling layer, two full-link layers, and an activation function, and the structure of the residual block after the SE-Net is added may refer to fig. 2, and the SE-Net performs feature selection on convolution features output by an original residual block in channel dimensions according to categories of logo defects through global pooling (globopoiling), full-link layers (FC), an activation function (ReLU), full-link layers (FC), and an activation function (Sigmoid), where the categories of the logo defects include text defects (i.e., text contents do not conform to the content of the target text and/or the typesetting order of the text does not conform to the typesetting order of the target text), pattern defects (i.e., pattern contents do not conform to the content of the target pattern and/or the color of the pattern does not conform to the target pattern) Texture defects (namely background textures do not accord with target background textures and/or background colors do not accord with target background colors), brightness defects (namely brightness values of logos do not accord with target brightness values of logos), confidence degrees of all feature channels are further obtained, importance of each feature is determined, then the feature channels are weighted on convolution features output by an original residual block channel by multiplication according to the confidence degrees of all the feature channels, feature selection is achieved, the original features input to the original residual block and the convolution features after feature selection are finally fused, and logo classification features are output. The SE-Net helps the network to learn important characteristic information by performing weight distribution on each characteristic channel, so that the classification accuracy is improved.
In order to further improve the classification performance of the network, the embodiment of the present application may further use a Deformable Convolution module (Deformable Convolution) to replace a common Convolution layer (Convolution) in the ResNet50 network. Specifically, a deformable convolution module may be used to replace a common convolution layer in the residual block, and a person skilled in the art may specifically select which residual block to replace according to actual situations, which is not specifically limited herein in the embodiment of the present application. The embodiment of the application considers that the ResNet50 network structure is not perfect, and due to the internal network structure, the modeling capability for unknown variations is insufficient, for example, when a logo image is captured, there are front captured logo images, and also side captured logo images, the logo image acquired at the side surface has certain deformation due to the angle, so that the size of the logo in the logo image acquired at the front surface is different from that of the logo image acquired at the side surface, the existing network structure has poor characteristic extraction capability for the deformed image, the deformed image is generally required to be manually adjusted, then input into the network for training and classification, and the convolution operation in the existing network has a very fixed geometrical structure, for any regular or irregular shaped object a sampling of irregular moments (usually squares) is used. Based on this, in order to improve classification efficiency and accuracy and avoid excessive manual interference, in the embodiment of the application, a deformable convolution module is adopted to replace a common convolution layer in a ResNet50 network, a feature receptive field in a forward propagation process is automatically adjusted for an image with deformation through the deformable convolution module, and feature extraction capability is enhanced to make up for influences brought by the deformed image, so that network classification capability is improved. The feature Field (received Field) is a size of an area where pixel points on a feature map (feature map) output by each layer in the neural network are mapped on an input picture (i.e., a training sample in this embodiment). The feature map is mathematically a two-dimensional matrix composed of the numerical values of a plurality of features.
It is understood that the training samples may be preprocessed, for example, normalized, after being obtained, and then the preprocessed training samples are input into the pre-constructed convolutional neural network for training. It should be noted that, the classification performance of the network can also be improved by adding the attention mechanism module only to the residual error network, and a person skilled in the art can select whether to adopt the deformable convolution module according to actual situations.
And S10232, calculating loss values of the logo classification features.
After the logo classification features are extracted, the full-link layer of the convolutional neural network (the full-link layer in the network has the function of a classifier) predicts the logo defect categories of the logo classification features, the probability of the logo classification features under each logo defect category is output, the logo defect category with the highest probability is the prediction label of the logo classification features, and then the loss value of the logo classification features between the prediction label and the real label is calculated through a loss function. The loss function may be a mean square error function, a cross entropy function, or the like.
It should be noted that, when the real label is obtained by obtaining the training sample, the real label is obtained by labeling according to the completeness of the logo in the training sample, and the real label includes a complete class and an incomplete class, where the incomplete class includes a text defect, a pattern defect, a texture defect, a brightness defect, and the like.
And S10233, if the loss value is greater than or equal to a preset threshold value, updating parameters in the neural network by using the loss value, and returning to execute the step of inputting the training sample into the pre-constructed neural network.
And S10234, if the loss value is smaller than a preset threshold value, determining that the neural network training is finished.
And S10235, taking the trained neural network as a logo defect detection model.
After the loss value is obtained through calculation, whether the network is converged can be determined by judging whether the loss value meets a preset threshold value, if the loss value is larger than or equal to the preset threshold value, the network is not converged, parameters (such as weight, bias and the like) in the neural network are updated through back propagation of the loss value, the training samples are returned to be input into the neural network which is constructed in advance, next round of training is carried out, the training is stopped until the loss value is smaller than the preset threshold value, and the trained convolutional neural network is used as a logo defect detection model.
Whether the convolutional neural network is converged can be judged by calculating the detection accuracy of the convolutional neural network, when the detection accuracy of the convolutional neural network does not reach a preset accuracy threshold, the convolutional neural network is judged not to be converged, parameters in the neural network are updated according to back propagation of loss values obtained through calculation, a training sample is returned to be input into the neural network constructed in advance, next round of training is carried out, the convolutional neural network is judged to be converged until the detection accuracy of the convolutional neural network reaches the preset accuracy threshold, the training is stopped, and the trained convolutional neural network is used as a logo defect detection model.
The current training times can be obtained, whether the convolutional neural network is converged or not is judged by judging whether the current training times reach the preset maximum training times or not, if the current training times do not reach the maximum training times, the convolutional neural network is judged not to be converged, the parameters of the convolutional neural network are updated through back propagation of loss values obtained through calculation, the training samples are returned to be input into the neural network which is constructed in advance, next round of training is carried out, the convolutional neural network is judged to be converged until the current training times reach the maximum training times, the training is stopped, and the trained convolutional neural network is used as a logo defect detection model.
And finally, deploying the trained logo defect detection model in a generation environment for the defect detection of the logo.
And 103, storing the defect detection result of the logo image to be detected in a preset database.
After the defect detection result of the logo image to be detected is obtained through detection, the defect detection result of the logo image to be detected can be stored in a preset database according to the date of shooting the logo image, and a user can conveniently check the defect detection result.
In the embodiment of the application, after the logo image to be detected is obtained, the logo image to be detected is subjected to feature extraction and defect detection through the logo defect detection model, the defect detection result is stored to a preset database for users to look up, the logo defect detection is performed through the logo defect detection model, the detection speed is high, manual logo defect detection is not needed, the detection efficiency is improved, the existing shop logo is generally inspected through manual timing, time and labor are wasted, and the efficiency is low.
The foregoing is an embodiment of a logo defect detection method provided in the present application, and the following is another embodiment of a logo defect detection method provided in the present application.
Referring to fig. 3, an embodiment of a logo defect detection method includes:
And step 203, storing the defect detection result of the logo image to be detected in a preset database.
The specific contents of step 201 to step 203 are the same as those of step 101 to step 103 in the foregoing embodiment, and are not described herein again.
And step 204, generating a logo defect report according to the defect detection result in the preset database, and sending the logo defect report to an auditing department for auditing.
The embodiment of the application can generate the logo defect reports of all stores according to the defect detection result in the preset database, and can set to generate the logo defect reports periodically, for example, the logo defect reports are generated once per week, and the generated logo defect reports can be automatically sent to an auditing department for auditing through mails.
In the embodiment of the application, after the logo image to be detected is obtained, the logo image to be detected is subjected to feature extraction and defect detection through the logo defect detection model, the defect detection result is stored to a preset database for users to look up, the logo defect detection is performed through the logo defect detection model, the detection speed is high, manual logo defect detection is not needed, the detection efficiency is improved, the existing shop logo is generally inspected through manual timing, time and labor are wasted, and the efficiency is low.
Furthermore, the embodiment of the application generates the logo defect report periodically and automatically sends the logo defect report to the auditing department for auditing, manual counting of logo defect conditions is not needed, users can conveniently audit and check, and auditing efficiency is improved.
The foregoing is another embodiment of the logo defect detection method provided in the present application, and the following is an embodiment of the logo defect detection apparatus provided in the present application.
Referring to fig. 4, an embodiment of the present disclosure provides a logo defect detecting apparatus, including:
the system comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring images containing logos in stores as to-be-detected logo images;
the defect detection unit is used for inputting the logo image to be detected into a logo defect detection model, so that the logo defect detection model performs feature extraction on the logo image to be detected, and identifies whether the logo image to be detected has defects or not based on the extracted features to obtain a defect detection result of the logo image to be detected;
and the storage unit is used for storing the defect detection result of the logo image to be detected to a preset database.
As a further improvement, the apparatus further comprises:
and the report generation unit is used for generating a logo defect report according to the defect detection result in the preset database and sending the logo defect report to an auditing department for auditing.
In the embodiment of the application, after the logo image to be detected is obtained, the logo image to be detected is subjected to feature extraction and defect detection through the logo defect detection model, the defect detection result is stored to a preset database for users to look up, the logo defect detection is performed through the logo defect detection model, the detection speed is high, manual logo defect detection is not needed, the detection efficiency is improved, the existing shop logo is generally inspected through manual timing, time and labor are wasted, and the efficiency is low.
Furthermore, the embodiment of the application generates the logo defect report periodically and automatically sends the logo defect report to the auditing department for auditing, manual counting of logo defect conditions is not needed, users can conveniently audit and check, and auditing efficiency is improved.
The embodiment of the application also provides logo defect detection equipment, which comprises a processor and a memory;
the memory is used for storing program codes and transmitting the program codes to the processor;
the processor is used for executing the logo defect detection method in the method embodiment according to the instructions in the program code.
The embodiment of the present application further provides a computer-readable storage medium, which is used for storing program codes, and when the program codes are executed by a processor, the method for detecting logo defects in the foregoing method embodiments is implemented.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The terms "first," "second," "third," "fourth," and the like in the description of the application and the above-described figures, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of operation in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be understood that in the present application, "at least one" means one or more, "a plurality" means two or more. "and/or" for describing an association relationship of associated objects, indicating that there may be three relationships, e.g., "a and/or B" may indicate: only A, only B and both A and B are present, wherein A and B may be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "at least one of the following" or similar expressions refer to any combination of these items, including any combination of single item(s) or plural items. For example, at least one (one) of a, b, or c, may represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for executing all or part of the steps of the method described in the embodiments of the present application through a computer device (which may be a personal computer, a server, or a network device). And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.
Claims (10)
1. A logo defect detection method is characterized by comprising the following steps:
collecting images containing logos in stores as logo images to be detected;
inputting the logo image to be detected into a logo defect detection model, enabling the logo defect detection model to perform feature extraction on the logo image to be detected, and identifying whether the logo image to be detected has defects or not based on the extracted features to obtain a defect detection result of the logo image to be detected;
and storing the defect detection result of the logo image to be detected into a preset database.
2. The logo defect detection method as claimed in claim 1, wherein the logo defect detection model is trained by:
acquiring an original image containing a logo;
constructing a training sample according to the completeness of the logo in the original image;
training the capability of a pre-constructed neural network for detecting the defects of the logo through the training sample until the neural network meets the preset training condition, and taking the trained neural network as a logo defect detection model.
3. The logo defect detection method as claimed in claim 2, wherein the training samples comprise positive samples and negative samples; the constructing of the training sample according to the completeness of the logo in the original image comprises:
detecting the completeness of the logo in the original image to take the original image with complete logo as a positive sample and the original image with incomplete logo as a negative sample;
respectively executing data enhancement operation on the positive sample and the negative sample to obtain a training sample consisting of the positive sample subjected to data enhancement and the negative sample subjected to data enhancement;
wherein the data enhancement operation includes, but is not limited to, random cropping, center cropping, random rotation, random horizontal flipping.
4. The logo defect detection method as claimed in claim 3, wherein the detecting the completeness of the logo in the original image to make the original image with complete logo as a positive sample and the original image with incomplete logo as a negative sample comprises:
detecting feature information of the logo in the original image, wherein the feature information comprises character information, pattern information, texture information and brightness information;
if the characteristic information does not meet the preset logo design requirement, determining that the currently detected logo is an incomplete logo, and taking the original image with the incomplete logo as a negative sample;
and if the characteristic information meets the logo design requirement, determining that the currently detected logo is a complete logo, and taking the original image with the complete logo as a positive sample.
5. The logo defect detection method as claimed in claim 4, wherein the text information comprises text content and a typesetting sequence of the text, the pattern information comprises pattern content and a color of the pattern, the texture information comprises background texture and a background color, and the brightness information comprises a brightness value of the logo; the logo design requirement comprises target character content, a typesetting sequence of the target characters, target pattern content, target pattern color, target background texture, target background color and target brightness value of the logo;
after the detecting the feature information of the logo in the original image, the method further includes:
when the character content is determined to be not in accordance with the target character content and/or the typesetting sequence of the characters is determined to be not in accordance with the typesetting sequence of the target characters, determining that the characteristic information does not meet the preset logo design requirement;
and/or the presence of a gas in the gas,
when the pattern content is determined not to accord with the target pattern content and/or the color of the pattern is determined not to accord with the color of the target pattern, determining that the characteristic information does not meet the preset logo design requirement;
and/or the presence of a gas in the gas,
when the background texture is determined not to accord with the target background texture and/or the background color is determined not to accord with the target background color, determining that the characteristic information does not meet the preset logo design requirement;
and/or the presence of a gas in the gas,
and when the brightness value of the logo is determined not to meet the target brightness value of the logo, determining that the characteristic information does not meet the preset logo design requirement.
6. The logo defect detection method according to any one of claims 2-5, wherein the training of the capability of the pre-constructed neural network to detect the defect of the logo through the training sample until the neural network meets the preset training condition, and the training of the neural network as the logo defect detection model comprises:
inputting the training sample into a pre-constructed neural network, extracting the characteristics of the logo in the training sample under multiple dimensions by using the neural network, and outputting to obtain logo classification characteristics fused by multiple characteristics;
calculating a loss value of the logo classification feature;
if the loss value is larger than or equal to a preset threshold value, updating parameters in the neural network by using the loss value, and returning to execute the step of inputting the training sample into a pre-constructed neural network;
if the loss value is smaller than a preset threshold value, determining that the neural network training is finished;
and taking the trained neural network as a logo defect detection model.
7. The logo defect detection method as claimed in claim 6, wherein the pre-constructed neural network comprises a residual network, an attention mechanism module, a deformable convolution module; inputting the training sample into a pre-constructed neural network, extracting features of the logo in the training sample under multiple dimensions by using the neural network, and outputting to obtain logo classification features fused by multiple features, wherein the method comprises the following steps:
performing convolution operation on the training sample by utilizing a plurality of residual blocks in the residual error network to obtain convolution characteristics;
feature selection is carried out on the convolution features according to the types of logo defects through the attention mechanism module, a plurality of convolution features after feature selection are fused, and logo classification features are output; the category of the logo defects comprises defects of the logo in any dimension of characters, patterns, textures and brightness;
the deformable convolution module is used for adjusting the characteristic receptive field of the training sample in the forward propagation process of the whole neural network.
8. An emblem defect detection apparatus, comprising:
the system comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring images containing logos in stores as to-be-detected logo images;
the defect detection unit is used for inputting the logo image to be detected into a logo defect detection model, so that the logo defect detection model performs feature extraction on the logo image to be detected, and identifies whether the logo image to be detected has defects or not based on the extracted features to obtain a defect detection result of the logo image to be detected;
and the storage unit is used for storing the defect detection result of the logo image to be detected to a preset database.
9. A logo defect detecting device, characterized in that the device comprises a processor and a memory;
the memory is used for storing program codes and transmitting the program codes to the processor;
the processor is configured to execute the logo defect detection method of any one of claims 1-7 according to instructions in the program code.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium is used for storing program code, which when executed by a processor implements the logo defect detection method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110956354.3A CN113610832B (en) | 2021-08-19 | 2021-08-19 | Logo defect detection method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110956354.3A CN113610832B (en) | 2021-08-19 | 2021-08-19 | Logo defect detection method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113610832A true CN113610832A (en) | 2021-11-05 |
| CN113610832B CN113610832B (en) | 2023-02-10 |
Family
ID=78341391
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110956354.3A Active CN113610832B (en) | 2021-08-19 | 2021-08-19 | Logo defect detection method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113610832B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114723748A (en) * | 2022-06-06 | 2022-07-08 | 深圳硅山技术有限公司 | Detection method, device and equipment of motor controller and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6134342A (en) * | 1993-12-27 | 2000-10-17 | Menicon Co., Ltd. | Visual inspection method and apparatus for contact lenses |
| CN1586069A (en) * | 2001-11-13 | 2005-02-23 | 皇家飞利浦电子股份有限公司 | Identification and evaluation of audience exposure to logos in a broadcast event |
| CN102436578A (en) * | 2012-01-16 | 2012-05-02 | 宁波江丰生物信息技术有限公司 | Formation method for dog face characteristic detector as well as dog face detection method and device |
| CN104077577A (en) * | 2014-07-03 | 2014-10-01 | 浙江大学 | Trademark detection method based on convolutional neural network |
| CN110910339A (en) * | 2019-07-10 | 2020-03-24 | 研祥智能科技股份有限公司 | Logo defect detection method and device |
| CN112200246A (en) * | 2020-10-09 | 2021-01-08 | 河北工业大学 | Training method of SVM classifier and petrochemical storage tank corrosion defect segmentation method |
-
2021
- 2021-08-19 CN CN202110956354.3A patent/CN113610832B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6134342A (en) * | 1993-12-27 | 2000-10-17 | Menicon Co., Ltd. | Visual inspection method and apparatus for contact lenses |
| CN1586069A (en) * | 2001-11-13 | 2005-02-23 | 皇家飞利浦电子股份有限公司 | Identification and evaluation of audience exposure to logos in a broadcast event |
| CN102436578A (en) * | 2012-01-16 | 2012-05-02 | 宁波江丰生物信息技术有限公司 | Formation method for dog face characteristic detector as well as dog face detection method and device |
| CN104077577A (en) * | 2014-07-03 | 2014-10-01 | 浙江大学 | Trademark detection method based on convolutional neural network |
| CN110910339A (en) * | 2019-07-10 | 2020-03-24 | 研祥智能科技股份有限公司 | Logo defect detection method and device |
| CN112200246A (en) * | 2020-10-09 | 2021-01-08 | 河北工业大学 | Training method of SVM classifier and petrochemical storage tank corrosion defect segmentation method |
Non-Patent Citations (2)
| Title |
|---|
| ZHOU LH ET AL: "《Detecting Motion Blurred Vehicle Logo in IoV Using Filter-DeblurGAN and VL-YOLO》", 《IEEE》 * |
| 陈宇: "《软包商标印刷缺陷检测系统研究》", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114723748A (en) * | 2022-06-06 | 2022-07-08 | 深圳硅山技术有限公司 | Detection method, device and equipment of motor controller and storage medium |
| CN114723748B (en) * | 2022-06-06 | 2022-09-02 | 深圳硅山技术有限公司 | Detection method, device and equipment of motor controller and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113610832B (en) | 2023-02-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108562589B (en) | Method for detecting surface defects of magnetic circuit material | |
| CN108960245B (en) | Tire mold character detection and recognition method, device, equipment and storage medium | |
| CN110956225B (en) | Contraband detection method and system, computing device and storage medium | |
| CN108830145B (en) | People counting method based on deep neural network and storage medium | |
| CN107169463B (en) | Method for detecting human face, device, computer equipment and storage medium | |
| WO2020167581A1 (en) | Method and apparatus for processing video stream | |
| CN112348117B (en) | Scene recognition method, device, computer equipment and storage medium | |
| CN110070029B (en) | Gait recognition method and device | |
| US11288552B2 (en) | Image quality assessment using adaptive non-overlapping mean estimation | |
| CN114494837B (en) | Intelligent density identification method and system for fishery resources | |
| CN110765990A (en) | Intelligent article detection method and system, computing device and storage medium | |
| JP6397379B2 (en) | CHANGE AREA DETECTION DEVICE, METHOD, AND PROGRAM | |
| CN110598019B (en) | Repeated image identification method and device | |
| CN110942456B (en) | Tamper image detection method, device, equipment and storage medium | |
| CN110097616B (en) | Combined drawing method and device, terminal equipment and readable storage medium | |
| CN109740539B (en) | 3D object identification method based on ultralimit learning machine and fusion convolution network | |
| CN108446688B (en) | Face image gender judgment method and device, computer equipment and storage medium | |
| CN112862757A (en) | Weight evaluation system based on computer vision technology and implementation method | |
| CN115131283A (en) | Defect detection and model training method, device, equipment and medium for target object | |
| CN108984576A (en) | A kind of patent document acquisition methods and system based on convolutional neural networks | |
| CN114596290A (en) | Defect detection method, defect detection device, storage medium, and program product | |
| CN111415339A (en) | Image defect detection method for complex texture industrial product | |
| CN111950457A (en) | Oil field safety production image identification method and system | |
| CN113610832B (en) | Logo defect detection method, device, equipment and storage medium | |
| CN114078127B (en) | Object defect detection and counting method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |