CN110826416B - Bathroom ceramic surface defect detection method and device based on deep learning - Google Patents

Bathroom ceramic surface defect detection method and device based on deep learning Download PDF

Info

Publication number
CN110826416B
CN110826416B CN201910963407.7A CN201910963407A CN110826416B CN 110826416 B CN110826416 B CN 110826416B CN 201910963407 A CN201910963407 A CN 201910963407A CN 110826416 B CN110826416 B CN 110826416B
Authority
CN
China
Prior art keywords
candidate
defect
image
frame
frames
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910963407.7A
Other languages
Chinese (zh)
Other versions
CN110826416A (en
Inventor
王志锋
周壮壮
卢清华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Foshan University
Original Assignee
Foshan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Foshan University filed Critical Foshan University
Priority to CN201910963407.7A priority Critical patent/CN110826416B/en
Publication of CN110826416A publication Critical patent/CN110826416A/en
Application granted granted Critical
Publication of CN110826416B publication Critical patent/CN110826416B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Abstract

The invention relates to the technical field of visual detection, in particular to a bathroom ceramic surface defect detection method and device based on deep learning, which comprises the steps of firstly obtaining an image dataset, wherein the image dataset is a set of surface images of bathroom ceramic to be detected; marking the defect type and the minimum defect circumscribed rectangle of the surface image, and generating an image feature set; dividing the image feature set into a training set, a verification set and a test set, and generating a candidate frame according to the minimum external rectangle of the defects of the training set; inputting the candidate boxes into a Faster R-CNN neural network for training to generate a surface defect detection model; and finally, inputting the test set into the detection model to detect the defect type of the bathroom ceramic to be detected.

Description

Bathroom ceramic surface defect detection method and device based on deep learning
Technical Field
The invention relates to the technical field of visual detection, in particular to a bathroom ceramic surface defect detection method and device based on deep learning.
Background
In the actual production line of factories, certain defects of products inevitably exist due to uncontrollable factors such as unstable product transportation, unstable manufacturing process, artificial interference and the like. Among them, the surface defects have a remarkable influence on the quality of the product, such as scratches, cracks, etc., and the defects not only affect the appearance of the product, but also cause that certain functions of the product cannot be normally realized. Therefore, surface defect detection is a key point for improving production efficiency and yield. For the closestool workpiece after glaze spraying firing, the surface of the workpiece is smooth and easy to reflect light, the contrast of a defect part is not obvious, and the defect part is difficult to identify under common ambient light irradiation. In addition, as toilets of different models have different molded surfaces and have outstanding edges, the characteristics can cause certain interference to the recognition algorithm.
The surface defect detection technology is an extremely important link in an intelligent production line, and the traditional manual quality inspection or off-line sampling inspection cannot keep long-term and high-efficiency detection efficiency due to high labor intensity, large workload and easy subjective influence, so that the production efficiency of factories is restricted to a certain extent.
The combination of the support vector machine and the traditional image processing technology is suitable for detecting defects of small samples, but the quality of detection results is greatly dependent on the quality of extracted features. Therefore, the method has higher requirements on relevant expertise of developers, and meanwhile, when the identification task changes, the robustness of the algorithm of the method needs to be further improved.
Therefore, how to accurately identify and locate the surface defect detection of the bathroom ceramic becomes a problem to be solved.
Disclosure of Invention
In order to solve the problems, the invention provides a bathroom ceramic surface defect detection method and device based on deep learning, which can accurately identify and position the bathroom ceramic surface defect detection.
In order to achieve the above object, the present invention provides the following technical solutions:
according to an embodiment of the first aspect of the invention, a bathroom ceramic surface defect detection method based on deep learning comprises the following steps:
acquiring an image data set, wherein the image data set is a set of surface images of bathroom ceramics to be detected;
labeling the defect type and the minimum defect circumscribed rectangle of the surface image to generate an image feature set;
dividing the image feature set into a training set, a verification set and a test set, and generating a candidate frame according to the minimum external rectangle of the defects of the training set;
inputting the candidate boxes into a Faster R-CNN neural network for training to generate a surface defect detection model;
inputting the test set into the detection model, and detecting the defect type of the bathroom ceramic to be detected.
According to some embodiments of the invention, before labeling the defect type and the minimum bounding rectangle of the defect of the surface image, the method comprises:
performing size normalization processing on the surface image to generate a normalized image;
removing noise contained in the normalized image by using Gaussian filtering, and enhancing the contrast of the normalized image by using Laplacian transformation;
and rotating the surface image by 90 degrees, 180 degrees and 270 degrees along the anticlockwise direction in sequence, and randomly cutting the surface image at each rotation angle to enhance the definition of the surface image.
According to some embodiments of the invention, the generating a candidate frame according to the minimum circumscribed rectangle of the defects of the training set includes:
clustering the minimum external rectangle of the defects by adopting a K-Means algorithm and taking the minimum intra-class distance as an evaluation index;
replacing the size of the minimum defect circumscribed rectangle with the size obtained by clustering, and generating a preselected frame of the minimum defect circumscribed rectangle;
and selecting the candidate frame with the minimum circumscribed rectangle of the defect from the preselected frames by adopting a Faster R-CNN neural network frame.
According to some embodiments of the invention, the inputting the candidate box into a fast R-CNN neural network for training, generating a surface defect detection model, includes:
performing multi-scale feature fusion on the candidate frames, and identifying the defect types and coordinates of feature graphs contained in the candidate frames;
screening the candidate frames to generate annotation frames;
inputting the verification set into a Faster R-CNN neural network for training to obtain a result frame of the surface defect;
calculating the deviation of the result frame and the labeling frame by adopting a multi-task loss function, wherein the multi-task loss function comprises a classification loss function and a regression loss function, and when the deviation is lower than a set threshold value, training of a fast R-CNN neural network is completed;
and taking the trained Faster R-CNN neural network as a detection model.
According to some embodiments of the invention, the performing multi-scale feature fusion on the candidate frame, identifying the defect type of the feature map and coordinates of the coordinate frame contained in the candidate frame, includes:
adding a feature pyramid network into a VGG16 convolutional neural network, merging a plurality of feature graphs contained in the candidate frame, wherein the feature graphs have respective sizes and receptive fields;
inputting a plurality of feature images into an RPN network, and identifying the defect type and the coordinate frame coordinates of the feature images;
and mapping the feature map into candidate frames according to the respective anchor points.
According to some embodiments of the invention, the filtering the candidate frames to generate the labeling frame includes:
step 2.1, eliminating candidate frames beyond the boundary of the surface image;
step 2.2, filtering out candidate frames with the width or the height smaller than a set threshold value;
step 2.3, calculating the overlapping degree of the candidate frames and the labeling frames, sorting the candidate frames according to the overlapping degree, and extracting N candidate frames sorted in the front;
step 2.4, filtering the candidate frames extracted in the step 2.3 by using a non-maximum suppression algorithm;
and 2.5, if the number of the candidate frames filtered in the step 2.4 is still greater than a set threshold S, selecting S candidate frames sequenced in the front as labeling frames.
According to a second aspect of the embodiment of the invention, a bathroom ceramic surface defect detection device based on deep learning comprises: the system comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the computer program is executed by the processor to realize the deep learning-based bathroom ceramic surface defect detection method according to the first aspect of the invention.
The beneficial effects of the invention are as follows: the invention discloses a bathroom ceramic surface defect detection method and device based on deep learning, which comprises the steps of firstly, acquiring an image data set, wherein the image data set is a set of surface images of bathroom ceramic to be detected; marking the defect type and the minimum defect circumscribed rectangle of the surface image, and generating an image feature set; dividing the image feature set into a training set, a verification set and a test set, and generating a candidate frame according to the minimum external rectangle of the defects of the training set; inputting the candidate boxes into a Faster R-CNN neural network for training to generate a surface defect detection model; and finally, inputting the test set into the detection model, and detecting the defect type of the bathroom ceramic to be detected. The invention can accurately identify and position the surface defect detection of the bathroom ceramic.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings that are needed in the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a bathroom ceramic surface defect detection method based on deep learning according to an embodiment of the invention;
FIG. 2 is a flowchart of step S300 according to an embodiment of the present invention;
FIG. 3 is an original block generated by the standard Faster R-CNN algorithm in an embodiment of the invention;
FIG. 4 is a candidate box generated based on a K-Means clustering algorithm in an embodiment of the invention;
FIG. 5 is a flowchart of step S400 according to an embodiment of the present invention;
FIG. 6 is a block diagram of a multi-scale feature fusion provided by an embodiment of the present invention.
Detailed Description
The conception, specific structure, and technical effects produced by the present disclosure will be clearly and completely described below in connection with the embodiments and the drawings to fully understand the objects, aspects, and effects of the present disclosure. It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be combined with each other.
According to a first aspect of the application, referring to fig. 1, a bathroom ceramic surface defect detection method based on deep learning provided in an embodiment of the application includes the following steps:
step S100, acquiring an image data set, wherein the image data set is a set of surface images of the bathroom ceramic to be detected.
In an alternative embodiment, an industrial camera is used to perform image acquisition on the surface of the bathroom ceramic under the irradiation of the coaxial light source, so as to acquire an image dataset of the surface defect of the bathroom ceramic to be detected, wherein the image dataset is a set of surface images of the bathroom ceramic to be detected.
And step 200, labeling the defect type and the minimum defect circumscribed rectangle of the surface image, and generating an image feature set.
In an alternative embodiment, the surface images are manually marked by using a labelImg image marking tool, so that a defect range is defined, the marked surface images are collected into an image feature set, and the image feature set is stored in an xml file format for subsequent model training. Optionally, the manually noted content includes: the defect type comprises pinholes, hard cracks, pits, pit bags and impurities.
And step S300, dividing the image feature set into a training set, a verification set and a test set, and generating a candidate frame according to the minimum defect circumscribed rectangle in the training set.
And step S400, inputting the candidate box into a Faster R-CNN neural network for training, and generating a surface defect detection model.
And S500, inputting the test set into the detection model, and detecting the defect type of the bathroom ceramic to be detected.
Firstly, acquiring an image data set, labeling a defect type and a minimum defect circumscribed rectangle of the surface image, and generating an image feature set so as to form a surface image sample for model training; performing preliminary processing on the surface image sample, dividing the image feature set into a training set, a verification set and a test set, and generating a candidate frame according to the minimum external rectangle of the defects in the training set; inputting the candidate boxes into a Faster R-CNN neural network for training to generate a surface defect detection model; and finally, inputting the test set into the detection model, and detecting the defect type of the bathroom ceramic to be detected. According to the embodiment provided by the invention, after the detection model is trained, the defect area and the defect type of the bathroom ceramic surface defect detection can be automatically identified only by inputting the surface image to be detected into the detection model, and the bathroom ceramic surface defect detection can be accurately identified and positioned without manual treatment in the detection process.
In a preferred embodiment, before step S200, it comprises:
performing size normalization processing on the surface image to generate a normalized image;
noise contained in the surface image is removed using gaussian filtering, and the contrast of the surface image is enhanced using laplace transform.
And rotating the surface image by 90 degrees, 180 degrees and 270 degrees along the anticlockwise direction in sequence, and randomly cutting the surface image at each rotation angle to enhance the definition of the surface image. Thereby enhancing the degree of differentiation of defective areas from background areas in the surface image. The problems of reduced precision and recall rate caused by lower contrast and smaller defects of the surface image are solved.
Referring to fig. 2, in a preferred embodiment, in step S300, generating a candidate frame according to the minimum bounding rectangle of defects in the training set includes:
step S310, clustering the minimum external rectangle of the defect by adopting a K-Means algorithm and taking the minimum intra-class distance as an evaluation index;
step S320, replacing the size of the minimum defect circumscribed rectangle with the size obtained by clustering, and generating a preselected frame of the minimum defect circumscribed rectangle;
and S330, selecting a candidate frame of the minimum-defect circumscribed rectangle from the preselected frames by adopting a Faster R-CNN neural network frame.
In this embodiment, the dimensions include a size and an aspect ratio, and by adjusting the minimum circumscribed rectangle of the defect, the dimensions and the aspect ratio of the candidate frame generated directly by adopting the RPN network are changed, so that the method is suitable for positioning and selecting the surface defect of the bathroom ceramic.
To demonstrate the improvement of the K-Means clustering algorithm on the size and aspect ratio of the candidate boxes, in a comparative example, as shown in fig. 3 and 4, where fig. 3 is the original box generated by the standard fast R-CNN algorithm and fig. 4 is the candidate box generated based on the K-Means clustering algorithm. As can be seen from fig. 3, most of the regions marked by the original boxes contain excessive background, which increases the algorithm calculation cost and increases the model convergence difficulty. As shown in fig. 4, most of the candidate frames are marked with the positions of the defects, the candidate frames with smaller sizes are favorable for identifying pinholes, pit bags, pits and impurity defects, the candidate frames with nearly rectangular shapes are favorable for identifying cracks and hard crack defects, and through the combination of various sizes and length-width ratios, the candidate frames can cover most types of defects and effectively reduce algorithm complexity.
Referring to fig. 5, in a preferred embodiment, the step S400 includes:
and step S410, carrying out multi-scale feature fusion on the candidate frame, and identifying the defect type and coordinates of the feature map contained in the candidate frame.
In this embodiment, the feature map means: after the characteristic extraction network is used for extracting, the generated characteristic value matrix is used for classifying the receptive field, the larger the value of the receptive field is, the larger the image range which can be contacted with the receptive field is, namely, the more abstract the characteristics contained in the neuron are, the higher the semantic level is; the smaller the value of the receptive field, the more localized and detailed the features it contains tend to be; the feature extraction network is an important component in the defect detection model, and the feature map acquisition is an important step of target detection.
In a depth network, the bottom layer features have less semantic information, but the target positioning is accurate, and the high layer features have rich semantic information and are suitable for detecting small targets, but the positioning effect is not ideal. In order to improve the positioning and detection precision of the model, the embodiment adopts a multi-feature fusion technology to provide more semantic information and fine-grained features for a subsequent network structure, increases the feature extraction capability of the model on small targets, and improves the precision to a certain extent.
In a specific embodiment, the step S410 includes:
1.1, adding a feature pyramid network into a VGG16 convolutional neural network, merging a plurality of feature graphs contained in the candidate frame, wherein the feature graphs have respective sizes and receptive fields;
1.2, inputting a plurality of feature images into an RPN network, and identifying the defect type and the coordinate frame coordinates of the feature images;
and 1.3, mapping the feature map into candidate frames according to the respective anchor points.
In this embodiment, receptive field means: the size of the area mapped by the pixel points on the characteristic map output by each layer of the convolutional neural network on the input picture. Anchor points refer to: and the feature extraction network extracts pixel points on the feature map. The RPN Network refers to a Region-pro-pos-Network (Region-pro-area-Network). The multi-scale feature fusion structure diagram provided by the embodiment of the invention is shown in fig. 6. Within the dashed box in the figure are cross-links, and the convolution kernel of 1*1 is used to adjust the dimension of the feature map so that it can be fused with the up-sampled feature matrix. First, the higher-layer features with more abstract and stronger semantic features are up-sampled by 2 times, and then are transversely connected to the upper-layer features through matrix addition. The convolution of 3*3 was used after fusion to check each new feature convolution to eliminate aliasing effects after upsampling. By fusing low-layer high-resolution features and high-layer high-semantic information features, features under all scales have rich semantic features, and independent prediction is performed on each fused feature layer, so that the detection capability of small targets is enhanced.
In this embodiment, the number of candidate frames directly generated by the anchor point is excessive, and a part of candidate frames do not frame-select the defect area. If all the candidate frames participate in training, the model training time cost is greatly increased, and the training of the algorithm model is difficult to converge.
And step S420, screening the candidate frames to generate labeling frames. The method specifically comprises the following steps:
step 2.1, eliminating candidate frames exceeding the boundary of the surface image;
step 2.2, filtering out candidate frames with the width or the height smaller than a set threshold value;
step 2.3, calculating the overlapping degree of the candidate frames and the labeling frames, sorting the candidate frames according to the overlapping degree, and extracting N candidate frames sorted in the front;
step 2.4, filtering the candidate frames extracted in the step 2.3 by using a non-maximum suppression algorithm;
and 2.5, if the number of the candidate frames filtered in the step 2.4 is still greater than a set threshold S, selecting S candidate frames sequenced in the front as labeling frames.
The value of N, S is set according to the actual size of the defect, for example, the top N candidate boxes may be selected according to a certain percentage, for example, the top 10% candidate boxes may be selected; or directly selecting a certain number of candidate frames, for example, sorting the candidate frames located in the first 1000, it is clear to those skilled in the art that the more the number of candidate frames is selected, the more advantageous the training accuracy of the multi-detection model is, so that the selected candidate frames need to satisfy a certain number, N is set to 1000, and S is set to 300.
And S430, inputting the verification set into a Faster R-CNN neural network for training to obtain a result frame of the surface defect.
And step S440, calculating the deviation of the result frame and the labeling frame by adopting a multi-task loss function, wherein the multi-task loss function comprises a classification loss function and a regression loss function, and when the deviation is lower than a set threshold value, training the Faster R-CNN neural network is completed.
And S450, taking the trained Faster R-CNN neural network as a detection model.
In an example, in step S500 of the present embodiment, the test set is input into the detection model, and the defect type of the bathroom ceramic to be detected is detected; the test results are shown in table 1, compared with the existing defect detection method based on the support vector machine.
Table 1: comparing the experimental results
Model Precision (precision) Recall (recovery) Detection time ms/sheet
Existing defect detection method 0.603 0.416 630
This embodiment 0.903 0.886 165
From the comparative data of table 1, it can be seen that: the method provided by the embodiment has higher precision and recall rate and shorter detection time, so that the precision of the detection of the surface defects of the bathroom ceramic is improved, and the perceptibility of the micro defects of the surface of the bathroom ceramic is improved.
According to a second aspect of the application, the invention further provides a bathroom ceramic surface defect detection device based on deep learning, which comprises: a memory, a processor and a computer program stored in the memory and executable on the processor, which when executed by the processor implements the deep learning based bathroom ceramic surface defect detection method described in the first aspect embodiment.
The bathroom ceramic surface defect detection device based on deep learning can be operated in computing equipment such as a desktop computer, a notebook computer, a palm computer and a cloud server. The device for detecting the surface defects of the bathroom ceramics based on deep learning can comprise, but is not limited to, a processor and a memory. It will be appreciated by those skilled in the art that the example is merely an example of a deep learning based bathroom ceramic surface defect detection apparatus, and is not limited to a deep learning based bathroom ceramic surface defect detection apparatus, and may include more or fewer components than examples, or may combine certain components, or different components, e.g., the deep learning based bathroom ceramic surface defect detection apparatus may further include input and output devices, network access devices, buses, etc.
The Processor may be a Central-Processing Unit (CPU), other general-purpose Processor, digital-Signal-Processor (DSP), application-Specific-Integrated-Circuit (ASIC), field-Programmable-Gate array (FPGA), other Programmable logic device, discrete Gate or transistor logic device, discrete hardware components, or the like. The general processor can be a microprocessor or any conventional processor, etc., and the processor is a control center of the deep learning-based bathroom ceramic surface defect detection device, and various interfaces and lines are utilized to connect various parts of the whole deep learning-based bathroom ceramic surface defect detection device operational device.
The memory may be used to store the computer program and/or module, and the processor may implement various functions of the deep learning-based bathroom ceramic surface defect detection device by running or executing the computer program and/or module stored in the memory and invoking data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like; the storage data area may store data (such as audio data, phonebook, etc.) created according to the use of the handset, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as a hard disk, memory, plug-in hard disk, smart-Media-Card (SMC), secure-Digital (SD) Card, flash Card (Flash-Card), at least one disk storage device, flash memory device, or other volatile solid-state storage device.
While the present disclosure has been described in considerable detail and with particularity with respect to several described embodiments, it is not intended to be limited to any such detail or embodiments or any particular embodiment, but is to be construed as providing broad interpretation of such claims by reference to the appended claims in view of the prior art so as to effectively encompass the intended scope of the disclosure. Furthermore, the foregoing description of the present disclosure has been presented in terms of embodiments foreseen by the inventor for the purpose of providing a enabling description for enabling the enabling description to be available, notwithstanding that insubstantial changes in the disclosure, not presently foreseen, may nonetheless represent equivalents thereto.

Claims (4)

1. A bathroom ceramic surface defect detection method based on deep learning is characterized by comprising the following steps:
acquiring an image data set, wherein the image data set is a set of surface images of bathroom ceramics to be detected;
labeling the defect type and the minimum defect circumscribed rectangle of the surface image to generate an image feature set;
dividing the image feature set into a training set, a verification set and a test set, and generating a candidate frame according to the minimum external rectangle of the defects of the training set;
inputting the candidate boxes into a Faster R-CNN neural network for training to generate a surface defect detection model;
inputting the test set into the detection model, and detecting the defect type of the bathroom ceramic to be detected;
generating a candidate frame according to the minimum defect circumscribed rectangle of the training set, including:
clustering the minimum external rectangle of the defects by adopting a K-Means algorithm and taking the minimum intra-class distance as an evaluation index;
replacing the size of the minimum defect circumscribed rectangle with the size obtained by clustering, and generating a preselected frame of the minimum defect circumscribed rectangle;
selecting a candidate frame of the minimum defect circumscribed rectangle from the preselected frames by adopting a Faster R-CNN neural network frame;
inputting the candidate box into a Faster R-CNN neural network for training, and generating a surface defect detection model, wherein the method comprises the following steps of:
performing multi-scale feature fusion on the candidate frames, and identifying the defect types and coordinates of feature graphs contained in the candidate frames;
screening the candidate frames to generate annotation frames;
inputting the verification set into a Faster R-CNN neural network for training to obtain a result frame of the surface defect;
calculating the deviation of the result frame and the labeling frame by adopting a multi-task loss function, wherein the multi-task loss function comprises a classification loss function and a regression loss function, and when the deviation is lower than a set threshold value, training of a fast R-CNN neural network is completed;
taking the trained Faster R-CNN neural network as a detection model;
the step of carrying out multi-scale feature fusion on the candidate frame and identifying the defect type of the feature map and coordinates of the coordinate frame contained in the candidate frame comprises the following steps:
adding a feature pyramid network into a VGG16 convolutional neural network, merging a plurality of feature graphs contained in the candidate frame, wherein the feature graphs have respective sizes and receptive fields;
inputting a plurality of feature images into an RPN network, and identifying the defect type and the coordinate frame coordinates of the feature images;
and mapping the feature map into candidate frames according to the respective anchor points.
2. The method for detecting the surface defects of the bathroom ceramic based on the deep learning according to claim 1, wherein before marking the defect type and the minimum circumscribed rectangle of the surface image, the method comprises the following steps:
performing size normalization processing on the surface image to generate a normalized image;
removing noise contained in the normalized image by using Gaussian filtering, and enhancing the contrast of the normalized image by using Laplacian transformation;
and rotating the surface image by 90 degrees, 180 degrees and 270 degrees along the anticlockwise direction in sequence, and randomly cutting the surface image at each rotation angle to enhance the definition of the surface image.
3. The method for detecting the surface defects of the bathroom ceramic based on deep learning according to claim 2, wherein the step of screening the candidate frames to generate the labeling frame comprises the following steps:
step 2.1, eliminating candidate frames beyond the boundary of the surface image;
step 2.2, filtering out candidate frames with the width or the height smaller than a set threshold value;
step 2.3, calculating the overlapping degree of the candidate frames and the labeling frames, sorting the candidate frames according to the overlapping degree, and extracting N candidate frames sorted in the front;
step 2.4, filtering the candidate frames extracted in the step 2.3 by using a non-maximum suppression algorithm;
and 2.5, if the number of the candidate frames filtered in the step 2.4 is still greater than a set threshold S, selecting S candidate frames sequenced in the front as labeling frames.
4. Bathroom ceramic surface defect detection device based on deep learning, characterized in that the device includes: a memory, a processor and a computer program stored in the memory and executable on the processor, which when executed by the processor implements the deep learning based bathroom ceramic surface defect detection method of any one of claims 1 to 3.
CN201910963407.7A 2019-10-11 2019-10-11 Bathroom ceramic surface defect detection method and device based on deep learning Active CN110826416B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910963407.7A CN110826416B (en) 2019-10-11 2019-10-11 Bathroom ceramic surface defect detection method and device based on deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910963407.7A CN110826416B (en) 2019-10-11 2019-10-11 Bathroom ceramic surface defect detection method and device based on deep learning

Publications (2)

Publication Number Publication Date
CN110826416A CN110826416A (en) 2020-02-21
CN110826416B true CN110826416B (en) 2023-05-30

Family

ID=69549223

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910963407.7A Active CN110826416B (en) 2019-10-11 2019-10-11 Bathroom ceramic surface defect detection method and device based on deep learning

Country Status (1)

Country Link
CN (1) CN110826416B (en)

Families Citing this family (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111652853A (en) * 2020-05-11 2020-09-11 西南科技大学 Magnetic powder flaw detection method based on deep convolutional neural network
CN111524135B (en) * 2020-05-11 2023-12-26 安徽继远软件有限公司 Method and system for detecting defects of tiny hardware fittings of power transmission line based on image enhancement
CN111709451B (en) * 2020-05-21 2023-10-17 五邑大学 Wine bottle surface defect detection method, electronic device and storage medium
CN111680750B (en) * 2020-06-09 2022-12-06 创新奇智(合肥)科技有限公司 Image recognition method, device and equipment
CN111754502A (en) * 2020-06-30 2020-10-09 浙江工业大学 Method for detecting surface defects of magnetic core based on fast-RCNN algorithm of multi-scale feature fusion
CN111967595B (en) * 2020-08-17 2023-06-06 成都数之联科技股份有限公司 Candidate frame labeling method and system, model training method and target detection method
CN111968093A (en) * 2020-08-19 2020-11-20 创新奇智(上海)科技有限公司 Magnetic shoe surface defect detection method and device, electronic equipment and storage medium
CN112767369A (en) * 2021-01-25 2021-05-07 佛山科学技术学院 Defect identification and detection method and device for small hardware and computer readable storage medium
CN112903703A (en) * 2021-01-27 2021-06-04 广东职业技术学院 Ceramic surface defect detection method and system based on image processing
CN112700442A (en) * 2021-02-01 2021-04-23 浙江驿公里智能科技有限公司 Die-cutting machine workpiece defect detection method and system based on Faster R-CNN
CN112950547B (en) * 2021-02-03 2024-02-13 佛山科学技术学院 Machine vision detection method for lithium battery diaphragm defects based on deep learning
CN112819796A (en) * 2021-02-05 2021-05-18 杭州天宸建筑科技有限公司 Tobacco shred foreign matter identification method and equipment
CN112907543B (en) * 2021-02-24 2024-03-26 胡志雄 Product appearance defect detection method based on random defect model
CN113077431A (en) * 2021-03-30 2021-07-06 太原理工大学 Laser chip defect detection method, system, equipment and storage medium based on deep learning
CN113654470A (en) * 2021-07-29 2021-11-16 佛山市科莱机器人有限公司 Method, system, equipment and medium for detecting accumulated water on surface of bathtub
CN113554240B (en) * 2021-08-20 2022-06-10 西藏众陶联供应链服务有限公司 Data analysis and prediction method and system for architectural ceramic tile surface decoration glazing
CN113724238A (en) * 2021-09-08 2021-11-30 佛山科学技术学院 Ceramic tile color difference detection and classification method based on feature point neighborhood color analysis
CN113724240B (en) * 2021-09-09 2023-10-17 中国海洋大学 Refrigerator caster detection method, system and device based on visual identification
CN113850335B (en) * 2021-09-28 2022-06-21 哈尔滨工业大学 Data augmentation method for bathroom ceramic defect detection
CN113850791B (en) * 2021-09-28 2022-07-05 哈尔滨工业大学 Bathroom ceramic defect detection method based on two-stage MobileNet
CN114066848B (en) * 2021-11-16 2024-03-22 苏州极速光学科技有限公司 FPCA appearance defect visual detection system
CN114354633B (en) * 2022-01-14 2024-04-12 广东猛犸象智能机器人制造有限公司 Ceramic bathroom appearance quality detection system and detection method
CN114841915A (en) * 2022-03-14 2022-08-02 阿里巴巴（中国）有限公司 Tile flaw detection method and system based on artificial intelligence and storage medium
CN117218097B (en) * 2023-09-23 2024-04-12 宁波江北骏欣密封件有限公司 Method and device for detecting surface defects of shaft sleeve type silk screen gasket part

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109711474A (en) * 2018-12-24 2019-05-03 中山大学 A kind of aluminium material surface defects detection algorithm based on deep learning
CN109767427A (en) * 2018-12-25 2019-05-17 北京交通大学 The detection method of train rail fastener defect
CN110276754A (en) * 2019-06-21 2019-09-24 厦门大学 A kind of detection method of surface flaw, terminal device and storage medium
CN110310262A (en) * 2019-06-19 2019-10-08 上海理工大学 A kind of method, apparatus and system for detection wheel tyre defect

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109711474A (en) * 2018-12-24 2019-05-03 中山大学 A kind of aluminium material surface defects detection algorithm based on deep learning
CN109767427A (en) * 2018-12-25 2019-05-17 北京交通大学 The detection method of train rail fastener defect
CN110310262A (en) * 2019-06-19 2019-10-08 上海理工大学 A kind of method, apparatus and system for detection wheel tyre defect
CN110276754A (en) * 2019-06-21 2019-09-24 厦门大学 A kind of detection method of surface flaw, terminal device and storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于Faster R-CNN的实木板材缺陷检测识别系统;范佳楠;刘英;胡忠康;赵乾;沈鹭翔;周晓林;;林业工程学报(第03期);全文 *

Also Published As

Publication number Publication date
CN110826416A (en) 2020-02-21

Similar Documents

Publication Publication Date Title
CN110826416B (en) Bathroom ceramic surface defect detection method and device based on deep learning
US20210374940A1 (en) Product defect detection method, device and system
CN108562589B (en) Method for detecting surface defects of magnetic circuit material
CN110599544B (en) Workpiece positioning method and device based on machine vision
CN111292305A (en) Improved YOLO-V3 metal processing surface defect detection method
CN111915704A (en) Apple hierarchical identification method based on deep learning
CN111260626A (en) Workpiece wear detection method and system based on deep learning
CN110942013A (en) Satellite image feature extraction method and system based on deep neural network
CN108647706B (en) Article identification classification and flaw detection method based on machine vision
JP7316731B2 (en) Systems and methods for detecting and classifying patterns in images in vision systems
CN116188475B (en) Intelligent control method, system and medium for automatic optical detection of appearance defects
CN112767369A (en) Defect identification and detection method and device for small hardware and computer readable storage medium
CN109584215A (en) A kind of online vision detection system of circuit board
CN114255223A (en) Deep learning-based method and equipment for detecting surface defects of two-stage bathroom ceramics
CN114723677A (en) Image defect detection method, image defect detection device, image defect detection equipment and storage medium
CN110942063B (en) Certificate text information acquisition method and device and electronic equipment
CN113609984A (en) Pointer instrument reading identification method and device and electronic equipment
CN115984662A (en) Multi-mode data pre-training and recognition method, device, equipment and medium
CN111178405A (en) Similar object identification method fusing multiple neural networks
CN111161213A (en) Industrial product defect image classification method based on knowledge graph
Zhou et al. An adaptive clustering method detecting the surface defects on linear guide rails
CN115984593B (en) Method, device, equipment and storage medium for detecting defects of cigarette filter sticks
CN112669300A (en) Defect detection method and device, computer equipment and storage medium
CN108985294B (en) Method, device and equipment for positioning tire mold picture and storage medium
Manuaba et al. The object detection system of balinese script on traditional Balinese manuscript with findcontours method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant