CN112766110A - Training method of object defect recognition model, object defect recognition method and device - Google Patents
Training method of object defect recognition model, object defect recognition method and device Download PDFInfo
- Publication number
- CN112766110A CN112766110A CN202110023478.6A CN202110023478A CN112766110A CN 112766110 A CN112766110 A CN 112766110A CN 202110023478 A CN202110023478 A CN 202110023478A CN 112766110 A CN112766110 A CN 112766110A
- Authority
- CN
- China
- Prior art keywords
- defect
- image
- model
- training
- identification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 600
- 238000012549 training Methods 0.000 title claims abstract description 139
- 238000000034 method Methods 0.000 title claims abstract description 83
- 230000002950 deficient Effects 0.000 claims abstract description 39
- 230000011218 segmentation Effects 0.000 claims abstract description 26
- 239000013598 vector Substances 0.000 claims description 28
- 238000012360 testing method Methods 0.000 claims description 26
- 230000008569 process Effects 0.000 claims description 12
- 238000004519 manufacturing process Methods 0.000 claims description 11
- 238000003860 storage Methods 0.000 claims description 11
- 238000007781 pre-processing Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 7
- 230000004807 localization Effects 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 5
- 238000013145 classification model Methods 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 10
- 238000004590 computer program Methods 0.000 description 5
- 238000007689 inspection Methods 0.000 description 5
- 230000008901 benefit Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 2
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000005728 strengthening Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000002645 vision therapy Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/088—Non-supervised learning, e.g. competitive learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a training method of an object defect recognition model, and an object defect recognition method and device, wherein the training method of the object defect recognition model comprises the following steps: acquiring a training sample image of an object; determining a defect area in the defective training image through a defect positioning segmentation sub-model in the object defect identification model, and segmenting the defect area from the defective training image to obtain a defect area image; identifying the defect type of the image of the defect area through a defect classification submodel in the object defect identification model to obtain a defect type identification result and a defect type identification loss value; and updating parameters of the object defect identification model according to the defect type identification loss value. The training method can avoid the interference of other irrelevant information in the image on the defect type identification, so that the identification efficiency and the accuracy of the model are high, the generalization capability of the trained model on different types of defects is good, and the cost of object defect identification is reduced.
Description
Technical Field
The application relates to the technical field of machine vision, in particular to a training method of an object defect identification model, and an object defect identification method and device.
Background
With the upgrading and transformation of industrial automation and intellectualization in China, at present, in the field of quality inspection, many enterprises still judge whether workpieces or products are qualified or not through experienced quality inspection personnel, however, the mode needs to depend on the subjective judgment of the quality inspection personnel, and the problems of low efficiency, high cost, easy omission, false inspection and the like exist.
In recent years, artificial intelligence is rapidly developed and developed, so that machine vision technology is more applied to the field of quality inspection, however, the generalization capability of the existing object defect identification model obtained based on machine vision training to different types of defects is poor, an algorithm for extracting defect features needs to be redeveloped aiming at each type of defects, the difficulty is high, and the cost is high.
Disclosure of Invention
In view of the above, the present application is proposed to provide a training method of an object defect recognition model, an object defect recognition method and apparatus that overcome the above problems or at least partially solve the above problems.
According to a first aspect of the present application, there is provided a training method of an object defect recognition model, the method comprising:
acquiring training sample images of an object, wherein the training sample images include defective training images and non-defective training images;
determining a defect region in the defect training image through a defect positioning segmentation sub-model in the object defect identification model, and segmenting the defect region from the defect training image to obtain a defect region image;
identifying the defect type of the defect area image through a defect classification submodel in the object defect identification model to obtain a defect type identification result and a defect type identification loss value, wherein the defect type identification result comprises a known defect image or an unknown defect image;
and updating the parameters of the object defect identification model according to the defect type identification loss value to obtain the trained object defect identification model.
Optionally, the defect localization and segmentation sub-model is trained by:
processing the defective training image through an encoder in the defect positioning and segmenting submodel to obtain a feature vector of the defective training image;
processing the feature vector of the defective training image through a decoder of the defect positioning and segmenting sub-model to obtain a reconstructed image;
and updating the parameters of the defect positioning segmentation sub-model according to the loss value between the defect training image and the reconstructed image to obtain the trained defect positioning segmentation sub-model.
Optionally, the method further comprises:
and preprocessing the training sample image to obtain a preprocessed image, determining a defect area in the preprocessed image through a defect positioning and segmenting sub-model in the object defect identification model, and segmenting the defect area from the preprocessed image to obtain the defect area image.
Optionally, the defect type identification result is obtained by:
calculating Euclidean distances between the defect characteristic vectors of the defect area images and the characteristic vectors of the known defect images;
determining the confidence degree of the defect region image belonging to each known defect image according to the Euclidean distance;
if the confidence coefficient exceeds a corresponding preset confidence coefficient threshold value, determining that the defect area image belongs to a corresponding known defect image;
if the confidence degrees do not exceed the corresponding preset confidence degree threshold, determining that the defective training image belongs to the unknown defective image;
and the preset confidence threshold is determined according to a service scene.
Optionally, the method further comprises:
acquiring a test sample image;
testing the trained object defect recognition model by using the test sample image to obtain a test result;
and if the test result passes, sending the trained object defect identification model to a production line system.
According to a second aspect of the present application, there is provided an object defect identification method, the method comprising:
acquiring an image to be detected of a target object;
extracting defect characteristics of the image to be detected through a defect positioning sub-model in the object defect identification model to obtain a defect characteristic vector, and determining a defect area in the image to be detected according to the defect characteristic vector;
dividing the defect region from the image to be detected through a defect division submodel in the object defect identification model to obtain a defect region image;
and identifying the defect type of the defect area image through a defect classification submodel in the object defect identification model to obtain an object defect type identification result, wherein the object defect type identification result comprises a known defect image or an unknown defect image, and the object defect identification model is obtained by training based on the training method of the object defect identification model in any item.
Optionally, the object defect identification model includes a plurality of models, and the method further includes:
receiving an object defect identification task;
determining the attribute type of the target object through an object attribute classification model according to the object defect identification task;
and calling a corresponding object defect identification model according to the attribute type of the target object, and carrying out defect identification on the target object through the object defect identification model to obtain an object defect identification result.
Optionally, the attribute type of the target object includes at least one of the following dimensions: a production line of the target object, a product type of the target object, a process section of the target object, and a panel type of the target object.
Optionally, the defect classification submodel includes a plurality of models, and identifying the defect type of the defect region image by the defect classification submodel in the object defect identification model to obtain the object defect identification result includes:
determining the size of the defect area;
and calling a corresponding defect classification submodel according to the size of the defect area so as to identify the defect type of the image of the defect area according to the corresponding defect classification submodel.
According to a third aspect of the present application, there is provided an object defect recognition model training apparatus, the apparatus including:
a first acquisition unit for acquiring a training sample image of an object, wherein the training sample image includes a defective training image and a non-defective training image;
the defect positioning and dividing unit is used for determining a defect area in the defect training image through a defect positioning and dividing submodel in the object defect identification model and dividing the defect area from the defect training image to obtain a defect area image;
the first defect classification unit is used for identifying the defect type of the defect area image through a defect classification submodel in the object defect identification model to obtain a defect type identification result and a defect type identification loss value;
and the updating unit is used for updating the parameters of the object defect identification model according to the defect type identification loss value to obtain the trained object defect identification model.
According to a fourth aspect of the present application, there is provided an object defect identifying apparatus, the apparatus comprising:
the second acquisition unit is used for acquiring an image to be detected of the target object;
the defect feature extraction unit is used for extracting defect features of the image to be detected through a defect positioning sub-model in the object defect identification model to obtain a defect feature vector, and determining a defect area in the image to be detected according to the defect feature vector;
the defect segmentation unit is used for segmenting the defect region from the image to be detected through a defect segmentation sub-model in the object defect identification model to obtain a defect region image;
and the second defect classification unit is used for identifying the defect type of the defect area image through a defect classification submodel in the object defect identification model to obtain an object defect type identification result, wherein the object defect type identification result comprises a known defect image or an unknown defect image, and the object defect identification model is obtained based on the training of the object defect identification model training device.
According to a fifth aspect of the present application, there is provided an electronic device comprising: a processor; and a memory arranged to store computer executable instructions that, when executed, cause the processor to perform a training method implementing an object defect recognition model as defined in any preceding item, or an object defect recognition method as defined in any preceding item.
According to a sixth aspect of the present application, there is provided a computer readable storage medium, wherein the computer readable storage medium stores one or more programs which, when executed by a processor, implement the training method of an object defect recognition model according to any preceding item, or the object defect recognition method according to any preceding item.
As can be seen from the above, the technical solution of the present application achieves at least the following technical effects: according to the training method of the object defect identification model, training sample images of an object are obtained firstly, and the training sample images specifically comprise a defect training image and a defect-free training image; determining a defect region in the defect training image through a defect positioning segmentation sub-model in the object defect identification model, and segmenting the defect region from the defect training image to obtain a defect region image; then, identifying the defect type of the image of the defect area through a defect classification submodel in the object defect identification model to obtain a defect type identification result and a defect type identification loss value, wherein the defect type identification result specifically comprises a known defect image or an unknown defect image; and finally, updating parameters of the object defect identification model according to the defect type identification loss value to obtain the trained object defect identification model. According to the training method of the object defect identification model, the defect regions in the image are segmented to identify the defect types, so that the interference of other irrelevant information in the image on the defect type identification is avoided, the identification efficiency and the accuracy of the model are high, the generalization capability of the object defect identification model obtained through training in the embodiment of the application to the defects of different types is good, an algorithm for extracting defect characteristics does not need to be re-developed for each type of defect, and the cost of object defect identification is greatly reduced.
The foregoing description is only an overview of the technical solutions of the present application, and the present application can be implemented according to the content of the description in order to make the technical means of the present application more clearly understood, and the following detailed description of the present application is given in order to make the above and other objects, features, and advantages of the present application more clearly understandable.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the application. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 illustrates a network architecture diagram of an object defect identification model according to one embodiment of the present application;
FIG. 2 illustrates a flow diagram of a method for training an object defect recognition model according to one embodiment of the present application;
FIG. 3 illustrates a schematic flow chart of training a defect localization segmentation sub-model according to an embodiment of the present application;
FIG. 4 shows a schematic flow diagram of an object defect identification method according to an embodiment of the present application;
FIG. 5 illustrates a schematic flow chart of object defect identification according to an embodiment of the present application;
FIG. 6 is a schematic structural diagram of a training apparatus for an object defect recognition model according to an embodiment of the present application;
FIG. 7 shows a schematic structural diagram of an object defect identification apparatus according to an embodiment of the present application;
FIG. 8 shows a schematic structural diagram of an electronic device according to an embodiment of the present application;
FIG. 9 shows a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present application.
Detailed Description
Exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
As shown in fig. 1, a schematic diagram of a network structure of an object defect identification model according to an embodiment of the present application is provided, and the object defect identification model according to the embodiment of the present application mainly includes two network structures, a defect location segmentation sub-model and a defect classification sub-model. The defect positioning and segmenting sub-model can be subdivided into a defect positioning sub-model and a defect segmenting sub-model and is used for positioning and segmenting defects in the image. The defect classification submodel is mainly used for realizing the identification of specific defect categories.
Based on this, an embodiment of the present application provides a training method for an object defect recognition model, as shown in fig. 2, the training method for the object defect recognition model includes the following steps S210 to S240:
step S210, training sample images of the object are obtained, wherein the training sample images comprise defective training images and non-defective training images.
When training the object defect recognition model, a certain number of training sample images may be obtained, where the training sample images may include labeled defective training images and non-defective training images. If the object defect recognition model is applied to a scene of workpiece defect recognition, the workpiece can be scanned or shot in all directions by adopting professional imaging equipment such as a high-definition camera and the like, and then a training sample image of the workpiece is obtained.
Of course, how to acquire the training sample image of the object is flexible, and those skilled in the art can select the training sample image according to the actual situation, and the training sample image is not limited in this embodiment.
Step S220, determining a defect area in the defect training image through a defect positioning and segmenting sub-model in the object defect identification model, and segmenting the defect area from the defect training image to obtain a defect area image.
After the training sample image of the object is obtained, the defect area in the defective training image can be located through a defect location segmentation sub-model in the object defect identification model, specifically, the position coordinates of the defect area in the defective training image can be included, and then the defect area is segmented from the defective training image to obtain a defect area image, so that the defect type of the defect area image can be identified later.
Step S230, identifying the defect type of the defect area image through a defect classification submodel in the object defect identification model to obtain a defect type identification result and a defect type identification loss value, wherein the defect type identification result comprises a known defect image or an unknown defect image.
After the defect area image is obtained, the defect classification of the defect area image can be carried out through a defect classification submodel in the object defect identification model, and then a defect classification identification result and a defect classification identification loss value are obtained. The object defect type identification result may specifically include whether the defect region in the defective training image belongs to a known defect or an unknown defect, and if the defect region belongs to a known defect, which type of known defect is specified. The defect class identification loss value can be understood as the error between the defect class identified by the object defect identification model and the defect class actually labeled in the defect training image.
And S240, updating parameters of the object defect identification model according to the defect type identification loss value to obtain the trained object defect identification model.
After the defect type identification loss value is obtained, the defect type identification loss value can be used for reversely updating the parameters of the object defect identification model, and if the updated object defect identification model can meet the performance requirement under a specific application scene, the training process of the object defect identification model is finished.
Compared with the existing training method of the object defect recognition model, the embodiment of the application only classifies the defect region after segmenting the defect region, so that the excessive learning of other irrelevant features in the image can be avoided, the overall training efficiency and the recognition accuracy of the model are further influenced, namely the training method of the object defect recognition model of the embodiment of the application can improve the recognition efficiency and the recognition accuracy of the model by strengthening and pertinently learning the features of the defect part, the generalization capability of the finally trained object defect recognition model to the defects of different types is better, an algorithm for extracting the defect features does not need to be redeveloped aiming at each type of defects, and the cost of object defect recognition is greatly reduced.
In an embodiment of the present application, the defect localization and segmentation sub-model in the object defect identification model may be obtained by training alone, where an automatic encoder in the prior art may be used as a basic skeleton network of the defect localization and segmentation sub-model. An Automatic Encoder (AE) is a type of artificial neural network used in unsupervised learning, and its principle is to perform characterization learning on input information by using the input information as a learning target. If some potential spatial representation exists in the input information (i.e., there is a correlation between the input features), such representation can be learned by the auto-encoder and the output can then be reconstructed from such representation. An auto-encoder typically comprises two parts, an encoder (encoder) for converting an input into an internal representation and a decoder (decoder) for converting the internal representation into an output.
As shown in fig. 3, when training the defect localization and segmentation sub-model, the defective training image is input into an encoder as input information for encoding, so as to obtain a feature vector of the defective training image as a potential spatial representation; then, learning and reconstructing the characteristic vector of the defective training image through a decoder to obtain a reconstructed image; and finally, calculating reconstruction loss according to the reconstructed image output by the decoder and the defective training image input into the encoder, and updating parameters of the encoder and the decoder according to the reconstruction loss so as to obtain a trained defect positioning and segmenting sub-model.
The purpose of the training process is to enable the automatic encoder to learn the potential spatial representation of the defect region in the defective training image, if the reconstruction loss obtained finally is smaller, the reconstructed image output by the decoder is more similar to the defective training image input into the encoder, namely, the automatic encoder has well learned the spatial representation of the defect feature in the defective training image, so that the trained defect positioning segmentation sub-model can realize pixel-level defect positioning and segmentation, the accuracy is higher, and a foundation is provided for accurate identification of the subsequent defect category. In addition, the defect positioning and dividing sub-model can automatically position and divide defects in a self-learning mode, and when the trained defect positioning and dividing sub-model is applied to an object defect recognition scene, the positions of the defects do not need to be manually marked in advance, so that the manual marking cost is greatly saved, the training difficulty of the model is simplified, and the model and the training efficiency are improved.
In an embodiment of the application, before the defect positioning and segmenting sub-model is used for positioning and segmenting the defect region in the defect training image, a series of preprocessing can be performed on the training sample image to obtain a preprocessed image, and then the preprocessed image can be used as the basis of subsequent defect positioning, segmenting and classifying, so that the overall training efficiency and accuracy of the model are improved.
The image preprocessing method adopted here can be various, for example, if the requirement of the actual service scene on the image color is not high, the acquired color image can be subjected to gray processing and converted into a gray image, so that excessive learning of color information is avoided, and the training efficiency of the model is improved. In addition, the method can also carry out image enhancement processing on the test sample image to enhance useful information in the image, wherein the image enhancement processing can be a distortion process and aims to improve the visual effect of the image, purposefully emphasize the overall or local characteristics of the image aiming at the application occasion of a given image, change the original unclear image into clear or emphasize certain interesting characteristics, enlarge the difference between different object characteristics in the image, inhibit the uninteresting characteristics, improve the image quality and enrich the information content, enhance the image interpretation and recognition effect and meet the requirements of certain special analysis.
Of course, besides the two image preprocessing methods listed above, those skilled in the art may set other preprocessing methods according to actual needs, and the method is not limited in detail here.
In one embodiment of the present application, when determining the defect type identification result, the euclidean distance between the defect feature vector of the defect region image and the feature vector of each known defect image may be calculated, and then the confidence that the defect region image belongs to each known defect image may be determined according to the euclidean distance, and if the euclidean distance between the defect feature vector of the defect region image and the feature vector of the defect image of a certain target type is closer, the confidence of dividing the defect region image into the target types is higher, otherwise, the confidence is lower.
Different confidence degree threshold values can be preset according to the actual service scene, and if the confidence degree exceeds the corresponding preset confidence degree threshold value, the defect area image is determined to belong to the corresponding category of the known defect image; if the confidence degrees do not exceed the corresponding preset confidence degree threshold, determining that the defect training image belongs to an unknown defect image, namely, indicating that the defect region image does not belong to any one of the currently labeled known defect images, and using the defect region image as the basis of a subsequent incremental training model and an optimization model.
In an embodiment of the present application, in order to test whether the identification performance of the object defect identification model meets the requirements of an actual service scene, the object defect identification model may be tested. A certain number of test sample images can be obtained, and then the test sample images are input into the trained object defect recognition model for testing to obtain a test result. The test result specifically comprises model performance evaluation indexes such as identification accuracy and recall rate, if the test result meets the requirements of an actual service scene, the trained object defect identification model can be sent to a corresponding production line system, so that the production line system can deploy the model on line, and if the test result does not meet the requirements, the model can be further optimized according to the test result.
In an embodiment of the present application, the training process is suitable for training object defect recognition models in different service scenes, so that corresponding object defect recognition models can be trained according to different service scene requirements. For example, for a type a workpieces, a number of type a workpiece images may be acquired and trained to obtain an object defect recognition model X1, and for a type B workpieces, a number of type B workpiece images may be acquired and trained to obtain an object defect recognition model X2.
An embodiment of the present application provides an object defect identification method, as shown in fig. 4, the object defect identification method includes at least the following steps S410 to S440:
step S410, an image to be detected of the target object is acquired.
When object defect identification is performed, an image to be detected of a target object can be acquired first, where the target object refers to any object needing defect identification in an actual application scene, for example, for workpiece defect identification, the target object can be a workpiece, and the workpiece is scanned or photographed in all directions by professional imaging equipment such as a high-definition camera, so as to obtain an image to be detected of the workpiece.
And step S420, extracting defect characteristics of the image to be detected through a defect positioning sub-model in the object defect identification model to obtain a defect characteristic vector, and determining a defect area in the image to be detected according to the defect characteristic vector.
After the image to be detected of the target object is obtained, defect feature extraction is carried out on the image to be detected through a defect positioning sub-model in the object defect identification model to obtain a defect feature vector, and then the defect feature vector is mapped back to the original image to be detected, so that a defect area in the image to be detected can be obtained.
And step S430, segmenting the defect region from the image to be detected through a defect segmentation sub-model in the object defect identification model to obtain a defect region image.
In order to avoid the interference of other irrelevant features in the image on defect identification, the defect region can be segmented from the image to be detected through a defect segmentation sub-model in the object defect identification model, and then the defect region image is obtained and used as the basis of subsequent defect type identification.
Step S440, identifying the defect type of the defect area image through a defect classification submodel in the object defect identification model to obtain an object defect type identification result, wherein the object defect type identification result comprises a known defect image or an unknown defect image, and the object defect identification model is obtained by training based on the training method of the object defect identification model.
After the defect area image is obtained, the defect classification of the defect area image can be carried out through a defect classification sub-model in the object defect identification model, and then the object defect classification identification result is obtained. The object defect identification result may specifically include whether the defect in the defect region image belongs to a known defect or an unknown defect, and if the defect is a known defect, which type of known defect is specific.
The object defect identification method of the embodiment of the application locates, segments and classifies the defects in the image to be detected by using the object defect identification model trained in advance, and compared with the existing defect identification method, the identification efficiency and the identification accuracy are higher.
As described above, in an actual application scenario, a plurality of object defect recognition models are trained according to different business requirements, and therefore, as shown in fig. 5, in an embodiment of the present application, an object defect recognition task may be received first, the object defect recognition task may be issued by a specific task system, and the object defect recognition task may carry information of a target object to be recognized, so that according to the received object defect recognition task, an attribute type of the target object may be determined by using a previously trained object attribute classification model; and finally, automatically adapting a corresponding object defect identification model according to the attribute type of the target object so as to identify the defect of the target object.
In an actual application scene, due to the fact that the workpiece attributes such as a production line, a product type, a process section and a panel type are different, the corresponding workpiece forms are also different, and further the scales for classifying the workpiece defects are different, therefore, in the training stage, the corresponding object defect recognition models can be respectively trained from any one or more dimensions of the production line, the product type, the process section and the panel type. Based on this, in an embodiment of the present application, when the corresponding object defect identification model is called according to the attribute type of the target object, the object defect identification model to be called may be determined from dimensions such as a production line, a product type, a process segment, and a panel type to which the target object belongs.
In addition, the sizes of the defect regions located by the defect location submodels are different, so that the defect classification results are also influenced, and in order to further reduce the misjudgment probability caused by the large span of the classification results, a plurality of defect classification submodels can be trained according to the defect regions with different sizes in the training stage. Based on this, in an embodiment of the present application, when the defect classification sub-model in the object defect identification model identifies the defect type of the defect region image, the size of the defect region may be determined first, and then the corresponding defect classification sub-model is automatically adapted according to the size of the defect region, so as to identify the defect type of the defect region image according to the corresponding defect classification sub-model.
Of course, besides the dimensional information such as the production line, the product type, the process segment, the panel type, and the size of the defect area, those skilled in the art can also flexibly set the information of other dimensions according to the actual requirements and perform corresponding model training, which is not listed here.
An embodiment of the present application provides an object defect recognition model training apparatus 600, as shown in fig. 6, the apparatus 600 includes:
a first acquiring unit 610 for acquiring a training sample image of an object, wherein the training sample image includes a defective training image and a non-defective training image;
a defect positioning and segmenting unit 620, configured to determine a defect region in the defective training image through a defect positioning and segmenting sub-model in the object defect identification model, and segment the defect region from the defective training image to obtain a defect region image;
a first defect classification unit 630, configured to perform defect classification identification on the defect region image through a defect classification submodel in the object defect identification model, so as to obtain a defect classification identification result and a defect classification identification loss value;
and the updating unit 640 is configured to update parameters of the object defect identification model according to the defect type identification loss value, so as to obtain a trained object defect identification model.
In one embodiment of the present application, the apparatus further comprises: and the preprocessing unit is used for preprocessing the training sample image to obtain a preprocessed image, determining a defect area in the preprocessed image through a defect positioning and segmenting sub-model in the object defect identification model, and segmenting the defect area from the preprocessed image to obtain the defect area image.
In one embodiment of the present application, the apparatus further comprises: a third acquisition unit for acquiring a test sample image; the testing unit is used for testing the trained object defect identification model by using the test sample image to obtain a test result; and the sending unit is used for sending the trained object defect identification model to a production line system if the test result passes.
The embodiment of the present application provides an object defect identifying apparatus 700, as shown in fig. 7, where the apparatus 700 includes:
a second obtaining unit 710, configured to obtain an image to be detected of the target object;
a defect feature extraction unit 720, configured to perform defect feature extraction on the image to be detected through a defect positioning sub-model in the object defect identification model to obtain a defect feature vector, and determine a defect region in the image to be detected according to the defect feature vector;
the defect segmentation unit 730 is used for segmenting the defect region from the image to be detected through a defect segmentation sub-model in the object defect identification model to obtain a defect region image;
a second defect classification unit 740, configured to perform defect classification on the defect region image through a defect classification submodel in the object defect identification model to obtain an object defect classification identification result, where the object defect classification identification result includes a known defect image or an unknown defect image, and the object defect identification model is obtained based on the training of the object defect identification model training apparatus as described above.
In one embodiment of the present application, the object defect identification model includes a plurality of models, and the apparatus further includes: the receiving unit is used for receiving an object defect identification task; the object attribute classification unit is used for determining the attribute type of the target object through an object attribute classification model according to the object defect identification task; and the calling unit is used for calling a corresponding object defect identification model according to the attribute type of the target object, and performing defect identification on the target object through the object defect identification model to obtain an object defect identification result.
In an embodiment of the application, the defect classification submodel includes a plurality of models, and the second defect classification unit 740 is specifically configured to: determining the size of the defect area; and calling a corresponding defect classification submodel according to the size of the defect area so as to identify the defect type of the image of the defect area according to the corresponding defect classification submodel.
It should be noted that, for the specific implementation of each apparatus embodiment, reference may be made to the specific implementation of the corresponding method embodiment, which is not described herein again.
To sum up, this application has reached following technological effect at least: according to the training method of the object defect identification model, the defect regions in the image are segmented to identify the defect types, so that the interference of other irrelevant information in the image on the defect type identification is avoided, the identification efficiency and the accuracy of the model are high, the generalization capability of the object defect identification model obtained through training in the embodiment of the application to the defects of different types is good, an algorithm for extracting defect characteristics does not need to be re-developed aiming at each type of defect, and the cost of subsequent object defect identification is greatly reduced.
It should be noted that:
the algorithms and displays presented herein are not inherently related to any particular computer, virtual machine, or other apparatus. Various general purpose devices may be used with the teachings herein. The required structure for constructing such a device will be apparent from the description above. In addition, this application is not directed to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of the present application as described herein, and any descriptions of specific languages are provided above to disclose the best modes of the present application.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the application may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the application, various features of the application are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the application and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: this application is intended to cover such departures from the present disclosure as come within known or customary practice in the art to which this invention pertains. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this application.
Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the application and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.
The various component embodiments of the present application may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It will be appreciated by those skilled in the art that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components of the training apparatus or object defect recognition apparatus for object defect recognition models in accordance with embodiments of the present application. The present application may also be embodied as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present application may be stored on a computer readable medium or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.
For example, fig. 8 shows a schematic structural diagram of an electronic device according to an embodiment of the present application. The electronic device 800 comprises a processor 810 and a memory 820 arranged to store computer executable instructions (computer readable program code). The memory 820 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read only memory), an EPROM, a hard disk, or a ROM. The memory 820 has a storage space 830 storing computer readable program code 831 for performing any of the method steps described above. For example, the storage space 830 for storing the computer-readable program code may include respective computer-readable program codes 831 for respectively implementing various steps in the above methods. The computer readable program code 831 may be read from or written to one or more computer program products. These computer program products comprise a program code carrier such as a hard disk, a Compact Disc (CD), a memory card or a floppy disk. Such a computer program product is typically a computer readable storage medium such as that shown in fig. 9. FIG. 9 shows a schematic diagram of a computer-readable storage medium according to an embodiment of the present application. The computer readable storage medium 900 stores computer readable program code 831 for executing the steps of the method according to the present application, which is readable by a processor 810 of the electronic device 800, and when the computer readable program code 831 is executed by the electronic device 800, causes the electronic device 800 to perform the steps of the method described above, and in particular, the computer readable program code 831 stored by the computer readable storage medium may perform the method shown in any of the embodiments described above. The computer readable program code 831 may be compressed in a suitable form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The application may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.
Claims (13)
1. A method for training a defect recognition model of an object, the method comprising:
acquiring training sample images of an object, wherein the training sample images include defective training images and non-defective training images;
determining a defect region in the defect training image through a defect positioning segmentation sub-model in the object defect identification model, and segmenting the defect region from the defect training image to obtain a defect region image;
identifying the defect type of the defect area image through a defect classification submodel in the object defect identification model to obtain a defect type identification result and a defect type identification loss value, wherein the defect type identification result comprises a known defect image or an unknown defect image;
and updating the parameters of the object defect identification model according to the defect type identification loss value to obtain the trained object defect identification model.
2. The method of claim 1, wherein the defect localization segmentation submodel is trained by:
processing the defective training image through an encoder in the defect positioning and segmenting submodel to obtain a feature vector of the defective training image;
processing the feature vector of the defective training image through a decoder of the defect positioning and segmenting sub-model to obtain a reconstructed image;
and updating the parameters of the defect positioning segmentation sub-model according to the loss value between the defect training image and the reconstructed image to obtain the trained defect positioning segmentation sub-model.
3. The method of claim 1, further comprising:
and preprocessing the training sample image to obtain a preprocessed image, determining a defect area in the preprocessed image through a defect positioning and segmenting sub-model in the object defect identification model, and segmenting the defect area from the preprocessed image to obtain the defect area image.
4. The method according to claim 1, wherein the defect type identification result is obtained by:
calculating Euclidean distances between the defect characteristic vectors of the defect area images and the characteristic vectors of the known defect images;
determining the confidence degree of the defect region image belonging to each known defect image according to the Euclidean distance;
if the confidence coefficient exceeds a corresponding preset confidence coefficient threshold value, determining that the defect area image belongs to a corresponding known defect image;
if the confidence degrees do not exceed the corresponding preset confidence degree threshold, determining that the defective training image belongs to the unknown defective image;
and the preset confidence threshold is determined according to a service scene.
5. The method of claim 1, further comprising:
acquiring a test sample image;
testing the trained object defect recognition model by using the test sample image to obtain a test result;
and if the test result passes, sending the trained object defect identification model to a production line system.
6. A method for identifying defects in an object, the method comprising:
acquiring an image to be detected of a target object;
extracting defect characteristics of the image to be detected through a defect positioning sub-model in the object defect identification model to obtain a defect characteristic vector, and determining a defect area in the image to be detected according to the defect characteristic vector;
dividing the defect region from the image to be detected through a defect division submodel in the object defect identification model to obtain a defect region image;
identifying the defect type of the defect area image through a defect classification submodel in the object defect identification model to obtain an object defect type identification result, wherein the object defect type identification result comprises a known defect image or an unknown defect image, and the object defect identification model is obtained by training based on the training method of the object defect identification model according to any one of claims 1 to 5.
7. The method of claim 6, wherein the object defect identification model comprises a plurality of models, the method further comprising:
receiving an object defect identification task;
determining the attribute type of the target object through an object attribute classification model according to the object defect identification task;
and calling a corresponding object defect identification model according to the attribute type of the target object, and carrying out defect identification on the target object through the object defect identification model to obtain an object defect identification result.
8. The method of claim 7, wherein the type of attribute of the target object comprises at least one of the following dimensions: a production line of the target object, a product type of the target object, a process section of the target object, and a panel type of the target object.
9. The method of claim 8, wherein the defect classification submodel includes a plurality of models, and the identifying the defect type of the defect region image by the defect classification submodel in the object defect identification model to obtain the object defect identification result includes:
determining the size of the defect area;
and calling a corresponding defect classification submodel according to the size of the defect area so as to identify the defect type of the image of the defect area according to the corresponding defect classification submodel.
10. An object defect recognition model training device, the device comprising:
a first acquisition unit for acquiring a training sample image of an object, wherein the training sample image includes a defective training image and a non-defective training image;
the defect positioning and dividing unit is used for determining a defect area in the defect training image through a defect positioning and dividing submodel in the object defect identification model and dividing the defect area from the defect training image to obtain a defect area image;
the first defect classification unit is used for identifying the defect type of the defect area image through a defect classification submodel in the object defect identification model to obtain a defect type identification result and a defect type identification loss value;
and the updating unit is used for updating the parameters of the object defect identification model according to the defect type identification loss value to obtain the trained object defect identification model.
11. An object defect identification apparatus, the apparatus comprising:
the second acquisition unit is used for acquiring an image to be detected of the target object;
the defect feature extraction unit is used for extracting defect features of the image to be detected through a defect positioning sub-model in the object defect identification model to obtain a defect feature vector, and determining a defect area in the image to be detected according to the defect feature vector;
the defect segmentation unit is used for segmenting the defect region from the image to be detected through a defect segmentation sub-model in the object defect identification model to obtain a defect region image;
a second defect classification unit, configured to perform defect classification on the defect region image through a defect classification submodel in the object defect identification model to obtain an object defect classification identification result, where the object defect classification identification result includes a known defect image or an unknown defect image, and the object defect identification model is obtained by training based on the object defect identification model training apparatus according to claim 10.
12. An electronic device, wherein the electronic device comprises: a processor; and a memory arranged to store computer executable instructions that, when executed, cause the processor to perform a method of training an object defect recognition model according to any one of claims 1 to 5, or a method of object defect recognition according to any one of claims 6 to 9.
13. A computer readable storage medium, wherein the computer readable storage medium stores one or more programs which, when executed by a processor, implement a training method of an object defect recognition model according to any one of claims 1 to 5, or an object defect recognition method according to any one of claims 6 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110023478.6A CN112766110A (en) | 2021-01-08 | 2021-01-08 | Training method of object defect recognition model, object defect recognition method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110023478.6A CN112766110A (en) | 2021-01-08 | 2021-01-08 | Training method of object defect recognition model, object defect recognition method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112766110A true CN112766110A (en) | 2021-05-07 |
Family
ID=75701008
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110023478.6A Pending CN112766110A (en) | 2021-01-08 | 2021-01-08 | Training method of object defect recognition model, object defect recognition method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112766110A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113344886A (en) * | 2021-06-11 | 2021-09-03 | 长江存储科技有限责任公司 | Wafer surface defect detection method and equipment |
| CN113536971A (en) * | 2021-06-28 | 2021-10-22 | 中科苏州智能计算技术研究院 | Target detection method based on incremental learning |
| CN113793343A (en) * | 2021-08-23 | 2021-12-14 | 南昌航空大学 | Defect positioning and segmentation method, system, terminal and storage medium based on image |
| WO2022236876A1 (en) * | 2021-05-14 | 2022-11-17 | 广州广电运通金融电子股份有限公司 | Cellophane defect recognition method, system and apparatus, and storage medium |
| CN116245846A (en) * | 2023-03-08 | 2023-06-09 | 华院计算技术(上海)股份有限公司 | Defect detection method and device for strip steel, storage medium and computing equipment |
| WO2024020994A1 (en) * | 2022-07-29 | 2024-02-01 | 宁德时代新能源科技股份有限公司 | Training method and training device for defect detection model of battery cell |
| TWI833326B (en) * | 2021-09-15 | 2024-02-21 | 日商日立全球先端科技股份有限公司 | Defect inspection system and defect inspection method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109583489A (en) * | 2018-11-22 | 2019-04-05 | 中国科学院自动化研究所 | Defect classifying identification method, device, computer equipment and storage medium |
| CN111627015A (en) * | 2020-05-29 | 2020-09-04 | 联想(北京)有限公司 | Small sample defect identification method, device, equipment and storage medium |
| CN111814867A (en) * | 2020-07-03 | 2020-10-23 | 浙江大华技术股份有限公司 | Defect detection model training method, defect detection method and related device |
-
2021
- 2021-01-08 CN CN202110023478.6A patent/CN112766110A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109583489A (en) * | 2018-11-22 | 2019-04-05 | 中国科学院自动化研究所 | Defect classifying identification method, device, computer equipment and storage medium |
| CN111627015A (en) * | 2020-05-29 | 2020-09-04 | 联想(北京)有限公司 | Small sample defect identification method, device, equipment and storage medium |
| CN111814867A (en) * | 2020-07-03 | 2020-10-23 | 浙江大华技术股份有限公司 | Defect detection model training method, defect detection method and related device |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022236876A1 (en) * | 2021-05-14 | 2022-11-17 | 广州广电运通金融电子股份有限公司 | Cellophane defect recognition method, system and apparatus, and storage medium |
| CN113344886A (en) * | 2021-06-11 | 2021-09-03 | 长江存储科技有限责任公司 | Wafer surface defect detection method and equipment |
| CN113536971A (en) * | 2021-06-28 | 2021-10-22 | 中科苏州智能计算技术研究院 | Target detection method based on incremental learning |
| CN113793343A (en) * | 2021-08-23 | 2021-12-14 | 南昌航空大学 | Defect positioning and segmentation method, system, terminal and storage medium based on image |
| CN113793343B (en) * | 2021-08-23 | 2023-06-06 | 南昌航空大学 | Image-based defect positioning and segmentation method, system, terminal and storage medium |
| TWI833326B (en) * | 2021-09-15 | 2024-02-21 | 日商日立全球先端科技股份有限公司 | Defect inspection system and defect inspection method |
| WO2024020994A1 (en) * | 2022-07-29 | 2024-02-01 | 宁德时代新能源科技股份有限公司 | Training method and training device for defect detection model of battery cell |
| CN116245846A (en) * | 2023-03-08 | 2023-06-09 | 华院计算技术(上海)股份有限公司 | Defect detection method and device for strip steel, storage medium and computing equipment |
| CN116245846B (en) * | 2023-03-08 | 2023-11-21 | 华院计算技术(上海)股份有限公司 | Defect detection method and device for strip steel, storage medium and computing equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112766110A (en) | Training method of object defect recognition model, object defect recognition method and device | |
| CN111325713B (en) | Neural network-based wood defect detection method, system and storage medium | |
| CN112348787A (en) | Training method of object defect detection model, object defect detection method and device | |
| CN110648310B (en) | Weak supervision casting defect identification method based on attention mechanism | |
| CN110264444B (en) | Damage detection method and device based on weak segmentation | |
| CN110533086B (en) | Semi-automatic image data labeling method | |
| CN114549997B (en) | X-ray image defect detection method and device based on regional feature extraction | |
| CN112819748B (en) | Training method and device for strip steel surface defect recognition model | |
| CN113591948A (en) | Defect pattern recognition method and device, electronic equipment and storage medium | |
| CN115830004A (en) | Surface defect detection method, device, computer equipment and storage medium | |
| CN114757925A (en) | Non-contact type high-voltage circuit breaker defect detection method and system | |
| CN116012291A (en) | Industrial part image defect detection method and system, electronic equipment and storage medium | |
| CN111814852A (en) | Image detection method, image detection device, electronic equipment and computer-readable storage medium | |
| CN115578326A (en) | Road disease identification method, system, equipment and storage medium | |
| CN110728269A (en) | High-speed rail contact net support pole number plate identification method | |
| CN113128522B (en) | Target identification method, device, computer equipment and storage medium | |
| CN110618129A (en) | Automatic power grid wire clamp detection and defect identification method and device | |
| CN114299040A (en) | Ceramic tile flaw detection method and device and electronic equipment | |
| CN112836724A (en) | Object defect recognition model training method and device, electronic equipment and storage medium | |
| CN115797314B (en) | Method, system, equipment and storage medium for detecting surface defects of parts | |
| CN116843611A (en) | Hardware surface flaw detection system based on difference and graph neural network | |
| CN113592859B (en) | Deep learning-based classification method for defects of display panel | |
| CN110751061A (en) | SAR image recognition method, device, equipment and storage medium based on SAR network | |
| US20230084761A1 (en) | Automated identification of training data candidates for perception systems | |
| CN114782822A (en) | Method and device for detecting state of power equipment, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210507 |