WO2020143592A1 - 缺陷识别模型训练方法、装置、计算机设备和存储介质 - Google Patents
缺陷识别模型训练方法、装置、计算机设备和存储介质 Download PDFInfo
- Publication number
- WO2020143592A1 WO2020143592A1 PCT/CN2020/070578 CN2020070578W WO2020143592A1 WO 2020143592 A1 WO2020143592 A1 WO 2020143592A1 CN 2020070578 W CN2020070578 W CN 2020070578W WO 2020143592 A1 WO2020143592 A1 WO 2020143592A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- training
- data
- defect
- image data
- sample
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
Definitions
- the present application relates to the field of computer technology, and in particular, to a method, device, computer equipment, and storage medium for defect recognition model training.
- the established defect recognition model learns images of products containing defects, and detects product defects through the images of products.
- the product can be of various types; the material of the product can include various types of materials, such as metal, plastic, and glass; the types of defects can include various types, such as lack, excess, bubbles, scratches, and irregular shapes, etc. .
- a defect recognition model training method includes:
- the initial model is trained according to the training sample set to obtain a defect recognition model.
- a defect recognition model training device includes:
- Data acquisition module for acquiring sample image training data
- An auxiliary acquisition module for acquiring auxiliary sample image data corresponding to the sample image training data
- a sample set construction module configured to construct a training sample set from the sample image training data and the auxiliary sample image data
- the model obtaining module is used to train the initial model according to the training sample set to obtain a defect recognition model.
- a computer device includes a memory, a processor, and a computer program stored on the memory and executable on the processor.
- the processor implements the computer program to implement the following steps:
- the initial model is trained according to the training sample set to obtain a defect recognition model.
- the initial model is trained according to the training sample set to obtain a defect recognition model.
- the above method, apparatus, computer equipment and storage medium for defect recognition model training obtain sample image training data and auxiliary sample image data corresponding to the sample image training data, without extracting feature data from the sample image training data, and using the sample image training data and Auxiliary sample image data, construct a training sample set, and then train the initial model according to the training sample set to obtain a defect recognition model.
- auxiliary sample image data corresponding to sample image training data the time taken to extract feature data during model training is reduced, and the training efficiency of the model is improved.
- FIG. 1 is an application environment diagram of a defect recognition model training method in an embodiment
- FIG. 2 is a schematic flowchart of a method for training a defect recognition model in an embodiment
- FIG. 3 is a schematic flowchart of steps of acquiring auxiliary sample image data in an embodiment
- FIG. 4 is a schematic flowchart of steps of constructing a training sample set in an embodiment
- FIG. 5 is a schematic flowchart of steps for obtaining a defect identification model in an embodiment
- FIG. 6 is a schematic flowchart of steps for obtaining a defect prediction model in an embodiment
- FIG. 7 is a schematic diagram of sample images in sample image training data in an embodiment
- FIG. 8 is a schematic diagram of preprocessed images in preprocessed image data in an embodiment
- FIG. 9 is a schematic diagram of a non-defective product image in the non-defective product image data in an embodiment
- FIG. 10 is a schematic diagram of defect labeling data in an embodiment
- FIG. 11 is a schematic flowchart of steps of using a defect identification model in an embodiment
- FIG. 12 is a schematic diagram of defect identification results in an embodiment
- FIG. 13 is a schematic diagram of a defect recognition model training in an embodiment
- FIG. 14 is a structural block diagram of a defect recognition model training device in an embodiment
- 15 is an internal structure diagram of a computer device in an embodiment.
- the defect recognition model training method provided in this application can be applied to the application environment shown in FIG. 1, and the application environment can include a terminal 102 and a server 104, and the terminal 102 communicates with the server 104 through a network. This method can be applied to both the terminal 102 and the server 104.
- the terminal 102 may be, but not limited to, various industrial computers, personal computers, notebook computers, smart phones, and tablet computers.
- the server 104 may be implemented by an independent server or a server cluster composed of multiple servers.
- a method for training a defect recognition model is provided.
- the method is applied to the terminal in FIG. 1 as an example for illustration, and includes the following steps:
- Step 202 Obtain sample image training data.
- the sample image training data is the sample data obtained by performing image acquisition on the product with quality defects to obtain the sample image, and adding the defect annotation data to the sample image, which is used to train the initial model.
- the terminal obtains the model training instruction triggered by the user, and parses the model training instruction to obtain the storage address of the sample image training data.
- the terminal accesses the storage space corresponding to the storage address, and extracts the stored sample image training data from the accessed storage space .
- the terminal reads the pre-configured model training script according to the model training instruction, extracts the storage address of the sample image data from the model training script, and reads the sample image training data from the storage space according to the extracted storage address.
- Step 204 Acquire auxiliary sample image data corresponding to the sample image training data.
- the auxiliary sample image data is image data related to the sample image training data, which is used to assist the initial model in training.
- the terminal parses the acquired model training instruction to obtain the auxiliary sample image data type corresponding to the sample image data and the storage address of the auxiliary sample image data.
- the terminal extracts auxiliary sample image data from the storage space according to the storage address corresponding to the auxiliary sample image data.
- the terminal generates a data acquisition request according to the model training instruction, and sends the data acquisition request to the server through the network.
- the server receives the data acquisition request, extracts the sample image training data and auxiliary sample image data from the database according to the data acquisition request, and sends the extracted sample image training data and auxiliary sample image data to the terminal through the network.
- Step 206 Construct a training sample set with sample image training data and auxiliary sample image data.
- the training sample set is a combination of sample image training data and auxiliary sample image data.
- the terminal after acquiring the sample image training data and the auxiliary sample image data, the terminal triggers the training sample set construction instruction, and combines the sample image training data and the auxiliary sample image data according to the training sample set construction instruction to obtain the training sample set.
- Step 208 Train the initial model according to the training sample set to obtain a defect recognition model.
- the initial model is a pre-established model without parameter adjustment.
- the defect recognition model is the model obtained after the initial model is trained.
- the training sample set is input into the initial model.
- the initial model is trained according to the input training sample set, and the model parameters are adjusted so that the defect prediction result output by the initial model continuously approaches the defect annotation data.
- the terminal gets the defect recognition model.
- the type of the initial model is not limited here, and may be at least one of a neural network model, a point cloud classification model, a support vector machine, and a logistic regression model.
- the number of types of auxiliary sample image data is not limited. For example, the terminal may obtain only one kind of auxiliary sample image data, construct a set of training sample sets, and train an initial model according to the training sample sets to obtain a defect recognition model.
- the sample image training data and the auxiliary sample image data corresponding to the sample image training data by acquiring the sample image training data and the auxiliary sample image data corresponding to the sample image training data, there is no need to extract additional feature data from the sample image training data, and the sample image training data and the auxiliary sample image data are used to construct the training sample set , Train the initial model according to the training sample set to obtain the defect recognition model.
- the time taken to extract feature data during model training is reduced, and the training efficiency of the model is improved.
- step 204 specifically includes the step of acquiring auxiliary sample image data.
- the step specifically includes the following steps:
- Step 302 Determine the auxiliary data type corresponding to the sample image training data.
- the terminal extracts the sample image training data identifier, queries the sample image training data identifier from the pre-stored model training script, and determines the auxiliary data type corresponding to the sample image training data identifier.
- the model training script is a format file used for training the model.
- the model training script includes data for controlling the terminal to perform a series of operations.
- the auxiliary data type may include at least one of preprocessed image data, non-defective product image data, and illumination transformation image data.
- Step 304 Acquire sample data addresses corresponding to each auxiliary data type.
- the sample data address is the storage address of each auxiliary data type in the storage space.
- the terminal after determining the auxiliary data type according to the model training script, the terminal extracts the sample data addresses corresponding to the determined auxiliary data types in the model training script, respectively.
- Step 306 Extract preprocessed image data, non-defective product image data, and illumination transformation image data according to the sample data addresses.
- the preprocessed image data is the preprocessed image obtained by performing visual processing on the sample image, and the image data obtained by adding the annotation data.
- the processing of the visual method may include at least one of edge extraction, connected domain, and RGB to HSB.
- RGB represents a color mode, and various colors are obtained by changing the three color channels of red (R), green (G), and blue (B) and superimposing each other;
- HSB indicates a color mode , H (hues) indicates hue, S (saturation) indicates saturation, B (brightness) indicates brightness, and the medium corresponding to the HSB mode is the human eye.
- the non-defective product image data is the image data obtained by adding the annotation data to the non-defective product image obtained by image acquisition of the intact product.
- Illumination-transformed image data is the image data obtained by adding the annotation data to the illumination-transformed image obtained by image acquisition under the illumination of a specific color for products with quality defects.
- different wavelengths of light can be used for illumination to collect images of products under specific illumination.
- the product defect type can be determined manually, and light of different wavelengths can be used to illuminate according to the product defect type. For example, when there are small scratches on the surface of the product, you can use short-wavelength light for irradiation, such as blue light.
- the terminal After the terminal obtains the sample data addresses corresponding to the pre-processed image data, the non-defective product image data, and the illumination transformation image data from the model training script, the terminal respectively accesses the storage space corresponding to each sample data address, and from the accessed storage space , Extract the pre-processed image data, the image data of the incomplete product and the illumination transformation image data separately.
- the sample image in the preparation stage of the auxiliary sample image data, may be copied first, and the copied sample image may be processed by a traditional visual method to obtain a preprocessed image; or a part of the sample image may be selected for The processing of traditional visual methods results in preprocessed images.
- Step 308 the pre-processed image data, the image data of the incomplete product and the image data of the illumination transformation are used as various auxiliary sample image data,
- the terminal marks the preprocessed image data, the non-defective product image data, and the illumination transformed image data as auxiliary sample image data, respectively.
- the sample data addresses corresponding to each auxiliary data type are obtained, and the pre-processed image data, the non-defective product image data, and the illumination transformation image data are extracted according to the sample data addresses, Then, the extracted pre-processed image data, non-defective product image data and illumination transformation image data are used as auxiliary sample image data, which improves the accuracy of obtaining auxiliary sample image data.
- step 206 specifically includes the step of constructing a training sample set.
- the step specifically includes the following steps:
- Step 402 Use the sample image training data as the first training sample set.
- the terminal after acquiring various auxiliary sample image data, the terminal triggers the training sample set construction instruction, and constructs the training sample set according to the training sample set construction instruction. First, the terminal constructs the training sample set solely from the sample image training data to obtain the first group of training sample sets.
- Step 404 Combine the sample image training data with the pre-processed image data, non-defective product image data, and illumination transformation image data in various auxiliary sample image data to obtain the second and third training sample sets And the fourth training sample set.
- the terminal separately combines the sample image training data with various auxiliary sample image data.
- the terminal obtains the second set of training sample sets, the third set of training sample sets, and the fourth set of training sample sets, respectively.
- the second set of training sample sets can be a combination of sample image data and preprocessed image data
- the third set of training sample sets can be a combination of sample image data and incomplete product image data
- the fourth set of training sample sets can be sample images The combination of data and lighting transform image data. It can be understood that the terms "first”, “second”, “third”, and “fourth” used in the present invention are only used to distinguish the training sample sets of each group, but the training sample sets of each group are not Limited by these terms.
- the sample image training data is used as the first training sample set, and the sample image training data is organically combined with the pre-processed image data, the non-defective product image data, and the illumination transformation image data in the auxiliary sample image data.
- the second set of training sample sets, the third set of training sample sets, and the fourth set of training sample sets are obtained, which improves the efficiency of obtaining the training sample set.
- step 208 specifically includes the step of obtaining a defect identification model.
- the step specifically includes the following steps:
- Step 502 Train each initial model separately according to each training sample set to obtain multiple defect prediction models.
- the defect prediction model is a model obtained by training the initial model.
- the terminal inputs each set of training sample sets into the corresponding initial model, and the initial model is trained according to the input training sample set.
- the terminal can divide the training sample set into multiple parts.
- Step 504 Construct a defect recognition model according to multiple defect prediction models.
- the terminal uses multiple defect prediction models as sub-models to form a defect prediction model cluster, and uses the obtained defect prediction model cluster as a parent model to obtain a defect recognition model.
- each initial model is trained according to each training sample set to obtain multiple defect pre-judgment models, and then a defect recognition model is constructed according to the multiple defect pre-judge models, which improves the efficiency of constructing the defect recognition model.
- step 502 specifically includes the step of obtaining a defect prediction model. This step specifically includes the following steps:
- Step 602 For each training sample set, extract model input data and defect annotation data in the training sample set.
- the defect annotation data is the annotation data of the defects in the sample image of the sample image training data, which is the expected output of the model during model training.
- the defects may include at least one of bubbles, scratches, impurities, defects, and excess.
- the terminal extracts the sample images and defect annotation data in the sample image training data and uses the sample images as model input data; for the second set of training sample sets, the terminal extracts the sample image training data Sample image and defect annotation data, pre-processed image and pre-processed image data in pre-processed image data, use the sample image and pre-processed image as model input data; for the third set of training sample set, the terminal extracts the sample image in the sample image training data And the defect annotation data, the defect-free product image and the annotation data in the defect-free product image data, and the sample image and the defect-free product image are used as the model input data; for the fourth set of training sample sets, the terminal extracts the sample images and defects in the sample image training data In the annotation data, the illumination transformation image and the annotation data in the illumination transformation image data, the sample image and the illumination transformation image are used as model input data.
- the sample image extracted by the terminal from the sample image training data is shown in FIG. 7.
- the pre-processed image extracted from the pre-processed image data by the terminal is shown in FIG. 8, wherein the pre-processed image is obtained through visual contour extraction.
- the non-defective product image extracted by the terminal from the non-defective product image data is shown in FIG. 9.
- the defect labeling data is shown in FIG. 10, where the defect 1002 and the defect 1004 are defects whose locations are marked.
- Step 604 Input the model input data into the corresponding initial model to obtain the defect prediction result output by the initial model.
- the defect prediction result is the prediction result of the initial model in the training process to the defect in the model input data.
- the terminal presets an initial model corresponding to each training sample set. After extracting the model input data and defect labeling data from the training sample set, the terminal inputs the model input data into the corresponding initial model, and the initial model processes the model input data to obtain the defect prediction result.
- Step 606 Determine the prediction error according to the defect prediction result and defect annotation data.
- the prediction error is a function of the difference between the defect prediction result and the defect annotation data as an independent variable.
- the terminal uses supervised learning to perform model training. After the terminal obtains the defect prediction result, it compares the defect prediction result with the defect annotation data, and performs calculation according to a preset error formula to obtain the prediction error.
- step 608 the initial model is adjusted according to the prediction error until the prediction error meets the training stop condition, and the defect prediction model corresponding to the training sample set is obtained.
- the training stop condition is a condition for stopping model training, and the training stop condition may be that the prediction error is less than a predetermined error threshold.
- the terminal acquires a predetermined error threshold, and compares the prediction error with the error threshold.
- the terminal adjusts the model parameters in the initial model according to the direction of reducing the prediction error.
- the terminal processes the input data of the model again to obtain the defect prediction result.
- the prediction error is obtained.
- Compare the prediction error with the error threshold If the prediction error is still greater than or equal to the error threshold , Adjust the model again, and iterate in this way until the prediction error is less than the error threshold, stop training, and use the model at the time of training as the defect prediction model corresponding to the training sample set.
- the training stop condition may be a preset number of iterations in model training.
- the terminal stops training and obtains a defect pre-judgment model.
- the model input data and defect labeling data in the training sample set are extracted, and the model input data is input into the corresponding initial model to obtain the defect prediction results output by the initial model.
- the defect prediction results and defects Annotate the data, determine the prediction error, adjust the initial model according to the prediction error, stop training when the prediction error meets the training stop conditions, obtain the defect pre-judgment model corresponding to the training sample set, and continuously adjust the initial model through training to improve acquisition The accuracy of the defect prediction model.
- a step of using a defect identification model is included.
- the step specifically includes the following steps:
- Step 1102 Obtain product image data of the product to be tested.
- the product image data is the image data of the product to be detected in the defect identification model when the defect identification model is used.
- the terminal acquires the triggered product detection instruction, and extracts the product image data of the product to be detected from the storage space according to the product detection instruction.
- the terminal is equipped with an image acquisition device. After the terminal obtains the product detection instruction, the image acquisition device is started to perform image acquisition on the product to be tested on the assembly line to obtain product image data.
- the product image data may include image data corresponding to a plurality of products to be tested, and each product to be tested may include multiple copies of image data.
- Step 1104 input the product image data into the defect recognition model.
- the terminal copies the acquired product image data according to the number of defect prediction models in the defect identification model, so that each defect prediction model corresponds to the same product image data.
- the terminal sequentially inputs the product image data into each defect prediction model in the defect identification model.
- Step 1106 Obtain defect prejudgment results respectively output by each defect prejudgment model in the defect identification model.
- the defect prejudgment result is the result of the defect prejudgment model identifying the defects in the product image data.
- each defect prediction model processes the product image data, identifies the defects in the product image data, and outputs the respective defect prediction results.
- the terminal obtains the defect prediction result of each defect prediction model for analysis and processing.
- Step 1108 Determine the defect recognition result according to the obtained defect prejudgment result.
- the defect identification result is the result of the defect identification model identifying the defects in the product image data.
- the terminal extracts the defect pre-judgment results of each defect pre-judgment model, and counts the occurrence frequency of each defect pre-judgment result, screens the defect pre-judgment results with the highest frequency, and screens the defect pre-judgment results with the highest frequency,
- the defect identification result as a defect identification model.
- the defect recognition result is shown in FIG. 12, which may be an image extraction of a defect part, where defect 1202 and defect 1204 are the recognized defects.
- a defect recognition failure information is generated, and the defect recognition failure information is passed through the display For display.
- the terminal compares the occurrence frequency of the pre-judgment result of the screening with the preset frequency threshold, and if the appearance frequency of the pre-judgment result of the defect is greater than or equal to the preset frequency threshold, the terminal confirms The judgment result is effective, and the pre-judgment result of the screening is used as the defect recognition result of the defect recognition model.
- the terminal when the terminal obtains only one type of auxiliary sample image data, the terminal obtains a defect recognition model after training an initial model.
- the defect recognition model When the terminal uses the defect recognition model, the product image data of the product to be inspected is input into the defect recognition model, and the defect recognition result output by the defect recognition model is directly obtained.
- the product image data of the product to be inspected is obtained, the product image data is input into the defect identification model, the defect prediction results respectively output by the defect prediction models in the defect identification model are obtained, and the defect identification is determined according to the defect prediction results
- the defect recognition results of the model improve the accuracy of obtaining defect recognition results.
- FIG. 13 is a schematic diagram of a defect recognition model training method in an embodiment.
- the first set of training sample sets may be composed of sample image training data
- the second set of training sample sets may be composed of sample image training data and preprocessed image data
- the third set of training sample sets may be composed of sample images
- the training data and the image data of the incomplete product are composed.
- the fourth training sample set may be composed of sample image training data and illumination transformation image data.
- the terminal extracts the model input data and defect labeling data in the training sample set, uses the model input data as the model input, and the defect labeling data as the model output to train each initial model.
- steps in the flowcharts of FIGS. 2-5 and 11 are displayed in order according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless clearly stated in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least some of the steps in FIGS. 2-5 and 11 may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but may be executed at different times. These sub-steps Or the execution order of the stages is not necessarily sequential, but may be executed in turn or alternately with other steps or sub-steps of the other steps or at least a part of the stages.
- a defect recognition model training device 1400 including: a data acquisition module 1402, an auxiliary acquisition module 1404, a sample set construction module 1406, and a model acquisition module 1408, where:
- the data acquisition module 1402 is used to acquire sample image training data.
- the auxiliary obtaining module 1404 is used to obtain auxiliary sample image data corresponding to the sample image training data.
- the sample set construction module 1406 is used to construct the training sample set from the sample image training data and the auxiliary sample image data.
- the model obtaining module 1408 is used to train the initial model according to the training sample set to obtain a defect recognition model.
- the sample image training data and the auxiliary sample image data corresponding to the sample image training data by acquiring the sample image training data and the auxiliary sample image data corresponding to the sample image training data, there is no need to extract additional feature data from the sample image training data, and the sample image training data and the auxiliary sample image data are used to construct the training sample set , Train the initial model according to the training sample set to obtain the defect recognition model.
- the time taken to extract feature data during model training is reduced, and the training efficiency of the model is improved.
- the auxiliary acquisition module 1404 specifically includes: a type determination module, an address acquisition module, a data extraction module, and an auxiliary determination module, where:
- the type determination module is used to determine the auxiliary data type corresponding to the sample image training data.
- the address obtaining module is used to obtain the sample data addresses corresponding to each auxiliary data type.
- the data extraction module is used to extract pre-processed image data, non-defective product image data and illumination transformation image data according to the sample data address.
- the auxiliary determination module is used to preprocess the image data, the image data of the incomplete product and the illumination transformation image data as various auxiliary sample image data.
- the sample data addresses corresponding to each auxiliary data type are obtained, and the pre-processed image data, the non-defective product image data, and the illumination transformation image data are extracted according to the sample data addresses, Then, the extracted pre-processed image data, non-defective product image data and illumination transformation image data are used as auxiliary sample image data, which improves the accuracy of obtaining auxiliary sample image data.
- the sample set construction module 1406 is further used to use the sample image training data as the first set of training sample sets; the sample image training data is respectively combined with the pre-processed image data and incomplete products in various auxiliary sample image data The image data and the light-transformed image data are combined to obtain a second set of training sample sets, a third set of training sample sets, and a fourth set of training sample sets.
- the sample image training data is used as the first training sample set, and the sample image training data is organically combined with the pre-processed image data, the non-defective product image data, and the illumination transformation image data in the auxiliary sample image data.
- the second set of training sample sets, the third set of training sample sets, and the fourth set of training sample sets are obtained, which improves the efficiency of obtaining the training sample set.
- the model obtaining module 1408 is further configured to train each initial model according to each set of training sample sets respectively to obtain multiple defect pre-judgment models; and construct a defect recognition model according to the multiple defect pre-judgment models.
- each initial model is trained according to each set of training sample sets to obtain multiple defect prediction models, and then a defect identification model is constructed based on the multiple defect prediction models, which improves the efficiency of constructing the defect identification model.
- the model obtaining module is further used to extract the model input data and defect annotation data in the training sample set for each set of training sample sets; input the model input data into the corresponding initial model to obtain the defect prediction results output by the initial model ; Determine the prediction error according to the defect prediction results and defect annotation data; adjust the initial model according to the prediction error until the prediction error meets the training stop condition, and obtain the defect prediction model corresponding to the training sample set.
- the model input data and defect labeling data in the training sample set are extracted, and the model input data is input into the corresponding initial model to obtain the defect prediction results output by the initial model.
- the defect prediction results and defects Annotate the data, determine the prediction error, adjust the initial model according to the prediction error, stop training when the prediction error meets the training stop conditions, obtain the defect pre-judgment model corresponding to the training sample set, and continuously adjust the initial model through training to improve acquisition The accuracy of the defect prediction model.
- the defect recognition model training device 1400 further includes a model usage module, which is used to obtain product image data of the product to be inspected; input the product image data into the defect recognition model; and obtain each defect pre-judgment in the defect recognition model Defect pre-judgment results respectively output by the model; the defect recognition results are determined according to the obtained defect pre-judgment results.
- a model usage module which is used to obtain product image data of the product to be inspected; input the product image data into the defect recognition model; and obtain each defect pre-judgment in the defect recognition model Defect pre-judgment results respectively output by the model; the defect recognition results are determined according to the obtained defect pre-judgment results.
- the product image data of the product to be inspected is obtained, the product image data is input into the defect identification model, the defect prediction results respectively output by the defect prediction models in the defect identification model are obtained, and the defect identification is determined according to the defect prediction results
- the defect recognition results of the model improve the accuracy of obtaining defect recognition results.
- Each module in the above-mentioned defect recognition model training device can be implemented in whole or in part by software, hardware, and a combination thereof.
- the above modules may be embedded in the hardware form or independent of the processor in the computer device, or may be stored in the memory in the computer device in the form of software so that the processor can call and execute the operations corresponding to the above modules.
- a computer device is provided.
- the computer device may be a terminal, and an internal structure diagram thereof may be as shown in FIG. 15.
- the computer equipment includes a processor, a memory, a network interface, a display screen, an input device, and an image acquisition device connected through a system bus.
- the processor of the computer device is used to provide computing and control capabilities.
- the memory of the computer device includes a non-volatile storage medium and an internal memory.
- the non-volatile storage medium stores an operating system and computer programs.
- the internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium.
- the network interface of the computer device is used to communicate with external terminals through a network connection.
- the display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen
- the input device of the computer device may be a touch layer covered on the display screen, or may be a button, a trackball, or a touchpad provided on the computer device housing , Can also be an external keyboard, touchpad or mouse.
- the image acquisition device is used to acquire product image data of the product to be tested.
- FIG. 15 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied.
- the specific computer device may Include more or less components than shown in the figure, or combine certain components, or have a different arrangement of components.
- a computer device which includes a memory, a processor, and a computer program stored on the memory and executable on the processor.
- the processor executes the computer program, the following steps are implemented: acquiring sample image training data; Obtain the auxiliary sample image data corresponding to the sample image training data; construct the training sample set from the sample image training data and the auxiliary sample image data; train the initial model according to the training sample set to obtain the defect recognition model.
- obtaining auxiliary sample image data corresponding to the sample image training data includes: determining the type of auxiliary data corresponding to the sample image training data; obtaining the sample data address corresponding to each auxiliary data type; Process image data, non-defective product image data and illumination transformation image data; use preprocessed image data, non-defective product image data and illumination transformation image data as various auxiliary sample image data.
- using the sample image training data and the auxiliary sample image data to construct the training sample set includes: using the sample image training data as the first set of training sample sets; and using the sample image training data separately from the various auxiliary sample image data The pre-processed image data, non-defective product image data and lighting transformation image data in the combination are combined to obtain the second set of training sample sets, the third set of training sample sets and the fourth set of training sample sets.
- training the initial model according to the training sample set to obtain a defect recognition model includes: training each initial model separately according to each set of training sample sets to obtain multiple defect prediction models; based on multiple defect predictions The model builds a defect recognition model.
- training each initial model according to each set of training sample sets to obtain multiple defect prediction models includes: for each set of training sample sets, extracting model input data and defect labeling data in the training sample set; Enter the initial model corresponding to the model input data to obtain the defect prediction result output by the initial model; determine the prediction error based on the defect prediction result and the defect annotation data; adjust the initial model according to the prediction error until the prediction error meets the training stop condition and obtain training The defect prediction model corresponding to the sample set.
- the processor after constructing a defect identification model based on multiple defect pre-judgment models, the processor also implements the following steps when executing the computer program: acquiring product image data of the product to be inspected; entering the product image data into the defect identification model; acquiring defects Defect pre-judgment results output by each defect pre-judgment model in the recognition model; determine the defect recognition result according to the obtained defect pre-judgment results.
- the sample image training data and the auxiliary sample image data corresponding to the sample image training data by acquiring the sample image training data and the auxiliary sample image data corresponding to the sample image training data, there is no need to extract additional feature data from the sample image training data, and the sample image training data and the auxiliary sample image data are used to construct the training sample set , Train the initial model according to the training sample set to obtain the defect recognition model.
- the time taken to extract feature data during model training is reduced, and the training efficiency of the model is improved.
- a computer-readable storage medium on which a computer program is stored.
- the computer program is executed by a processor, the following steps are achieved: acquiring sample image training data; acquiring auxiliary samples corresponding to the sample image training data Image data; construct the training sample set based on the sample image training data and the auxiliary sample image data; train the initial model according to the training sample set to obtain the defect recognition model.
- obtaining auxiliary sample image data corresponding to the sample image training data includes: determining the type of auxiliary data corresponding to the sample image training data; obtaining the sample data address corresponding to each auxiliary data type; Process image data, non-defective product image data and illumination transformation image data; use preprocessed image data, non-defective product image data and illumination transformation image data as various auxiliary sample image data.
- using the sample image training data and the auxiliary sample image data to construct the training sample set includes: using the sample image training data as the first set of training sample sets; and using the sample image training data separately from the various auxiliary sample image data The pre-processed image data, non-defective product image data and lighting transformation image data in the combination are combined to obtain the second set of training sample sets, the third set of training sample sets and the fourth set of training sample sets.
- training the initial model according to the training sample set to obtain a defect recognition model includes: training each initial model separately according to each set of training sample sets to obtain multiple defect prediction models; based on multiple defect predictions The model builds a defect recognition model.
- training each initial model according to each set of training sample sets to obtain multiple defect prediction models includes: for each set of training sample sets, extracting model input data and defect labeling data in the training sample set; Enter the initial model corresponding to the model input data to obtain the defect prediction result output by the initial model; determine the prediction error based on the defect prediction result and the defect annotation data; adjust the initial model according to the prediction error until the prediction error meets the training stop condition and obtain training The defect prediction model corresponding to the sample set.
- the computer program is executed by the processor and further implements the following steps: obtaining product image data of the product to be inspected; inputting the product image data into the defect recognition model; obtaining Defect pre-judgment results respectively output by each defect pre-judgment model in the defect recognition model; the defect recognition result is determined according to the obtained defect pre-judged results.
- the sample image training data and the auxiliary sample image data corresponding to the sample image training data by acquiring the sample image training data and the auxiliary sample image data corresponding to the sample image training data, there is no need to extract additional feature data from the sample image training data, and the sample image training data and the auxiliary sample image data are used to construct the training sample set , Train the initial model according to the training sample set to obtain the defect recognition model.
- the time taken to extract feature data during model training is reduced, and the training efficiency of the model is improved.
- Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
- Volatile memory can include random access memory (RAM) or external cache memory.
- RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
- SRAM static RAM
- DRAM dynamic RAM
- SDRAM synchronous DRAM
- DDRSDRAM double data rate SDRAM
- ESDRAM enhanced SDRAM
- SLDRAM synchronous chain (Synchlink) DRAM
- SLDRAM synchronous chain (Synchlink) DRAM
- Rambus direct RAM
- DRAM direct memory bus dynamic RAM
- RDRAM memory bus dynamic RAM
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
- Image Analysis (AREA)
Abstract
Description
Claims (10)
- 一种缺陷识别模型训练方法,所述方法包括:获取样本图像训练数据;获取与所述样本图像训练数据对应的辅助样本图像数据;以所述样本图像训练数据和所述辅助样本图像数据,构建训练样本集;根据所述训练样本集对初始模型进行训练,得到缺陷识别模型。
- 根据权利要求1所述的方法,其特征在于,所述获取与所述样本图像训练数据对应的辅助样本图像数据包括:确定所述样本图像训练数据对应的辅助数据类型;获取各辅助数据类型分别对应的样本数据地址;根据所述样本数据地址分别提取预处理图像数据、无缺产品图像数据和光照变换图像数据;以所述预处理图像数据、所述无缺产品图像数据和所述光照变换图像数据,作为各种辅助样本图像数据。
- 根据权利要求2所述的方法,其特征在于,所述以所述样本图像训练数据和所述辅助样本图像数据,构建训练样本集包括:以所述样本图像训练数据作为第一组训练样本集;将所述样本图像训练数据,分别与所述各种辅助样本图像数据中的预处理图像数据、无缺产品图像数据和光照变换图像数据进行组合,得到第二组训练样本集、第三组训练样本集和第四组训练样本集。
- 根据权利要求3所述的方法,其特征在于,所述根据所述训练样本集对初始模型进行训练,得到缺陷识别模型包括:分别根据每组训练样本集对各初始模型进行训练,得到多个缺陷预判模型;根据所述多个缺陷预判模型构建缺陷识别模型。
- 根据权利要求4所述的方法,其特征在于,所述分别根据每组训练样本集对各初始模型进行训练,得到多个缺陷预判模型包括:对于每组训练样本集,提取所述训练样本集中的模型输入数据和缺陷标注数据;将所述模型输入数据输入对应的初始模型,得到所述初始模型输出的缺陷预测结果;根据所述缺陷预测结果与所述缺陷标注数据,确定预测误差;根据所述预测误差对所述初始模型进行调整,直至所述预测误差满足训练停止条件,得到所述训练样本集对应的缺陷预判模型。
- 根据权利要求1-5中任一项所述的方法,其特征在于,所述根据所述多个缺陷预判模型构建缺陷识别模型之后,还包括:获取待检测产品的产品图像数据;将所述产品图像数据输入所述缺陷识别模型;获取所述缺陷识别模型中各缺陷预判模型分别输出的缺陷预判结果;根据获取到的缺陷预判结果确定缺陷识别结果。
- 一种缺陷识别模型训练装置,所述装置包括:数据获取模块,用于获取样本图像训练数据;辅助获取模块,用于获取与所述样本图像训练数据对应的辅助样本图像数据;样本集构建模块,用于以所述样本图像训练数据和所述辅助样本图像数据,构建训练样本集;模型得到模块,用于根据所述训练样本集对初始模型进行训练,得到缺陷识别模型。
- 根据权利要求7所述的装置,其特征在于,所述辅助获取模块包括:类型确定模块,用于确定所述样本图像训练数据对应的辅助数据类型;地址获取模块,用于获取各辅助数据类型分别对应的样本数据地址;数据提取模块,用于根据所述样本数据地址分别提取预处理图像数据、无缺产品图像数据和光照变换图像数据;辅助确定模块,用于以所述预处理图像数据、所述无缺产品图像数据和所述光照变换图像数据,作为各种辅助样本图像数据。
- 一种计算机设备,包括存储器、处理器及存储在存储器上并可在处理器 上运行的计算机程序,其特征在于,所述处理器执行所述计算机程序时实现权利要求1至6中任一项所述方法的步骤。
- 一种计算机可读存储介质,其上存储有计算机程序,其特征在于,所述计算机程序被处理器执行时实现权利要求1至6中任一项所述的方法的步骤。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910012264.1A CN109829483B (zh) | 2019-01-07 | 2019-01-07 | 缺陷识别模型训练方法、装置、计算机设备和存储介质 |
CN201910012264.1 | 2019-01-07 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2020143592A1 true WO2020143592A1 (zh) | 2020-07-16 |
Family
ID=66860187
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2020/070578 WO2020143592A1 (zh) | 2019-01-07 | 2020-01-07 | 缺陷识别模型训练方法、装置、计算机设备和存储介质 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN109829483B (zh) |
WO (1) | WO2020143592A1 (zh) |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112734693A (zh) * | 2020-12-18 | 2021-04-30 | 平安科技(深圳)有限公司 | 一种管道焊缝缺陷检测方法及相关装置 |
CN113192021A (zh) * | 2021-04-26 | 2021-07-30 | 深圳中科飞测科技股份有限公司 | 检测方法及装置、检测设备和存储介质 |
CN113538341A (zh) * | 2021-03-31 | 2021-10-22 | 联合汽车电子有限公司 | 自动光学检测辅助方法、装置及存储介质 |
CN113610787A (zh) * | 2021-07-27 | 2021-11-05 | 广东省科技基础条件平台中心 | 图像缺陷检测模型的训练方法、装置、计算机设备 |
CN113706462A (zh) * | 2021-07-21 | 2021-11-26 | 南京旭锐软件科技有限公司 | 产品表面缺陷检测方法、装置、设备及存储介质 |
CN114083770A (zh) * | 2021-10-29 | 2022-02-25 | 北京百度网讯科技有限公司 | 工艺参数调整及模型训练方法、装置、设备与存储介质 |
CN116523845A (zh) * | 2023-04-10 | 2023-08-01 | 江苏濠汉信息技术有限公司 | 一种基于电缆隧道的缺陷检测方法和系统 |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109829483B (zh) * | 2019-01-07 | 2021-05-18 | 鲁班嫡系机器人(深圳)有限公司 | 缺陷识别模型训练方法、装置、计算机设备和存储介质 |
CN112348040B (zh) * | 2019-08-07 | 2023-08-29 | 杭州海康威视数字技术股份有限公司 | 模型训练方法、装置及设备 |
CN112907498B (zh) * | 2019-11-18 | 2024-04-30 | 中国商用飞机有限责任公司 | 一种孔隙识别方法、装置、设备及存储介质 |
CN111402220B (zh) * | 2020-03-11 | 2023-06-09 | 北京百度网讯科技有限公司 | 用于获取信息的方法及装置 |
CN112926437B (zh) * | 2021-02-22 | 2024-06-11 | 深圳中科飞测科技股份有限公司 | 检测方法及装置、检测设备和存储介质 |
CN113570007B (zh) * | 2021-09-27 | 2022-02-15 | 深圳市信润富联数字科技有限公司 | 零件缺陷识别模型构建优化方法、装置、设备及存储介质 |
CN115240031B (zh) * | 2022-07-21 | 2023-10-27 | 无锡锤头鲨智能科技有限公司 | 一种基于生成对抗网络的板材表面缺陷生成方法及系统 |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106530284A (zh) * | 2016-10-21 | 2017-03-22 | 广州视源电子科技股份有限公司 | 基于图像识别的焊点类型检测和装置 |
US20180144216A1 (en) * | 2016-11-23 | 2018-05-24 | Industrial Technology Research Institute | Classification method, classification module and computer program product using the same |
CN108562589A (zh) * | 2018-03-30 | 2018-09-21 | 慧泉智能科技(苏州)有限公司 | 一种对磁路材料表面缺陷进行检测的方法 |
CN108615230A (zh) * | 2018-03-16 | 2018-10-02 | 北京邮电大学 | 一种轮毂表面异常检测方法和系统 |
CN108764372A (zh) * | 2018-06-08 | 2018-11-06 | Oppo广东移动通信有限公司 | 数据集的构建方法和装置、移动终端、可读存储介质 |
CN109829483A (zh) * | 2019-01-07 | 2019-05-31 | 鲁班嫡系机器人(深圳)有限公司 | 缺陷识别模型训练方法、装置、计算机设备和存储介质 |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7130776B2 (en) * | 2002-03-25 | 2006-10-31 | Lockheed Martin Corporation | Method and computer program product for producing a pattern recognition training set |
CN103760229B (zh) * | 2014-01-07 | 2016-03-30 | 天津大学 | 基于支持向量机的焊接缺陷巨磁电阻涡流检测方法 |
CN106528417A (zh) * | 2016-10-28 | 2017-03-22 | 中国电子产品可靠性与环境试验研究所 | 软件缺陷智能检测方法和系统 |
CN107316295A (zh) * | 2017-07-02 | 2017-11-03 | 苏州大学 | 一种基于深度神经网络的织物瑕疵检测方法 |
CN107392896B (zh) * | 2017-07-14 | 2019-11-08 | 佛山市南海区广工大数控装备协同创新研究院 | 一种基于深度学习的木材缺陷检测方法和系统 |
CN108896567B (zh) * | 2018-08-06 | 2024-06-04 | 慧泉智能科技(苏州)有限公司 | 一种对钮扣电池表面压伤缺陷进行检测的方法 |
-
2019
- 2019-01-07 CN CN201910012264.1A patent/CN109829483B/zh active Active
-
2020
- 2020-01-07 WO PCT/CN2020/070578 patent/WO2020143592A1/zh active Application Filing
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106530284A (zh) * | 2016-10-21 | 2017-03-22 | 广州视源电子科技股份有限公司 | 基于图像识别的焊点类型检测和装置 |
US20180144216A1 (en) * | 2016-11-23 | 2018-05-24 | Industrial Technology Research Institute | Classification method, classification module and computer program product using the same |
CN108615230A (zh) * | 2018-03-16 | 2018-10-02 | 北京邮电大学 | 一种轮毂表面异常检测方法和系统 |
CN108562589A (zh) * | 2018-03-30 | 2018-09-21 | 慧泉智能科技(苏州)有限公司 | 一种对磁路材料表面缺陷进行检测的方法 |
CN108764372A (zh) * | 2018-06-08 | 2018-11-06 | Oppo广东移动通信有限公司 | 数据集的构建方法和装置、移动终端、可读存储介质 |
CN109829483A (zh) * | 2019-01-07 | 2019-05-31 | 鲁班嫡系机器人(深圳)有限公司 | 缺陷识别模型训练方法、装置、计算机设备和存储介质 |
Cited By (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112734693A (zh) * | 2020-12-18 | 2021-04-30 | 平安科技(深圳)有限公司 | 一种管道焊缝缺陷检测方法及相关装置 |
CN112734693B (zh) * | 2020-12-18 | 2024-06-07 | 平安科技(深圳)有限公司 | 一种管道焊缝缺陷检测方法及相关装置 |
CN113538341A (zh) * | 2021-03-31 | 2021-10-22 | 联合汽车电子有限公司 | 自动光学检测辅助方法、装置及存储介质 |
CN113538341B (zh) * | 2021-03-31 | 2024-04-30 | 联合汽车电子有限公司 | 自动光学检测辅助方法、装置及存储介质 |
CN113192021A (zh) * | 2021-04-26 | 2021-07-30 | 深圳中科飞测科技股份有限公司 | 检测方法及装置、检测设备和存储介质 |
CN113706462A (zh) * | 2021-07-21 | 2021-11-26 | 南京旭锐软件科技有限公司 | 产品表面缺陷检测方法、装置、设备及存储介质 |
CN113610787A (zh) * | 2021-07-27 | 2021-11-05 | 广东省科技基础条件平台中心 | 图像缺陷检测模型的训练方法、装置、计算机设备 |
CN114083770A (zh) * | 2021-10-29 | 2022-02-25 | 北京百度网讯科技有限公司 | 工艺参数调整及模型训练方法、装置、设备与存储介质 |
CN114083770B (zh) * | 2021-10-29 | 2024-03-22 | 北京百度网讯科技有限公司 | 工艺参数调整及模型训练方法、装置、设备与存储介质 |
CN116523845A (zh) * | 2023-04-10 | 2023-08-01 | 江苏濠汉信息技术有限公司 | 一种基于电缆隧道的缺陷检测方法和系统 |
CN116523845B (zh) * | 2023-04-10 | 2023-11-07 | 江苏濠汉信息技术有限公司 | 一种基于电缆隧道的缺陷检测方法和系统 |
Also Published As
Publication number | Publication date |
---|---|
CN109829483B (zh) | 2021-05-18 |
CN109829483A (zh) | 2019-05-31 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2020143592A1 (zh) | 缺陷识别模型训练方法、装置、计算机设备和存储介质 | |
Zheng et al. | Adaptive color deconvolution for histological WSI normalization | |
CN109961421B (zh) | 数据生成装置、数据生成方法以及数据生成记录介质 | |
CN109767418B (zh) | 检査装置、数据生成装置、数据生成方法及存储介质 | |
US10445557B2 (en) | Learning pixel visual context from object characteristics to generate rich semantic images | |
WO2020134991A1 (zh) | 纸质表单的自动录入方法、装置、计算机设备和存储介质 | |
EP3904866A1 (en) | Defect inspecting device, defect inspecting method, and program for same | |
JP2021532434A (ja) | 顔特徴抽出モデル訓練方法、顔特徴抽出方法、装置、機器および記憶媒体 | |
JP7477517B2 (ja) | 画像分析システムおよび画像分析システムを使用する方法 | |
Muñoz et al. | Computer image analysis as a tool for classifying marbling: A case study in dry-cured ham | |
CN110956628B (zh) | 图片等级分类方法、装置、计算机设备和存储介质 | |
RU2013134132A (ru) | Способ и устройство для определения эталонного набора данных класса для классификации ценных документов | |
US20210337073A1 (en) | Print quality assessments via patch classification | |
Shete et al. | TasselGAN: An application of the generative adversarial model for creating field-based maize tassel data | |
CN117392042A (zh) | 缺陷检测方法、缺陷检测设备及存储介质 | |
CN109656808A (zh) | 一种基于混合式主动学习策略的软件缺陷预测方法 | |
Dubosclard et al. | Automated visual grading of grain kernels by machine vision | |
CN106960188B (zh) | 天气图像分类方法及装置 | |
JP7154786B2 (ja) | 画像処理装置、画像処理方法及びプログラム | |
CN107729635A (zh) | 半导体芯片合格性检查方法及装置 | |
CN116824419A (zh) | 一种着装特征识别方法、识别模型的训练方法及装置 | |
TWM579762U (zh) | Fabric information mediation system | |
US20230306714A1 (en) | Chromatic undertone detection | |
CN109829898B (zh) | 一种用于互联网检测中基于神经网络的测量检测系统及方法 | |
CN117726627B (zh) | 芯片表面缺陷检测方法及设备 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20738496 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20738496 Country of ref document: EP Kind code of ref document: A1 |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 06.12.2021) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20738496 Country of ref document: EP Kind code of ref document: A1 |