CN111024708A

CN111024708A - Method, device, system and equipment for processing product defect detection data

Info

Publication number: CN111024708A
Application number: CN201911273653.6A
Authority: CN
Inventors: 沈小勇; 张文杰; 刘刚
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2019-09-06
Filing date: 2019-09-06
Publication date: 2020-04-17
Anticipated expiration: 2039-09-06
Also published as: CN111024708B; CN110554047B; CN110554047A

Abstract

The application relates to a method, a device, a system and equipment for processing product defect detection data, which relate to a block chain technology, and the method comprises the following steps: acquiring a picture to be detected corresponding to a product to be detected; inputting the picture to be detected into a target defect detection model to obtain a defect detection result of the product to be detected; and storing the defect detection result in a block chain, and generating a result query code corresponding to the defect detection result, wherein the result query code is used for querying the defect detection result in the block chain. The method ensures the safety and reliability of the defect detection result.

Description

Method, device, system and equipment for processing product defect detection data

The present application is a divisional application entitled "method, apparatus, system and device for processing data for detecting product defects" filed by the chinese patent office on 09/06/2019 with application number 201910844469.6, the entire contents of which are incorporated herein by reference.

Technical Field

The present application relates to the field of product inspection, and in particular, to a method, an apparatus, a system, and a device for processing product defect inspection data.

Background

The product quality is one of the most important production indexes in the manufacturing industry, and in order to ensure the product quality, the defect detection of the product in the production process of the product becomes an indispensable procedure. For example, in a camera production plant, a defective camera generated in the production process needs to be selected, so as to prevent the defective camera from directly flowing into the next process.

At present, when the defect of a product is detected, a technician is relied on for visual identification, and factors such as personal understanding difference and fatigue of defect detection standards exist in manual identification, so that the defect identification efficiency is low.

Disclosure of Invention

In view of the above, it is necessary to provide a product defect detection data processing method, apparatus, system and device for solving the problem of low defect identification efficiency.

On one hand, the method for processing the product defect detection data comprises the steps of obtaining a picture to be detected corresponding to a product to be detected; inputting the picture to be detected into a target defect detection model to obtain a defect detection result of the product to be detected; and storing the defect detection result in a block chain, and generating a result query code corresponding to the defect detection result, wherein the result query code is used for querying the defect detection result in the block chain.

In one aspect, a product defect detection data processing system is provided, the system comprising: the model training node receives a product defect detection model generation request, the product defect detection model generation request carries model training parameters, corresponding training pictures are obtained according to the model training parameters, defect types corresponding to the training pictures are obtained, training samples are obtained, model training is carried out according to the training samples, a target defect detection model is generated, the model training parameters are determined according to parameter configuration operation of a user, and the model training parameters comprise at least one of defect learning types or training picture information; the product defect detection node acquires a to-be-detected picture corresponding to a to-be-detected product, inputs the to-be-detected picture into a target defect detection model to obtain a defect detection result of the to-be-detected product, stores the defect detection result in a block chain, and generates a result query code corresponding to the defect detection result, wherein the result query code is used for querying the defect detection result in the block chain.

On one hand, the device for processing the product defect detection data comprises a to-be-detected picture acquisition module, a to-be-detected picture acquisition module and a to-be-detected picture acquisition module, wherein the to-be-detected picture acquisition module is used for acquiring a to-be-detected picture corresponding to a to-be-detected product; the input module is used for inputting the picture to be detected into a target defect detection model to obtain a defect detection result of the product to be detected; and the storage module is used for storing the defect detection result in a block chain and generating a result query code corresponding to the defect detection result, wherein the result query code is used for querying the defect detection result in the block chain.

In one aspect, a computer device is provided, which comprises a memory and a processor, wherein the memory stores a computer program, and the computer program, when executed by the processor, causes the processor to execute the steps of the above-mentioned product defect detection data processing method.

In one aspect, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, causes the processor to perform the steps of the above-mentioned product defect detection data processing method.

According to the product defect detection data processing method, device, system and equipment, the defect detection result of the product to be detected is stored in the data block of the block chain, so that the safety and reliability of the defect detection result can be guaranteed, and the defect detection result of the product can be prevented from being tampered by others for the benefit due to the non-tamper property of the block chain, so that the authenticity of the product defect detection result obtained through the inquiry of the result inquiry code is guaranteed.

Drawings

FIG. 1 is a diagram of an application environment of a product defect detection data processing method provided in some embodiments;

FIG. 2A is a flow diagram of a method for processing product defect detection data in some embodiments;

FIG. 2B is a schematic diagram of a parameter configuration interface in accordance with certain embodiments;

FIG. 3 is a schematic diagram of an interface for configuring product related information in some embodiments;

FIG. 4 is a flow diagram of a method for processing product defect detection data in some embodiments;

FIG. 5 is a schematic diagram of a model evaluation interface in some embodiments;

FIG. 6 is a block diagram of a product defect detection data processing system in some embodiments;

FIG. 7 is a block diagram of a product defect detection data processing system in accordance with certain embodiments;

FIG. 8 is a schematic diagram of an application interface corresponding to a service system in some embodiments;

FIG. 9 is a diagram of a product module for model training in some embodiments;

FIG. 10 is a schematic diagram of an overview page in some embodiments;

FIG. 11 is a flow diagram of defect detection and model training in some embodiments;

FIG. 12 is a diagram illustrating a system architecture corresponding to a product defect detection system in some embodiments;

FIG. 13 is a diagram illustrating a deployment architecture for a product defect detection system in accordance with certain embodiments;

FIG. 14 is a block diagram of a product defect detection data processing apparatus in some embodiments;

FIG. 15 is a block diagram of the internal architecture of a computing device in some embodiments.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

The following explains a technique related to a product defect detection data processing method according to an embodiment of the present application.

Artificial Intelligence (AI) is a theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human Intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making.

The artificial intelligence technology is a comprehensive subject and relates to the field of extensive technology, namely the technology of a hardware level and the technology of a software level. The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, machine learning/deep learning and other directions.

Computer Vision technology (CV) Computer Vision is a science for researching how to make a machine "see", and further refers to that a camera and a Computer are used to replace human eyes to perform machine Vision such as identification, tracking and measurement on a target, and further image processing is performed, so that the Computer processing becomes an image more suitable for human eyes to observe or transmitted to an instrument to detect. As a scientific discipline, computer vision research-related theories and techniques attempt to build artificial intelligence systems that can capture information from images or multidimensional data. Computer vision techniques typically include image processing, image recognition, and the like. According to the method provided by the embodiment of the application, the defect detection is carried out on the picture to be detected by using the computer vision technology, so that the defect of the product to be detected can be determined.

Machine Learning (ML) is a multi-domain cross discipline, and relates to a plurality of disciplines such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and the like. The special research on how a computer simulates or realizes the learning behavior of human beings so as to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve the performance of the computer. Machine learning is the core of artificial intelligence, is the fundamental approach for computers to have intelligence, and is applied to all fields of artificial intelligence. Machine learning and deep learning generally include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and formal education learning.

The scheme provided by the embodiment of the application relates to detecting product defects based on artificial intelligence and computer vision, and is specifically explained by the following embodiment.

The scheme provided by the embodiment of the application can also relate to a block chain correlation technology, for example, the defect detection result of the product to be detected can be stored in the block chain, and the defect detection can also be executed by the block chain link points. The defect detection result of the product to be detected is stored in the data block of the block chain, so that the safety and reliability of the defect detection result can be guaranteed, and the defect detection result of the product can be prevented from being tampered by others for the benefit due to the non-tampering property of the block chain, so that the authenticity of the product defect detection result obtained by inquiry is guaranteed.

In some embodiments, when the defect detection result of the product to be detected is stored in the block chain, the block chain link point may generate a result query code corresponding to the defect detection result of the product, and the result query code may be obtained by hashing the defect detection result of the product, so that the uniqueness of the result query code may be ensured. When the block chain node receives a defect detection result query request carrying a result query code, a corresponding product defect detection result can be obtained from the data block according to the result query code and returned to the query end.

The blockchain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism and an encryption algorithm. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product services layer, and an application services layer.

The block chain underlying platform can comprise processing modules such as user management, basic service, intelligent contract and operation monitoring. The user management module is responsible for identity information management of all blockchain participants, and comprises public and private key generation maintenance (account management), key management, user real identity and blockchain address corresponding relation maintenance (authority management) and the like, and under the authorization condition, the user management module supervises and audits the transaction condition of certain real identities and provides rule configuration (wind control audit) of risk control; the basic service module is deployed on all the block chain node equipment and used for verifying the validity of the service request and recording the valid request after the valid request is identified in common to storage. The service request can be a product defect detection result storage request, for a new service request, the basic service firstly carries out interface adaptation analysis and authentication processing (interface adaptation), then encrypts service information (consensus management) through a consensus algorithm, completely and consistently transmits the encrypted service information to a shared account book (network communication), and records and stores the encrypted service information; the intelligent contract module is responsible for registering and issuing contracts, triggering the contracts and executing the contracts, developers can define contract logics through a certain programming language, issue the contract logics to a block chain (contract registration), call keys or other event triggering and executing according to the logics of contract clauses, complete the contract logics and simultaneously provide the function of upgrading and canceling the contracts; the operation monitoring module is mainly responsible for deployment, configuration modification, contract setting, cloud adaptation in the product release process and visual output of real-time states in product operation, such as: alarm, monitoring network conditions, monitoring node equipment health status, and the like.

The platform product service layer provides basic capability and an implementation framework of typical application, and developers can complete block chain implementation of business logic based on the basic capability and the characteristics of the superposed business. The application service layer provides the application service based on the block chain scheme for the business participants to use.

Fig. 1 is a diagram of an application environment of a product defect detection data processing method provided in some embodiments, as shown in fig. 1, in the application environment, including a service node 110, a product defect detection node 120, a model training node 130, and a device 140 corresponding to a manufacturing enterprise. The Manufacturing enterprise corresponding equipment 140 may be equipment in a Manufacturing Execution System (MES). The device 140 corresponding to the manufacturing enterprise may be referred to as a manufacturing terminal, and may be, for example, a computer of a manager in a factory, in which a corresponding production management application may be installed, or may be manufacturing equipment for manufacturing a product. The product defect detection data processing system may include a service node 110, a product defect detection node 120, and a model training node 130. The service node 110 is used for communicating with a device 110 corresponding to a production enterprise and providing a product defect detection service. For example, the defect detection result is sent to the equipment 140 corresponding to the manufacturing enterprise, the product defect detection model generation request sent by the equipment 140 corresponding to the manufacturing enterprise is received, and the model training node 130 is controlled to perform model training according to the product defect detection model generation request. The model training node 130 is used for performing model training to obtain a target defect detection model, and the product defect detection node 120 is deployed with the target defect detection model and used for performing defect detection according to the picture of the product to be detected to obtain the defects of the product.

The nodes such as the service node 110, the product defect detection node 120, or the model training node 130 may be independent servers, may also be a server cluster formed by a plurality of physical servers, and may be cloud servers providing basic cloud computing services such as a cloud server, a cloud database, a cloud storage, and a CDN.

The nodes may be connected by communication connection methods such as a network, and the application is not limited herein. The device 140 corresponding to the manufacturing enterprise may be connected to the service node 110 through a network. Of course, the device 140 corresponding to the manufacturing enterprise may also be connected to the product defect detecting node 120, and may also be connected to the model training node 130.

It is to be understood that the above application environment is only an example and is not to be construed as limiting the product defect detection data processing system in the present application, for example, the product defect detection data processing system may further include a picture marking system and a picture storage system. The product defect detection data processing system may also not include a service node. But rather by product defect detection node 120 interacting with a corresponding device 140 of the manufacturing enterprise.

As shown in fig. 2A, in some embodiments, a method for processing product defect detection data is provided, and this embodiment is mainly illustrated by applying the method to the product defect detection data processing system in fig. 1. Of course, all the steps may be performed by some of the nodes, for example, by the model training node 130, which may specifically include the following steps:

step S202, a product defect detection model generation request is received, the product defect detection model generation request carries model training parameters, the model training parameters are determined according to parameter configuration operation of a user, and the model training parameters comprise at least one of defect learning categories or training picture information.

Specifically, the product defect detection model generation request is used for requesting generation of a product defect detection model. The request for generating the product defect detection model can be sent by a production terminal, and the production terminal is a terminal corresponding to a production user. The production user refers to a user corresponding to a factory that produces the product. Such as a research and development engineer or a process engineer in a factory. A production user can log in the product defect detection data processing system on a production terminal through an account and a password. The model training parameters refer to parameters related to model training. The method may include at least one of a defect learning category or training picture information, and may further include at least one of a type of the model, a number of hidden layers in the model, an accuracy requirement of the model, test set information for performing performance evaluation on the model, or computer resources required to be configured during model training. The type of model may be, for example, CNN (Convolutional Neural Networks). The defect learning type corresponding to the model refers to a defect type to be learned by the model. The number of the defect learning categories may be set as needed, for example, 1 or more. The plurality means two or more, and two or more includes two. The product defect detection model is a classification model. The classification model may be binary, and the classification result may be either defect-included or defect-not-included. The classification model may be more than three classifications. For example, for three classifications, the classification result may be three types, i.e., no defect, a defect and B defect. The A defect and the B defect are carried in the product defect detection model generation request. The training picture information is used to determine a training picture. For example, the product defect detection model generation request may carry at least one of a training picture, an identification of the training picture, or a storage location of the training picture.

The model training parameters are determined according to the parameter configuration operation of the user, the configuration operation can be at least one of a selection operation or an addition operation, the selection operation is to select a target parameter from the candidate parameters, and the addition operation is to add a new parameter. For example, a category may be selected from the defect learning categories displayed on the interface, and a new defect learning category may be added. The parameter configuration operation may be one or more of a screen touch operation, a gesture operation, a voice operation, or a mouse input operation. The product defect detection model generation request can be sent by a production terminal corresponding to the user. The user can configure the model training parameters in a parameter configuration interface displayed in the production terminal. For example, a production management application may be installed on the production terminal. The production terminal may display a parameter configuration interface. The user can configure the model training parameters in the parameter configuration interface according to the requirement. The defects of the product may be different due to any one or more of the differences in the production time, the station, or the process parameters during the production of the product. Or different products, the corresponding defect detection criteria may also be different. And different defect detection standards, the obtained training pictures are different. For example, if the defect detection standard is high, strict labeling standards are also required to be implemented when labeling the defects of the training picture. If the defects of the products are different, the defect types to be learned by the model are also different. Through the parameter configuration interface, the model training parameters can be flexibly configured according to the production business scene. For example, the defect learning category is configured according to the defect corresponding to the product to be tested. And configuring a training picture according to the product defect detection standard, wherein the defect marking standard of the training picture is determined according to the product detection marking. The training pictures can be stored in the system in advance, and the training pictures corresponding to different product defect detection standards can be configured in the system. Or the user selects the corresponding training picture according to the product defect detection standard.

As shown in fig. 2B, which is a schematic diagram of a parameter configuration interface, a production user may select a defect learning category and input a training picture storage path as needed, and may add a new defect. Clicking the "browse" button may enter an interface for selecting a picture from a server or locally stored pictures. After the parameters are configured, a 'submit model generation request' can be clicked to send a product defect detection model generation request.

In some embodiments, the product defect detection model generation request may also include other information, such as the name of the model.

And step S204, acquiring corresponding training pictures and defect types corresponding to the training pictures according to the model training parameters to obtain training samples.

Specifically, the training samples are used for model training, and include training pictures and corresponding labels, where the labels are corresponding defect classes. There may be a plurality of training samples, for example 1 ten thousand. When the model training parameters include the defect learning categories, pictures corresponding to the defect learning categories can be obtained as training pictures, and certainly, pictures without defects can also be obtained as negative samples. When the model training parameters include training picture information, the training pictures can be obtained according to the training picture information. For example, the training pictures included in the model training parameters are pictures stored in the folder a, and the defect learning categories are H and B. Then the picture with the defect type H and the picture with the defect type B may be obtained in the a folder as training pictures.

In some embodiments, sample pictures for model training may be stored in advance, for example, pictures taken of a product during a production process may be obtained, and the type of the defect or whether the defect is included may be manually marked. And after the marking is finished, uploading the mark to a process storage in the system. When the model needs to be generated, the system can return option information for selecting the training picture to the production terminal, the option information comprises the picture identification, such as the folder name of the picture, and the user selects the picture to train according to the option information.

And S206, performing model training according to the training sample to generate a target defect detection model, wherein the target defect detection model is used for performing defect detection on the product to be detected according to the picture to be detected corresponding to the product to be detected to obtain a defect detection result of the product to be detected.

Specifically, the target defect detection model is a model for performing defect detection. When model training is carried out, a supervised training method is adopted. Inputting training pictures in the training samples into the model to obtain defect types predicted by the model, calculating according to the difference between the defect types predicted by the model and the actual defect types of the training pictures to obtain a loss value, and adjusting model parameters towards the direction of reducing the loss value by using a gradient descent method until the model converges to obtain a target detection model. The model convergence may include at least one of a loss value of the model being less than a preset value or a number of training times reaching a preset number. The defect detection result of the product to be detected may be whether the product includes a defect. In some embodiments, a category of defects may be included. The product to be detected is a product needing defect detection.

The product defect detection data processing method provided by the embodiment can be used for carrying out model training to obtain a product defect detection model, and detecting the defects of the product based on the product defect detection model, so that the efficiency is high. Moreover, the model training parameters are carried by the product defect detection model generation request and are determined according to the parameter configuration operation of the user, and the parameters comprise at least one of defect learning categories or training picture information, so that the user can configure the training parameters according to the actual product defect detection requirement, the product defect detection model obtained by model training meets the production requirement, the product defects can be automatically detected, and the method is high in flexibility and detection efficiency.

The method for how to train the model is configured on the product defect detection data processing system, so that the complex model training process can be simplified, and when a model generation request is received, the model training can be performed according to the model training method, so that a production user can configure at least one of defect learning categories or training picture information according to production needs under the condition that the production user does not have related machine learning professional knowledge, and a model training node can automatically perform model training according to the training parameters of the model by adopting the configured model training method to generate a defect detection model, thereby reducing the threshold of the production user for performing defect detection by using the model and improving the generation efficiency of the model. For example, the product defect detection data processing system may be configured with a structure of the model, initial values of model parameters, a function for calculating loss values, rules for adjusting model parameters according to the loss values during model training, and model convergence conditions. And when the training picture is obtained, the training picture is automatically input into the model to obtain a predicted defect, the system calculates a loss value according to the predicted defect, the actual defect and a loss value calculation function, and the model parameters are adjusted according to the loss value and the rule for adjusting the model parameters. The rule for adjusting the parameters of the model may include at least one of a step size or a learning rate for each adjustment of the parameters of the model. The steps of automatically inputting the training pictures into the model to obtain predicted defects, calculating by the system according to the predicted defects, the actual defects and the loss value calculation function to obtain loss values, and adjusting the model parameters according to the loss values and the rules of adjusting the model parameters can be repeated for multiple times until the model converges. Therefore, the configured model training method is equivalent to a model of the model, and the model suitable for different service scenes (different production requirements) can be trained by inputting at least one of different defect learning categories or training picture information through the model.

In some embodiments, the system may already include a defect detection model, and thus may continue training on the original defect detection model. For example, hidden layer parameters in the model may be randomly set, and model training may be performed based on the hidden layer parameters that are randomly set. The parameter configuration interface may also configure whether to continue training an existing model or retrain a model. Therefore, the model can be created from the beginning, and the parameters of the existing model can be updated, namely the model can be continuously updated iteratively through new training pictures along with the change of production.

In some embodiments, when the system has deployed the defect detection model, the deployed defect detection model is also trained according to the pictures. Therefore, the option information corresponding to the historical pictures and the new pictures respectively can be displayed on the production terminal. The user can select at least one of the historical pictures and the new pictures as the pictures for the model training at this time. The historical picture is a training picture used in training an existing defect detection model. The new picture is a picture other than the historical picture, and for example, a picture obtained by shooting a product when the product is produced after the historical picture. When the user selects the historical picture and the new picture for training, the trained model can learn to obtain parameters for detecting the latest defect and can also keep the detection capability of the existing defect.

In some embodiments, the obtaining of the training sample includes: the method comprises the steps of obtaining training pictures corresponding to defect learning categories and defect categories corresponding to the training pictures respectively to obtain training samples, wherein the target defect detection model takes the defect learning categories as candidate defect categories, and the target defect categories of a product to be detected are screened out from the candidate defect categories according to the pictures to be detected.

Specifically, when the model training parameters include a defect learning type, a training picture corresponding to the defect learning type and a defect type corresponding to the training picture may be obtained. Therefore, the defect learning category in the obtained target defect detection model is used as a candidate defect category, after a to-be-detected picture obtained by shooting a to-be-detected product is obtained, the to-be-detected picture can be input into the target defect detection model, the target defect detection model can output the probability corresponding to each candidate defect category, and the defect category with the highest probability is used as the target defect category.

In some embodiments, the request for generating the product defect detection model further carries product related information corresponding to the model, and the method for processing the product defect detection data further includes: and establishing a corresponding relation between the target defect detection model and the product related information according to the product related information corresponding to the model.

Specifically, the product-related information is information related to a product. Such as the time of manufacture of the product, the lot number of the product, etc. The product related information may be configured on a product information configuration interface. The product information configuration interface and the model parameter configuration interface can be the same interface or different interfaces. And when the product defect detection model generation request also carries the product related information corresponding to the model, establishing the corresponding relation between the target defect detection model and the product related information.

In some embodiments, the product-related information may include at least one of process information, category information, or production environment information corresponding to the product to be tested. The process information represents a current manufacturing process of the product. A product may go through multiple manufacturing processes. For example, a camera includes three main components, a lens holder, a chip, and a substrate. The production process may include three processes of flip chip, heating after flip chip, and mounting the lens holder, and the defect detection of the product produced by the flip chip process may be required in the flip chip process. flip chip refers to a flip chip, mounting a chip in a substrate. The category information indicates a category of the product. The products can be sorted as desired. For example, products are classified according to manufacturing parameters. Different manufacturing parameters correspond to different categories. Or may be classified according to the function of the product. For example, liquid crystal panel products can be classified into twisted nematic panels and wide viewing angle panels. The production environment information is a parameter related to the production environment of the product, such as the plant area where the product is located, production line information, and the like. And can also include one or more of temperature, plant cleanliness, or humidity.

In the embodiment of the application, the corresponding relation between the target defect detection model and the relevant information of the product is established, so that the defect detection models corresponding to different service scenes can be distinguished. For example, different product categories, corresponding defects or model parameters may be different, and if the same model is used for defect detection, the defect accuracy is low. In a factory, the types of the produced products are generally multiple, the corresponding product related information may be continuously changed, and the production personnel need to flexibly determine a defect detection strategy according to the produced products, so that the corresponding defect detection models are different, and by establishing the corresponding relation between the target defect detection model and the product related information, the corresponding detection models can be pertinently utilized to detect the defects of the product pictures.

FIG. 3 is a schematic diagram of an interface for configuring product-related information in some embodiments. In fig. 3, a site represents a manufacturing process where a product is located, and a factory floor represents production environment information. Different plants represent different production environments. As shown in fig. 3, the configuration process of the product related information of different levels can be shown in the form of a tree diagram. When the mouse is hovered to any hierarchy, the corresponding operation flow can be activated, namely, the configuration process of the product related information of the hierarchy is activated. In FIG. 3, a factory floor belongs to a first hierarchy, and a factory floor may have one or more categories, i.e., different factories. Different plants may have one or more sites. Different sites may also have one or more categories of products. For example, in fig. 3, T01 represents a factory floor, and b001 and b002 represent factory differences. 3051. 3052, 3053 and 3054 denote sites. CP 001-CP 005 represent the product category. When the site name is clicked on, products may be added, removed, or the site deleted under the site. As shown in fig. 3, an industry configuration interface may also be displayed on the production terminal, for configuring an industry corresponding to the product. A task type configuration interface and a computing resource configuration interface may also be included. The task type configuration interface is used for configuring the types of tasks, and the tasks can include the types of dividing pictures, classifying the pictures, dividing the pictures and the like. The computing resource configuration interface is configured to configure resources used in model training or resources used in model using, where the resources may be at least one of GPU (Graphics processing unit) or CPU (central processing unit) resources.

In some embodiments, as shown in fig. 4, the product defect detection data processing method may further include the steps of:

step S402, receiving a product defect detection task, wherein the product defect detection task carries the related information of the target product.

Specifically, the product defect detection task is used to trigger product defect detection. The target product related information refers to product related information of a product corresponding to the product defect detection task. The target product related information may include at least one of process information, category information, or production environment information corresponding to the product to be tested. The product defect detection task may be sent by the MES system.

In some embodiments, the product defect detection task may also carry a product identifier of the product to be detected and a picture identifier of a picture corresponding to the product identifier. Therefore, when the detection result is obtained according to the picture, the corresponding product identification can be obtained according to the picture identification, and the detection result and the corresponding product identification are returned. For example, the liquid crystal panel identifier and the corresponding picture identifier may be carried.

And S404, determining a target defect detection model corresponding to the related information of the target product according to the corresponding relation between the target defect detection model and the related information of the product.

Specifically, when a product defect detection task is received, a target defect detection model corresponding to the relevant information of the target product can be obtained according to the corresponding relationship between the target defect detection model and the relevant information of the product. For example, a plurality of target defect detection models are deployed on the system. Model 1 corresponds to process 1 and model 2 corresponds to process 2. When the product-related information includes the process 2, the model 2 is used as a target defect detection model.

In some embodiments, the defect detection task is real-time, and when the manufacturing device begins production, a picture is taken by a capture device on the manufacturing device and stored to a picture storage system. And the MES system sends a message to the product defect detection data processing system, namely a defect detection task. And the product defect detection data processing system determines a corresponding target defect detection model according to the product related information carried in the task.

Step S406, inputting the picture to be detected corresponding to the product defect detection task into a target defect detection model corresponding to the target product related information to obtain the target defect category.

Specifically, there may be one or more pictures to be measured of a product. For example, for a liquid crystal panel, since the liquid crystal panel is relatively large, a plurality of pictures can be taken to perform overall defect detection on the liquid crystal panel. The product defect detection task can carry the picture to be detected and also can carry the storage position information of the picture to be detected. Therefore, a picture to be detected corresponding to the product defect detection task can be obtained, the picture to be detected is input into a target defect detection model corresponding to the relevant information of the target product, and the model processes the picture to obtain a target defect type which is used as the defect type of the product to be detected corresponding to the picture.

In the embodiment of the application, the corresponding target defect detection model is obtained according to the relevant information of the product, and the target defect detection model corresponding to the relevant information of the product is utilized for carrying out model detection, so that the product detection is more targeted, and the refined detection is realized.

In some embodiments, the picture storage location information corresponding to the product to be tested, and the picture of the product may be pre-stored in the picture storage system. Inputting the picture to be detected corresponding to the product defect detection task into a target defect detection model corresponding to the target product related information, and obtaining the target defect category comprises the following steps: acquiring a corresponding target picture from a product picture storage node according to the picture storage position information, wherein the target picture is used as a picture of a product to be detected corresponding to the product defect detection task, and the product picture storage node is used for storing a picture obtained by shooting the product to be detected; and inputting the target picture into a target defect detection model corresponding to the relevant information of the target product to obtain the target defect category.

Specifically, the picture storage location information is used to indicate a location where the picture is stored, and may be, for example, a storage path of the picture. The picture obtained by shooting the product to be detected is stored in the picture storage system. When the product is shot, the product can be shot and transmitted to a picture storage system, and when the product needs to be detected, a product defect detection task is triggered and the corresponding picture storage position information of the product to be detected is carried. The detection task and the uploading of the pictures are respectively carried out, the pictures are stored in the picture storage nodes, and the picture storage positions are carried in the detection task, so that the data volume of the detection task can be reduced. After the picture corresponding to the picture storage position information is obtained, the product defect detection node can take the picture as a target picture, input the target picture into a target defect detection model corresponding to the target product related information, and obtain the defect type of the product corresponding to the picture, namely the target defect type.

In some embodiments, model training is performed based on training samples, and generating the target defect detection model includes: acquiring a defective area corresponding to each training picture as a candidate area, wherein the candidate area is obtained by screening from the training pictures according to the picture characteristics of the defect type corresponding to the training pictures; and performing model training according to the candidate area corresponding to each training picture and the corresponding defect type to generate a target defect detection model.

Specifically, the candidate region is a defective region, a picture feature corresponding to each defect type may be preset, and the picture feature is used to represent a characteristic of the picture, and may be the picture itself, or at least one of a color feature and a texture feature. The candidate picture can be segmented to obtain a plurality of image areas, similarity calculation is performed on each image area in the training picture based on the picture characteristics of the defect types, and the area with the similarity larger than the preset similarity is used as the candidate area. For example, the candidate region may be obtained by screening using a template matching method. The template matching refers to that for the template picture and the search picture, a region similar to the template picture is found in the search picture as a candidate region, and the similarity may refer to that the similarity is greater than a preset similarity. In the embodiment of the application, the search picture is a training picture, and the template picture is a picture corresponding to the defect type.

The defect class corresponding to the training picture is determined, for example, manually labeled. The picture characteristics corresponding to each defect type are preset. And after the candidate region is obtained, performing model training by taking the candidate region as the input of a model and the corresponding defect type as the expected output of the model, obtaining a loss value according to the difference between the output obtained by model prediction and the expected output of the model, and adjusting model parameters in the direction that the loss value becomes smaller to obtain the target defect detection model.

In the embodiment of the application, after the training picture is obtained, the candidate region with the defect is obtained by screening from the training picture according to the picture characteristics of the defect type corresponding to the training picture, and then the candidate region is utilized to carry out model training, so that the model obtained by training is more accurate. Generally, the training picture is larger, the defect is smaller, and the training picture also comprises a region without the defect besides the defect, so that a candidate region which can reflect the characteristics of the defect more effectively can be screened, and model training can be performed by using the candidate region, so that more accurate model parameters can be learned. Moreover, the candidate region is automatically determined by computer equipment, so that an accurate model can be trained without manually marking the position of the defect in the picture, and the labor cost is reduced.

In some embodiments, the number of the products to be detected is multiple, the defect detection result includes a target defect type of the product to be detected, and the product defect detection data processing method may further include the following steps: the method comprises the steps of obtaining a processing mode corresponding to a target defect type of a product to be detected, classifying product identifications of the product to be detected according to the processing mode corresponding to the target defect type of the product to be detected to obtain a product identification set corresponding to each processing mode, and outputting the product identification set corresponding to each processing mode to a production terminal.

Specifically, the product identification is used to identify the product. The product identifier may be generated according to a preset rule. The identification is generated, for example, according to the production order of the products and the discharge position. The processing modes corresponding to different defect types are different. Some defective products need to be scrapped. Some defects can be further optimized, and the optimization mode can be different, for example, the optimization mode can be glue supplementing or dirt removing. For example, if the defect is a glass corner defect, the glass needs to be disposed of. The processing modes corresponding to the defect categories may be preset. The product identification set may include identifications corresponding to one or more products. For each processing mode, a product identification set can be corresponded. After the product identification set is obtained, the product identification set corresponding to each processing mode is output to the production terminal, so that production personnel can determine each product to be processed and the corresponding processing mode according to the product identification, and process the product according to the corresponding processing mode, and the method is convenient and efficient.

In some embodiments, the performance of the model may be evaluated by using a test set, the trained intermediate defect detection model may be tested according to the test sample set to obtain predicted defects corresponding to each test picture in the test sample set, a test picture with a defect identification error may be determined according to the predicted defects and actual defects corresponding to the test pictures, and the predicted defects and the actual defects corresponding to the test picture with the defect identification error may be output to a production terminal that sends a request for generating the product defect detection model. Therefore, the predicted defects and the actual defects corresponding to the test pictures with the wrong defect identification can be displayed on the production terminal. Therefore, a user can determine whether the pictures in the test set are marked wrongly according to the predicted defects and the actual defects corresponding to the test pictures with the wrong defect identification, so that production personnel can adjust the marking standard according to the marking condition, and the test pictures with the wrong defect identification can refer to pictures which are manually marked and are inconsistent with the output of the model.

In some embodiments, the image with the wrong label may be output, so that the production personnel re-labels the image, and performs the model training again by using the re-labeled image.

In some embodiments, a performance evaluation result obtained after the performance evaluation of the model by using the test set, such as an accuracy and a recall rate corresponding to the model, may also be output. The production terminal can display the accuracy and the recall rate corresponding to the model. So that the producer can know the training condition of the model. For example, as shown in fig. 5, the performance evaluation result of the model may be displayed in the production terminal. The performance evaluation result may be corresponding to each sequence, and a sequence may refer to a training picture corresponding to a defect type. When a sequence is clicked, sample information corresponding to the sequence may be displayed. For example, the misjudgment sample is information of a sample with a defect recognition error, the true type is a type of a true defect of the picture, and the estimation type is a type of a model output. The resolution of the training pictures may be 600 × 400.

In some embodiments, the following method provided by the present application is described by taking a product as a liquid crystal panel as an example, and includes the following steps:

1. and uploading the training pictures and the corresponding defect categories.

For example, in the production process of a liquid crystal panel, if a manufacturer finds that a new defect (hereinafter referred to as an X defect) occurs in the liquid crystal panel and needs to detect the X defect, a picture of the liquid crystal panel with the X defect may be obtained, the defect of the picture may be labeled, and the defect type corresponding to the picture may be labeled as X. And then uploading the pictures marked with the defect categories to a training picture storage system.

2. Receiving a product defect detection model generation request, wherein the product defect detection model generation request carries model training parameters, the model training parameters are determined according to parameter configuration operation of a user, and the model training parameters comprise at least one of defect learning categories or training picture information.

For example, a production user may add H as a new defect learning category and select a storage path of a picture on a parameter interface, and send a product defect detection model generation request to a model training node, where the model training parameters include the storage path of a training picture and the types of defects to be learned as H defects and B defects.

In some embodiments, the product defect detection model generation request also includes product-related information, such as an identification, e.g., name, of the D factory E site assuming that the H defect and the B defect were generated at the D factory E site.

3. And acquiring a corresponding training picture according to the model training parameters and acquiring a defect type corresponding to the training picture to obtain a training sample.

For example, a training picture corresponding to an H defect and a training picture corresponding to a B defect may be obtained according to the storage path. Of course, a picture without defects can be obtained as a training picture.

4. And carrying out model training according to the training samples to generate a target defect detection model.

For example, the target defect detection model is a model for detecting H defects and B defects of the liquid crystal panel.

5. And establishing a corresponding relation between the target defect detection model and the relevant information of the product.

For example, the target defect detection model corresponds to the product-related information dpower E site.

6. And receiving a product defect detection task, wherein the product defect detection task carries the relevant information of the target product, the product identification of the liquid crystal panel, the storage path of the picture to be detected and the identification of the picture.

For example, when a liquid crystal panel is produced in the D factory E site, the name of the D factory E site is carried in the product defect detection task.

7. And determining a target defect detection model corresponding to the relevant information of the target product according to the corresponding relation between the target defect detection model and the relevant information of the product.

For example, according to the name of the D factory E site, a corresponding target defect detection model of the D factory E site is obtained.

8. And inputting the picture to be detected corresponding to the product defect detection task into a target defect detection model corresponding to the target product related information to obtain the target defect category.

For example, the image corresponding to the identifier of the image may be obtained according to the storage path of the image to be detected to the position corresponding to the storage path, and the image may be input into the target defect detection model corresponding to the related information of the target product, so as to obtain the target defect type. For example, it can be obtained that the defect type corresponding to the picture 1 is an H defect, and the defect type corresponding to the picture 2 is a B defect.

9. The method comprises the steps of obtaining a processing mode corresponding to a target defect type of a product to be detected, classifying product identifications of the product to be detected according to the processing mode corresponding to the target defect type of the product to be detected to obtain a product identification set corresponding to each processing mode, and outputting the product identification set corresponding to each processing mode to a production terminal.

Specifically, it is assumed that the processing method corresponding to the H defect is reject, and the processing method corresponding to the B defect is repair. Then the product identifier corresponding to the H defect can be obtained to form a product identifier set corresponding to the H defect. And acquiring product identifications corresponding to the B defects to form a product identification set corresponding to the B defects.

The product defect detection data processing method can be used for detecting defects of products produced in factories. The method provided by the embodiment of the application can package a complex model training method, and at least one of the defect learning categories or training picture information related to model training supports a user to flexibly configure according to needs, so that personnel in a factory can configure parameters according to production needs at any time, and realize rapid model construction from 0 to 1 and from 1 to N to obtain a product defect detection model meeting the needs, wherein N refers to the version of the model.

In some embodiments, as shown in FIG. 6, a product defect detection data processing system is provided, comprising a model training node 130 and a product defect detection node 120.

A model training node 130 for receiving a request for generating a product defect detection model, the request carrying model training parameters, obtaining corresponding training pictures according to the model training parameters and obtaining defect types corresponding to the training pictures to obtain training samples, performing model training according to the training samples to generate a target defect detection model, the model training parameters being determined according to parameter configuration operations of a user, the model training parameters including at least one of defect learning types or training picture information,

and the product defect detection node 120 acquires a to-be-detected picture corresponding to the to-be-detected product according to the product defect detection task, and puts the to-be-detected picture into the target defect detection model to obtain a defect detection result of the to-be-detected product.

In some embodiments, the product defect detection model generation request further carries product related information corresponding to the model; and the model training node 130 establishes a corresponding relationship between the target defect detection model and the product-related information according to the product-related information corresponding to the model, wherein the product-related information corresponding to the model includes at least one of process information, category information, production time or production environment information corresponding to the product to be detected.

In some embodiments, the product defect detection node 120 receives a product defect detection task, where the product defect detection task carries target product related information, determines a target defect detection model corresponding to the target product related information according to a correspondence between the target defect detection model and the product related information, and inputs a to-be-detected picture corresponding to the product defect detection task into the target defect detection model corresponding to the target product related information to obtain a target defect category.

In some embodiments, the number of the products to be detected is multiple, the defect detection result includes a target defect type of the product to be detected, the product defect detection data processing system further includes a service node 110, obtains a processing mode corresponding to the target defect type of the product to be detected, classifies the product identifier of the product to be detected according to the processing mode corresponding to the target defect type of the product to be detected, obtains a product identifier set corresponding to each processing mode, and outputs the product identifier set corresponding to each processing mode to the production terminal.

In some embodiments, the product defect detection data processing system may further include a picture storage system for storing a picture taken of the product. The picture storage system can comprise a training picture storage system and a picture storage system to be tested. The training picture storage system is used for storing test pictures and training pictures required by training. The picture storage system to be tested is used for storing pictures corresponding to products to be tested.

In some embodiments, as shown in fig. 7, which is an architecture diagram of a product defect detection data processing system in some embodiments, as shown in fig. 7, the product defect detection data processing system includes a service system (corresponding to the service node of fig. 1), a product defect detection system (corresponding to the product defect detection node of fig. 1), (corresponding to the model training node of fig. 1), a picture storage system, and a picture labeling system. Each system can be decoupled with each other in a micro-service mode, so that the deployment cost is reduced, and the maintainability of the system is improved. Of course, the specific architecture is not so limited, and adjustments may be made in the actual customer deployment and the corresponding functionality customized. For example, a product defect detection system and a model training system may be included. The various systems can be connected through HTTP/TCP communication. The basic idea of microservices is to consider creating applications around business domain components that can be developed, managed and accelerated independently. The use of microservice cloud architectures and platforms in decentralized components makes deployment, management, and service function delivery simpler. An API (Application programming interface) is a predefined function. The alarm processing may be to give an alarm when the defect result satisfies a preset condition, where the preset condition may be that the number of defective products is greater than a preset value or that the defect rate is greater than a preset defect rate. Subscription push may push subscribed information to the terminal. Automatic bad classification may be based on treatment.

The service system is a whole set of automatic defect detection system, provides a management and control console interface based on Web (network), can be deployed according to a factory, and can undertake tasks such as real-time monitoring, scheduling, distribution, MES integration, equipment operation data distribution and detection result distribution of a defect detection task corresponding to the factory; the service system can be directly connected with an intelligent manufacturing system in a factory through an FTP (File transfer protocol) to perform detection tasks and message exchange, so that perfect connection between an artificial intelligent model and an information system at a factory end is realized. The service system can also provide the functions of model version management, calling and configuring custom defect calculation rules and load balancing the requests of the AI inference system. The defect calculation rule is a correspondence between a processing method and a condition to be satisfied by a defect. For example, for a certain defect, it may be set that the defect is repaired when the defect satisfies the condition a, and the defect is scrapped when the defect satisfies the condition B. As a specific example, the dirt may need to be discarded when the dirt area is larger than the first threshold, and may be cleaned when the dirt area is smaller than the second threshold. Since the defect detection system may include a plurality of devices and the defect detection task may also include a plurality of services, the devices of the defect detection system may be polled to determine whether the defect detection task may be provided, so as to implement load balancing of the defect detection system.

As shown in fig. 8, the service system may provide management of different models, site ordering rules, processing mode determination rule configuration, real-time data statistics tools, and re-judgment tool components. The processing mode determination rule is configured to determine a processing mode of the product. The data statistics component can count the online data report conditions in real time, and check the different types of defect proportion conditions of the images obtained by shooting by using Automatic Optical Inspection (AOI) and the online effect of the model in real time. The picture to be detected can be a picture corresponding to a product with defects primarily judged by AOI. The 'order proposal management' is used for configuring a strategy document which determines whether a product information sheet needs to be opened for the relevant personnel to review according to different defects. The re-judgment tool component supports the pictures judged by the model to carry out secondary manual verification and re-judgment so as to confirm the real processing effect of the model. The 'account number authority' is used for configuring different account number authorities. Automated optical inspection equipment is equipment that is based on optical principles to detect defects encountered in manufacturing processes.

In some embodiments, the service system provides statistical analysis of tasks corresponding to systems such as a data large screen of the defect detection system and the monitoring system. The data is visualized through a data large screen. For example, the product quantity of each defect type in the detection result is counted and displayed according to a preset rule.

The image marking system can provide image marking service, wherein image marking can be semi-automatic, defect types of products can be marked manually, and then candidate areas with defects are determined by the image marking system. And model training is carried out according to the candidate region, so that the labor labeling cost and the labeling period can be greatly reduced. The image marking system can also have a manual checking function, namely, the marked image can be returned to the terminal for manual checking, model training is carried out by using the image with accurate marking, and the accuracy and the effectiveness of the training image are ensured.

The product defect detection system is a deployment platform of a trained target detection model, and can provide service encapsulation layers of a calculation engine, a defect detection model, calculation interface calling, GPU resource management and the like. Intelligent adjustment can be performed, load balancing is achieved, and high availability of service is achieved.

The product defect detection system is a platform for predicting defects by using a model, and the corresponding platform can comprise a service configuration management module and a micro-service management module. For the service configuration management module, a model service configuration can be created on a page for performing model service configuration, a bootable model service single instance can be configured and defined by packaging the model service with required computing resources (such as CPU and GPU resources), and the model service configuration can be launched to complete the deployment of the model service. For the micro-service management module, the service operation condition can be checked in the corresponding page. In the process of starting the service, the service range can be adjusted manually or automatically according to the service requirement, so that the service can be finely controlled and managed.

The model training system can provide functions of new model training, model iteration promotion, model on-line evaluation, question bank testing, model mirror image management and the like, so that a user can create a new defect detection model according to new data and historical data, label pictures with wrong judgment and retrain the model. The model mirroring function refers to a method in which model training is set. The model training can be performed by using a model training method according to model training parameters configured by a user.

The platform corresponding to the model training system may include seven modules of overview page display, user management, master data management, data set management, model mirroring, model library and online evaluation, and a login page may be provided before login, as shown in fig. 9. The model library may store a plurality of different models. Online evaluation is used to evaluate the performance of the model from the test data set. Can be expressed in terms of accuracy as well as recall. The question bank refers to a test sample, and model comparison can be performed by comparing the performances of different models, so that the optimal model is selected. The question bank data set refers to a test data set. Therefore, the model training system integrates data processing, model training, evaluation and prediction, and provides the full-process capability of model training. The page corresponding to the overview may be as shown in fig. 10, the main data module is a module for configuring product related information, and the industry main data refers to data related to industry.

The product defect detection data processing system has the advantages of good real-time performance and universality, high expandability, universality, industry and custom algorithm, capability of meeting requirements of different service scenes, capability of continuously improving the model precision through the model training system, high defect identification speed, high accuracy and high efficiency. The target defect detection model may be a deep learning neural network model, which is processed by a deep learning technique and combined with an image processing technique, such as an image segmentation technique. The automatic detection of product defects can be realized, and the labor cost is saved. And the waiting time of the semi-finished products in the production process can be reduced, so that the product yield per unit time is improved. Furthermore, the detection accuracy is high, so that the yield of each process can be improved, and the overall final yield is improved.

The system provided by the embodiment of the application can be applied to the manufacturing fields of liquid crystal panels, semiconductors, consumer electronics, automobiles, new energy sources (photovoltaic panels, power batteries and the like) and the like. For example, a mobile phone screen can be detected for defects.

In some embodiments, a flow chart for performing defect detection and model training may be as shown in FIG. 11. The numbers in the boxes in the squares represent steps in the model training flow, and the numbers in the boxes in the circles represent steps in the defect detection flow. As can be seen from fig. 11, the MES system may exchange messages with the service system through the message bus, the service system may parse the messages to obtain a product defect detection task, send a product defect detection request to the product defect detection system in a http (HyperText Transfer Protocol) request manner, and the product defect detection system obtains a corresponding picture from the product picture storage system storing the produced product to perform defect detection, and obtains a detection result and returns the detection result to the service system. The service system classifies the product identification of the product to be detected according to the processing mode (business rule) corresponding to the target defect category to obtain a product identification set corresponding to each processing mode, and outputs the product identification set corresponding to the processing mode to the MES system. The service system can schedule a model in the product defect detection system, the product defect detection system can perform virtualization processing, and the picture storage system stores pictures and corresponding description files. The model training system can be used for newly building a model, can also be used for iteratively improving the model, and can be used for testing the model.

In the model training flow, the service system may add training pictures (training data sets) to the training picture storage system. Test pictures (test data sets) may also be added to the training picture storage system. Wherein the added test data set and training data set are configured by the user through the production terminal. The image marking system can output the image to a corresponding terminal, and the image is marked manually to obtain a label of the image, namely a defect type. And then automatically detecting the corresponding defective area of the obtained picture as a candidate area, thereby realizing semi-automatic labeling, and certainly outputting a training picture labeled with the candidate area, and manually checking to determine that the candidate area is the defective area. And the model training node performs model training and model performance evaluation according to the product defect detection model generation request, obtains a target detection model and deploys the target detection model to a product defect detection system.

As shown in fig. 12, which is a system architecture diagram corresponding to the product defect detection system in some embodiments, the product defect detection system is a cluster formed by a plurality of devices, the load balancing device performs load balancing processing on a received product defect detection request (also referred to as an inference request), the inference service node is used for providing a product defect detection service, and may include a plurality of inference service nodes, one inference service node may provide a plurality of inference services, and one inference service may be deployed with a product defect detection model. The management terminal manages the ke (kubernets engine) console and the inference service console, for example, sets related parameters. KE may provide a container-centric, highly scalable, high-performance container management service. The reasoning service background is used for issuing the tasks to an application program interface Server (API Server), and the application program interface Server has a scheduling function and can schedule the corresponding reasoning service model to provide product defect detection service. The API Server receives the request sent by the user and distributes the request according to the routing rule. The distributed storage system may be an etcd storage system, which is a key-value storage system used for configuration sharing and service discovery. The API Server may interact with the etcd system. The API Server may call a corresponding plug-in or component. For example, it may be a node component such as a plug-in corresponding to kubel or gpu (Graphics Processing Unit) or a docker. kubernets is a distributed cluster management system for life cycle management of containers. Docker refers to an application container engine. The database may be a MYSQL database. MySQL is a relational database management system.

FIG. 13 is a diagram illustrating a deployment architecture for a product defect detection system according to some embodiments. The product defect detection system may include a service providing node, a GPU management node, a storage system, and a KE management node. There may be more than one KE management node. The management user can manage the service providing node through the management terminal, and can complete the configuration of the parameters of the product defect detection system through the service providing node. The service providing node can correspond to a plurality of MYSQL databases and can perform containerization processing on the databases. The terminal corresponding to the production user can send a product defect detection request to the product defect detection system. One or more of the ke (kubernets engine) management node and the GPU management node may be provided, and are specifically set as required. The Storage system may be a NAS (network attached Storage) Storage system. The service providing node and the KE management node may be the same node. The storage system, the KE management node, and the GPU management node may interact through a switch. The service providing node is used for providing a product defect detection service, and the KE management node can provide a cloud container management service.

As shown in fig. 14, in some embodiments, a product defect inspection data processing apparatus is provided, which may be integrated in the product defect inspection data processing system described above, and specifically may include a generation request receiving module 1402, a training sample obtaining module 1404, and a model training module 1406.

A generation request receiving module 1402, configured to receive a request for generating a product defect detection model, where the request for generating the product defect detection model carries model training parameters, the model training parameters are determined according to parameter configuration operations of a user, and the model training parameters include at least one of defect learning categories or training picture information;

a training sample obtaining module 1404, configured to obtain a corresponding training picture according to the model training parameters and obtain a defect type corresponding to the training picture, so as to obtain a training sample;

the model training module 1406 is configured to perform model training according to the training samples to generate a target defect detection model, and the target defect detection model is configured to perform defect detection on the product to be detected according to the picture to be detected corresponding to the product to be detected to obtain a defect detection result of the product to be detected.

In some embodiments, the model training parameters include a defect learning category, and the training sample derivation module 1404 is configured to: the method comprises the steps of obtaining training pictures corresponding to defect learning categories and defect categories corresponding to the training pictures respectively to obtain training samples, wherein the target defect detection model takes the defect learning categories as candidate defect categories, and the target defect categories of a product to be detected are obtained by screening from the candidate defect categories according to the pictures to be detected.

In some embodiments, the product defect detection model generation request further carries product related information corresponding to the model, and the product defect detection data processing apparatus further includes: and the corresponding relation establishing module is used for establishing the corresponding relation between the target defect detection model and the product related information according to the product related information corresponding to the model.

In some embodiments, the product defect detection data processing apparatus further comprises:

the detection task receiving module is used for receiving a product defect detection task, and the product defect detection task carries relevant information of a target product;

the target defect detection model determining module is used for determining a target defect detection model corresponding to the relevant information of the target product according to the corresponding relation between the target defect detection model and the relevant information of the product;

and the target defect type obtaining module is used for inputting the to-be-detected picture corresponding to the product defect detection task into a target defect detection model corresponding to the target product related information to obtain a target defect type.

In some embodiments, the product-related information corresponding to the model includes at least one of process information, category information, or production environment information corresponding to the product to be tested.

In some embodiments, the product defect detection task further carries image storage location information corresponding to the product to be detected, and the target defect type obtaining module is configured to: acquiring a corresponding target picture from a product picture storage node according to the picture storage position information, wherein the target picture is used as a picture of a product to be detected corresponding to the product defect detection task, and the product picture storage node is used for storing a picture obtained by shooting the product to be detected; and inputting the target picture into a target defect detection model corresponding to the relevant information of the target product to obtain the target defect category.

In some embodiments, the model training module 1406 is configured to:

acquiring a defective area corresponding to each training picture as a candidate area, wherein the candidate area is obtained by screening from the training pictures according to the picture characteristics of the defect type corresponding to the training pictures;

and performing model training according to the candidate area corresponding to each training picture and the corresponding defect type to generate a target defect detection model.

In some embodiments, the number of the products to be tested is plural, the defect detection result includes a target defect type of the products to be tested, and the apparatus further includes: and the processing mode determining module is used for acquiring the processing modes corresponding to the target defect types of the products to be detected, classifying the product identifications of the products to be detected according to the processing modes corresponding to the target defect types, obtaining product identification sets corresponding to the processing modes, and outputting the product identification sets corresponding to the processing modes to the production terminal.

In some embodiments, the generate request receive module 1402 is to: and receiving a product defect detection model generation request sent by a production terminal corresponding to a user, wherein the model training parameters are configured in a parameter configuration interface displayed by the production terminal.

FIG. 15 is a diagram illustrating an internal structure of a computer device in some embodiments. The computer device may specifically be a node in fig. 1, such as model training node 130. As shown in fig. 15, the computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may further store a computer program which, when executed by the processor, causes the processor to implement the product defect detection data processing method. The internal memory may also have a computer program stored therein, which when executed by the processor, causes the processor to perform a method of processing product defect detection data.

Those skilled in the art will appreciate that the architecture shown in fig. 15 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In some embodiments, the product defect detection data processing apparatus provided by the present application may be implemented in the form of a computer program, which is executable on a computer device as shown in fig. 15. The memory of the computer device may store various program modules that make up the product defect detection data processing apparatus, such as a generation request receiving module 1402, a training sample obtaining module 1404, and a model training module 1406 shown in FIG. 14. The computer program constituted by the respective program modules causes the processor to execute the steps in the product defect detection data processing method of the respective embodiments of the present application described in the present specification.

For example, the computer device shown in fig. 14 may receive a product defect detection model generation request through the generation request receiving module 1402 in the product defect detection data processing apparatus shown in fig. 14, where the product defect detection model generation request carries model training parameters, the model training parameters are determined according to parameter configuration operations of a user, and the model training parameters include at least one of a defect learning category or training picture information; obtaining a corresponding training picture and a defect type corresponding to the training picture according to the model training parameters through a training sample obtaining module 1404 to obtain a training sample; model training is performed according to the training samples through the model training module 1406 to generate a target defect detection model, and the target defect detection model is used for performing defect detection on the product to be detected according to the picture to be detected corresponding to the product to be detected to obtain a defect detection result of the product to be detected.

In some embodiments, a computer device is provided, comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of the above-described product defect detection data processing method. Here, the steps of the product defect detection data processing method may be the steps in the product defect detection data processing methods of the respective embodiments described above.

In some embodiments, a computer readable storage medium is provided, storing a computer program which, when executed by a processor, causes the processor to perform the steps of the above-described product defect detection data processing method. Here, the steps of the product defect detection data processing method may be the steps in the product defect detection data processing methods of the respective embodiments described above.

It should be understood that, although the steps in the flowcharts of the embodiments of the present application are shown in sequence as indicated by the arrows, the steps are not necessarily performed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps in various embodiments may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware related to instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, the computer program can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. A method of processing product defect detection data, the method comprising:

acquiring a picture to be detected corresponding to a product to be detected;

inputting the picture to be detected into a target defect detection model to obtain a defect detection result of the product to be detected;

and storing the defect detection result in a block chain, and generating a result query code corresponding to the defect detection result, wherein the result query code is used for querying the defect detection result in the block chain.

2. The method of claim 1, wherein the generating the result query code corresponding to the defect detection result comprises:

and carrying out Hash calculation on the defect detection result to obtain a result inquiry code corresponding to the defect detection result.

3. The method of claim 1, further comprising:

receiving a defect detection result query request carrying the result query code;

responding to the defect detection result query request, and acquiring the defect detection result from the data block corresponding to the block chain according to the result query code;

and returning the defect detection result to the query end.

4. The method of claim 1, wherein the step of training the target defect detection model comprises:

receiving a product defect detection model generation request, wherein the product defect detection model generation request carries model training parameters, the model training parameters are determined according to parameter configuration operation of a user, and the model training parameters comprise at least one of defect learning categories or training picture information;

acquiring a corresponding training picture according to the model training parameters and acquiring a defect type corresponding to the training picture to obtain a training sample;

and carrying out model training according to the training sample to obtain the target defect detection model.

5. The method according to claim 4, wherein the model training parameters include defect learning categories, and the obtaining of the training samples according to the model training parameters and the obtaining of the defect categories corresponding to the training pictures comprises:

and acquiring training pictures corresponding to the defect learning categories and defect categories corresponding to the training pictures respectively to obtain training samples, wherein the target defect detection model takes the defect learning categories as candidate defect categories, and the target defect categories of the product to be detected are obtained by screening from the candidate defect categories according to the picture to be detected.

6. The method of claim 4, wherein the request for generating the product defect detection model further carries product-related information corresponding to the model, and the method further comprises:

and establishing a corresponding relation between the target defect detection model and the product related information according to the product related information corresponding to the model.

7. The method of claim 6, further comprising:

receiving a product defect detection task, wherein the product defect detection task carries relevant information of a target product;

and determining a target defect detection model corresponding to the relevant information of the target product according to the corresponding relation between the target defect detection model and the relevant information of the product.

8. The method of claim 6, wherein the product-related information corresponding to the model comprises at least one of process information, category information, or production environment information corresponding to the product to be tested.

9. The method of claim 4, wherein the performing model training according to the training samples to obtain the target defect detection model comprises:

acquiring a defective area corresponding to each training picture as a candidate area, wherein the candidate area is obtained by screening from the training pictures according to picture characteristics of defect categories corresponding to the training pictures;

and performing model training according to the candidate area corresponding to each training picture and the corresponding defect type to obtain the target defect detection model.

10. The method of claim 4, wherein the product under test is a plurality of products under test, the defect detection result comprises a target defect category of the product under test, and the method further comprises:

and acquiring a processing mode corresponding to the target defect type of the product to be detected, classifying the product identification of the product to be detected according to the processing mode corresponding to the target defect type to obtain a product identification set corresponding to each processing mode, and outputting the product identification set corresponding to each processing mode to a production terminal.

11. The method of claim 4, wherein receiving a product defect detection model generation request comprises:

and receiving a product defect detection model generation request sent by a production terminal corresponding to a user, wherein the model training parameters are configured in a parameter configuration interface displayed by the production terminal by the user.

12. A product defect detection data processing system, the system comprising:

the model training node receives a product defect detection model generation request, the product defect detection model generation request carries model training parameters, corresponding training pictures are obtained according to the model training parameters, defect types corresponding to the training pictures are obtained, training samples are obtained, model training is carried out according to the training samples, a target defect detection model is generated, the model training parameters are determined according to parameter configuration operation of a user, and the model training parameters comprise at least one of defect learning types or training picture information;

the product defect detection node acquires a to-be-detected picture corresponding to a to-be-detected product, inputs the to-be-detected picture into a target defect detection model to obtain a defect detection result of the to-be-detected product, stores the defect detection result in a block chain, and generates a result query code corresponding to the defect detection result, wherein the result query code is used for querying the defect detection result in the block chain.

13. A product defect detection data processing apparatus, the apparatus comprising:

the to-be-detected picture acquisition module is used for acquiring a to-be-detected picture corresponding to a to-be-detected product;

the input module is used for inputting the picture to be detected into a target defect detection model to obtain a defect detection result of the product to be detected;

and the storage module is used for storing the defect detection result in a block chain and generating a result query code corresponding to the defect detection result, wherein the result query code is used for querying the defect detection result in the block chain.

14. A computer arrangement, comprising a memory and a processor, the memory having stored therein a computer program which, when executed by the processor, causes the processor to carry out the steps of the product defect detection data processing method of any one of claims 1 to 11.

15. A computer-readable storage medium, having stored thereon a computer program which, when executed by a processor, causes the processor to carry out the steps of the method for processing product defect detection data according to any of claims 1 to 11.