WO2021120181A1 - 推理计算装置、模型训练装置、推理计算系统 - Google Patents

推理计算装置、模型训练装置、推理计算系统 Download PDF

Info

Publication number
WO2021120181A1
WO2021120181A1 PCT/CN2019/127048 CN2019127048W WO2021120181A1 WO 2021120181 A1 WO2021120181 A1 WO 2021120181A1 CN 2019127048 W CN2019127048 W CN 2019127048W WO 2021120181 A1 WO2021120181 A1 WO 2021120181A1
Authority
WO
WIPO (PCT)
Prior art keywords
model
inference
training
reasoning
calculation device
Prior art date
Application number
PCT/CN2019/127048
Other languages
English (en)
French (fr)
Inventor
李昭月
柴栋
路元元
王洪
Original Assignee
京东方科技集团股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 京东方科技集团股份有限公司 filed Critical 京东方科技集团股份有限公司
Priority to CN201980003145.8A priority Critical patent/CN113597618A/zh
Priority to JP2021571382A priority patent/JP7442550B2/ja
Priority to US17/044,276 priority patent/US20210209488A1/en
Priority to KR1020227001528A priority patent/KR20220117194A/ko
Priority to PCT/CN2019/127048 priority patent/WO2021120181A1/zh
Priority to EP19955399.1A priority patent/EP4080407A4/en
Publication of WO2021120181A1 publication Critical patent/WO2021120181A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/04—Inference or reasoning models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • This application relates to the field of cloud computing technology, in particular to a reasoning calculation device, a model training device, and a reasoning calculation system.
  • Cloud computing technology has the characteristics of strong computing power, high availability, and high scalability.
  • applications in cloud computing are deployed on cloud devices, and users can obtain application services from cloud devices on demand, which improves the convenience for users to obtain application services. Performance, and improve the overall utilization rate of computing resources.
  • an inference calculation device which includes at least one processor and a memory; the memory stores program instructions.
  • the program instructions may be executed by the at least one processor to cause the inference calculation device to perform the following operations: receive a first inference model from the model training device, wherein the first inference model is based on a first training sample library, and The model training device is obtained through model training; the first training sample library includes training samples from historical data generated in the manufacturing stage; the model training device includes cloud equipment; and manufacturing is performed based on the first inference model
  • the inference calculation of the to-be-processed data generated in the stage, and the inference result is obtained, and the inference result is sent to the user-side device, and the inference calculation device is closer to the user-side device than the model training device; evaluates the first inference model To determine whether the first inference model needs to be updated, and if so, update the first inference model.
  • the historical data includes product images marked with defective information content; the to-be-processed data includes original product images generated in the manufacturing stage; the inference result includes identifying the original product images Defect type information.
  • updating the first inference model includes: performing model training based on the second training sample library to obtain the second inference model; or, sending a model update request to the model training device to obtain the first inference model.
  • the second training sample library includes training samples from historical data and/or training samples from the inference results that have undergone re-judgment. When the second inference model satisfies the update condition, the first inference model is updated with the second inference model.
  • the update condition includes: testing the second inference model, and the second inference model passes the test.
  • the test includes evaluating the performance of the second reasoning model based on test samples, and if the performance meets the evaluation requirements, it is determined to pass the test.
  • the update condition further includes: performing gray-scale deployment of the second inference model, and evaluating its performance during the gray-scale deployment process, and the performance of the second inference model conforms to the evaluation Claim.
  • the update condition further includes: verifying the configuration information of the second inference model, and the configuration information matches the inference calculation to be performed.
  • the inference calculation device before performing model training to obtain the second inference model; or before sending a model update request to the model training device to obtain the second inference model, the inference calculation device further performs the following operations: judging the model being performed Whether the training parameters required for training are within the set training parameter range; if so, perform the model training; if not, send the model update request to the model training device.
  • the training parameters include: at least one of the data size required in the model training, the training duration, and the computing ability.
  • the evaluation parameters used to evaluate the performance of the inference model include: at least one of the accuracy, precision, recall, and F score of the inference model in the inference calculation process.
  • the situation where the first reasoning model needs to be updated according to the performance evaluation includes: performance fluctuation or performance degradation of the first reasoning model, wherein, according to the change of the evaluation parameter in a continuous period of time, the first reasoning model is judged Whether the inference model has performance fluctuations or performance degradation.
  • the condition that the first inference model needs to be updated according to the performance evaluation includes: receiving a third inference model, where the third inference model is that the model training device has not received the model update request In the case of the model training based on the updated first training sample library, the updated first training sample library includes the training samples from the inference result that have undergone re-judgment. Compare the performance of the first inference model and the third inference model, if the performance of the third inference model is higher than that of the first inference model, or if the first inference model is abnormal, use all The third reasoning model updates the first reasoning model model.
  • the inference calculation device in the case of updating the first inference model with the second inference model, the inference calculation device further performs the following operations: perform performance evaluation on the second inference model, and compare the first inference model.
  • the performance of a reasoning model and the second reasoning model in the case that the performance of the second reasoning model is worse than the performance of the first reasoning model, the second reasoning model is rolled back to the first reasoning model Reasoning model; re-do model training, or re-request the model training device to perform model training.
  • the inference calculation device further performs the following operations: according to the inference result, a decision instruction is generated.
  • the defect information includes: the identified original product image with defects, its defect location and defect type;
  • the decision instruction includes: according to the defect information of the original product image, the original product with defects The product corresponding to the image undergoes corresponding defect processing.
  • a model training device which includes at least one processor and a memory; the memory stores program instructions, and the program instructions can be executed by the at least one processor, so that the model training device performs the following operations: Model training is performed based on the first training sample library to obtain the first inference model; the first training sample library includes training samples from historical data generated in the manufacturing stage. Sending the first inference model to the inference calculation device; in the case of receiving a model update request from the inference calculation device, perform model training based on the updated first training sample library to obtain a second inference model; The updated first training sample library includes re-judged training samples from the inference result of the first inference model. The second inference model is sent to the inference calculation device.
  • the model training device performs the following operations: generating training task information; The training task information is described to generate training configuration data; training samples are obtained from the corresponding training sample library, and based on the training samples and the training configuration data, model training is performed to obtain a corresponding inference model.
  • the model training device in the process of obtaining training samples from the corresponding training sample library, performing model training based on the training samples and the training configuration data, and obtaining the corresponding inference model, the model training device further Perform the following operations: monitor the process of model training; update the training configuration data when an abnormality occurs in the process of model training; perform model training based on the training samples and the updated training configuration data.
  • the model training device before sending the first inference model or the second inference model to the inference calculation device, the model training device further performs the following operations: perform the following operations on the model version information of the corresponding inference model Check to determine whether the model version information is correct, if so, send the corresponding reasoning model to the reasoning calculation device, if not, find the corresponding reasoning model whose model version meets the requirements; It is sent to the inference calculation device, or, if it is not found, notification information is sent to the inference calculation device.
  • the model training device further includes the first training sample library, and the model training device further performs the following operations: updating the first training sample library; and when the model update request is not received In the case of performing model training based on the updated first training sample library, a third inference model is obtained; and the third inference model is sent to the inference calculation device.
  • a computer-readable storage medium is provided, and program instructions are stored; the program instructions can be executed by a processor to cause the inference calculation device to perform the following operations: receive a first inference model, the first inference model Based on the first training sample library, obtained through model training; the first training sample library includes training samples from historical data. Based on the first reasoning model, the reasoning calculation of the to-be-processed data in industrial production is performed, and the reasoning result is obtained. The performance of the first reasoning model is evaluated to determine whether the first reasoning model needs to be updated, and if so, the first reasoning model is updated.
  • a computer-readable storage medium which stores program instructions; the program instructions can be executed by a processor to cause the model training device to perform the following operations: perform model training based on a first training sample library, and obtain the first training sample library.
  • An inference model ; the first training sample library includes training samples from historical data.
  • the first inference model is sent to the inference calculation device.
  • perform model training based on the updated first training sample library to obtain a second inference model the updated first training sample library is used from the The inference result of the first inference model and the re-judged training samples are obtained by updating the first training sample library.
  • the second inference model is sent to the inference calculation device.
  • a reasoning calculation system including: the reasoning calculation device described in any of the above embodiments, and the model training device described in any of the above embodiments.
  • Figure 1 is a schematic structural diagram of a user-side cloud computing architecture according to related technologies
  • Figure 2 is a schematic structural diagram of a cloud user side-edge side-cloud computing architecture according to related technologies
  • Fig. 3 is a schematic structural diagram of an inference calculation device according to some embodiments of the present disclosure.
  • FIG. 4 is a schematic diagram of interaction between an inference calculation device and a model training device according to some embodiments of the present disclosure
  • Fig. 5 is a schematic structural diagram of a model training device according to some embodiments of the present disclosure.
  • Fig. 6 is a schematic diagram of interaction between another inference calculation device and a model training device according to some embodiments of the present disclosure
  • Fig. 7 is a schematic diagram of interaction between yet another inference calculation device and a model training device according to some embodiments of the present disclosure
  • Fig. 8 is a flowchart of the operation steps of a model training device according to some embodiments of the present disclosure.
  • FIG. 9 is a flowchart of the operation steps of another model training device according to some embodiments of the present disclosure.
  • Fig. 10 is a schematic structural diagram of an inference calculation system according to some embodiments of the present disclosure.
  • Fig. 11 is a structural diagram of an inference calculation device and a structural diagram of a model training device according to some embodiments of the present disclosure.
  • Machine learning algorithms include support vector machines (SVM), cascade algorithms (Adaboost), decision trees, random forests, and deep learning algorithms based on neural networks.
  • Cloud Computing is a type of distributed computing. Cloud computing can decompose data computing processing programs into multiple small computing processing programs through the network "cloud". It has the characteristics of strong computing power, high availability, and high scalability. . During the production process of the factory, a large amount of production data will be generated. Cloud computing can train an inference model for processing this large amount of production data, and perform inference calculations on the production data through the inference model.
  • a system composed of multiple servers that is, The cloud device
  • performs inference calculation on the data to be processed uploaded by the user-side device that is, the production equipment in the factory
  • analyzes the inference result to obtain the cloud decision result
  • sends the cloud decision result to the user side Equipment which can realize the Internet of Things in the production process.
  • the aforementioned computing architecture is referred to as the user-side cloud computing architecture.
  • the user-side cloud computing architecture is exemplarily introduced.
  • the user-side device 3' uploads the data to be processed (that is, the production data generated by the production equipment) to the cloud device 2', and the cloud device 2'performs inference calculations on the uploaded data to be processed, and obtains the decision result, and then the decision result Send back to the user side device 3'.
  • the user-side device 3' takes corresponding actions according to the received decision result, so that the unified management of the user-side device 3'can be realized.
  • the client-side device 3' uploads a large amount of data to be processed.
  • the network bandwidth and network management authority used for uploading will cause greater restrictions on the upload speed, resulting in the timeliness of data processing. Poor sex.
  • a large amount of to-be-processed data needs to be stored first after being uploaded to the cloud device 3', which will occupy more resources of the cloud device 3'and bring a greater burden to the cloud device.
  • the user-side-edge-cloud computing architecture can be used to process the production data generated during the production preparation process.
  • an inference calculation device 100 which includes at least one processor 110 and a memory 120.
  • the memory 120 stores program instructions.
  • the program instruction may be executed by the at least one processor 110 to cause the inference calculation device 100 to perform the following operations S10 to S40:
  • the first inference model is based on a first training sample library, which is obtained by the model training device 200 through model training.
  • the first training sample database includes training samples from historical data generated during the manufacturing stage.
  • the model training device 200 includes cloud equipment. . That is, the model training apparatus 200 may be a cloud device.
  • the model training device 200 trains the neural network model, which mainly includes the following steps: selecting the network topology; using a set of training data representing the problem being modeled by the network; and adjusting the weights until the network model targets the training data set All instances of is shown to have the smallest error. For example, during a supervised learning training process for a neural network, the output produced by the network in response to an input representing an example in the training data set is compared with the "correct" labeled output of the example; the calculation represents the output The error signal of the difference from the marked output; and when the error signal is propagated backwards through the layers of the network, the weights associated with the connection are adjusted to minimize the error. When the error of each output generated from the instance of the training data set is minimized, the neural network model is considered "trained” and can be used for artificial intelligence inference tasks.
  • GPU Graphics Processing Unit
  • the inference model can be a neural network model, for example, a feedforward neural network model, a convolutional neural network (Convolutional Neural Network, referred to as CNN) model, a recurrent neural network (RNN, Recurrent Neural Network) model, a generative confrontation network (GAN, Generative Adversarial Network model, but not limited to this, other neural network models known to those skilled in the art can also be used.
  • a feedforward neural network model for example, a feedforward neural network model, a convolutional neural network (Convolutional Neural Network, referred to as CNN) model, a recurrent neural network (RNN, Recurrent Neural Network) model, a generative confrontation network (GAN, Generative Adversarial Network model, but not limited to this, other neural network models known to those skilled in the art can also be used.
  • CNN convolutional Neural Network
  • RNN Recurrent Neural Network
  • GAN Generative Adversarial Network model
  • the inference calculation device 100 is closer to the model training device 200 than the model training device 200 User side equipment.
  • the inference computing device 100 is the edge-side device 1'
  • the model training device 200 is the cloud device 2'
  • the deployment position of the edge-side device 1' is between the user-side device 3'and the cloud device 2'.
  • the edge-side device 1 uses the inference model to perform inference calculations on the data to be processed from the user-side device 3', and obtains the edge-side decision result based on this, and then combines the edge-side decision result Send to the user-side device 3'; or, the edge-side device 1'uploads the calculation result after the inference calculation to the cloud device 2', and the cloud device 2'processes and analyzes the uploaded calculation result to obtain the cloud decision result, and then The cloud decision result is sent to the user side device 3'.
  • the edge-side device 1' can be integrated into the user-side device 3', that is, the user-side device 3'will generate production data and at the same time perform inference calculations on the generated production data.
  • the edge-side device 1' is closer to the user-side device 3'. Therefore, the transmission of the to-be-processed data consumes relatively less time, which increases the timeliness of data processing.
  • the resources of the cloud device 2' are less occupied, thereby reducing the burden of the cloud device.
  • the cloud device 2' can make full use of its powerful computing capabilities to perform model training without real-time feedback decision-making.
  • the inference model in the edge device 1' refers to the algorithm structure that uses a certain algorithm (such as machine learning algorithm) for inference calculation.
  • model training is required to meet the inference calculation requirements of the edge device 1' Reasoning model.
  • model training refers to the extraction of specific features of these training samples through multiple iterations and calculations of a large number of training samples, and finally obtain the algorithm structure and parameter values that can converge on the data set with the specific features, etc.
  • an inference model that can identify data with specific characteristics is obtained.
  • training data set samples and verification data sets used for model training and a test data set used to test the trained model.
  • All three include original image samples and corresponding labeled information.
  • the original image samples in the three can completely overlap, have an intersection, or even have no intersection at all.
  • the training data set, the verification data set, and the test data set completely overlap, have an intersection, or even have no intersection at all. No intersection at all is an ideal situation, so the trained training model is relatively reliable.
  • the display panel production process it is necessary to take pictures of the display panel after each process is completed to obtain the image of the display panel (such as the image of the array substrate).
  • the image of the display panel such as the image of the array substrate.
  • Display the defect information of the panel mark the display panel with defects and its defect information, and then repair them according to different defects using corresponding repair processes.
  • Image recognition can realize computer vision recognition through the neural network model of deep learning.
  • the inference model is sent to the edge-side device 2', and the image generated by the user side 3'directly passes through the edge-side device 2' , Quickly make inference calculations, eliminating the bandwidth pressure and delay of transmission to cloud computing.
  • the method of recognizing the defect content in the image through the neural network model is as follows. First, the product image is scaled to a fixed pixel size of M ⁇ N (or may not be scaled), and then the M ⁇ N image is sent to the deep convolutional nerve.
  • Networks such as VGGNet, ResNet, MobileNet, etc.; through multi-layer convolutional layer, activation layer, pooling layer, the feature map of the entire image (feature maps) is obtained; the feature map is input to the screening area network (ZF/SSD/RPN) Etc.), after calculation, the proposal region (proposal region) is obtained; then, operations such as convolution pooling are performed on the proposal region to obtain the proposal feature of the proposed region, and then the proposal feature map is sent into Subsequent full connection and softmax network are used for classification (classification is what kind of defect is the classification proposal), the defect category with the largest probability is obtained as the final classification result, and the category and probability are recorded.
  • the coordinates and size of the proposal region represent the location and size of the defect.
  • the method for identifying the content of product defects based on the defect model can adopt similar variations of the above methods or other methods known to those skilled in the art, and the present disclosure is not limited herein.
  • the inference model requires a large number of training samples to be obtained after model training.
  • the training sample can be regarded as a data set formed by at least one piece of data, and the training sample may include various forms of data.
  • the training data may include images, videos, audios, etc., which are not specifically limited here.
  • a large number of training samples used to train the first inference model are from the first training sample library, and the training samples in the first training sample library include training samples from historical data.
  • historical data refers to production data that has been judged and marked with a correct result.
  • the production data is judged by manual judgment or other equipment capable of correct judgment to obtain the aforementioned historical data. Since historical data has been marked with correct results, it can be used as a training sample in model training.
  • the to-be-processed data to be processed by the first reasoning model includes the original product image generated in the manufacturing stage, and the reasoning result of the first reasoning model includes defect information in the original product image .
  • the original product image is an unprocessed image
  • the first inference model needs to perform inference calculations on the original product image to obtain defect information in the original product image, for example, to identify the defect location and defect type.
  • the historical data as described above includes product images marked with a defect type, and here, the marked defect type is a correct result.
  • the inference calculation device 100 evaluates the performance of the first inference model, it compares the inference result obtained by the inference calculation of the first inference model with the correct result, so as to realize the performance evaluation of the first inference model.
  • the correct result may be input to the inference calculation device 100 after manual judgment.
  • the staff judges the original product image of the display screen on the production line within a fixed period of time (for example, 14:00-15:00 every day) to determine defective products
  • the original display screen image is marked with the defect type, and then the result of manual judgment is input to the inference calculation device 100 as the correct result.
  • a fixed number of original product images are randomly selected for manual judgment to obtain the result of manual judgment.
  • the original product image of the display screen of the specified batch is manually judged to obtain the result of the manual judgment.
  • S30 is performed to evaluate the performance of the first inference model.
  • the reasoning calculation device 100 by performing performance evaluation on the first reasoning model, it can be known whether the performance of the first reasoning model meets the current reasoning requirements, or the performance of the first reasoning model can be evaluated according to the performance evaluation result. Pre-judge the trend of change in order to determine whether the first reasoning model needs to be updated.
  • Update the first reasoning model When it is known that the performance of the first reasoning model does not meet the current reasoning requirements, or the performance of the first reasoning model will not meet the reasoning requirements through pre-judgment, that is, when it is determined that the first reasoning model needs to be updated , Update the first reasoning model.
  • the performance of the first inference model can be monitored, and when its performance does not meet the inference requirements, it can be known in time and updated, thereby ensuring that the performance of the first inference model used by the inference computing device 100 meets the requirements.
  • the accuracy of the inference result of the inference calculation device 100 is improved. Furthermore, the accuracy of the decision instruction made based on the inference result can be improved.
  • the application scenario of the inference calculation apparatus 100 is not specifically limited.
  • the inference calculation apparatus 100 may be used as an edge-side device.
  • the inference calculation device 100 may be integrated in a user-side device (ie, a production device).
  • the inference computing device 100 obtains the original product image of the display panel in the manufacturing stage from the user-side device, and the original product image is used as the unprocessed data that the inference computing device 100 needs to process.
  • User-side equipment may include: Automated Optical Inspection (AOI) equipment, Distributed File System (DFS), or image server.
  • the user-side equipment may use, for example, based on File Transfer Protocol (File Transfer). Protocol (FTP abbreviation) method, capture the image of the display panel in the production process in real time, as the original product image, that is, as the data to be processed. It should be noted that here only the data to be processed is used as an image form for an exemplary description.
  • AOI Automated Optical Inspection
  • DFS Distributed File System
  • FTP abbreviation File Transfer Protocol
  • the data to be processed may include data in various forms, for example, it may include document files, photos, drawings, microfilms, and videos.
  • At least one of unstructured data such as audio and audio may also include at least one of structured information such as information managed by a database, which is not specifically limited here.
  • the inference calculation device 100 performs inference calculation according to the first inference model.
  • the neural network model identifies defects from the original product image, and then marks them with defect information.
  • the marked defect information includes at least one of the following:
  • the types of defects include: particle, remain, line, hole, splash, electrostatic breakdown, wrinkle, film color And bubble (bubble) classification.
  • these classifications can be further divided into several levels.
  • the defect type is fine particles, and P represents fine particles.
  • this type of defects can be further divided into multiple levels, such as P0 for ordinary Particles (Normal Particles), P1 stands for Wall Particles, P2 stands for Sand Particles, etc.
  • the defect location includes: at least one of the circuit board where the defect is located, the level where the defect is located, or the mask layer where the defect is located, etc. that can show the location of the defect, so that the identified defect is located in a certain block.
  • the defect location can show the specific coordinates of the defect on the circuit board, or the defect can be selected by the display frame of a rectangular or circular frame, and the coordinates of a certain point related to the display frame For example, the coordinates of a vertex of a rectangular frame are used to represent the coordinates of the defect, or the coordinates of the center of a circular frame are used to represent the coordinates of the defect.
  • the relationship between defects and the circuit elements of the array substrate For example, the number of gates covered by the defect area, whether the defect completely falls within the gate, intersects with it, or is near the outside, etc.
  • Defect size It can be based on the length of the defect or the area of the defect region (for example, the area of the effective display area occupied by the defect).
  • the defect information described above may be expressed in various forms such as codes (for example, codes P0 to P2 corresponding to the classification of the defect type of particles), coordinate values, numbers, and text descriptions.
  • the first reasoning model will mark the defect type and defect location on the identified defect.
  • the type code of the identified defect is used to correspondingly mark the defect type; the rectangular wire frame is used to mark the defect position in the original picture, and the coordinate information of the rectangular wire frame in the corresponding picture is recorded.
  • the performance of the first inference model is evaluated during the inference calculation of the data to be processed by the first inference model (S30).
  • exemplary evaluation parameters used to evaluate the performance of the first inference model include: Accuracy, Precision, and Recall of the first inference model in inference calculations , And at least one of F Score (F Score).
  • the accuracy rate can be calculated by the following formula (1)
  • the accuracy rate can be calculated by the following formula (2)
  • the recall rate can be calculated by the following formula (3)
  • the F score can be calculated by the following formula (4):
  • F Score [(1+ ⁇ 2 ) ⁇ Precision ⁇ Recall]/( ⁇ 2 ⁇ Precision+Recall) (4).
  • TP Ture Positive, true
  • A A represents a result
  • the actual result It is also A. At this time, the calculated result is consistent with the actual result.
  • FP False Positive
  • FN False Negative
  • TN Ture Negative
  • precision rate precision rate
  • recall rate recall rate
  • F score evaluation parameters that characterize the performance of the first reasoning model.
  • the selected evaluation parameters can be determined according to the actual inference calculation requirements of the inference calculation device 100.
  • the accuracy is used as an evaluation parameter for evaluating the performance of the inference model.
  • the F score is used to evaluate the performance of the first reasoning model.
  • the F score is an evaluation parameter obtained by weighting the precision rate and the recall rate.
  • the ⁇ used to calculate the F score is a weighting coefficient.
  • the size of ⁇ can be adjusted according to different application scenarios, for example, , ⁇ is set to 1, so that in the performance evaluation of the first reasoning model, the precision rate and the recall rate have the same impact on the evaluation result, that is, the precision rate and the recall rate have the same emphasis ratio.
  • the inference calculation device 100 needs to determine whether the 1000 pictures are cats or dogs through the inference calculation of the first inference model.
  • the judgment result of the inference calculation device 100 is the decision made by it.
  • the 1000 pictures are all dog pictures, that is, the actual result is: all the 1000 pictures are dogs.
  • the first reasoning model is calculated by reasoning, and the calculated results are: 720 pictures judged to be dogs, 80 pictures judged to be cats, and the remaining 200 pictures cannot be judged.
  • the picture is 80, that is, the FP is 80; the calculation result is not a dog (including the sum of the number of pictures with a cat and the number of pictures that cannot be judged) ,
  • the actual result is that there are 280 pictures of dogs, the calculation result is 0 pictures that are not cats, and the actual result is cats, that is, FN is 280; the calculation results are not dogs, the actual results are not dogs, and the pictures are 0, the calculation result is not
  • the lower limit value is set for the evaluation parameter.
  • the evaluation result determines that the performance of the first inference model is degraded; if the obtained evaluation parameters are all higher than or If it is equal to the lower limit, the evaluation result determines that the performance of the first reasoning model meets the requirements.
  • the evaluation result determines the performance fluctuation of the inference model; if the obtained parameters are all If it is higher than or equal to the lower limit, the evaluation result determines that the performance of the first reasoning model meets the requirements.
  • the lower limit of the accuracy is set to 0.95.
  • the accuracy rate of the first inference model is 0.92, and the evaluation result is that the performance is degraded.
  • the accuracy rate of the first inference model is 0.96, 093, 097, 0.92, 0.95, then the evaluation result is a performance fluctuation.
  • the situations in which the first inference model needs to be updated include: performance fluctuations or performance degradation of the first inference model.
  • the change of the evaluation parameter in a continuous time period it is judged whether the first reasoning model has performance fluctuation or performance degradation.
  • the update of the first reasoning model in S40 includes:
  • S401a Perform model training based on the second training sample library to obtain a second inference model; or, S401b, send a model update request to the model training device 200 to obtain the second inference model.
  • the second training sample library includes: training samples from historical data, and/or training samples from inference results that have undergone re-judgment.
  • the above-mentioned “inference result” refers to the inference result obtained after inference calculation by the first inference model in S20.
  • the “from the inference result and the re-judgment” mentioned here and later refers to the re-judgment of the reasoning result of the first reasoning model, for example, the artificial judgment processing is performed on the reasoning result of the first reasoning model, and the mark is correct.
  • the resulting production data refers to the inference result obtained after inference calculation by the first inference model in S20.
  • the data to be processed is the original product image of the display panel, and the original product image of the display panel is sent to the inference calculation device 100, and the inference calculation is performed on it through the first inference model , The inference result is obtained, and after the inference result is re-judged, the product image of the display panel that has been marked with the correct result is obtained, and this is used as the source of the training samples of the second training library 150.
  • the second training sample library includes the training samples from the inference results that have undergone re-judgment. Since the inference results are continuously generated by the inference calculation device 100 during the production and preparation process, this is used as the source of the training samples of the second training sample library. It will make the training samples of the second training sample library newer, including training samples with new features of the production data during the production preparation process. In this way, it can be ensured that the inference calculation device 100 uses newer training samples in model training. Therefore, compared with the first inference model, the obtained second inference model is more sensitive to the newly-appearing and new features to be processed. Data reasoning calculation ability is strong. In the case where the first inference model is updated with the second inference model, the accuracy of the inference result of the inference calculation device 100 can be increased.
  • the inference calculation device 100 includes the second training sample library, and the second training sample library is stored in the memory 120.
  • the second training sample library is stored in another memory, and the inference computing device 100 can obtain the second training sample library from the other storage device.
  • the inference computing device 100 can be used as an edge-side device, and the model training device can be used as a cloud device, that is, the location of the inference computing device 100 (edge-side device) is more relative to the model training device 200 (cloud device). Close to the user side equipment.
  • the inference calculation apparatus 100 further includes a coupling interface 150, which can enable the inference calculation apparatus 100 to communicate with the user-side equipment in a wired or wireless manner to realize information interaction.
  • the coupling interface 150 may be any hardware or software interface capable of realizing wired or wireless connection.
  • the inference computing device 100 also includes a network interface 140, which communicates with the model training device 200 to realize information interaction.
  • the inference computing device 100 can use Transmission Control Protocol/Internet Protocol (TCP/IP for short) or Hypertext Transfer Protocol (HTTP for short) through the network interface 140.
  • TCP/IP Transmission Control Protocol/Internet Protocol
  • HTTP Hypertext Transfer Protocol
  • the inference computing device 100 communicates with the model training device 200 through the network interface 140, so as to be able to receive the first inference model, or to send a model update request to the model training device 200 in S401b, and obtain the second inference model. That is, information interaction with the model training device 200 is realized.
  • the update condition that the second inference model shown in FIG. 6 needs to meet includes: testing the second inference model, and the second inference model passes the test.
  • the test includes: evaluating the performance of the second reasoning model based on the test sample. If the performance meets the evaluation requirements, it is judged to pass the test.
  • the source of the test sample is not specifically limited.
  • the test sample comes from the historical data, that is, the production data that has been marked with correct identification information; for another example, the test sample comes from a manual judgement. Data to be processed; for another example, part of the test sample comes from the historical data, and part of the data to be processed is manually judged.
  • the sources of the test samples described above are not limited to these.
  • a specific test sample may be selected from a plurality of test samples, and at least one test plan is generated accordingly, and then the inference calculation device 100 performs the calculation according to the at least one test plan.
  • the second reasoning model described above is tested.
  • the above-mentioned update condition further includes: performing grayscale deployment of the second inference model, and evaluating the performance of the second inference model during the grayscale deployment process, the second inference model The performance of the model meets the evaluation requirements.
  • “Gray deployment” refers to that within a preset time, the inference computing device 100 uses the second inference model to perform simulation processing of the data to be processed.
  • the inference computing device 100 simultaneously uses the first inference model and the second inference model to process the to-be-processed data sent from the user-side device.
  • the reasoning calculation device 100 will output the reasoning result of the first reasoning model, and the reasoning result of the second reasoning model is only used for evaluating its performance, and will not be output as the real reasoning result, that is, the reasoning calculation device 100 does not
  • decision-making instructions are generated. Therefore, the gray-scale deployment of the second inference model will not affect the processing of the data to be processed by the inference computing device 100, which has an impact on the production line. There is no impact on the normal production.
  • the performance of the second inference model By evaluating the performance of the second inference model in the simulation process, it is estimated whether the performance of the second inference model can meet the requirements.
  • the data to be processed for the inference calculation performed by the second inference model requires manual judgment.
  • the result of the manual judgment is regarded as the correct result for the inference calculation device 100 to compare the result of the manual judgment and the second inference
  • the inference results of the models are compared, so as to evaluate the performance of the second inference model during the gray-scale deployment process.
  • one copy of the original product image of the display panel sent from the user-side device is sent to the inference calculation device 100 for simulation processing of the second inference model in the gray-scale deployment process, and one copy is used for manual judgment
  • the correct result is obtained, and the reasoning calculation device 100 compares the reasoning result with the correct result, thereby realizing the performance evaluation of the second reasoning model.
  • the to-be-processed data marked with the correct result can be sent to the first training sample library (the first training sample library is the training sample library based on the model training device 200 for model training) and/or the second training sample library, Using it as a training sample in model training, it is possible to update the first training sample database and/or the second training sample database.
  • the above-mentioned update condition further includes: verifying the configuration information of the second inference model, and the configuration information of the second inference model matches the inference calculation to be performed.
  • the inference calculation to be performed refers to the inference calculation performed by the inference calculation device 100 on the data to be processed.
  • the “inference calculation to be performed” means that the inference calculation device 100 needs to identify defects in the original product image and further identify the defect types. In this way, by verifying the configuration information of the second inference model, it can be detected whether the data of the second inference model is damaged or tampered with during transmission, thereby ensuring that the second inference model and the inference calculation obtained by the inference calculation device 100 The inference calculation requirements of the device 100 are matched.
  • the inference computing device 100 before performing model training (S401a) based on the second training sample library; or before sending a model update request (S401b) to the model training device 200, the inference computing device 100 also performs the following operations :
  • S35 Determine whether the training parameters required for the model training performed are within the set training parameter range. If yes, perform the model training (S401a), that is, in this case, the inference calculation device 100 performs model training to obtain the second inference model; if not, it sends a model update request to the model training device 200, that is, the In this case, the model training device 200 performs model training to obtain the second inference model.
  • the "set training parameter range” refers to the training parameter range corresponding to the training ability of the inference computing device 100 to meet the requirements of the model training performed, that is, the training parameters of the model training performed If it is within the set training parameter range, the training ability of the inference computing device 100 meets the requirements of the model training performed. In this case, the model training ability of the inference computing device 100 itself can complete the model training. Therefore, The inference calculation device 100 performs this model training to obtain the second inference model. Conversely, if the training parameters of the model training performed are not within the set training parameter range, the model training ability of the inference computing device 100 does not meet the requirements of the model training performed.
  • the inference computing device 100 itself The training ability is not enough to complete this model training, so it is necessary to send a model update request to the model training device 200 so that the model training device 200 can complete this model training and obtain the second inference model.
  • the model training apparatus 200 is a cloud device.
  • the above-mentioned training parameters include: at least one of the required data size, training duration, and computing power in model training.
  • the required data scale in a certain model training is 2000 images and corresponding 2000 labels, and the required training duration is 15 minutes; set The data scale of is 3000 images and 3000 corresponding labels, and the set training time is 20min.
  • the data scale required in model training is within the range of the set data scale, and the required training duration is within the range of the set training duration. That is, the model of the inference calculation device 100 If the training ability meets the model training requirements, the inference computing device 100 performs this model training (that is, performs S401a).
  • the inference computing device 100 sends a model update request to the model training device 200 so that the model training device 200 can perform this model training.
  • the inference computing device 100 when the model training capability of the inference computing device 100 meets the requirements of one model training, the inference computing device 100 is allowed to perform model training, which saves the time of sending requests to the model training device 200 and receiving data from the model training device 200.
  • the reasoning calculation device 100 can obtain the reasoning model that meets the reasoning calculation requirement in a more timely manner, and update the reasoning model, thereby further improving the accuracy of the reasoning result.
  • model training apparatus 200 described above is a cloud device
  • the inference computing device 100 meets the model training requirements, let the inference computing device 100 Autonomous model training can reduce the burden of cloud devices and reduce unnecessary occupation of cloud device resources, thereby improving the resource utilization of cloud devices.
  • the inference calculation apparatus 100 further performs the following operations:
  • the second inference model is rolled back to the first inference model and the model training is performed again, or the model training device 200 is requested to perform model training again.
  • the inference calculation device 100 can roll back the updated second inference model to the first inference model before the update. Reasoning model. Then the inference computing device 100 performs model training again, or requests the model training device 200 to perform model training again.
  • the situations in which the first inference model needs to be updated include:
  • the third inference model is obtained by the model training device 200 based on the updated first training sample library for model training without receiving the model update request of the inference calculation device 100.
  • the updated first training sample library includes re-judgment training samples from the inference result, that is, the updated first training sample library uses the re-judgment training samples from the inference result Obtained by updating the first training sample library.
  • the updated training samples of the first training sample library are relatively new, including training samples with new features appearing in the production data during the production preparation process, which can ensure that the model training device 200 uses newer training samples in model training. . Therefore, compared with the first inference model, the obtained third inference model has stronger inference calculation ability for newly-appearing data to be processed with new features. In the case where the first inference model is updated with the third inference model, the accuracy of the inference result of the inference calculation device 100 can be increased.
  • the abnormality of the first inference model means that the first inference model cannot output correct inference results, for example, the performance of the first inference model suddenly drops; or the first inference model does not output the inference results as required; or, the first inference model
  • the reasoning calculation time of the reasoning model is too long and the timeout is serious.
  • the third reasoning model can be used as a backup reasoning model, and when an abnormal situation occurs in the first reasoning model, it can be replaced.
  • the first inference model needs to be updated, that is, the performance of the first inference model no longer meets the requirements.
  • the inference computing device 100 or model training can be used During the period of model training performed by the device 200, the third inference model is used to replace the first inference model whose performance does not meet the requirements to process the data to be processed, so as to ensure the normal operation of the inference computing device 100.
  • the inference calculation device 100 after obtaining the inference result (S20), the inference calculation device 100 further performs the following operations: S70. Generate a decision instruction according to the inference result.
  • the defect information includes: the identified original product image with the defect, its defect location and defect type.
  • the decision instruction includes: according to the defect information of the original product image, perform corresponding defect processing on the product corresponding to the defective original product image. For example, according to the identified defect type, a repair strategy corresponding to the type of defect is given; another example, based on historical data analysis, the possible cause of the type of defect is given (such as a failure of one or several process equipment, or Some process parameter settings are unreasonable, etc.); for example, when an unknown defect or a large area defect occurs, an alarm is issued.
  • the decision instruction may not be generated by the inference calculation device 100.
  • the inference calculation device 100 obtains the inference result, it uploads the inference result to the cloud device, and the cloud device performs calculation and analysis on the inference result, generates a decision result instruction, and then sends it back to the inference calculation device 100.
  • the inference computing device 100 serves as the edge-side device in the user-side-edge-side-cloud computing architecture mentioned above, and the edge-side device sends the inference result to the cloud after preliminary processing (inference calculation) on the data to be processed Devices, cloud devices can use their powerful computing capabilities to make decision instructions and send them back to edge devices. In this way, unified planning, management and configuration of factory production equipment can be realized.
  • some embodiments of the present disclosure also provide a model training device 200, which includes at least one processor 210 and a memory 220.
  • the memory 220 stores program instructions, which can be executed by the at least one processor 210, and the model training device 200 performs the following operations A10 to A40:
  • the first training sample library includes training samples from historical data generated during the manufacturing stage.
  • historical data also refers to production data that has been manually judged, and is production data marked with correct results.
  • the model training device 200 includes the first training sample library, and the first training sample library is stored in the memory 220.
  • the first training sample library is stored in a memory other than the memory 220.
  • A20 Send the first inference model to the inference calculation device 100.
  • A30 Perform model training based on the updated first training sample library to obtain a second inference model.
  • the updated first training sample library includes the training samples from the inference result of the first inference model and which have undergone re-judgment.
  • A40 Send the second inference model to the inference calculation device 100.
  • the updated training samples of the first training sample library are relatively new, including training samples with new features appearing in the production data during the production preparation process, it can be ensured that the model training device 200 uses newer training samples in the model training. .
  • the second inference model obtained after model training based on the updated first training sample library has strong inference calculation capabilities for newly-appearing data to be processed with new features. In the case where the first inference model is updated with the second inference model, the accuracy of the inference result of the inference calculation device 100 can be increased.
  • the model training apparatus 200 further includes a network interface 240.
  • the model training apparatus 200 can communicate with other equipment (for example, the user-side equipment and/or the model training apparatus 200) through the network interface 240 to realize information Interactive.
  • the model training device 200 can use Transmission Control Protocol/Internet Protocol (TCP/IP) or Hypertext Transfer Protocol (Hyper Text Transfer Protocol, HTTP for short) through the network interface 240.
  • TCP/IP Transmission Control Protocol/Internet Protocol
  • HTTP Hypertext Transfer Protocol
  • the model training device 200 communicates with the inference calculation model 100 through the network interface 240, so that the first inference model can be sent to the inference calculation device 100, or receive a model update request from the inference calculation device 100, or send the second inference model To the inference calculation device 100.
  • the model training device 200 performs the following operations:
  • the "corresponding training sample library” and “corresponding inference model” mentioned in the following are relative to the model training performed, that is, if A10 is performed, training samples will be obtained from the first training sample library After the model training, the first inference model is obtained. If it is A30, the training samples will be obtained from the updated first training sample library, and after model training, the second inference model will be obtained.
  • the training task information is generated based on the received model update request sent by the inference computing device 100, which includes training requirements for model training.
  • the second inference model obtained through this model training needs to have specific inferences. Calculation function, and can get the calculation result.
  • the training configuration data includes training scheduling information generated according to training requirements.
  • the training scheduling information the training samples required for this model training can be known.
  • the training scheduling information includes a data identifier, through which the model can be found The training samples to be called for training.
  • model training device 200 during the model training process in B3, the model training device 200 also performs the following operations:
  • the model The training device 100 before sending the first inference model to the inference computing device 100 (that is, before A20 in FIG. 6), or before sending the second inference model to the inference computing device 100 (that is, before A40 in FIG. 6), the model The training device 100 also performs the following operations:
  • the corresponding inference model is sent to the inference calculation device 100.
  • the first reasoning model is sent, if it is judged that the model version information is correct, then proceed to A20; if the second reasoning model is being sent, if it is judged that the model version information is correct, then proceed to A40.
  • model version information is not correct, find the corresponding reasoning model whose model version meets the requirements. If it is found, it is sent to the inference calculation device 100; or, if it is not found, the notification information is sent to the inference calculation device 100.
  • the inference computing device 100 may request the model training device to perform model training again according to the received notification information, or may stop the operation.
  • the model training device 200 includes a first training sample library, and the model training device 200 further performs the following operations:
  • model update request sent by the inference calculation device 100 If the model update request sent by the inference calculation device 100 is not received, perform model training based on the updated first training sample library to obtain a third inference model.
  • A70 Send the third inference model to the inference calculation device 100.
  • the third inference model is obtained by the model training device 200 through model training without receiving the model update request, that is, the third inference model is obtained by the model training device 200 according to the updated first training sample Obtained after the library conducts model training on its own.
  • Some embodiments of the present disclosure also provide a computer-readable storage medium, which is provided in the inference calculation device 100 in any of the above embodiments.
  • the computer-readable storage medium stores program instructions, and the program instructions can be calculated by inference.
  • At least one processor 110 in the device 100 executes to cause the inference calculation device 100 to perform the following operations S10 to S40:
  • the first inference model is based on a first training sample library and is obtained through model training, and the first training sample library includes training samples from historical data.
  • the computer-readable storage medium also has the effect of improving the accuracy of the inference result, and will not be repeated here.
  • Some embodiments of the present disclosure also provide a computer-readable storage medium, which is set in the model training device 200 described in any of the above embodiments, and the program instructions can be used by at least one processor 210 in the model training device 200.
  • the model training device 200 is executed to perform the following operations A10 to A40:
  • the first training sample library includes training samples from historical data.
  • A20 sends the first inference model to the inference calculation device 100.
  • A30 Perform model training based on the updated first training sample library to obtain a second inference model.
  • the later first training sample library is obtained by updating the first training sample library using the inference result from the first inference model and the re-judged training samples.
  • A40 Send the second inference model to the inference calculation device 100.
  • the computer-readable storage medium also has the effect of improving the accuracy of the inference result, and will not be repeated here.
  • Some embodiments of the present disclosure also provide an inference calculation system 1000, referring to FIG. 10, which includes: the inference calculation device 100 as described in any of the above embodiments, and the model training device 200 as described in any of the above embodiments.
  • the reasoning calculation device 100 and the model training device 200 included in the reasoning calculation system 1000 have the effect of improving the accuracy of the reasoning result. Therefore, the model deployment system 1000 also has the effect of improving the accuracy of the reasoning result, which will not be repeated here.
  • processors for example, the processor 110 and the processor 210) mentioned in the embodiments of the present disclosure may be a central processing unit (CPU), a general-purpose processor, or a digital signal processor ( Digital Signal Processor, DSP), Application-Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof . It can implement or execute various exemplary logical blocks and modules described in conjunction with the present disclosure.
  • the processor 210 may also be a combination that implements computing functions, for example, including a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and so on.
  • the memory for example, the memory 120 and the memory 220 mentioned in the embodiments of the present disclosure may be random access memory (Random Access Memory, RAM), flash memory, read-only memory (Read Only Memory, ROM), and erasable memory. Except for programmable read-only memory (Erasable Programmable ROM, EPROM), electrically erasable programmable read-only memory (Electrically EPROM, EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any known in the art Other forms of storage media.
  • RAM Random Access Memory
  • flash memory read-only memory
  • Read Only Memory Read Only Memory
  • ROM Read Only Memory
  • erasable memory erasable memory
  • programmable read-only memory Erasable Programmable ROM, EPROM
  • Electrically erasable programmable read-only memory Electrically erasable programmable read-only memory
  • registers hard disks, mobile hard disks, CD-ROMs, or any known in the art Other forms of storage media.
  • the inference calculation device 300 includes: a performance evaluation module 11, an inference model module 13, and a model task management module 14.
  • the model task management module 14 is configured to receive the first inference model from the model training device 400; the inference model module 13 is configured to perform the to-be-processed data generated in the manufacturing stage based on the received first inference model (from the user side) The inference calculation of the device 3) obtains the inference result, and sends the inference result to the user-side device 3; the performance evaluation module 11 is configured to evaluate the performance of the first inference model to determine whether the first inference model needs to be evaluated Update, if yes, the model task management module 14 updates the first reasoning model.
  • the inference calculation device 300 further includes a second model training module 12 and a second training sample library 19.
  • the second model training module 12 is configured to perform model training based on the second training sample library 19 to obtain the second inference model; or, the performance evaluation module 11 is also configured to provide information to the model
  • the training device 400 sends a model update request to obtain the second inference model.
  • the second training sample library 19 stores training samples required for model training, and the training samples may come from the inference model module 13 and/or the model task management module 14.
  • the model task management module 14 is further configured to receive the second inference model from the model training device 400, and perform testing, grayscale deployment, or configuration information verification on the received second inference model. At least one of.
  • the model task management module 14 includes a verification unit 141, a testing unit 142 and a gray scale deployment unit 143.
  • the inference calculation device 300 further includes a model training decision module 18 configured to determine whether the training parameters required for the model training performed are within the set training parameter range. If so, the performance evaluation module 11 is made to send a model update request to the second model training module 12, that is, the second model training module 12 is made to perform model training to obtain the second inference model; if not, the performance evaluation module 11 is made to send the model
  • the training device 400 sends a model update request, that is, makes the model training device 400 perform model training to obtain the second inference model.
  • the inference calculation device 300 further includes a backup module 15 configured to receive the third inference model.
  • the backup module 15 sets the third inference model The model is sent to the model task management module 14, and the model task management module 14 updates the first reasoning model with the third reasoning model.
  • the performance evaluation module 11 is further configured to perform performance evaluation on the second inference model, and compare the performance of the first inference model and the second inference model. In the case where the performance of the second inference model is worse than the performance of the first inference model, the model task management module 14 is further configured to roll back the second inference model to the first inference model. The performance evaluation module 11 is also configured to send a model training request to the second model training module 12 or the model training device 400 again.
  • the reasoning computing device 300 further includes a function expansion management module 16, and the function expansion management module 16 is configured to expand or modify the functions of the reasoning model module 13 and the task management module 14.
  • the function expansion module 16 receives the imported reasoning model, and controls the reasoning model in the reasoning model module 13 to switch to the imported reasoning model.
  • the function expansion module 16 can also expand the function of the first training sample library.
  • the function expansion module 16 includes a function expansion interface.
  • the inference calculation device 300 further includes: an original data summary module 17 and an inference model module 13.
  • the raw data summary module 17 is configured to summarize the raw data from the user-side device 3 to obtain the data to be processed, and send the data to be processed to the inference model module 13.
  • the model training device 400 includes: a training management module 21, a first model training module 22, and a first training sample library 23.
  • the first model training module 22 is configured to perform model training based on the first training sample library 23 to obtain a first inference model, and/or to perform model training based on the updated first training sample library 23 to obtain a second inference model .
  • the training management module 21 is configured to send the first training model and/or the second training model to the inference calculation device 300.
  • the training management module 21 includes: a task management unit 211, a training scheduling unit 212 and a branch management unit 213.
  • the task management unit 211 is configured to generate training task information according to the model update request sent by the inference computing device 300; the training scheduling unit 212 is configured to generate training configuration data according to the training task information, and obtain training configuration data from the corresponding training sample library Training samples; the branch management unit 213 is configured to verify the model version information of the corresponding inference model to determine whether the model version information is correct. In the case that the model version information is correct, the branch management unit 213 is further configured to send a corresponding inference model with correct version information to the inference calculation device 300.
  • the training scheduling unit 212 is also configured to monitor the model training process of the first model training module 22, and in the case of an abnormality in the model training process, update the training configuration data, and send the updated training configuration data to The first model training module 22.
  • the first model training module 22 is also configured to perform model training according to the updated training configuration data and training samples.
  • the model training device 400 further includes a first training sample library 23 configured to receive training samples sent by the inference computing device 300 with correct results marked, For example, the inference result of the inference model module 13 of the inference calculation device 300 is re-judged and marked with a correct result, and then sent to the first training sample library 23, thereby updating the first training sample library 23.
  • the first model training module 22 is also configured to perform model training based on the updated first training sample library 23 to obtain the third inference model when the model update request is not received.
  • the training management module 21 includes a branch management unit 213
  • the first model training module 22 is configured to send the third inference model to the branch management unit 213, and the branch management unit 213 sends the third inference model To the inference calculation device 300.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Neurology (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Image Analysis (AREA)
  • General Factory Administration (AREA)

Abstract

一种推理计算装置，包括至少一个处理器和存储器。存储器存储程序指令，程序指令可由所述至少一个处理器执行，使推理计算装置进行以下操作：接收来自模型训练装置的第一推理模型，第一推理模型基于第一训练样本库，由模型训练装置通过模型训练得到，第一训练样本库包括来自生产制造阶段产生的历史数据的训练样本，模型训练装置包括云端设备；基于第一推理模型，进行生产制造阶段产生的待处理数据的推理计算，得到推理结果，将推理结果发送至用户侧设备；评估第一推理模型的性能，以确定是否需要对第一推理模型进行更新，若是，则更新第一推理模型。

Description

推理计算装置、模型训练装置、推理计算系统 技术领域
本申请涉及云计算技术领域，尤其涉及一种推理计算装置、模型训练装置、推理计算系统。
背景技术
云计算技术具有计算能力强、高可用性、高扩展性等特点，同时，云计算中应用程序被部署在云端设备，用户能够按需从云端设备处获取应用服务，提高了用户获取应用服务的便捷性，并提高了计算资源的整体率用率。
发明内容
一方面，提供一种推理计算装置，包括至少一个处理器和存储器；所述存储器存储有程序指令。所述程序指令可由所述至少一个处理器执行，使所述推理计算装置进行以下操作：接收来自模型训练装置的第一推理模型，其中，所述第一推理模型基于第一训练样本库，由所述模型训练装置通过模型训练得到；所述第一训练样本库包括来自生产制造阶段产生的历史数据的训练样本，所述模型训练装置包括云端设备；基于所述第一推理模型，进行生产制造阶段产生的待处理数据的推理计算，并得到推理结果，将所述推理结果发送至用户侧设备，所述推理计算装置相较于模型训练装置更靠近用户侧设备；评估所述第一推理模型的性能，以确定是否需要对所述第一推理模型进行更新，若是，则更新所述第一推理模型。
在一些实施例中，所述历史数据包括标记有缺陷信息内容的产品图像；所述待处理数据包括在生产制造阶段所产生的原始产品图像；所述推理结果包括识别所述原始产品图像中的缺陷类型信息。
在一些实施例中，对所述第一推理模型进行的更新包括：基于第二训练样本库，进行模型训练，得到第二推理模型；或者，向所述模型训练装置发送模型更新请求，得到第二推理模型，所述第二训练样本库包括来自历史数据的训练样本，和/或来自所述推理结果且经过复判的训练样本。在所述第二推理模型满足更新条件的情况下，用所述第二推理模型更新所述第一推理模型。
在一些实施例中，所述更新条件包括：对所述第二推理模型进行测试，所述第二推理模型通过所述测试。其中，所述测试包括基于测试样本对所述第二推理模型的性能进行评估，若性能符合评估要求，则判定为通过所述测试。
在一些实施例中，所述更新条件还包括：对所述第二推理模型进行灰度部署，并在所述灰度部署过程中对其性能进行评估，所述第二推理模型的性能符合评估要求。
在一些实施例中，所述更新条件还包括：对所述第二推理模型的配置信息进行校验，所述配置信息与需进行的推理计算相匹配。
在一些实施例中，在进行模型训练，得到第二推理模型；或者，向模型训练装置发送模型更新请求，得到第二推理模型之前，所述推理计算装置还进行以下操作：判断所进行的模型训练所需要的训练参数是否在设定的训练参数范围内；若是，则进行所述模型训练；若否，则向所述模型训练装置发送所述模型更新请求。
在一些实施例中，所述训练参数包括：模型训练中所需数据规模、训练时长、及计算能力中的至少一种。
在一些实施例中，用于评估所述推理模型的性能的评估参数包括：所述推理模型在推理计算过程中的准确率、精确率、召回率、及F分数中的至少一者。根据性能评估需要对所述第一推理模型进行更新的情况包括：第一推理模型出现性能波动或性能下降，其中，根据所述评估参数在连续的时间段内的变化情况，判断所述第一推理模型是否出现性能波动或性能下降。
在一些实施例中，根据性能评估需要对所述第一推理模型进行更新的情况包括：接收第三推理模型，所述第三推理模型为所述模型训练装置在未接收到所述模型更新请求的情况下，基于更新后的第一训练样本库进行模型训练得到，所述更新后的第一训练样本库包括来自所述推理结果且经过复判的训练样本。比较所述第一推理模型与所述第三推理模型的性能，若所述第三推理模型性能高于所述第一推理模型的性能，或者，若所述第一推理模型出现异常，用所述第三推理模型更新所述第一推理模型模型。
在一些实施例中，在用所述第二推理模型更新所述第一推理模型的情况下，所述推理计算装置还进行以下操作：对所述第二推理模型进行性能评估，比较所述第一推理模型与所述第二推理模型的性能；在所述第二推理模型的性能比所述第一推理模型的性能差的情况下，将所述第二推理模型回滚至所述第一推理模型；重新进行模型训练，或者重新请求所述模型训练装置进行模 型训练。
在一些实施例中，在得到所述推理结果之后，所述推理计算装置还进行以下操作：根据所述推理结果，生成决策指令。其中，所述缺陷信息包括：识别出的具有缺陷的原始产品图像，及其缺陷位置和缺陷类型；所述决策指令包括：根据所述原始产品图像的缺陷信息，对所述具有缺陷的原始产品图像对应的产品进行相应的缺陷处理。
另一方面，提供一种模型训练装置，包括至少一个处理器和存储器；所述存储器存储有程序指令，所述程序指令可由所述至少一个处理器执行，使所述模型训练装置进行以下操作：基于第一训练样本库进行模型训练，得到第一推理模型；所述第一训练样本库包括来自生产制造阶段产生的历史数据的训练样本。将所述第一推理模型发送至推理计算装置；在接收来自所述推理计算装置的模型更新请求的情况下，基于更新后的第一训练样本库进行模型训练，得到第二推理模型；所述更新后的第一训练样本库包括来自所述第一推理模型的推理结果且经过复判的训练样本。将所述第二推理模型发送至所述推理计算装置。
在一些实施例中，在基于所述第一训练样本库，或所述更新后的第一训练样本库进行模型训练的过程中，所述模型训练装置进行以下操作：生成训练任务信息；根据所述训练任务信息，生成训练配置数据；从相应的训练样本库获取训练样本，基于所述训练样本和所述训练配置数据，进行模型训练，得到相应的推理模型。
在一些实施例中，在所述从相应的训练样本库获取训练样本，基于所述训练样本和所述训练配置数据，进行模型训练，得到相应的推理模型的过程中，所述模型训练装置还进行以下操作：监控模型训练的过程；在模型训练的过程出现异常的情况下，更新所述训练配置数据；基于所述训练样本和更新后的训练配置数据进行模型训练。
在一些实施例中，在将所述第一推理模型或所述第二推理模型发送至所述推理计算装置之前，所述模型训练装置还进行以下操作：对相应的推理模型的模型版本信息进行校验，判断所述模型版本信息是否正确，若是，则向所述推理计算装置发送相应的推理模型，若否，则查找模型版本符合要求的相应的推理模型；在查找到的情况下，将其发送至所述推理计算装置，或者，在未查找到的情况下，向所述推理计算装置发送通知信息。
在一些实施例中，所述模型训练装置还包括所述第一训练样本库，所述模 型训练装置还进行以下操作：对所述第一训练样本库进行更新；在未接收所述模型更新请求的情况下，基于更新后的第一训练样本库进行模型训练，得到第三推理模型；将所述第三推理模型发送至所推理计算装置。
再一方面，提供一种计算机可读存储介质，存储有程序指令；所述程序指令可被处理器执行，使所述推理计算装置进行以下操作：接收第一推理模型，所述第一推理模型基于第一训练样本库，通过模型训练得到；所述第一训练样本库包括来自历史数据的训练样本。基于所述第一推理模型，进行工业生产中的待处理数据的推理计算，并得到推理结果。评估所述第一推理模型的性能，以确定是否需要对所述第一推理模型进行更新，若是，则更新所述第一推理模型。
再一方面，提供一种计算机可读存储介质，存储有程序指令；所述程序指令可被处理器执行，使所述模型训练装置进行以下操作：基于第一训练样本库进行模型训练，得到第一推理模型；所述第一训练样本库包括来自历史数据的训练样本。将所述第一推理模型发送至推理计算装置。在接收来自所述推理计算装置的模型更新请求的情况下，基于更新后的第一训练样本库进行模型训练，得到第二推理模型；所述更新后的第一训练样本库为使用来自所述第一推理模型的推理结果且经过复判的训练样本对所述第一训练样本库进行更新得到的。将所述第二推理模型发送至所述推理计算装置。
再一方面，提供一种推理计算系统，包括：如上任一实施例所述的推理计算装置，以及如上任一实施例所述的模型训练装置。
附图说明
图1为根据相关技术的一种用户侧-云端计算架构的结构示意图；
图2为根据相关技术的一种云端用户侧-边缘侧-云端计算架构的结构示意图；
图3为根据本公开的一些实施例的一种推理计算装置的结构示意图；
图4为根据本公开的一些实施例的一种推理计算装置和模型训练装置之间的交互示意图；
图5为根据本公开的一些实施例的一种模型训练装置的结构示意图；
图6为根据本公开的一些实施例的另一种推理计算装置和模型训练装置之间的交互示意图；
图7为根据本公开的一些实施例的再一种推理计算装置和模型训练装置 之间的交互示意图；
图8为根据本公开的一些实施例的一种模型训练装置的操作步骤的流程图；
图9为根据本公开的一些实施例的另一种模型训练装置的操作步骤的流程图；
图10为根据本公开的一些实施例的一种推理计算系统的结构示意图；
图11为根据本公开的一些实施例的一种推理计算装置的结构图和一种模型训练装置的结构图。
具体实施方式
下面将结合附图，对本公开的一些实施例进行描述。显然，所描述的实施例仅仅是本公开一部分实施例，而不是全部的实施例。基于本公开中的实施例，本领域普通技术人员所获得的所有其他实施例，都属于本公开保护的范围。
在工业生产中，由于设备、参数、操作、环境干扰等环节存在的问题，会导致生产的产品存在不符合工艺要求甚至不良出现，所以需要在每道工序之后及时把不符合要求的不良缺陷产品找出，避免不良的继续产生。进一步地，还可以根据生产设备中的实时监测数据，如温度、压力、湿度等的变化，预测生产系统的运行状态，提前预防产品不良产生或者设备损坏等问题。
随着人工智能的不断发展，工厂可根据已经产生的大量生产数据利用机器学习的方法进行模型训练，得到推理模型，并将该模型应用在后续的生产过程中进行推理和预测，从而代替工厂的操作人员和数据分析人员，提高生产效率，减少人力成本。机器学习算法包括支持向量机(support vector machine，SVM)、级联算法(Adaboost)、决策树、随机森林、以及基于神经网络的深度学习算法等。
云计算(Cloud Computing)是分布式计算的一种，云计算能够通过网络“云”将数据计算处理程序分解成多个小的计算处理程序，具有计算能力强、高可用性、高扩展性等特点。在工厂的生产过程中会产生大量的生产数据，云计算能够训练出处理该大量的生产数据的推理模型，并通过推理模型对生产数据进行推理计算，这样，多部服务器组成的系统(也即云端设备)对用户侧设备(即工厂中的生产设备)上传的待处理数据进行推理计算得到推理结果，然后对该推理结果进行分析，得到云端决策结果，并将该云端决策结果发送给用户侧设备，从而能够实现生产过程中的物联网。这里，称上述这种计算架构 为用户侧-云端计算架构。
下面参照图1，对用户侧-云端计算架构进行示例性的介绍。用户侧设备3’将待处理数据(即生产设备所产生的生产数据)上传至云端设备2’，云端设备2’对所上传的待处理数据进行推理计算，并得到决策结果，而后将决策结果发送回用户侧设备3’。用户侧设备3’根据所接收到的决策结果，做出相应的动作，从而能够实现对用户侧设备3’的统一管理。
然而，用户端侧设备3’上传的待处理数据数量较多，在待处理数据的上传过程中，上传所用的网络带宽及网管权限等会对上传速度造成较大的限制，造成数据处理的时效性较差。此外，大量的待处理数据被上传至云端设备3’后需先被存储，这会占用云端设备3’较多的资源，给云端设备带来较大负担。
为解决上述用户侧-云端计算架构存在的问题，可采用用户侧-边缘侧-云端计算架构对生产制备过程中产生的生产数据进行处理。
参见图3～图5，本公开的一些实施例提供了一种推理计算装置100，包括至少一个处理器110和存储器120。存储器120中存储有程序指令。其中，该程序指令可由所述至少一个处理器110执行，使推理计算装置100进行以下操作S10～S40：
S10、接收来自模型训练装置200的第一推理模型。其中，第一推理模型基于第一训练样本库，由模型训练装置200通过模型训练得到，第一训练样本库包括来自历生产制造阶段产生的史数据的训练样本，该模型训练装置200包括云端设备。也即，模型训练装置200可以是云端设备。
在一个示例中，模型训练装置200对神经网络模型进行训练，主要包括以下步骤：选择网络拓扑；使用表示被网络建模的问题的一组训练数据；以及调节权重，直到网络模型针对训练数据集的所有实例表现为具有最小误差。例如，在用于神经网络的监督式学习训练过程期间，将由网络响应于表示训练数据集中的实例的输入所产生的输出与该实例的“正确”的已标记输出相比较；计算表示所述输出与已标记输出之间的差异的误差信号；以及当将误差信号向后传播穿过网络的层时，调节与所述连接相关联的权重以最小化该误差。当从训练数据集的实例中生成的每个输出的误差被最小化时，该神经网络模型被视为“已经过训练”，并可以用于人工智能推理任务。
神经网络模型的推理和训练过程中，涉及大量参数、激活值、梯度值的缓冲区，其每个值在每一次训练迭代中都要被完全更新，对计算和吞吐能力的需求非常高。而GPU(Graphics Processing Unit，图形处理器)具有简便的编程 模型、强并行能力和高显存带宽，在处理与训练深度神经网络相关联的计算时极其高效。因此，使用多个GPU集成的GPU集群可以有效地提高神经网络模型的训练和推理速度。
推理模型可以是神经网络模型，例如，前馈神经网络模型、卷积神经网络(Convolutional Neural Network，简称为CNN)模型、循环神经网络(RNN，Recurrent Neural Network)模型、生成式对抗网络(GAN，Generative Adversarial Network)模型，但不限于此，也可以采用本领域技术人员公知的其他神经网络模型。
S20、基于第一推理模型，进行生产制造阶段产生的待处理数据的推理计算，得到推理结果，将该推理结果发送至用户侧设备，所述推理计算装置100相较于模型训练装置200更靠近用户侧设备。
在一个示例中，参见图2，推理计算装置100为边缘侧设备1’，模型训练装置200为云端设备2’，边缘侧设备1’的部署位置位于用户侧设备3’和云端设备2’之间，且靠近用户侧设备3’的一端，边缘侧设备1’利用推理模型对来自用户侧设备3’的待处理数据进行推理计算，据此得到边缘侧决策结果，并将该边缘侧决策结果发送给用户侧设备3’；或者，边缘侧设备1’将推理计算后的计算结果上传至云端设备2’，由云端设备2’处理和分析该上传的计算结果，得到云端决策结果，而后将云端决策结果发送给用户侧设备3’。在另一个示例中，可将边缘侧设备1’集成于用户侧设备3’中，也即用户侧设备3’会产生生产数据，同时对所产生的生产数据进行推理计算。
相对于云端设备2’而言，边缘侧设备1’更靠近用户侧设备3’，因此，待处理数据的传输过程中耗时相对较少，这使得数据处理的时效性增加。并且，由于无需将待处理数据上传至云端设备2’，因此对于云端设备2’的资源的占用较少，从而减轻了云端设备的负担。此外，在上述用户侧-边缘侧-云端计算架构中，云端设备2’可充分利用其强大的计算能力，进行无需实时反馈决策的模型训练工作。
需要说明的是，边缘侧设备1’中的推理模型指的是利用一定的算法(如机器学习算法)进行推理计算的算法结构，这里，需要通过模型训练得到满足边缘设备1’的推理计算需求的推理模型。其中，模型训练指的是，通过对大量训练样本进行多次的迭代和计算，提取出这些训练样本的特定特征，最终得到能够在具有该特定特征的数据集上收敛的算法结构和参数值等，也即，得到能够识别具有特定特征的数据的推理模型。
其中，用于模型训练的大量训练数据集样本以及验证数据集，以及用于测试训练出的模型的测试数据集。三者都包括原始图像样本和对应的标注的信息。三者中的原始图像样本可以完全重合、有交集、或者甚至完全没有交集，相应地，训练数据集、验证数据集和测试数据集完全重合、有交集、或者甚至完全没有交集。完全没有交集是理想的情况，这样训练出的训练模型相对可靠。
例如，在显示面板生产过程中，需要对各工艺环节完成后显示面板进行拍照以得到显示面板的图像(如阵列基板的图像)，通过识别图像中是否有缺陷、以及缺陷的种类和位置，得到显示面板的缺陷信息，并将具有缺陷的显示面板和其缺陷信息进行标记，然后根据不同缺陷分别使用相应的修复工艺对其进行修复。图像识别可通过深度学习的神经网络模型实现计算机视觉识别，通过云端设备1’进行样本训练后，将推理模型发送给边缘侧设备2’，用户侧3’产生的图像直接通过边缘侧设备2’，快速的做出推理计算，省去了传输到云端计算的带宽压力和延迟。
示例性的，通过神经网络模型识别图像中的缺陷内容的实现方式如下，首先将产品图像缩放至固定像素大小M×N(也可不进行缩放)，然后将M×N图像送入深度卷积神经网络，例如，VGGNet、ResNet和MobileNet等；通过多层卷积层、激活层、池化层，获得整个图像的特征图(feature maps)；把特征图输入到筛选区域网络(ZF/SSD/RPN等)，经过计算，获得建议区域(proposal region)；然后，针对建议区域进行卷积池化等操作，获得建议区域的区域特征图(proposal feature)，再将区域特征图(proposal feature)送入后续全连接和softmax网络作分类(classification即分类proposal到底是什么缺陷)，获得最大概率的缺陷类别作为最后分类结果，记录类别和概率。另外建议区域(proposal region)的坐标和尺寸代表了缺陷的位置和大小。基于缺陷模型识别产品缺陷内容的方法可以采用上述方法的类似变形或者其他本领域技术人员公知的方法，本公开在此不做限定。
S30、评估第一推理模型的性能，以确定是否需要对第一推理模型进行更新。
随着来自用户端侧设备3’的待处理数据不断产生和更新，具有新特征(指模型训练时未出现过的特征)的待处理数据不断出现，导致推理模型的推理计算能力逐渐下降，推理模型所得到的边缘侧决策结果的准确性随之下降。以显示面板的生产制备为例，新型号产品的生产制备、生产工艺的更新升级、生产设备的更新换代等因素，可能会导致所制备的显示面板出现新种类的缺陷，该 新种类的缺陷是推理模型在模型训练时的训练样本中未出现过的，这样会导致推理模型无法对新种类的缺陷进行识别，从而使推理模型不再满足生产制备过程中的缺陷识别和分类要求。
正如前文中所提到的，推理模型需要通过大量的训练样本，经模型训练之后得到。其中，训练样本可视为由至少一条数据形成的数据集，且训练样本可以包括多种形式的数据，示例性的，训练数据可包括图像、视频、音频等，此处不做具体限定。用于训练第一推理模型的大量的训练样本是来自第一训练样本库的，第一训练样本库中的训练样本包括来自历史数据的训练样本。这里，历史数据指的是已经过判断，且标记有正确结果的生产数据，示例性的，通过人工判断或其他能够正确判断的设备对生产数据进行判断，得到如上所述的历史数据。由于历史数据已被标记有正确结果，因此其能够作为模型训练中的训练样本。
示例性的，以显示面板的生产制备为例，第一推理模型要处理的待处理数据包括在生产制造阶段所产生的原始产品图像，第一推理模型的推理结果包括原始产品图像中的缺陷信息。这里，原始产品图像为未经过处理的图像，第一推理模型需要对该原始产品图像进行推理计算，以得到该原始产品图像中的缺陷信息，例如，识别出缺陷位置和缺陷类型。如上所述的历史数据包括标记有缺陷类型的产品图像，这里，所标记的缺陷类型为正确结果。需要说明的是，推理计算装置100在评估第一推理模型的性能时，会将第一推理模型经推理计算后得到的推理结果与正确结果进行比较，以此实现对第一推理模型的性能评估，这里，正确结果可以为经人工判断后，被输入至推理计算装置100。
示例性的，以显示面板的生产制备为例，工作人员对固定时段(例如，每天的14：00-15：00)内生产线上的显示屏的原始产品图像进行判断，以判断出具有缺陷的原始显示屏图像，并标记出缺陷类型，而后将人工判断的结果作为正确结果输入至推理计算装置100。在另一些示例中，随机抽取固定数量的原始产品图像进行人工判断，以得到人工判断的结果。在另一些示例中，将指定批次的显示屏的原始产品图像进行人工判断，以得到人工判断的结果。
示例性的，在S20的基于第一推理模型，进行生产制造阶段产生的待处理数据的推理计算的过程中，进行S30，评估第一推理模型的性能。
S40、若是，则更新第一推理模型。
在如上所述的推理计算装置100中，通过对第一推理模型进行性能评估， 能够得知第一推理模型的性能是否满足当前的推理需求，或者能够根据性能评估结果对第一推理模型的性能的变化趋势进行预判，以此可以确定是否需要对第一推理模型进行更新。在得知第一推理模型的性能不满足当前的推理需求，或通过预判得知第一推理模型的性能将不满足推理需求的情况下，即确定需要对第一推理模型进行更新的情况下，更新第一推理模型。这样能够对第一推理模型的性能进行监测，在其性能不满足推理需求时，能够及时知晓，并对其进行更新，从而保证了推理计算装置100所使用的第一推理模型的性能符合要求，提高了推理计算装置100的推理结果的准确性。进而，能够提高基于该推理结果所做出的决策指令的正确性。
需要说明的是，对于推理计算装置100的应用场景并不做具体限定，例如，推理计算装置100可以作为边缘侧设备。又例如，推理计算装置100可以被集成于用户侧设备(即生产设备)中。
示例性的，以显示面板的生产制备为例，推理计算装置100从用户侧设备处获取显示面板在制造阶段的原始产品图像，该原始产品图像作为推理计算装置100需要处理的未处理数据。用户侧设备可以包括：光学自动检测(Automated Optical Inspection，缩写AOI)设备、分布式文件系统(Distributed File System，简称DFS)或图像服务器，该用户侧设备可以采用，例如基于文件传输协议(File Transfer Protocol，缩写FTP)的方式，实时抓取生产过程中的显示面板的图像，做为原始产品图像，也即作为待处理数据。需要说明的是，这里仅以待处理数据为图像形式进行了示例性的说明，实际上待处理数据可包括多种形式的数据，例如其可包括文档文件、照片、图纸资料、缩微胶片、视频和音频等非结构化数据中的至少一种，也可以包括如数据库所管理的信息等结构化信息中的至少一种，此处不做具体限定。
而后，推理计算装置100根据第一推理模型进行推理计算。这里，以第一推理模型为神经网络模型为例，神经网络模型从原始产品图像中识别出缺陷，而后对其进行缺陷信息的标记，所标记的缺陷信息包括以下几种中的至少一种：
缺陷类型。示例性的，缺陷类型包括：微粒(particle)、残留(remain)、线不良(line)、孔洞(hole)、溅落(splash)、静电击穿、褶皱(wrinkle)、膜层变色(film color)和气泡(bubble)等分类。此外，这些分类还可进一步划分为若干各等级，例如缺陷类型为微粒，以P代表微粒，则可按照所形成的微粒的不同形态，将该类缺陷进一步划分为多个等级，如P0代表普通微粒 (Normal Particle)、P1代表沉积性微粒(Wall Particle)、P2代表散沙微粒(Sand Particle)等。
缺陷位置。示例性的，缺陷位置包括：识别出的缺陷所在的电路板、缺陷所在的层级或缺陷所在的掩膜层等可示出缺陷位置的信息中的至少一种，以识别出的缺陷位于某块电路板为例，缺陷位置可示出该缺陷在该电路板上的具体坐标，也可通过矩形框或圆形框等显示边框对该缺陷框选，并以与显示边框相关的某点的坐标，如用矩形框的某个顶点坐标表示该缺陷的坐标，或用圆形框的圆心坐标表示该缺陷的坐标。
缺陷与阵列基板(Array Substrate)的电路元件的关系。例如，缺陷区域所覆盖的栅极的数量、缺陷是否完全落在栅极内、与之相交、还是其外附近等。
缺陷大小。可以以缺陷的长度或者缺陷的区域面积(例如，缺陷所占有效显示区的面积)。
此外，示例性的，如上所述的缺陷信息可以以代码(例如与微粒这一缺陷类型的分类对应的代码P0～P2)、坐标值、数字、文字描述等多种形式表示。
在一个示例中，进行原始产品图像中的缺陷识别后，第一推理模型会对识别出的缺陷进行缺陷类型和缺陷位置的标记。这里，通过识别出的缺陷的类型代码，对应标记出缺陷类型；使用矩形线框在原始图片中标记缺陷位置，并记录对应图片中矩形线框的坐标信息。
基于此，在一些实施例中，在第一推理模型对待处理数据进行推理计算的过程中，评估第一推理模型的性能(S30)。在这种情况下，示例性的，用于评估第一推理模型的性能的评估参数包括：第一推理模型在推理计算中的准确率(Accuracy)、精确率(Precision)、召回率(Recall)、及F分数(F Score)中的至少一者。
这里，可以通过如下公式(1)计算出准确率，通过如下公式(2)计算出精确率，通过如下公式(3)计算出召回率，之后通过如下公式(4)计算出F分数：
Accuracy＝(TP+TN)/(TP+FP+TN+FN)              (1)；
Precision＝TP/(TP+FP)                        (2)；
Recall＝TP/(TP+FN)                           (3)；
F Score＝[(1+β 2)·Precision·Recall]/(β 2·Precision+Recall)      (4)。
其中，TP(Ture Positive，真正)表示计算结果为正，实际结果也为正，也就是说，推理模型经推理计算后，得到的计算结果为A(这里，A代表一种 结果)，实际结果也为A，此时计算结果与实际结果一致。
FP(False Positive，假正)表示计算结果为正，实际结果为负，也就是说，推理模型经推理计算后，得到的计算结果A，但实际结果为非A，此时计算结果和实际结果不一致。
FN(False Negative，假负)表示计算结果为负，实际结果为正，也就是说，推理模型经推理计算后，得到的计算结果为非A，但实际结果是A，此时计算结果和实际结果不一致。
TN(Ture Negative，真负)表示推理计算结果为负，实际结果也为负，也就是说，推理模型经推理计算后，得到的计算结果为非A，实际结果也为非A，此时计算结果与实际结果一致。
需要说明的是，精确率、精确率、召回率和F分数为表征第一推理模型的性能的评估参数。对第一推理模型进行性能评估时，所选用的评估参数可以根据推理计算装置100的实际的推理计算需求来确定。
示例性的，若推理计算装置100的推理计算需求更侧重于精确率，则将精确率作为评估推理模型性能的评估参数。
作为另一种示例，采用F分数对第一推理模型进行性能评估。其中，F分数是对于精确率和召回率加权后得到的评估参数。这里，计算F分数所用的β是一个权重系数，通过调整β的大小，能够调整性能评估时的精确率和召回率的侧重比例，β的大小可以根据不同的应用场景按照做相应的调整，例如，β设置为1，这样在对第一推理模型进行性能评估是，精确率和召回率对于评估结果的影响相同，也即精确率和召回率的侧重比例相同。
在一个示例中，推理计算装置100需要通过第一推理模型的推理计算，判断1000张图片中是猫或是狗，这里，推理计算装置100的判断结果即为其作出的决策。经过人工判断，该1000张图片均为狗的图片，也即实际结果为：这1000张图片中均是狗。第一推理模型经过推理计算，得到的计算结果为：判断为狗的图片为720张，判断为猫的图片为80张，剩余的200张图片无法做出判断。
在该种情况下，计算结果是狗、实际结果也是狗的图片为720张，计算结果是猫、实际结果不是猫的图片为0张，即TP为720；计算结果是狗、实际结果不是狗的图片为0张，计算结果是猫、实际结果不是猫的图片为80张，即FP为80；计算结果不是狗(包括计算结果为猫的图片的数量和无法判断的图片的数量之和)、实际结果是狗的图片为280张，计算结果不是猫、实际结 果是猫的图片为0张，即FN为280；计算结果不是狗、实际结果也不是狗的图片为0张，计算结果不是猫(包括计算结果为狗的图片的数量和无法判断的图片的数量之和)、实际结果也不是猫的图片为920张，即TN为920。
将TP、FP、FN和TN的值代入公式(1)、(2)、(3)和(4)中，其中，β为1，能够得到：准确率为0.82，精确率为0.9，召回率为0.72，F分数为0.8。
在一些实施例中，根据推理计算装置100的实际推理计算需求，为评估参数设定下限值。
例如，在对第一推理模型的至少一次性能评估中，若得到的评估参数值均低于该下限值，则评估结果判定第一推理模型的性能下降；若得到的评估参数均高于或等于该下限值，则评估结果判定第一推理模型性能符合要求。
又例如，在几次连续的性能评估中，若得到的评估参数时而高于或等于该下限值，时而低于该下限值，则评估结果判定推理模型的性能波动；若得到的参数均高于或等于该下限值，则评估结果判定第一推理模型性能符合要求。
示例性的，以评估参数为精确率为例，精确率的下限值设定为0.95。在一次性能评估中，得到该第一推理模型的精确率为0.92，则评估结果为性能下降。在另一个示例中，若在五次连续的性能评估中，得到该第一推理模型的精确率依次为0.96、093、097、0.92、0.95，则评估结果为性能波动。
在一些实施例中，根据S30中的对第一推理模型的性能进行的评估，需要对第一推理模型进行更新的情况包括：第一推理模型出现性能波动或性能下降。其中，根据评估参数在连续的时间段内的变化情况，判断第一推理模型是否出现性能波动或性能下降。
在一些实施例中，参见图3和图6，S40中对第一推理模型进行的更新，包括：
S401a、基于第二训练样本库，进行模型训练，得到第二推理模型；或者，S401b、向模型训练装置200发送模型更新请求，得到第二推理模型。其中，第二训练样本库包括：来自历史数据的训练样本，和/或来自推理结果且经过复判的训练样本。
S402、在第二推理模型满足更新条件的情况下，用第二推理模型更新第一推理模型。
需要说明的是，上述“推理结果”指的是即为在S20中，由第一推理模型进行推理计算后得到推理结果。这里以及后文中提到的“来自推理结果且经过复 判”指的是，对第一推理模型的推理结果进行复判，例如对第一推理模型的推理结果进行人工判断处理，得到标记有正确结果的生产数据。示例性的，以显示面板的生产制备为例，待处理数据为显示面板的原始产品图像，将这些显示面板的原始产品图像发送给推理计算装置100，并通过第一推理模型对其进行推理计算，得到推理结果，而后将该推理结果经过复判后，即得到已标记有正确结果的显示面板的产品图像，将此作为第二样训练本库150的训练样本的来源。
第二训练样本库包括来自推理结果且经过复判的训练样本，由于推理结果会在生产制备过程中不断地由推理计算装置100生成，因此将此作为第二训练样本库的训练样本的来源，会使第二训练样本库的训练样本较新，包括有生产制备过程中生产数据新出现的特征的训练样本。这样，能够保证推理计算装置100在模型训练中使用的为较新的训练样本，因此，与第一推理模型相比，所得到的第二推理模型对新出现的、带有新特征的待处理数据的推理计算能力较强。在用第二推理模型对第一推理模型进行更新的情况下，能够增加推理计算装置100的推理结果的准确性。
这里，对于S401a中的第二训练样本库，在一个示例中，如图3所示，推理计算装置100包括该第二训练样本库，第二训练样本库存储于存储器120中。而在另一个示例中，第二训练样本库存储于其他存储器中，推理计算装置100可从该其他存储设备中获取到第二训练样本库。
示例性的，参见图3，推理计算装置100可作为边缘侧设备，模型训练装置可作为云端设备，即推理计算装置100(边缘侧设备)所处位置相对于模型训练装置200(云端设备)更靠近用户侧设备。
进一步的，推理计算装置100还包括耦合接口150，耦合接口150可通过有线或无线的方式使推理计算装置100与用户侧设备通信，实现信息交互。此处，耦合接口150可以是任何硬件的或软件的、能够实现有线或者无线的连接的接口。
推理计算装置100还包括网络接口140，网络接口140与模型训练装置200进行通信连接，实现信息交互。此处，推理计算装置100通过该网络接口140，可利用传输控制协议/网际协议(Transmission Control Protocol/Internet Protocol，简称为TCP/IP)或超文本传输协议(Hyper Text Transfer Protocol，简称为HTTP)等协议中的至少一种，与外界(例如模型训练装置200)进行信息交互。
例如，推理计算装置100通过网络接口140与模型训练装置200通信连接，从而能够接收第一推理模型，或能够进行S401b中的向模型训练装置200发送模型更新请求，并获得第二推理模型。也即，实现了与模型训练装置200的信息交互。
在一些实施例中，图6所示的第二推理模型需要满足的更新条件包括：对第二推理模型进行测试，第二推理模型通过测试。
其中，所述测试包括：基于测试样本，对第二推理模型的性能进行评估。若性能符合评估要求，则判定为通过该测试。这里，对于测试样本的来源不做具体限定，例如，该测试样本来自所述历史数据，也即已被标记有正确的识别信息的生产数据；又例如，该测试样本来自已经过人工判断后的待处理数据；又例如，该测试样本部分来自所述历史数据，部分来自经过人工判断的待处理数据。当然，如上所述的测试样本的来源不仅限于这些。
此外，也可以根据一定的测试频率和一定的测试程度，在多个测试样本中挑选出特定的测试样例，据此生成至少一个测试计划，而后推理计算装置100根据所述至少一个测试计划对如上所述的第二推理模型进行测试。
基于此，在一些实施例中，如上所述的更新条件还包括：对第二推理模型进行灰度部署，并在灰度部署过程中对该第二推理模型的性能进行评估，该第二推理模型的性能符合评估要求。
“灰度部署”指的是在预设时间内，推理计算装置100使用第二推理模型进行待处理数据的模拟处理。示例性的，灰度部署过程中，推理计算装置100同时使用第一推理模型和第二推理模型对用户侧设备发送来的待处理数据进行处理。在此过程中，推理计算装置100会输出第一推理模型的推理结果，而第二推理模型的推理结果仅作为评估其性能使用，而不会作为真正的推理结果输出，即推理计算装置100不会基于灰度部署过程中第二推理模型得到的推理结果，生成决策指令，因此，对于第二推理模型的灰度部署将不会影响到推理计算装置100对于待处理数据的处理，其对生产线的正常生产没有影响。
通过对第二推理模型在模拟处理过程中的性能进行评估，预估第二推理模型的性能是否能够满足要求。灰度部署过程中，第二推理模型所进行推理计算的待处理数据均还需要进行人工判断，这里，人工判断的结果作为正确结果，以供推理计算装置100将人工判断的结果和第二推理模型的推理结果进行比较，从而在灰度部署过程中对该第二推理模型的性能进行评估。
示例性的，将用户侧设备发送来的显示面板的原始产品图像，一份发送给推理计算装置100，用于灰度部署过程中的第二推理模型的的模拟处理，一份用于人工判断得到正确结果，推理计算装置100比较推理结果和正确结果，从而实现对第二推理模型的性能评估。
此外，本领域技术人员应当理解，灰度部署中对第二推理模型的性能进行评估时，同样可以将准确率(Accuracy)、精确率(Precision)、召回率(Recall)、及F分数(FScore)中的至少一者做为评估参数，对第二推理模型的性能进行评估，此处不做具体限定。
需要说明的是，在如上所述的测试和灰度部署中，经人工判断的待处理数据已被标记了正确结果。因此可将该标记了正确结果的待处理数据发送至第一训练样本库(第一训练样本库为模型训练装置200进行模型训练时所基于的训练样本库)和/或第二训练样本库，以将其作为模型训练中的训练样本，这样能够实现对第一训练样本库和/或第二训练样本库的更新。
在一些实施例中，如上所述的更新条件还包括：对第二推理模型的配置信息进行校验，第二推理模型的配置信息与需进行的推理计算相匹配。
这里，“需进行的推理计算”指的是推理计算装置100对待处理数据所进行的推理计算。以显示面板的生产制备过程中，需要识别缺陷和缺陷类型为例，则“需进行的推理计算”指的是推理计算装置100需要对原始产品图像中的缺陷进行识别，并进一步识别缺陷类型。这样，通过对第二推理模型的配置信息进行校验，能够检测出第二推理模型在传输过程中其数据是否被破坏或篡改，从而保证推理计算装置100所得到的第二推理模型与推理计算装置100的推理计算需求相匹配。
在一些实施例中，参见图6，在基于第二训练样本库，进行模型训练(S401a)之前；或者，向模型训练装置200发送模型更新请求(S401b)之前，推理计算装置100还进行以下操作：
S35、判断所进行的模型训练所需要的训练参数是否在设定的训练参数范围内。若是，则进行所述模型训练(S401a)，也即该种情况下，推理计算装置100进行模型训练，得到第二推理模型；若否，则向模型训练装置200发送模型更新请求，也即该种情况下，由模型训练装置200进行模型训练，得到第二推理模型。
这里，“设定的训练参数范围”指的是，使得推理计算装置100的训练能力符合所进行的模型训练的要求时所对应的训练参数范围，也就是说，所进行 的模型训练的训练参数若在设定的训练参数范围内，那么推理计算装置100的训练能力符合所进行的模型训练的要求，这种情况下，推理计算装置100本身的模型训练能力可以完成此次模型训练，因此，推理计算装置100进行此次模型训练即可得到第二推理模型。反之，所进行的模型训练的训练参数若不在设定的训练参数范围内，那么推理计算装置100的模型训练能力不符合所进行的模型训练的要求，这种情况下，推理计算装置100本身的训练能力不足以完成此次模型训练，因此需要向模型训练装置200发送模型更新请求，使模型训练装置200来完成此次模型训练，得到第二推理模型。示例性的，模型训练装置200为云端设备。
在一些实施例中，如上所述的训练参数包括：模型训练中所需数据规模、训练时长、及计算能力中的至少一种。
示例性的，以训练参数为模型训练中所需数据规模和预计训练时长为例，某次模型训练中所需数据规模为2000张图像和对应2000个标签，所需训练时长为15min；设定的数据规模为3000张图像和3000个对应标签，设定的训练时长为20min。这种情况下，模型训练中所需的数据规模在设定的数据规模的范围之内，所需的训练时长在设定的训练时长的范围之内，也就是说，推理计算装置100的模型训练能力符合模型训练要求，则推理计算装置100进行此次模型训练(也即进行S401a)。
反之，若在另一次模型训练中所需的数据规模不在设定的数据规模的范围之内，所需的训练时长不在设定的训练时长的范围之内，说明边缘侧设备的模型训练能力不符合此次模型训练要求，则推理计算装置100向模型训练装置200发送模型更新请求，以使模型训练装置200进行此次模型训练。
如此一来，在推理计算装置100的模型训练能力符合一次模型训练要求的情况下，让推理计算装置100进行模型训练，节省了向模型训练装置200发送请求的时间和接收来自模型训练装置200的第二推理模型的时间，推理计算装置100能够更及时地得到满足推理计算需求的推理模型，对推理模型进行更新，从而进一步提高了推理结果的准确性。
此外，在如上所述的模型训练装置200为云端设备的情况下，与均需请求云端设备进行模型训练的方案相比，在推理计算装置100符合模型训练要求的情况下，让推理计算装置100自主进行模型训练，能够减轻云端设备的负担，减少对云端设备资源的不必要的占用，从而提高了云端设备的资源利用率。
在一些实施例中，参见图6，在用第二推理模型更新第一推理模型的情况下，也即在进行了S402之后，推理计算装置100还进行以下操作：
对第二推理模型进行性能评估，比较第一推理模型与第二推理模型的性能。
若第二推理模型的性能比第一推理模型的性能差，则将第二推理模型回滚至第一推理模型，并重新进行模型训练，或重新请求模型训练装置200进行模型训练。
如此一来，在实际处理待处理数据的过程中，若第二推理模型的性能不如第一推理模型的性能，推理计算装置100能够将更新后的第二推理模型回滚至更新前的第一推理模型。而后推理计算装置100重新进行模型训练，或者重新请求模型训练装置200进行模型训练。
在一些实施例中，参见图7，根据S30中的对第一推理模型的性能评估，需要对第一推理模型进行更新的情况包括：
S80、接收第三推理模型。
该第三推理模型为模型训练装置200在未接收到推理计算装置100的模型更新请求的情况下，基于更新后的第一训练样本库进行模型训练得到。其中，更新后的第一训练样本库包括来自所述推理结果且经过复判的训练样本，也就是说，更新后的第一训练样本库为使用来自所述推理结果且经过复判的训练样本对第一训练样本库进行更新得到的。
S81、比较第一推理模型与第三推理模型的性能。
S82、若第三推理模型性能高于第一推理模型的性能，或者，若第一推理模型出现异常，用第三推理模型更新第一推理模型模型。
这样，更新后的第一训练样本库的训练样本较新，包括有生产制备过程中生产数据新出现的特征的训练样本，能够保证模型训练装置200在模型训练中使用的为较新的训练样本。因此，与第一推理模型相比，所得到的第三推理模型对新出现的、带有新特征的待处理数据的推理计算能力较强。在用第三推理模型对第一推理模型进行更新的情况下，能够增加推理计算装置100的推理结果的准确性。
此外，第一推理模型出现异常指的是，第一推理模型无法输出正确的推理结果，例如，第一推理模型性能突然下降；或者，第一推理模型不按要求输出推理结果；或者，第一推理模型的推理计算时间过长、超时严重。也就是说，第三推理模型可作为备份推理模型，在第一推理模型出现异常状况时，对其进 行替换。并且，在对第一推理模型的性能进行评估后，需要对第一推理模型进行更新，也即第一推理模型的性能不再符合要求，这种情况下，可在推理计算装置100或模型训练装置200进行模型训练的这段时间内，用第三推理模型代替性能不符合要求的第一推理模型进行待处理数据的处理，以保证推理计算装置100的正常运行。
在一些实施例中，参见图6，在得到推理结果(S20)之后，推理计算装置100还进行以下操作：S70、根据推理结果，生成决策指令。
其中，缺陷信息包括：识别出的具有缺陷的原始产品图像，及其缺陷位置和缺陷类型。决策指令包括：根据原始产品图像的缺陷信息，对具有缺陷的原始产品图像对应的产品进行相应的缺陷处理。例如，根据识别出的缺陷类型，给出对应该类型缺陷的修复策略；又例如，基于历史数据分析，给出可能产生该类型缺陷的原因(如某一台或几台工艺设备出现故障，或某些工艺参数设置不合理等)；又例如，出现未知缺陷或大面积缺陷时，进行报警等。
需要说明的是，决策指令也可以不是推理计算装置100生成的。例如，推理计算装置100在得到推理结果后，将推理结果上传至云端设备，由云端设备对该推理结果进行计算分析，并生成决策结果指令，再发送回推理计算装置100，在这种情况下，推理计算装置100做为前文提到的用户侧-边缘侧-云端计算架构中的边缘侧设备，并且边缘侧设备在对待处理数据做了初步处理(推理计算)后，将推理结果发送给云端设备，云端设备可通过其强大的计算能力，作出决策指令发送回边缘侧设备。这样，能够实现对于工厂生产设备的统一规划、管理和配置等。
参见图5和图6，本公开的一些实施例还提供了一种模型训练装置200，包括至少一个处理器210和存储器220。其中，存储器220中存储有程序指令，该程序令可由所述至少一个处理器210执行，模型训练装置200进行以下操作A10～A40：
A10、基于第一训练样本库进行模型训练，得到第一推理模型。其中，第一训练样本库包括来自生产制造阶段产生的历史数据的训练样本。
这里，历史数据同样是指已经过人工判断的生产数据，为标记有正确结果的生产数据。
示例性的，模型训练装置200包括该第一训练样本库，第一训练样本库存储于存储器220中。作为另一种示例，第一训练样本库被存储于存储器220以外的其他存储器中。
A20、将第一推理模型发送至推理计算装置100。
在接收来自推理计算装置100的模型更新请求的情况下，
A30、基于更新后的第一训练样本库进行模型训练，得到第二推理模型。其中，更新后的第一训练样本库包括来自第一推理模型的推理结果且经过复判的训练样本。
A40、将第二推理模型发送至推理计算装置100。
由于更新后的第一训练样本库的训练样本较新，包括有生产制备过程中生产数据新出现的特征的训练样本，因此能够保证模型训练装置200在模型训练中使用的为较新的训练样本。基于更新后的第一训练样本库进行模型训练后得到的第二推理模型，对新出现的、带有新特征的待处理数据的推理计算能力较强。在用第二推理模型对第一推理模型进行更新的情况下，能够增加推理计算装置100的推理结果的准确性。
示例性的，参见图5，模型训练装置200还包括网络接口240，模型训练装置200能够通过网络接口240与其他设备(例如，用户侧设备和/或模型训练装置200)进行通信连接，实现信息交互。
此处，模型训练装置200通过该网络接口240，可利用传输控制协议/网际协议(Transmission Control Protocol/Internet Protocol，简称为TCP/IP)或超文本传输协议(Hyper Text Transfer Protocol，简称为HTTP)等协议中的至少一种，与外界(例如推理计算装置100)进行信息交互。
例如，模型训练装置200通过网络接口240与推理计算模型100通信，从而可以将第一推理模型发送至推理计算装置100，或接收来自推理计算装置100的模型更新请求，或将第二推理模型发送至推理计算装置100。
在一些实施例中，在基于第一训练样本库进行模型训练的过程中(即A10中)，或基于更新后的第一训练样本库进行模型训练的过程中(即A30中)，参见图8，模型训练装置200进行以下操作：
B1、生成训练任务信息。
B2、根据训练任务信息，生成训练配置数据。
B3、从相应的训练样本库获取训练样本，基于训练样本和训练配置数据，进行模型训练，得到相应的推理模型。
这里，及后文中提到的“相应的训练样本库”和“相应的推理模型”是相对于所进行的模型训练而言的，即若进行A10，则会从第一训练样本库获取训练样本，经过模型训练后，得到第一推理模型。若进行的是A30，则会从更新后 的第一训练样本库获取训练样本，经过模型训练后，得到第二推理摸模型。
此外，训练任务信息是根据接收的推理计算装置100发送来的模型更新请求而生成的，其包括模型训练的训练要求，例如，通过此次模型训练所获得的第二推理模型需要具有特定的推理计算功能，并能够得到计算结果。
训练配置数据中包括根据训练要求生成训练调度信息，根据训练调度信息能够得知此次模型训练所需调用的训练样本，示例性的，训练调度信息包括数据标识，通过该数据标识能够查找到模型训练所需调用的训练样本。
示例性的，参见图9，在B3中进行模型训练的过程中，模型训练装置200还进行以下操作：
B31、监控模型训练的过程。
B32、在模型训练的过程出现异常的情况下，更新训练配置数据。
B33、基于训练样本和更新后的训练配置数据进行模型训练。
在一些实施例中，在将第一推理模型发送至推理计算装置100之前(即图6中A20之前)，或第二推理模型发送至推理计算装置100之前(即图6中A40之前)，模型训练装置100还进行以下操作：
对相应的推理模型的模型版本信息进行校验，判断模型版本信息是否正确。
若模型版本信息是正确的，则向推理计算装置100发送相应的推理模型。
这里，在发送的是第一推理模型的情况下，若判断模型版本信息正确，则进行A20；在发送的是第二推理模型的情况下，若判断模型版本信息正确，则进行A40。
若模型版本信息不是正确的，则查找模型版本符合要求的相应的推理模型。在查找到的情况下，将其发送至推理计算装置100；或者，在未查找到的情况下，向推理计算装置100发送通知信息。
推理计算装置100可以根据接收到的通知信息，来请求模型训练装置重新进行模型训练，也可以停止操作。
这样，在将第一推理模型和或第二推理模发送之前，检测其模型版本信息是否正确，能够保证将正确版本的第一推理模型或第二推理模型发送至推理计算装置100。
在一些实施例中，如图5和图7所示，模型训练装置200包括第一训练样本库，模型训练装置200还进行以下操作：
A50、对第一训练样本库进行更新。
A60、在未接收到推理计算装置100发送来的模型更新请求的情况下，基于更新后的第一训练样本库进行模型训练，得到第三推理模型。
A70、将第三推理模型发送至推理计算装置100。
需要说明的是，第三推理模型是模型训练装置200在未接收模型更新请求的情况下通过模型训练得到的，也就是说，第三推理模型是模型训练装置200根据更新后的第一训练样本库自行进行模型训练后得到的。
由于前文已经对第三推理模型进行了详细介绍，因此此处不再对其进行赘述。
本公开的一些实施例中还提供了一种计算机可读存储介质，设置于如上任一实施例中的推理计算装置100中，该计算机可读存储介质中存储有程序指令，程序指令可由推理计算装置100中的至少一个处理器110执行以使该推理计算装置100进行以下操作S10～S40：
S10、接收第一推理模型。其中，第一推理模型基于第一训练样本库，通过模型训练得到，第一训练样本库包括来自历史数据的训练样本。
S20、基于第一推理模型，进行工业生产中的待处理数据的推理计算，并得到推理结果。
S30、评估第一推理模型的性能，以确定是否需要对第一推理模型进行更新。
S40、若是，则更新第一推理模型。
该计算机可读存储介质同样具有提高推理结果准确性的效果，此处不再赘述。
本公开的一些实施例中还提供了一种计算机可读存储介质，设置于如上任一实施例中所述的模型训练装置200中，该程序指令可由模型训练装置200中的至少一个处理器210执行使模型训练装置200进行以下操作A10～A40：
A10、基于第一训练样本库进行模型训练，得到第一推理模型。第一训练样本库包括来自历史数据的训练样本。
A20将第一推理模型发送至推理计算装置100。
在接收来自推理计算装置100的模型更新请求的情况下，
A30、基于更新后的第一训练样本库进行模型训练，得到第二推理模型。该更后的第一训练样本库为使用来自第一推理模型的推理结果且经过复判的训练样本对第一训练样本库进行更新得到的。
A40、将第二推理模型发送至推理计算装置100。
该计算机可读存储介质同样具有提高推理结果准确性的效果，此处不再赘述。
本公开的一些实施例中还提供了一种推理计算系统1000，参见图10，包括：如上任一实施例所述的推理计算装置100，以及如上任一实施例所述的模型训练装置200。
该推理计算系统1000中所包括的推理计算装置100和模型训练装置200具有提高推理结果准确性的效果，因此该模型部署系统1000同样具有提高推理结果准确性的效果，此处不再赘述。
需要说明的是，本公开实施例中所提到的处理器(例如，处理器110和处理器210)，可以是中央处理器(Central Processing Unit，CPU)，通用处理器，数字信号处理器(Digital Signal Processor，DSP)，专用集成电路(Application-Specific Integrated Circuit，ASIC)，现场可编程门阵列(Field Programmable Gate Array，FPGA)或者其他可编程逻辑器件、晶体管逻辑器件、硬件部件或者其任意组合。其可以实现或执行结合本公开所描述的各种示例性的逻辑方框和模块。处理器210也可以是实现计算功能的组合，例如包含一个或多个微处理器组合，DSP和微处理器的组合等等。
此外，本公开实施例所提到的存储器(例如，存储器120和存储器220)，可以是随机存取存储器(Random Access Memory，RAM)、闪存、只读存储器(Read Only Memory，ROM)、可擦除可编程只读存储器(Erasable Programmable ROM，EPROM)、电可擦可编程只读存储器(Electrically EPROM，EEPROM)、寄存器、硬盘、移动硬盘、只读光盘(CD-ROM)或者本领域熟知的任何其它形式的存储介质。
本公开的一些实施例提供了一种推理计算装置300。在采用对应各个功能划分各个功能模块的情况下，参见图11，该推理计算装置300包括：性能评估模块11、推理模型模块13和模型任务管理模块14。
其中，模型任务管理模块14被配置为接收来自模型训练装置400的第一推理模型；推理模型模块13被配置为基于接收的第一推理模型，进行生产制造阶段产生的待处理数据(来自用户侧设备3)的推理计算，得到推理结果，并将推理结果发送至用户侧设备3；性能评估模块11被配置为评估所述第一推理模型的性能，以确定是否需要对所述第一推理模型进行更新，若是，则由模型任务管理模块14更新所述第一推理模型。
在一些实施例中，推理计算装置300还包括第二模型训练模块12和第二 训练样本库19。在需要更新第一推理模型的情况下，第二模型训练模块12被配置为基于第二训练样本库19，进行模型训练，得到第二推理模型；或者，性能评估模块11还被配置为向模型训练装置400发送模型更新请求，得到第二推理模型。第二训练样本库19存储有模型训练所需的训练样本，该训练样本可来自推理模型模块13，和/或模型任务管理模块14。
基于此，在一些实施例中，模型任务管理模块14还被配置为接收来自模型训练装置400的第二推理模型，并对接收的第二推理模型进行测试、灰度部署或配置信息校验中的至少一者。
示例性的，参见图11，模型任务管理模块14包括：校验单元141、测试单元142和灰度部署单元143。
其中，校验单元141被配置为对接收的第二推理模型进行测试；测试单元142被配置为对接收的第二推理模型进行测试；灰度部署单元143被配置为对接收的第二推理模型进行灰度部署。
在一些实施例中，推理计算装置300还包括模型训练决策模块18，被配置为判断所进行的模型训练所需要的训练参数是否在设定的训练参数范围内。若是，则使性能评估模块11向第二模型训练模块12发送模型更新请求，即，使第二模型训练模块12进行模型训练，得到第二推理模型；若否，则使性能评估模块11向模型训练装置400发送模型更新请求，即，使模型训练装置400进行模型训练，得到第二推理模型。
在一些实施例中，推理计算装置300还包括备份模块15，备份模块15被配置为接收第三推理模型。在第三推理模型性能高于推理模型模块11中的第一推理模型的性能的情况下，或者，在推理模型模块11中的第一推理模型出现异常的情况下，备份模块15将第三推理模型发送给模型任务管理模块14，由模型任务管理模块14用第三推理模型更新第一推理模型。
在一些实施例中，性能评估模块11还被配置为对第二推理模型进行性能评估，比较第一推理模型与第二推理模型的性能。在第二推理模型的性能比第一推理模型的性能差的情况下，模型任务管理模块14还被配置为将第二推理模型回滚至第一推理模型。性能评估模块11还被配置为重新向第二模型训练模块12或者模型训练装置400发送模型训练请求。
在一些实施例中，参见图11，推理计算装置300还包括功能扩展管理模块16，功能扩展管理模块16被配置为对推理模型模块13和任务管理模块14的功能进行扩展或修改。
例如，功能扩展模块16接收导入的推理模型，并控制推理模型模块13中的推理模型切换为该导入的推理模型。
此外，参见图11，在模型训练装置400包括第一训练样本库23的情况下，功能扩展模块16还可以对第一训练样本库的功能进行扩展。
示例性的，功能扩展模块16包括功能扩展接口。
在一些实施例中，参见图11，推理计算装置300还包括：原始数据汇总模块17和推理模型模块13。
其中，原始数据汇总模块17被配置为对来自用户侧设备3的原始数据进行汇总，得到待处理数据，并将待处理数据发送至推理模型模块13。
本公开的一些实施例提供了一种模型训练装置400。在采用对应各个功能划分各个功能模块的情况下，参见图11，该模型训练装置400包括：训练管理模块21、第一模型训练模块22和第一训练样本库23。
其中，第一模型训练模块22被配置为基于第一训练样本库23进行模型训练，得到第一推理模型，和/或基于更新后的第一训练样本库23进行模型训练，得到第二推理模型。
训练管理模块21被配置为将第一训练模型和/或第二训练模型发送至推理计算装置300。
示例性的，参见图11，训练管理模块21包括：任务管理单元211、训练调度单元212和分支管理单元213。
其中，任务管理单元211被配置为根据推理计算装置300发送的模型更新请求，生成训练任务信息；训练调度单元212被配置为根据训练任务信息，生成训练配置数据，以及从相应的训练样本库获取训练样本；分支管理单元213被配置为对相应的推理模型的模型版本信息进行校验，判断模型版本信息是否正确。在模型版本信息正确的情况下，分支管理单元213还被配置为向推理计算装置300发送版本信息正确的相应的推理模型。
示例性的，训练调度单元212还被配置为监控第一模型训练模块22的模型训练过程，在模型训练的过程出现异常的情况下，更新训练配置数据，并将更新后的训练配置数据发送给第一模型训练模块22。在这种情况下，第一模型训练模块22还被配置为根据更新后的训练配置数据和训练样本进行模型训练。
在一些实施例中，参见图11，模型训练装置400还包括第一训练样本库23，该第一训练样本库23被配置为接收由推理计算设备300发送来的已标记 正确结果的训练样本，例如，将推理计算设备300的推理模型模块13的推理结果经过复判标记有正确结果后，发送至第一训练样本库23，由此对第一训练样本库23进行更新。在这种情况下，第一模型训练模块22还被配置为在未接收到模型更新请求时，基于更新后的第一训练样本库23进行模型训练，得到第三推理模型。
示例性的，在训练管理模块21包括分支管理单元213的情况下，第一模型训练模块22被配置为将第三推理模型发送给分支管理单元213，由分支管理单元213将第三推理模型发送给推理计算装置300。
以上所述，仅为本发明的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应以所述权利要求的保护范围为准。

Claims (20)

  1. 一种推理计算装置，包括至少一个处理器和存储器；所述存储器存储有程序指令，所述程序指令可由所述至少一个处理器执行，使所述推理计算装置进行以下操作：
    接收来自模型训练装置的第一推理模型，其中，所述第一推理模型基于第一训练样本库，由所述模型训练装置通过模型训练得到；所述第一训练样本库包括来自生产制造阶段产生的历史数据的训练样本，所述模型训练装置包括云端设备；
    基于所述第一推理模型，进行生产制造阶段产生的待处理数据的推理计算，得到推理结果，所述推理结果发送至用户侧设备，所述推理计算装置相较于模型训练装置更靠近用户侧设备；
    评估所述第一推理模型的性能，以确定是否需要对所述第一推理模型进行更新，若是，则更新所述第一推理模型。
  2. 根据权利要求1所述的推理计算装置，其中，
    所述历史数据包括标记有缺陷信息的产品图像；
    所述待处理数据包括在生产制造阶段所产生的原始产品图像；
    所述推理结果包括所述原始产品图像中的缺陷信息。
  3. 根据权利要求1或2所述的推理计算装置，其中，对所述第一推理模型进行的更新包括：
    基于第二训练样本库，进行模型训练，得到第二推理模型；或者，向所述模型训练装置发送模型更新请求，得到第二推理模型；所述第二训练样本库包括来自历史数据的训练样本，和/或来自所述推理结果且经过复判的训练样本；
    在所述第二推理模型满足更新条件的情况下，用所述第二推理模型更新所述第一推理模型。
  4. 根据权利要求3所述的推理计算装置，其中，所述更新条件包括：
    对所述第二推理模型进行测试，所述第二推理模型通过所述测试；
    其中，所述测试包括基于测试样本对所述第二推理模型的性能进行评估；若性能符合评估要求，则判定为通过所述测试。
  5. 根据权利要求4所述的推理计算装置，其中，所述更新条件还包括：
    对所述第二推理模型进行灰度部署，并在所述灰度部署过程中对其性能进行评估，所述第二推理模型的性能符合评估要求。
  6. 根据权利要求5所述的推理计算装置，其中，所述更新条件还包括：
    对所述第二推理模型的配置信息进行校验，所述配置信息与需进行的推理计算相匹配。
  7. 根据权利要求3所述的推理计算装置，其中，在进行模型训练，得到第二推理模型；或者，向模型训练装置发送模型更新请求，得到第二推理模型之前，所述推理计算装置还进行以下操作：
    判断所进行的模型训练所需要的训练参数是否在设定的训练参数范围内；若是，则进行所述模型训练；若否，则向所述模型训练装置发送所述模型更新请求。
  8. 根据权利要求7所述的推理计算装置，其中，所述训练参数包括：模型训练中所需数据规模、训练时长、及计算能力中的至少一种。
  9. 根据权利要求1所述的推理计算装置，其中，用于评估所述推理模型的性能的评估参数包括：所述第一推理模型在推理计算过程中的准确率、精确率、召回率、及F分数中的至少一者；
    根据性能评估需要对所述第一推理模型进行更新的情况包括：第一推理模型出现性能波动或性能下降；其中，根据所述评估参数在连续的时间段内的变化情况，判断所述第一推理模型是否出现性能波动或性能下降。
  10. 根据权利要求3所述的推理计算装置，其中，根据性能评估需要对所述第一推理模型进行更新的情况包括：
    接收第三推理模型，所述第三推理模型为所述模型训练装置在未接收到所述模型更新请求的情况下，基于更新后的第一训练样本库进行模型训练得到；所述更新后的第一训练样本库包括来自所述推理结果且经过复判的训练样本；
    比较所述第一推理模型与所述第三推理模型的性能，若所述第三推理模型性能高于所述第一推理模型的性能，或者，若所述第一推理模型出现异常，用所述第三推理模型更新所述第一推理模型模型。
  11. 根据权利要求3所述的推理计算装置，其中，在用所述第二推理模型更新所述第一推理模型的情况下，所述推理计算装置还进行以下操作：
    对所述第二推理模型进行性能评估，比较所述第一推理模型与所述第二推理模型的性能；
    在所述第二推理模型的性能比所述第一推理模型的性能差的情况下，将所述第二推理模型回滚至所述第一推理模型；重新进行模型训练，或者重新请求所述模型训练装置进行模型训练。
  12. 根据权利要求2所述的推理计算装置，其中，在得到所述推理结果之后，所述推理计算装置还进行以下操作：
    根据所述推理结果，生成决策指令；
    其中，所述缺陷信息包括：识别出的具有缺陷的原始产品图像，及其缺陷位置和缺陷类型；所述决策指令包括：根据所述原始产品图像的缺陷信息，对所述具有缺陷的原始产品图像对应的产品进行相应的缺陷处理。
  13. 一种模型训练装置，包括至少一个处理器和存储器；所述存储器存储有程序指令，所述程序指令可由所述至少一个处理器执行，使所述模型训练装置进行以下操作：
    基于第一训练样本库进行模型训练，得到第一推理模型；所述第一训练样本库包括来自生产制造阶段产生的历史数据的训练样本；
    将所述第一推理模型发送至推理计算装置；
    在接收来自所述推理计算装置的模型更新请求的情况下，基于更新后的第一训练样本库进行模型训练，得到第二推理模型；所述更新后的第一训练样本库包括来自所述第一推理模型的推理结果且经过复判的训练样本；
    将所述第二推理模型发送至所述推理计算装置。
  14. 根据权利要求13所述的模型训练装置，其中，在基于所述第一训练样本库，或基于所述更新后的第一训练样本库进行模型训练的过程中，所述模型训练装置进行以下操作：
    生成训练任务信息；
    根据所述训练任务信息，生成训练配置数据；
    从相应的训练样本库获取训练样本，基于所述训练样本和所述训练配置数据，进行模型训练，得到相应的推理模型。
  15. 根据权利要求14所述的模型训练装置，其中，在所述从相应的训练样本库获取训练样本，基于所述训练样本和所述训练配置数据，进行模型训练，得到相应的推理模型的过程中，所述模型训练装置还进行以下操作：
    监控模型训练的过程；
    在模型训练的过程出现异常的情况下，更新所述训练配置数据；
    基于所述训练样本和更新后的训练配置数据进行模型训练。
  16. 根据权利要求13～15中任一项所述的模型训练装置，其中，在将所述第一推理模型或所述第二推理模型发送至所述推理计算装置之前，所述模型训练装置还进行以下操作：
    对相应的推理模型的模型版本信息进行校验，判断所述模型版本信息是否正确；
    若是，则向所述推理计算装置发送相应的推理模型；
    若否，则查找模型版本符合要求的相应的推理模型；在查找到的情况下，将其发送至所述推理计算装置；或者，在未查找到的情况下，向所述推理计算装置发送通知信息。
  17. 根据权利要求13所述的模型训练装置，还包括所述第一训练样本库；所述模型训练装置还进行以下操作：
    对所述第一训练样本库进行更新；
    在未接收所述模型更新请求的情况下，基于更新后的第一训练样本库进行模型训练，得到第三推理模型；
    将所述第三推理模型发送至所述推理计算装置。
  18. 一种计算机可读存储介质，存储有程序指令；所述程序指令可被处理器执行，使所述推理计算装置进行以下操作：
    接收第一推理模型，所述第一推理模型基于第一训练样本库，通过模型训练得到；所述第一训练样本库包括来自历史数据的训练样本；
    基于所述第一推理模型，进行工业生产中的待处理数据的推理计算，并得到推理结果；
    评估所述第一推理模型的性能，以确定是否需要对所述第一推理模型进行更新，若是，则更新所述第一推理模型。
  19. 一种计算机可读存储介质，存储有程序指令；所述程序指令可被处理器执行，使所述模型训练装置进行以下操作：
    基于第一训练样本库进行模型训练，得到第一推理模型；所述第一训练样本库包括来自历史数据的训练样本；
    将所述第一推理模型发送至推理计算装置；
    在接收来自所述推理计算装置的模型更新请求的情况下，基于更新后的第一训练样本库进行模型训练，得到第二推理模型；所述更新后的第一训练样本库为使用来自所述第一推理模型的推理结果且经过复判的训练样本对所述第一训练样本库进行更新得到的；
    将所述第二推理模型发送至所述推理计算装置。
  20. 一种推理计算系统，包括：如权利要求1～12中任一项所述的推理计算装置，以及如权利要求13～17中任一项所述的模型训练装置。
PCT/CN2019/127048 2019-12-20 2019-12-20 推理计算装置、模型训练装置、推理计算系统 WO2021120181A1 (zh)

Priority Applications (6)

Application Number Priority Date Filing Date Title
CN201980003145.8A CN113597618A (zh) 2019-12-20 2019-12-20 推理计算装置、模型训练装置、推理计算系统
JP2021571382A JP7442550B2 (ja) 2019-12-20 2019-12-20 推論演算装置、モデル訓練装置、及び推論演算システム
US17/044,276 US20210209488A1 (en) 2019-12-20 2019-12-20 Inference computing apparatus, model training apparatus, inference computing system
KR1020227001528A KR20220117194A (ko) 2019-12-20 2019-12-20 추론 연산 장치, 모델 훈련 장치, 및 추론 연산 시스템
PCT/CN2019/127048 WO2021120181A1 (zh) 2019-12-20 2019-12-20 推理计算装置、模型训练装置、推理计算系统
EP19955399.1A EP4080407A4 (en) 2019-12-20 2019-12-20 COMPUTER INFERENCE DEVICE, PATTERN LEARNING DEVICE AND COMPUTER LEARNING SYSTEM

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/127048 WO2021120181A1 (zh) 2019-12-20 2019-12-20 推理计算装置、模型训练装置、推理计算系统

Publications (1)

Publication Number Publication Date
WO2021120181A1 true WO2021120181A1 (zh) 2021-06-24

Family

ID=76478141

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/127048 WO2021120181A1 (zh) 2019-12-20 2019-12-20 推理计算装置、模型训练装置、推理计算系统

Country Status (6)

Country Link
US (1) US20210209488A1 (zh)
EP (1) EP4080407A4 (zh)
JP (1) JP7442550B2 (zh)
KR (1) KR20220117194A (zh)
CN (1) CN113597618A (zh)
WO (1) WO2021120181A1 (zh)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11574246B2 (en) * 2020-01-21 2023-02-07 Microsoft Technology Licensing, Llc Updating training examples for artificial intelligence
KR20220030339A (ko) * 2020-08-27 2022-03-11 삼성디스플레이 주식회사 이물 불량 검출 방법 및 이물 불량 검출 시스템
KR20220049165A (ko) * 2020-10-14 2022-04-21 삼성에스디에스 주식회사 추론 데이터 기반 예측 모델 보정 시스템 및 방법
US12008487B2 (en) * 2020-12-15 2024-06-11 International Business Machines Corporation Inference model optimization
CN114741269B (zh) * 2022-04-14 2022-09-23 网思科技股份有限公司 一种推理系统业务性能评估的方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101242549A (zh) * 2007-11-22 2008-08-13 中国移动通信集团山东有限公司 通信网络告警关联的神经网络构建方法
CN103078897A (zh) * 2012-11-29 2013-05-01 中山大学 一种实现Web业务细粒度分类与管理的系统
CN107256009A (zh) * 2017-06-30 2017-10-17 武汉理工大学 一种基于深度学习的产品数字模型智能装配系统
CN109635171A (zh) * 2018-12-13 2019-04-16 成都索贝数码科技股份有限公司 一种新闻节目智能标签的融合推理系统和方法
CN109740661A (zh) * 2018-12-28 2019-05-10 沈阳工业大学 一种基于模糊评估的油井动液面自适应预测方法

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI385492B (zh) * 2008-12-16 2013-02-11 Ind Tech Res Inst 機台設備的維護分析系統及其方法
US11127020B2 (en) * 2009-11-20 2021-09-21 Palo Alto Research Center Incorporated Generating an activity inference model from contextual data
WO2012054339A1 (en) * 2010-10-19 2012-04-26 3M Innovative Properties Company Computer-aided assignment of ratings to digital samples of a manufactured web product
US8700658B2 (en) * 2011-09-01 2014-04-15 Knf Technologies, Inc. Relational meta model and associated domain context-based knowledge inference engine for knowledge discovery and organization
US10614373B1 (en) * 2013-12-23 2020-04-07 Groupon, Inc. Processing dynamic data within an adaptive oracle-trained learning system using curated training data for incremental re-training of a predictive model
US20150324686A1 (en) * 2014-05-12 2015-11-12 Qualcomm Incorporated Distributed model learning
US20170154273A1 (en) 2015-11-30 2017-06-01 Seematics Systems Ltd System and method for automatically updating inference models
WO2018208791A1 (en) * 2017-05-08 2018-11-15 Aquifi, Inc. Systems and methods for inspection and defect detection using 3-d scanning
US10698372B2 (en) * 2017-06-02 2020-06-30 Aspen Technology, Inc. Computer system and method for building and deploying predictive inferential models online to predict behavior of industrial processes
CN107316083B (zh) * 2017-07-04 2021-05-25 北京百度网讯科技有限公司 用于更新深度学习模型的方法和装置
CN109754105B (zh) * 2017-11-07 2024-01-05 华为技术有限公司 一种预测方法及终端、服务器
JP6924413B2 (ja) 2017-12-25 2021-08-25 オムロン株式会社 データ生成装置、データ生成方法及びデータ生成プログラム
JP7056259B2 (ja) 2018-03-14 2022-04-19 オムロン株式会社 検査システム、識別システム、及び識別器評価装置
US11030071B2 (en) * 2018-04-16 2021-06-08 Armory, Inc. Continuous software deployment
WO2019229910A1 (ja) 2018-05-30 2019-12-05 株式会社ウフル 顔認証を用いた入国審査システム、入国審査方法、プログラム、および認証装置
US20200193313A1 (en) * 2018-12-14 2020-06-18 Parallel Machines, Inc. Interpretability-based machine learning adjustment during production
US10481579B1 (en) * 2019-02-28 2019-11-19 Nanotronics Imaging, Inc. Dynamic training for assembly lines
CN110378900B (zh) * 2019-08-01 2020-08-07 北京迈格威科技有限公司 产品缺陷的检测方法、装置及系统
CN110554047B (zh) * 2019-09-06 2021-07-02 腾讯科技（深圳）有限公司 产品缺陷检测数据处理方法、装置、系统和设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101242549A (zh) * 2007-11-22 2008-08-13 中国移动通信集团山东有限公司 通信网络告警关联的神经网络构建方法
CN103078897A (zh) * 2012-11-29 2013-05-01 中山大学 一种实现Web业务细粒度分类与管理的系统
CN107256009A (zh) * 2017-06-30 2017-10-17 武汉理工大学 一种基于深度学习的产品数字模型智能装配系统
CN109635171A (zh) * 2018-12-13 2019-04-16 成都索贝数码科技股份有限公司 一种新闻节目智能标签的融合推理系统和方法
CN109740661A (zh) * 2018-12-28 2019-05-10 沈阳工业大学 一种基于模糊评估的油井动液面自适应预测方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4080407A4 *

Also Published As

Publication number Publication date
KR20220117194A (ko) 2022-08-23
JP7442550B2 (ja) 2024-03-04
JP2023514466A (ja) 2023-04-06
EP4080407A4 (en) 2023-02-01
EP4080407A1 (en) 2022-10-26
CN113597618A (zh) 2021-11-02
US20210209488A1 (en) 2021-07-08

Similar Documents

Publication Publication Date Title
WO2021120181A1 (zh) 推理计算装置、模型训练装置、推理计算系统
CN110554047B (zh) 产品缺陷检测数据处理方法、装置、系统和设备
Zhang et al. Crowdlearn: A crowd-ai hybrid system for deep learning-based damage assessment applications
US11042145B2 (en) Automatic health indicator learning using reinforcement learning for predictive maintenance
US11176691B2 (en) Real-time spatial and group monitoring and optimization
JP7393515B2 (ja) 分散型製品欠陥分析システム、方法及びコンピュータ可読記憶媒体
US20190294975A1 (en) Predicting using digital twins
US20230083744A1 (en) Combined Learned and Dynamic Control System
CN110663030A (zh) 用于处理极端数据的边缘设备、系统和方法
JP2018513490A (ja) 自動化システムを動作させる方法及び装置
WO2021120179A1 (zh) 产品制造消息处理方法、设备和计算机存储介质
CN114787831B (zh) 改进分类模型的准确性
US20230176562A1 (en) Providing an alarm relating to anomaly scores assigned to input data method and system
JP2023547849A (ja) ラベルなしセンサデータを用いた産業システム内の稀な障害の自動化されたリアルタイムの検出、予測、及び予防に関する、方法または非一時的コンピュータ可読媒体
Chen Production planning and control in semiconductor manufacturing: Big data analytics and industry 4.0 applications
US11995477B2 (en) Verification of updated analytical procedures in monitoring systems
Dahiya et al. Artificial Intelligence of Things: AIoT in Various Markets of IoT Deployments.
Wang et al. Human-AI Interaction
WO2022177567A1 (en) Dependability assessment framework for railroad asset management
US20230205158A1 (en) Building data platform with analytics development
US20240046161A1 (en) Method and system for training of artificial intelligence and machine learning models
US20230196248A1 (en) Systems and methods for improving quality of artificial intelligence model
WO2024043888A1 (en) Real time detection, prediction and remediation of machine learning model drift in asset hierarchy based on time-series data

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19955399

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021571382

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2019955399

Country of ref document: EP

Effective date: 20220720