WO2022108029A1

WO2022108029A1 - Machine tool predictive maintenance system using query-based deep learning inference system, and method for same

Info

Publication number: WO2022108029A1
Application number: PCT/KR2021/007830
Authority: WO
Inventors: 이준혁
Original assignee: (주)한국플랫폼서비스기술
Priority date: 2020-11-20
Filing date: 2021-06-22
Publication date: 2022-05-27
Also published as: KR102568010B1; KR20220069738A

Abstract

The present invention can diagnose or predict faults in machine tools by using a query-based deep learning inference system wherein a deep learning framework is connected to an information database, in a plug-in form, and data stored in the information database according to a request query of a user is learned via a deep learning method, and thus data corresponding to the query can be inferred.

Description

Machine tool predictive maintenance system and method using query-based deep learning inference system

The present invention relates to a machine tool predictive maintenance system using a query-based deep learning inference system and a method therefor, and more particularly, a deep learning frame so that even users without professional knowledge on deep learning can provide necessary information to users without difficulty Using a query-based deep learning inference system that allows the work to be connected to the information database in the form of a plug-in and learn the data stored in the information database by the user's request query in a deep learning method to infer the data corresponding to the query, A system and method for diagnosing or predicting machine tool failure.

A variety of rotation system and feed system parts are damaged or deteriorated in the parts of the machine tool that are in mechanical motion, and the loss may increase, and thus the output of the machine tool may be reduced. If machine tool failures can be predicted, productivity will increase and safety accidents will be reduced. Machine learning or deep learning technology may be used to predict machine tool failure.

The learning engine using deep learning shows significantly superior intelligence performance than the learning engine based on other existing AI technologies. However, in order to create a learning engine that provides intelligence based on deep learning technology, there are various difficulties such as deep network design, learning function setting, and parameter tuning. These problems cannot be easily done without a deep learning expert, so it is difficult for anyone to easily have a deep learning-based learning engine.

Also, whenever a learning engine is created, common elements of deep learning are used repeatedly, so there is a problem that the same process must be repeated.

(Patent Document 1) Korean Patent Registration 10-205812B1

It is an object of the present invention to solve the above-mentioned problems, that the deep learning framework is connected to the information database in the form of a plug-in so that even users without professional knowledge on deep learning can provide necessary information to the user without difficulty. Query-based deep learning that diagnoses or predicts machine tool failures using a query-based deep learning inference system that learns the data stored in the information database by a request query in a deep learning method and infers data corresponding to the query To provide a machine tool predictive maintenance system and method using an inference system.

According to an embodiment of the present invention, a machine tool predictive maintenance method using a query-based deep learning inference system of a deep learning framework interworking with a user terminal and a database includes the steps of: receiving a learning query from the user terminal; A plurality of failure diagnosis predictions are made by using at least a part of the training data set, which is a state information label indicating the machine tool's vibration amount, noise level, and electric power provided by the data collector, and whether the machine tool is faulty as a suitability judgment data set a determination step of whether a suitable model exists for determining whether there is a failure diagnosis predictive model suitable for the data set for determination of suitability among models; when there is no suitable failure diagnosis predictive model as a result of determining whether the suitable model exists, initializing a network according to the learning query and configuring the network; a learning execution step of executing learning using the collected learning data set when all layers of the configured network are initialized; and providing a result when the learning execution is completed and storing the finished failure diagnosis predictive model.

In addition, receiving an inference query (Call Test) from the user terminal; collecting a diagnostic data set having an amount of vibration, noise, and power of the machine tool from the data collector; calling an appropriate failure diagnosis predictive model from among the plurality of failure diagnosis predictive models by analyzing the diagnostic data set; a reasoning execution step of constructing a network according to the suitable failure diagnosis predictive model and executing reasoning based on the diagnosis data set when all layers of the network are initialized; And when the reasoning execution is finished, diagnosing the failure of the machine tool and predicting the failure; may further include. In addition, the learning execution step acquires batch data until the end of the learning (Get Batch Data) and repeats (Iteration) to store the results and model (Store Result & Model), and when exporting information or data from the database to the outside A model weight file including ONNX, NNEF, and training parameter bias may be converted into a structured format through the model converter. In addition, in the reasoning execution step, the reasoning test may be executed, and test data may be obtained (Get Test Data) to feedforward. In addition, in the step of receiving the learning query or inference query, the deep learning framework uses a model converter for compatibility with an external framework, and brings the pre-trained failure diagnosis predictive model of the existing framework, or When exporting information or data from the database to the outside, the model weight file including ONNX, NNEF, and training parameter bias can be converted into a structured format through the model converter. In addition, the model converter converts a network structure and model data defined in the structured format into a network model table format of the database, or vice versa, converts a network model of the database into the structured model format can be converted.

A machine tool predictive maintenance system using a query-based deep learning inference system according to an embodiment of the present invention is a detection unit that measures the amount of vibration, noise, and power of the machine tool, and the state of checking the state information of the machine tool a data collector for collecting the vibration amount, noise amount, power amount, and state information label from the recognition unit as a learning data set; a database for storing the training data set, a training network model, training parameters, and a learning result; a deep learning framework connected to the database in a plug-in manner, checking, correcting, and adding new data to information or data stored in the database; and a user terminal that inputs a query through the deep learning framework and receives an inference result corresponding to the query from the database through the deep learning framework, wherein the deep learning framework executes the query Upon receiving the input, it checks and corrects the failure diagnosis predictive model stored in the database, or creates a failure diagnosis predictive model for new learning, selects information or data and a failure diagnosis predictive model according to the input query, and sets the learning parameters. Set to execute machine learning, provide learning intermediate results and final results, select data and a pre-learned failure diagnosis predictive model through the input query to execute machine inference, and use the inference result to diagnose failure and failure It can be provided as an example.

In addition, the failure diagnosis predictive model is a model used for machine learning, and is composed of input/output and parameters defining the inside of the model, and parameters necessary for machine learning and inference, and is stored in the database in a relational data format, and through a converter It may be translatable to other deep learning frameworks. In addition, the database stores all input and output data used for machine learning and machine inference, stores models used for machine learning and machine inference, and provides a procedure corresponding to a user's query request, query Based deep learning inference systems are available. In addition, the deep learning framework is used for compatibility with external frameworks, and includes ONNX, NNEF and learning parameter bias when importing a pre-trained model of an existing framework or exporting information or data from the database to the outside A model converter using a model weight file in a structured format to A network structure and model data may be converted into a network model table format of the database, or, conversely, a failure diagnosis predictive model of the database may be converted into the structured model format. In addition, the data set is a set of information or data having the same format, and the information or data may be used for machine learning, including numbers, characters, images, images, and voices. In addition, the procedure is an Insert Network, an Insert Layer, a Make Project, an Input Data Loader, a Train, a Save Model, and a test. (Test) may be included. In addition, the database may include a dataset table, a network table, a project table, a job table, and a common table. In addition, the deep learning framework executes, when a learning query is input from the user terminal (Call Train), network initialization (Init Network), network configuration (Construct Network), and network update (Update Network), to all layers When the initialization (Initialize all layers) is made, training is executed, and batch data is obtained until the end of training (Get Batch Data), and the result and model are stored by iteration (Store Result & Model), and training is performed. Upon completion, the learning result may be provided to the user terminal.

According to the present invention, by using the query-based machine learning technology, the deep learning framework is connected to the database in the form of a plug-in and performs machine learning and inference using the data stored in the database according to the user's request query. can diagnose and predict the failure of

Therefore, even a user without professional knowledge on deep learning can easily provide necessary information without difficulty. In addition, it is possible to check the learning parameters of the currently executing or waiting learning plan, and check the middle and results of the currently executing learning plan. Also, you can stop the currently running learning plan and start a waiting learning plan.

1 is a configuration diagram schematically showing the overall configuration of a query-based deep learning inference system according to an embodiment of the present invention;

2 is a flowchart showing a flow of performing a query-based machine learning technique according to an embodiment of the present invention;

3 is a diagram illustrating an internal data flow of a database according to an embodiment of the present invention;

4 is a view showing an operation flowchart for explaining a query-based deep learning inference method according to an embodiment of the present invention;

5 is a diagram showing an execution flow of a learning procedure according to an embodiment of the present invention;

6 is a diagram showing an execution flow of an inference procedure according to an embodiment of the present invention;

7 is a view showing a conversion operation of a model converter according to an embodiment of the present invention;

8 is a schematic diagram of an internal structure of a QML framework according to an embodiment of the present invention;

9 is a configuration diagram of a machine tool predictive maintenance system using a query-based deep learning inference system according to an embodiment of the present invention;

10 is a block diagram of the data collector of FIG. 9, and

11 and 12 are flowcharts of a method for machine learning and failure diagnosis and prediction of a machine tool predictive maintenance system using a query-based deep learning inference system according to an embodiment of the present invention.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.

When an element is referred to as being “connected” or “connected” to another element, it is understood that it may be directly connected or connected to the other element, but other elements may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle. Also, when the first and second components on the network are connected or connected, it means that data can be exchanged between the first and second components by wire or wirelessly.

In addition, the suffixes "module" and "part" for the components used in the following description are given simply in consideration of the ease of writing the present specification, and do not impart a particularly important meaning or role by themselves. Accordingly, the terms “module” and “unit” may be used interchangeably.

When these components are implemented in actual applications, two or more components may be combined into one component, or one component may be subdivided into two or more components as needed. The same reference numerals are given to the same or similar components throughout the drawings, and detailed descriptions of the components having the same reference numerals may be omitted instead of being replaced with the descriptions of the above-described components.

Furthermore, the invention encompasses all possible combinations of the embodiments indicated herein. Various embodiments of the present invention are different, but not mutually exclusive. One embodiment of a particular shape, structure, function, and characteristic described herein may be embodied in another embodiment. For example, components mentioned in the first and second embodiments may perform all functions of the first and second embodiments.

1 is a configuration diagram schematically showing the overall configuration of a query-based deep learning inference system according to an embodiment of the present invention. Referring to FIG. 1 , a query-based deep learning inference system 100 according to an embodiment of the present invention may apply a query-based machine learning technique. To this end, the query-based deep learning inference system 100 may include a database (DB, 110), a deep learning framework (QML, 120), and a user terminal (130).

In the query-based deep learning technology, the deep learning framework 120 is connected to the database 110 in the form of a plug-in, and machine learning, inference, etc. are performed using data stored in the database 110 by a user's request query. it is technology

Deep learning can be a set of machine learning (machine learning) algorithms that attempt high-level abstraction through a combination of several nonlinear transformation methods. Machine learning is a field of artificial intelligence and can refer to the field of developing algorithms and technologies that allow computers to learn. Artificial intelligence refers to a computer system equipped with the functions of human intelligence, and may refer to artificially implemented human intelligence, such as a machine. In this specification, 'deep learning' is not limited to the deep learning technology itself, and can be interpreted and extended to machine learning or artificial intelligence.

The input/output module 130 may be its own interface module. The input/output module 130 may include an input device and an output device, respectively. The input/output module 130 may be implemented as one input/output device such as a touch screen.

The input/output module 130 may be implemented as a user terminal 130 as shown in FIG. 1 . In an embodiment of the present invention, the input/output module 130 will be described as an example of the user terminal 130 .

The database 110 may store a data set, a learning network model, a learning parameter, and a learning result.

A data set is a set of information or data having the same format. The information or data includes numbers, text, images, images, and voices, and may be any type of information or data used in machine learning.

The learning network model is a model used in deep learning and may be composed of input/output, parameters defining the inside of the model, and parameters necessary for machine learning and/or inference. The training network model can be stored in a database in the form of relational data. The learning network model may be convertible to other deep learning frameworks through a converter.

In this embodiment, the learning network model is a decision model that can be learned based on a data set based on an artificial intelligence algorithm, and may be a model based on a neural network. The present judgment model can be designed to simulate a human brain structure on a computer. The judgment model may include a plurality of network nodes having weights that simulate neurons of a human neural network. The plurality of network nodes may each form a connection relationship so that the neuron simulates a synaptic activity of a neuron that transmits and receives a signal through a synapse. The judgment model may include a machine learning model, a neural network model, and/or a deep learning model.

The learning network model can recognize text input by the user. The training network model can recognize speech and text contained in images, audio and video. The learning network model may analyze user intentions from the recognized speech and text.

As for the learning result, the intermediate output value is stored in the database 110 while the machine learning is in progress so that the user can check it. During machine learning, the model parameter values are stored in the database 110 so that the user can check them. The evaluation index value of the model calculated while the machine learning is in progress is stored in the database 110 so that the user can check it. The machine reasoning result value is stored in the database 110 so that the user can check it.

The database 110 may store all input/output data used for machine learning and machine inference, store models used for machine learning and machine inference, and provide a procedure corresponding to a user's query request. .

Procedures include Insert Network, Insert Layer, Make Project, Input Data Loader, Train, Save Model, and Test. can be provided

The deep learning framework 120 may be used for compatibility with external frameworks. The deep learning framework 120 may include a model converter that uses an Open Neural Network Exchange (ONNX) model format when importing a pre-trained model of an existing framework or exporting information or data from the database to the outside.

The model converter may convert the network structure and model data defined in the ONNX model format into the network model format of the database or, conversely, convert the network model of the database into the ONNX model format. In addition, the model converter may convert Open Network Exchange (ONNX), Neural Network Exchange Format (NNEF), and a model weight file having training parameters and biases into a structured format in addition to the ONNX model format. The model converter converts the model weight file into a structured format so that the training model can be input (Imported) or output (exported) from the database.

The deep learning framework 120 may be connected to the database 110 in a plug-in manner. The deep learning framework 120 may check, correct, and add new data to information or data stored in the database 110 . The deep learning framework 120 may be, for example, quantum machine learning (QML).

QML is a deep learning framework installed as a plug-in in the database 110 and may be executed by a database call. When QML is called, it may receive various data from the database 110 as an argument and return an execution result. QML can interpret the network model defined in the relational data format to compose the network within the framework. QML may receive learning parameters and learning data from the database 110 as factors, perform learning of a network configured in the framework, and return a learning result. QML may receive input data from the database 110 as an argument, perform machine inference using a network configured in the framework, and return a result.

The deep learning framework 120 may receive a query input from the user terminal 130 . The deep learning framework 120 may generate a learning network model for confirmation, correction, and new learning of the learning network model stored in the database 110 based on the input query. The deep learning framework 120 selects information or data and a learning network model according to the input query, sets learning parameters, executes machine learning, and may provide an intermediate learning result and a final result. The deep learning framework 120 may select data and a pre-trained learning network model through an input query to execute machine inference, and provide the inference result.

The user terminal 130 may input a query through the deep learning framework 120 and receive an inference result corresponding to the query from the database 110 through the deep learning framework 120 . The user terminal 130 may request various functions from the database 110 through a query and receive a response from the database 110 . The user terminal 130 may check and correct data stored in the database 110 through a query, and may add new data. The user terminal 130 may check and modify the network model stored in the database 110 through a query, and may generate a network model for new learning. The user terminal 130 may select data and a learning network model through a query, request machine learning by setting learning parameters, and check an intermediate learning result and a final result. The user terminal 130 may request machine inference by selecting data and a pre-learned network model through a query, and check the inference result.

2 is a flowchart illustrating an execution flow of a query-based machine learning technique according to an embodiment of the present invention. Referring to FIG. 2 , the query-based machine learning technology according to an embodiment of the present invention converts an ONNX format or a pre-learned model converted to an ONNX format into a QML format through a converter, and learns from the user terminal 130 or It receives an inference query and transmits information from the database 110 to QML, so that learning and inference can be performed in QML. And, if the learning or inference result is stored in the database 110 , the user terminal 130 may check the result stored in the database 110 .

The user terminal 130 may input (Import) the learning model or receive an output (Export) from the database 110 (①). In addition, when inputting or outputting the learning model, it can be converted according to the schema structure of the database 110 through the model converter (②). In addition, the database 110 may interpret the query to perform an appropriate operation (③). In addition, the deep learning framework 120 may perform a plug-in to the database 110 and perform learning and inference through information received from the database 110 (④). Also, the user terminal 130 may request learning or inference from the database 110 through a query (⑤). In addition, the user terminal 130 may inquire the table of the database 110 to inquire the learning-related information (⑥). And, the model data may be stored in the database 110 as a QML schema (⑦).

3 is a diagram illustrating an internal data flow of a database according to an embodiment of the present invention. Referring to FIG. 3 , the database 110 according to an embodiment of the present invention stores data related to machine learning, provides functions necessary for machine learning as a procedure, and performs machine learning according to a user's request. can do.

In the database 110, the table is largely provided with a dataset table, a network table, a project table, a job table, and a common table. can do.

The data set has a data type, the network has a network type and LeNet (one of the CNN structures), and the project can copy the information of the network to perform learning or inference work.

The work table may include user information, project status, log, and the like. The common table may include a lookup table such as a layer type and an error code. The network table may store network model information.

The project table may store project information for actual learning or inference copied from the network table. Since the project table has a configuration separate from the network table after the project is created, it is not affected even if the underlying network used in the project is modified. A large number of variable data (input/output data and weight information) is a binary large object (BLOB) or text type, and a small number of variable data (eg, each layer parameter) may be stored by dividing records.

The procedures required for machine learning are Insert Network, Insert Layer, Make Project, Input Data Loader, Init Network, Train, and Model. It may have a Save Model and a Test.

The insert network may create a network including a network name, a network type, a dataset name, an optimizer type, an optimizer parameter, a learning rate, a batch size, a number of trainings, and an output layer index. The insert layer may register a layer including a network ID, a layer name, a layer type, a layer index, a layer parameter, and an input layer index.

A make project can create a project containing the project name, dataset name, network name, training or inference flags, and number of GPUs. The input data loader may input a query according to the selection of the network input (layer index, query type (2: training table, 0: training data, 4: validation table, 3: validation data)).

Network initialization can configure the network model. The train may start learning with a project ID, number of learning generations, batch size, followed by whether or not to learn, save interval, verification interval, and GPU synchronization interval.

Model saving can copy the network information of the project table to the network table (project name, network name). The test can start making inferences that include the project ID, and a flag whether to save the results of all layers.

4 is a diagram illustrating an operation flowchart for explaining a query-based deep learning inference method according to an embodiment of the present invention. Referring to FIG. 4 , the query-based deep learning inference system 100 according to an embodiment of the present invention is a query-based deep learning inference method in the deep learning framework 120 interworking with the user terminal 130 and the database 110 . can run

The deep learning framework 120 may receive a learning query (Call Train) or an inference query (Call Test) from the user terminal (S410). The deep learning framework 120 may use a model converter for compatibility with external frameworks. The deep learning framework 120 may convert the pre-learned model of the existing framework into the ONNX model format through a model converter when importing a pre-trained model of an existing framework or exporting information or data from a database to the outside.

The model converter may convert a network structure and model data defined in the ONNX model format into a network model table format of a database. Conversely, the model converter can convert the network model of the database into the ONNX model format.

The deep learning framework 120 may execute a network initialization (Init Network), a network configuration (Construct Network), and a network update (Update Network) according to a learning query or an inference query ( S420 ).

The deep learning framework 120 may execute training or inference when initialization of all layers is performed (S430). The deep learning framework 120 may obtain batch data (Get Batch Data) and iterate until the end of learning and store the results and model (Store Result & Model). Also, the deep learning framework 120 may execute a test, obtain test data (Get Test Data), feedforward, and store an inference result (Store Result).

The deep learning framework 120 may provide a learning result or an inference result to the user terminal 130 at the end of learning or inference (S440).

The above-described learning or inference process may be separately described below.

5 is a diagram illustrating an execution flow of a learning procedure according to an embodiment of the present invention. Referring to FIG. 5 , the deep learning framework 120 according to an embodiment of the present invention executes the following steps for a user's learning request, and the database 110 calls the QML 120 to print data. can be passed to to return the result.

The deep learning framework 120 receives a learning query from the user terminal 130 and calls learning (Call Train) (S50), network initialization (Init Network) (S51), network configuration (Construct Network) (S52) , a network update (Update Network) ( S53 ) may be executed.

The deep learning framework 120 may execute layer initialization (Init Layer) (S55) until initialization of all layers is made (S54-No). The deep learning framework 120 may obtain initialized layer information (Initialized Layer Info), and execute a layer update (Update Layer) ( S56 ).

When the deep learning framework 120 is initialized for all layers (S54-Yes), it can execute training (S57). The deep learning framework 120 acquires batch data (Get Batch Data) (S59) until the end of learning (S58-No), iterates over it (S60), and stores the result and model (Store Result & Model) ) ( S610 ). The deep learning framework 120 may provide the learning result to the user terminal 130 at the end of the learning (S58-Yes) (S62).

6 is a diagram illustrating an execution flow of an inference procedure according to an embodiment of the present invention. Referring to FIG. 6 , the deep learning framework 120 according to an embodiment of the present invention executes the following steps in response to a user's inference request, and the database 110 calls the QML 120 to print data. can be passed to to return the result.

When an inference query is input from the user terminal 130 and inference is called (Call Test) (S63), the deep learning framework 120 performs a network initialization (Init Network) (S64), a network configuration (Construct Network) (S65) , and network update (Update Network) ( S66 ).

The deep learning framework 120 can execute Initialize all layers (S67-No), Initialize Layers (S68) and Update Layers (S69) for all layers. have.

The deep learning framework 120 is initialized for all layers (S67-Yes), executes an inference test (Test) (S70), obtains inference data (Get Test Data) (S71), and feeds forward ( feed forward) (S72), store the result (Store Result) (S73), and provide the inference result to the user terminal 130 (S74).

7 is a diagram illustrating a conversion operation of a model converter according to an embodiment of the present invention. Referring to FIG. 7 , the network model stored in the database 110 according to an embodiment of the present invention may require a model converter for compatibility with external frameworks (tensorflow, pytorch, caffe, etc.). The ONNX (Open Neural Network Exchange) format can be used when importing pre-trained models from existing frameworks or exporting them out of a database. Here, in addition to ONNX, Open Network Exchange (ONNX), Neural Network Exchange Format (NNEF), and a model weight file including training parameters and biases may be converted into a structured format.

The model converter may convert the network structure and model data (weight, bias) defined in the structured format including the ONNX model into the network model table format of the database 110 (a). Conversely, the network model of the database 110 may be converted into a structured format including the ONNX model (b).

The machine-learning model in the existing framework (Caffe, tensorflow, pytorch, etc.) can be uploaded to the database 110 through the Converter (Import) function after the user converts it to a saved format including the ONNX model.

The model learned from the QML 120 can be stored in a structured format including the ONNX model or a CVS file in the database 110 through the Converter (Export) function. The ONNX model and structured format stored in the database 110 can be converted into a target framework desired by the user and used.

On the other hand, QML is a deep learning framework 120 being developed in C language. It is connected to the database 110 through a User Defined Function (UDF) and can be executed by a call. Functions defined in the deep learning framework 120 may be registered in the database 110 through the UDF, and the deep learning framework 120 may be executed through the registered UDF call. The types of argument variables that can be used in UDF are defined as integer, real number, and string, and in QML, they can be used as follows. Integer is an integer value among the essential parameters constituting the network model, and the address value of the structure memory defined inside QML. Real number is a real value among essential parameters constituting the network model. String is a variable number of parameters and BLOB data (binary data).

The QML framework follows the channel-first data format, NCHW (N:batch, C: channel, H:height, W:width) format. The layer type supports the layers used in ONNX, and the parameters defined in each layer also follow the structured format including the ONNX format.

In the QML framework, a backpropagation algorithm is implemented so that the network model can be learned. A gradient calculation algorithm, which is an essential element of backpropagation, and an optimization algorithm for updating model parameters (weight, bias) are implemented. There are two ways to learn a network model, Train from scratch and fine tuning can be supported. Train from scratch trains a network model from scratch. Weights of each layer may be determined through a weight initialization algorithm. Fine tuning can set the initial weight of the layer by reading the weights of the pre-learned model (the weights stored in the database through the import function or obtained through previous learning attempts) and can proceed with learning.

8 is a diagram schematically illustrating an internal structure of a QML framework according to an embodiment of the present invention. Referring to FIG. 8 , in the deep learning framework 120 according to an embodiment of the present invention, a QML network t (qml_network_t) is composed of a plurality of QML layers t (qml_network_t), and one QML layer t (qmll_network_t) is It may be composed of a plurality of QML tensors t (qml_tensor_t).

The object qml_networks_t may have qml_network_t, and N qml_network_t may be included in qml_networks_t when a network model is trained with a multi GPU, and one qml_network_t may be included when a network model is inferred.

Object qml_network_t may have several qml_layer_t and network parameters. Object qml_layer_t has input/output tensors (qml_tensor_t), Object qml_tensor_t is a 4D tensor configured in NCHW format, and dtype, qml_shape_t, data, name, etc. may be provided inside.

The query-based deep learning inference system 100 according to an embodiment of the present invention may manage clients, members, datasets, networks, learning, learning execution, etc. as follows.

[Client]

The query-based deep learning inference system 100 according to an embodiment of the present invention may provide a function to manage a dataset and a machine learning process with the user terminal 130 and to check the result.

[Member Management]

The query-based deep learning inference system 100 may grant rights to create and modify data of the database 110 and network models through member management and leave a change history.

[Dataset Management]

The query-based deep learning inference system 100 may provide a function for creating a new table to manage a dataset, and querying, modifying, and uploading data. The system 100 may automatically create a new table and upload data when creating a new dataset. The present system 100 may access a table of the database to inquire data or show a result of inquiring data in the database through a query written by a user. The system 100 may modify data according to authority. The system 100 may receive numerical data from a user or read one or more files to upload data. The system 100 may provide a function of tagging the training data.

[Network Management]

The query-based deep learning inference system 100 may provide a function for managing a network model as follows. The present system 100 may create a new network model by adding supported layers and adjusting layer parameters. The present system 100 may inquire a list of previously created network models. The system 100 may create a new network model by adding a new layer to the previously created network model. The present system 100 may provide a function to visualize and show the network model.

[Learning Management]

The query-based deep learning inference system 100 may provide a function for managing learning as follows. The present system 100 may generate or modify learning by adjusting a network model, a dataset, and a learning parameter. The present system 100 may output the trained network model through a converter function. The system 100 may check the resources of the server currently being used.

[Manage Learning Execution]

The query-based deep learning inference system 100 may provide a function for performing learning and inference as follows and confirming the result. The system 100 may check the resources of the server. The present system 100 may inform the user of whether learning and inference can be performed. The present system 100 may inquire a list of learning plans currently being executed or waiting. The system 100 may generate a learning plan by setting the registered network model, dataset, and learning parameters. The system 100 may check the learning parameters of the currently executing or waiting learning plan. The present system 100 can check the middle and results of the currently executing learning plan. The system 100 may stop the currently running learning plan. The system 100 may start a pending learning plan. The present system 100 may generate an inference plan by setting a registered network model and dataset. The system 100 may check the result of the executed inference plan.

As described above, according to the present invention, it is possible to provide necessary information to a user without difficulty even for a user without professional knowledge on deep learning. The present invention is a query-based deep learning framework that is connected to an information database in the form of a plug-in, learns data stored in the information database by a user's request query in a deep learning method, and infers data corresponding to the query. A learning inference system and method thereof may be realized.

9 is a block diagram of a machine tool predictive maintenance system using a query-based deep learning inference system according to an embodiment of the present invention. FIG. 10 is a block diagram of the data collector of FIG. 9 . Reference is made to FIGS. 1 to 8 .

9, the machine tool predictive maintenance system using the query-based deep learning inference system includes a data collector 200, a database (DB, 110), a deep learning framework (QML, 120) and a user terminal 130 can do. The database (DB, 110), the deep learning framework (QML, 120), and the user terminal 130 are components provided in the query-based deep learning inference system 100 of FIGS. 1 to 8, and a detailed description is provided in FIG. 1 to FIG. 8 .

The data collector 200 may collect the vibration amount, noise amount, electric power amount, and state information label of the machine tool 300 as a data set.

The machine tool 300 may be a machine capable of making a desired shape by processing a material, but is not limited thereto, and may be a variety of machines. The machine tool 300 may be an assembly of numerous parts using electrical energy as a power source. The machine tool 300 may include a rotation system component rotating at a desired speed, and a transmission system component capable of transferring to an accurate position. The types of the machine tool 300 include a machining center, a lathe, a milling, a drill machine, a grinder, a laser cutting machine, a NCT punching machine (Numerically Controlled). Turret), a CNC bending machine, a slotter, a shaper, and the like. The machine tool 300 may be a numerically controlled (Numerical Control) or computerized numerically controlled (Computerized Numerical Control) machine.

Various rotation system and feed system parts are subjected to mechanical motion, and the parts of the machine tool 300 may fail. Failure or aging of the parts increases the loss, and a phenomenon in which the output of the machine tool 300 is lowered may appear. In addition, failure or aging of parts of the machine tool 300 may cause a safety accident of an administrator or a user to be a problem.

Referring to FIG. 10 , the data collector 200 may include a collector control unit 210 , a collector communication unit 220 , and a collector detection unit 250 .

The collector communication unit 220 may communicate with an external device through wired/wireless communication. The external device communicating with the collector communication unit 220 may correspond to the machine tool 300 , the user terminal 130 , the database 110 , and the like. The external device is not limited thereto, and other servers that manage the database 110 may also be applicable.

Collector communication unit 220 may communicate with a variety of machine tools, in addition to the specific machine tool (300). Accordingly, the data collector 200 may collect data from a plurality of machine tools in a data set format. The data set collected from such a plurality of machine tools can build a learning network model for various types of machine tools. The collected data set can make it possible to build various learning network models for homogeneous machine tools. Various learning network models for the same type of machine tool can be built with various models according to the surrounding environment in which the machine tool is installed or the amount of operation.

The collector communication unit 220 may include a short-range communication module, a wireless Internet module, and a mobile communication module 127 for wireless communication. Wired communication technologies may include Power Line Communication (PLC), USB communication, Ethernet, serial communication, and optical/coaxial cables. The short-distance communication module refers to a module for short-distance communication. Bluetooth, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, Near Field Communication (NFC), ultrasonic communication as short-range communication technologies (Ultra Sound Communication: USC), Visible Light Communication (VLC), Wi-Fi (Wi-Fi), Wi-Fi Direct (Wi-Fi Direct), etc. may be used. The wireless Internet module refers to a module for wireless Internet access. As wireless Internet technologies, WLAN (Wireless LAN) (Wi-Fi), DLNA (Digital Living Network Alliance), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave) Access), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), IEEE 802.16, Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), broadband wireless mobile communication A service (Wireless Mobile Broadband Service: WMBS) may be used. The mobile communication module may transmit/receive a wireless signal to/from at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include various types of data such as voice, video, photo, text, and combinations thereof. Mobile communication module is GSM (Global System for Mobile communication), CDMA (Code Division Multi Access), CDMA2000 (Code Division Multi Access 2000), EV-DO (Enhanced Voice-Data Optimized or Enhanced Voice-Data Only), WCDMA (Wideband) CDMA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), Long Term Evolution-Advanced (LTEA), and the like can be used. In addition, the collector communication unit 220 may include a low-power wide area network module for IoT such as NB-IoT, LTE-M, LoRa, and Sigfox. The low-power wide area network module may use the aforementioned short-distance/wireless Internet/mobile communication module.

The collector detection unit 250 may include a vibration sensor module 260 , a noise sensor module 270 , and a power amount measurement module 280 . The collector detection unit 250 may include a plurality of each of these modules as many as the number of a plurality of machine tools connected to the data collector 200 .

The vibration sensor module 260 and the noise sensor module 270 each include a plurality of sensors, and may be attached to various positions of the machine tool 300 . Data measured by the vibration sensor module 260 and the noise sensor module 270 may be used to predict failure or aging of the machine tool 300 , or failure. The vibration sensor module 260 may include a plurality of vibration sensors. It is preferable that the vibration sensor can measure the vibrations of the x, y, and z axes, respectively. The plurality of vibration sensors may be attached to a frame of the machine tool 300 or a portion performing a mechanical motion.

The noise sensor module 270 may include a plurality of noise sensors. The plurality of noise sensors may be attached to a frame of the machine tool 300 or a portion that performs a mechanical motion. The noise sensor module 270 may measure data that allows the collector control unit 210 to analyze the volume, frequency, etc. of the sound generated by the machine tool 300 .

The power amount measurement module 280 may measure the power consumption of the machine tool 300 . The measured power consumption may be a basis for comparison with a normal power consumption pattern, and/or whether or not there is an abnormal waveform in the measured power consumption signal. The measured power consumption may help the machine tool 300 to determine a normal operation or an abnormal operation.

The amount of vibration measured by the vibration sensor module 260 of the collector detection unit 250, the amount of noise measured by the noise sensor module 270, and the amount of power measured by the power measurement module 280 may be defined as a diagnostic data set. .

The collector control unit 210 may control the overall operation of the data collector 200 by generally controlling the operation of the respective units. The collector control unit 210 diagnoses the amount of vibration measured by the vibration sensor module 260 of the collector detection unit 250 , the amount of noise measured by the noise sensor module 270 , and the amount of power measured by the power measurement module 280 . It may be collected as a data set and transmitted to the database 110 .

The data collector 200 may further include a state recognition unit 230 for generating a learning network model for predicting failure or failure of the machine tool 300 .

The state recognition unit 230 may check the state information label. The state information label may be a label (measured value, flag) indicating whether the machine tool 300 has a failure, a failure site, and the like.

The state recognition unit 230 may receive the state information label for the transfer shaft and the rotary shaft through an encoder or a vision. The state recognition unit 230 may receive information about the surrounding temperature, pressure, humidity, inclination and location of the machine tool 300 through various sensors. The state recognizing unit 230 may receive an input of whether or not there is a failure and a portion of the failure by the user.

The collector control unit 210 may collect the diagnostic data set and the state information label as a learning data set and transmit it to the database 110 .

The database 110 may store the data set collected by the data collector 200 . The database 110 may store a learning network model, a learning parameter, and a learning result.

The deep learning framework 120 may be connected to the database 110 in a plug-in manner. The deep learning framework 120 may check, correct, and add new data to information or data stored in the database 110 .

The user terminal 130 may input a query through the deep learning framework 120 . The user terminal 130 may receive an inference result corresponding to the query from the database 110 through the deep learning framework 120 .

When receiving a query, the deep learning framework 120 may check and/or modify the learning network model stored in the database 110 , or generate a learning network model for new learning. The deep learning framework 120 may execute machine learning by selecting information or data and a learning network model according to an input query, and setting learning parameters. The deep learning framework 120 may provide an intermediate learning result and a final result according to the degree of machine learning.

The deep learning framework 120 may select a data set and a pre-learned learning network model through an input query to execute machine inference, and provide the inference result as a failure diagnosis and/or failure prediction.

The database 110 may store a plurality of failure diagnosis predictive models 140 including first and second failure diagnosis predictive models 140 - 1 and 140 - 2 . The failure diagnosis predictive model 140 may correspond to the above-described learning network model. For a detailed description, refer to FIGS. 1 to 8 .

The failure diagnosis predictive model 140 may refer to a classification model learned to output a result classified as a failure or failure prediction of the machine tool 300 . The failure diagnosis predictive model 140 may be generated through learning using one of various learning algorithms and at least one of a plurality of diagnostic data sets.

The diagnostic data set may be collected from the data collector 200 having the state recognition unit 230 or may be obtained from a public database.

11 and 12 are flowcharts of a method for machine learning and failure diagnosis and prediction of a machine tool predictive maintenance system using a query-based deep learning inference system according to an embodiment of the present invention. See Figures 9 and 10.

Referring to FIG. 11 , the deep learning framework 120 may receive a training query (Call Train) from the user terminal 130 (S510). The data collector 200 includes a state recognition unit 230 for checking the state information label of the machine tool 300, and a detection unit for measuring the amount of vibration, noise, and power of the machine tool 300, the state information The label, the amount of vibration, the amount of noise, and the amount of power may be collected as a training data set and provided to the database 110 .

The deep learning framework 120 may designate at least a part of the training data set as the suitability determination data set (S520).

The deep learning framework 120 may determine whether there is a model suitable for the data set DS for determining suitability among the plurality of pre-stored learning network models 140 or whether a suitable model exists ( S530 ). Whether a suitable model exists can be determined based on whether the data set of the failure diagnosis predictive model 140 and the data set for determination of suitability correspond, and the degree of accuracy of failure diagnosis or prediction according to the data set for determination of suitability. have.

When there is no suitable model as a result of determining whether a suitable model exists ( S530 ), the deep learning framework 120 may generate a suitable failure diagnosis predictive model 140 . To this end, the deep learning framework 120 initializes (S540) a network (NN) according to the learning query, configures the network (NN), and updates the network (NN) as necessary ( S550) can be done. The deep learning framework 120 may initialize all layers of the configured network NN (S560). For network initialization, configuration, and update, and layer initialization, refer to the description of FIG. 5 .

When all layers are initialized, the deep learning framework 120 may perform learning using the training data set (S580).

If there is a suitable model as a result of determining whether a suitable model exists (S530), the deep learning framework 120 checks the corresponding model, for example, the first failure diagnosis predictive model 140-1, and predicts the first failure diagnosis It may be determined whether the model 140-1 needs to be modified (S535). If it is necessary to modify the first failure diagnosis predictive model 140-1, the deep learning framework 120 performs steps S540, S550, S560, and S580 to add it to the first failure diagnosis predictive model 140-1 can be taught with

Criteria for determining whether the first failure diagnosis predictive model 140-1 needs to be modified may vary. For example, there may be examples such as the creation date of the first failure diagnosis predictive model 140 - 1 being older than a preset period, or a tag indicating that correction is required.

The determination of whether the first failure diagnosis predictive model 140 - 1 is modified may be simultaneously performed in the determination of whether the first failure diagnosis predictive model 140 - 1 is a suitable model ( S530 ). In this case, the network of the first failure diagnosis predictive model 140-1 may be initialized and configured, and the layers may be initialized. Therefore, if correction is necessary, the deep learning framework 120 may directly execute the learning ( S580 ) on the first failure diagnosis predictive model 140 - 1 .

When the learning execution (S580) is finished, the deep learning framework 120 may provide a result to the user terminal 130 (S590) and store the finished learning network model in the database 110 .

Referring to FIG. 12 , the deep learning framework 120 may receive an inference query (Call Test) from the user terminal 130 ( S610 ).

The deep learning framework 120 may collect a diagnostic data set from the data collector 200 (S620).

The deep learning framework 120 may analyze the training data set and call ( S640 ) a suitable first failure diagnosis predictive model 140 - 1 from among the plurality of failure diagnosis predictive models 140 . The first failure diagnosis predictive model 140 - 1 may be selected and called by a user input.

The deep learning framework 120 configures the network NN according to the first failure diagnosis predictive model 140-1 (S650), and when initialization (S660) for all layers of the network NN is made, for diagnosis Inference may be executed based on the data set (S670).

When the reasoning execution (S670) is finished, the deep learning framework 120 may diagnose and predict the failure of the machine tool 300 (S680).

The present invention may be implemented in hardware or software. Implementation The present invention can also be implemented as computer-readable codes on a computer-readable recording medium. That is, it may be implemented in the form of a recording medium including instructions executable by a computer. The computer readable medium includes any type of medium in which data that can be read by a computer system is stored. Computer-readable media may include computer storage media and communication storage media. Computer storage media includes all storable media implemented as any method or technology for information storage, such as computer-readable instructions, data structures, program modules, and other data, and includes volatile/nonvolatile/hybrid memory. Whether or not, it is not limited to whether the detachable / non-separable type. Communication storage media includes a modulated data signal or transmission mechanism, such as a carrier wave, any information delivery media, and the like. And functional programs, codes, and code segments for implementing the present invention can be easily inferred by programmers in the technical field to which the present invention pertains.

In addition, although preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention as claimed in the claims In addition, various modifications are possible by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or prospect of the present invention.

100: query-based deep learning inference system 110: database

120: deep learning framework 130: user terminal

140: diagnostic predictive model 200: data collector

Claims

As a machine tool predictive maintenance method using a query-based deep learning inference system of a deep learning framework that works with user terminals and databases,

receiving a learning query (Call Train) from the user terminal;

A plurality of failure diagnosis predictions are made by using at least a part of the training data set, which is a state information label indicating the machine tool's vibration amount, noise level, and electric power provided by the data collector, and whether the machine tool is faulty as a suitability judgment data set. a determination step of whether a suitable model exists for determining whether there is a failure diagnosis predictive model suitable for the data set for determination of suitability among models;

when there is no suitable failure diagnosis predictive model as a result of determining whether the suitable model exists, initializing a network according to the learning query and configuring the network;

a learning execution step of executing learning using the collected learning data set when all layers of the configured network are initialized; and

When the learning execution is finished, providing a result and storing the finished failure diagnosis predictive model; a machine tool predictive maintenance method using a query-based deep learning inference system comprising a.
The method of claim 1,

receiving an inference query (Call Test) from the user terminal;

collecting a diagnostic data set having an amount of vibration, noise, and power of the machine tool from the data collector;

calling an appropriate failure diagnosis predictive model from among the plurality of failure diagnosis predictive models by analyzing the diagnostic data set;

a reasoning execution step of constructing a network according to the suitable failure diagnosis predictive model and executing reasoning based on the diagnostic data set when all layers of the network are initialized; and

Diagnosing the failure of the machine tool when the reasoning execution is finished, and predicting the failure; Machine tool predictive maintenance method using a query-based deep learning inference system further comprising a.
The method of claim 1,

The learning execution step acquires batch data until the end of the learning (Get Batch Data) and stores the results and model by iteration (Store Result & Model),

Machine tool predictive maintenance method using a query-based deep learning inference system that converts a model weight file including ONNX, NNEF and learning parameter bias into a structured format through the model converter when information or data is exported from the database to the outside.
3. The method of claim 2,

The reasoning execution step is a machine tool predictive maintenance method using a query-based deep learning inference system that executes the reasoning test (Test), obtains test data (Get Test Data), and feeds forward (feedforward).
The vibration amount, noise amount, power amount, and state information label are collected as a learning data set from a detection unit that measures the amount of vibration, noise, and electric power of the machine tool, and a state recognition unit that checks the state information of the machine tool data collector;

a database for storing the training data set, a training network model, training parameters, and a learning result;

a deep learning framework connected to the database in a plug-in manner, checking, correcting, and adding new data to information or data stored in the database; and

A user terminal that inputs a query through the deep learning framework and receives an inference result corresponding to the query from the database through the deep learning framework; includes,

The deep learning framework, upon receiving the query, checks and corrects the failure diagnosis predictive model stored in the database, or generates a failure diagnosis predictive model for new learning, and includes information or data according to the input query. Execute machine learning by selecting a failure diagnosis predictive model and setting learning parameters, providing an intermediate learning result and final result, and executing machine inference by selecting data and a pre-learned failure diagnosis predictive model through the input query and a machine tool predictive maintenance system using a query-based deep learning inference system that provides the inference result as failure diagnosis and failure prediction.
6. The method of claim 5,

The failure diagnosis predictive model is a model used for machine learning, and is composed of input/output and parameters defining the inside of the model, and parameters necessary for machine learning and inference, and is stored in the database in a relational data format, and other deep Machine tool predictive maintenance system using a query-based deep learning inference system that can be converted into a learning framework.
6. The method of claim 5,

The database stores all input and output data used for machine learning and machine inference, stores models used for machine learning and machine inference, and provides a procedure corresponding to a user's query request. Machine tool predictive maintenance system using an inference system.