KR102568010B1 - Predition system for fault of machine tool using deep learning inference system based on query and method thereof - Google Patents

Abstract

In the present invention, a deep learning framework is connected to an information database in the form of a plug-in, and query-based deep learning enables inference of data corresponding to a query by learning data stored in the information database by a user's requested query using a deep learning method. Inference systems can be used to diagnose or predict machine tool failures.

Description

Machine tool predictive maintenance system using query-based deep learning inference system and its method

The present invention relates to a machine tool predictive maintenance system and method using a query-based deep learning inference system, and more particularly, to a deep learning frame so that even a user without expert knowledge on deep learning can provide necessary information to the user without difficulty. Using a query-based deep learning inference system in which a work is connected to an information database in the form of a plug-in and learns data stored in the information database by a user's request query using a deep learning method to infer data corresponding to the query, It is to diagnose or predict machine tool failure.

Damage or deterioration that occurs in parts of a machine tool that performs mechanical motion in various rotation system and feed system components increases loss, and the output of the machine tool may decrease. If machine tool failures can be predicted, productivity will increase and safety accidents will decrease. Machine learning or deep learning technology may be used to predict machine tool failure.

Learning engines using deep learning show significantly superior intelligence performance compared to learning engines based on other existing AI technologies.

However, in order to create a learning engine that provides intelligence based on deep learning technology, there are various difficulties such as deep network design, learning function setting, and parameter tuning. These problems cannot be easily solved unless you are a deep learning expert, so it is difficult for anyone to easily have a deep learning-based learning engine.

In addition, whenever a learning engine is created, common elements of deep learning are used repeatedly, and the same process must be repeated.

KR 10-2058124 B1

An object of the present invention for solving the above problems is that a deep learning framework is connected to an information database in the form of a plug-in so that even a user without professional knowledge about deep learning can provide necessary information to the user without difficulty, so that the user's Query-based deep learning that diagnoses or predicts machine tool failures by using a query-based deep learning inference system that enables inference of data corresponding to a query by learning data stored in an information database by a request query using a deep learning method. An object of the present invention is to provide a machine tool predictive maintenance system and method using an inference system.

A machine tool predictive maintenance method using a query-based deep learning inference system according to an embodiment of the present invention is a machine tool predictive maintenance method using a query-based deep learning inference system of a deep learning framework that interworks with a user terminal and a database. Receiving a learning query (Call Train) from a user terminal; A plurality of failure diagnosis predictions by using at least a part of the learning data set, which is a status information label indicating the amount of vibration, noise, and power of the machine tool provided by the data collector, and whether or not the machine tool is out of order, as a data set for determining suitability. determining whether a suitable model exists or not, determining whether there is a failure diagnosis prediction model suitable for the suitability determination data set among the models; initializing a network according to the learning query and configuring the network when there is no suitable failure diagnosis prediction model as a result of determining whether or not the suitable model exists; a learning execution step of executing learning using the collected training data set when all layers of the configured network are initialized; and providing a result when the learning execution is terminated and storing the terminated failure diagnosis prediction model.

In addition, the step of receiving an inference query (call test) from the user terminal; collecting a diagnostic data set including vibration, noise, and power of the machine tool from the data collector; analyzing the diagnosis data set and calling an appropriate failure diagnosis prediction model among the plurality of failure diagnosis prediction models; an inference execution step of constructing a network according to the appropriate failure diagnosis predictive model and executing inference based on the diagnosis data set when all layers of the network are initialized; and diagnosing a failure of the machine tool and predicting a failure when the reasoning execution is terminated.

In addition, the learning execution step acquires batch data (Get Batch Data) and repeats (Iteration) to store results and models (Store Result & Model) until the end of the learning, and when exporting information or data from the database to the outside A model weight file including ONNX, NNEF, and learning parameter bias can be converted into a structured format through the model converter.

In addition, the reasoning execution step may execute the reasoning test (Test), obtain test data (Get Test Data), and feed forward the reasoning test.

In addition, in the step of receiving the learning query or inference query, the deep learning framework uses a model converter for compatibility with an external framework, imports a pre-learned failure diagnosis prediction model of an existing framework, or When information or data is exported from the database, the model weight file including ONNX, NNEF, and learning parameter biases can be converted into a structured format through the model converter.

In addition, the model converter converts the network structure and model data defined in the structured format into the network model table format of the database, or conversely converts the network model of the database into the structured model. format can be converted.

A machine tool predictive maintenance system using a query-based deep learning inference system according to an embodiment of the present invention includes a sensing unit that measures vibration, noise, and power of a machine tool, and state recognition that checks state information of the machine tool. a data collector for collecting the vibration amount, noise amount, power amount, and status information label from the unit as a data set for learning; a database for storing the learning data set, learning network model, learning parameters, and learning results; a deep learning framework that is connected to the database in a plug-in manner and checks, corrects, and adds new data to information or data stored in the database; and a user terminal inputting a query through the deep learning framework and receiving an inference result corresponding to the query from the database through the deep learning framework, wherein the deep learning framework performs the query When an input is received, the failure diagnosis prediction model stored in the database is checked and corrected, or a failure diagnosis prediction model for new learning is created, information or data and a failure diagnosis prediction model are selected according to the input query, and learning parameters are set. set to execute machine learning, provide intermediate results and final results of learning, select data and pre-learned failure diagnosis prediction models through the input query, execute machine inference, and use the inference results to diagnose failures and failures A foreshadowing can be provided.

In addition, the failure diagnosis prediction model is a model used for machine learning, and is composed of input and output, parameters defining the inside of the model, and parameters necessary for machine learning and inference, stored in the database in a relational data format, and through a converter. It may be convertible to other deep learning frameworks.

In addition, the database stores all input and output data used in machine learning and machine reasoning, stores models used in machine learning and machine reasoning, and provides a procedure corresponding to a user's query request. based deep learning inference system.

In addition, the deep learning framework is used for compatibility with external frameworks, and includes ONNX, NNEF, and learning parameter bias when a pre-learned model of an existing framework is imported or information or data is exported from the database A network structure defined in the model format (using the model weight file including the ONNX, NNEF, and learning parameter biases in a structured format); The network structure and model data may be converted into the network model table format of the database, or conversely, the failure diagnosis prediction model of the database may be converted into the structured model format.

In addition, the data set is a set of information or data having the same format, and the information or data includes numbers, texts, images, video, and audio and can be used for machine learning.

In addition, the procedure includes Insert Network, Insert Layer, Make Project, Input Data Loader, Train, Save Model and Test (Test).

In addition, the database may include a dataset table, a network table, a project table, a job table, and a common table.

In addition, the deep learning framework executes network initialization (Init Network), network configuration (Construct Network), and network update (Update Network) when a learning query is input from the user terminal (Call Train), and all layers When initialization (Initialize all layers) is done, training (Train) is executed, batch data is acquired (Get Batch Data) until training ends, results and models are stored (Store Result & Model) by repetition (Iteration), and training Upon completion, learning results may be provided to the user terminal.

According to the present invention, by using a query-based machine learning technology, a deep learning framework is connected to a database in the form of a plug-in and performs machine learning, inference, etc. using data stored in the database by a user's request query. failure can be diagnosed and predicted.

Therefore, even a user without expert knowledge of deep learning can easily provide necessary information without difficulty.

In addition, it is possible to check the learning parameters of the currently executed or waiting learning plan, and the middle and result of the currently executing learning plan.

Also, you can stop the currently running learning plan and start the waiting learning plan.

1 is a configuration diagram schematically showing the overall configuration of a query-based deep learning inference system according to an embodiment of the present invention.
2 is a flowchart showing the execution flow of a query-based machine learning technique according to an embodiment of the present invention.
3 is a diagram illustrating an internal data flow of a database according to an embodiment of the present invention.
4 is a flowchart illustrating an operation for explaining a query-based deep learning inference method according to an embodiment of the present invention.
5 is a diagram showing an execution flow of a learning procedure according to an embodiment of the present invention.
6 is a diagram illustrating an execution flow of an inference procedure according to an embodiment of the present invention.
7 is a diagram illustrating a conversion operation of a model converter according to an embodiment of the present invention.
8 is a diagram schematically illustrating the internal structure of a QML framework according to an embodiment of the present invention.
9 is a block diagram of a machine tool predictive maintenance system using a query-based deep learning inference system according to an embodiment of the present invention.
FIG. 10 is a block diagram of the data collector of FIG. 9 .
11 and 12 are flowcharts of a method for machine learning, fault diagnosis, and edge of a machine tool predictive maintenance system using a query-based deep learning inference system according to an embodiment of the present invention.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention. The terms and/or include any combination of a plurality of related recited items or any of a plurality of related recited items.

It is understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle. In addition, that the first component and the second component on the network are connected or connected means that data can be exchanged between the first component and the second component in a wired or wireless manner.

In addition, the suffixes "module" and "unit" for the components used in the following description are simply given in consideration of ease of preparation of this specification, and do not themselves give a particularly important meaning or role. Accordingly, the “module” and “unit” may be used interchangeably.

When these components are implemented in actual applications, two or more components may be combined into one component, or one component may be subdivided into two or more components as needed. The same reference numerals have been assigned to the same or similar components throughout the drawings, and detailed descriptions of components having the same reference numerals may be omitted as they are replaced with descriptions of the components described above.

Furthermore, the present invention covers all possible combinations of the embodiments shown herein. The various embodiments of the present invention are different but not mutually exclusive. One embodiment of the particular shape, structure, function, and characteristic described herein may be implemented in another embodiment. For example, components mentioned in the first and second embodiments may perform all functions of the first and second embodiments.

1 is a configuration diagram schematically showing the overall configuration of a query-based deep learning inference system according to an embodiment of the present invention.

Referring to FIG. 1 , a query-based deep learning inference system 100 according to an embodiment of the present invention may apply query-based machine learning technology. To this end, the query-based deep learning inference system 100 may include a database (DB, 110), a deep learning framework (QML, 120), and a user terminal (130).

Here, in the query-based machine learning technology, the deep learning framework 120 is connected to the database 110 in the form of a plug-in, and machine learning, inference, etc. It is a technique that is performed

The input/output module 130 may be its own interface module. The input/output module 130 may separately include an input device and an output device. The input/output module 130 may be implemented as one input/output device such as a touch screen.

The input/output module 130 may be implemented as a user terminal 130 as shown in FIG. 1 . In an embodiment of the present invention, the input/output module 130 will be described as the user terminal 130 as an example.

The database 110 may store a data set, a learning network model, a learning parameter, and a learning result.

A data set is a collection of information or data having the same format. Information or data includes numbers, texts, images, videos, and voices, and may be any type of information or data used in machine learning.

A learning network model is a model used for machine learning and can be composed of input/output, parameters defining the inside of the model, and parameters necessary for machine learning and inference. Learning network models can be stored in databases in relational data format. The learning network model may be convertible to other deep learning frameworks through a converter.

The learning network model can recognize text input by a user. The learning network model can recognize voice and text contained in images, audio, and video. The learning network model can analyze the user's intention from the recognized voice and text.

As for the learning result, the intermediate output value is stored in the database 110 while machine learning is in progress, so that the user can check it. While machine learning is in progress, model parameter values are stored in the database 110 and can be checked by the user. The evaluation index value of the model calculated while machine learning is in progress is stored in the database 110 and can be checked by the user. The machine reasoning result value is stored in the database 110 and can be checked by the user.

The database 110 may store all input/output data used in machine learning and machine reasoning, store models used in machine learning and machine reasoning, and provide a procedure corresponding to a user's query request. .

The procedure includes Insert Network, Insert Layer, Make Project, Input Data Loader, Train, Save Model and Test. can include

The deep learning framework 120 may be used for compatibility with external frameworks. The deep learning framework 120 may include a model converter that uses an Open Neural Network Exchange (ONNX) model format when importing a pretrained model of an existing framework or exporting information or data from the database.

The model converter may convert a network structure and model data defined in the ONNX model format into the network model format of the database, or convert the network model of the database into the ONNX model format.

In addition, the model converter may convert an Open Network Exchange (ONNX), a Neural Network Exchange Format (NNEF), and a model weight file including learning parameters and biases into a structured format in addition to the ONNX model format.

The model converter converts the model weight file into a structured format so that the learning model can be imported or exported from the database.

The deep learning framework 120 is connected to the database 110 in a plug-in manner, and can check, correct, and add new data to information or data stored in the database 110 . The deep learning framework 120 may be, for example, Quantum Machine Learning (QML).

QML is a deep learning framework installed as a plug-in in the database 110 and can be executed by calling the database. When QML is called, it can receive various data from the database 110 as arguments and return execution results. QML can compose a network inside a framework by interpreting a network model defined in a relational data format.

QML may receive learning parameters and learning data from the database 110 as factors, perform learning of the network configured inside the framework, and return learning results. QML may receive input data from the database 110 as a factor, perform machine inference using a network configured inside the framework, and return a result.

In addition, when a query is input from the user terminal 130, the deep learning framework 120 checks and corrects the learning network model stored in the database 110, and creates a learning network model for new learning. Selects information or data and a learning network model according to the query, executes machine learning by setting learning parameters, provides intermediate and final results of learning, and selects data and a pre-trained learning network model through the input query. It can execute machine reasoning and provide the reasoning result.

The user terminal 130 may input a query through the deep learning framework 120 and receive an inference result corresponding to the query from the database 110 through the deep learning framework 120 .

The user terminal 130 may request various functions to the database 110 through a query and receive a response from the database 110 . The user terminal 130 can check and modify data stored in the database 110 through a query, and can add new data. The user terminal 130 can check and modify the network model stored in the database 110 through a query and create a new network model for learning. The user terminal 130 may select data and a learning network model through a query, set learning parameters, request machine learning, and check intermediate and final results of learning. The user terminal 130 may select data and a pre-learned network model through a query, request machine inference, and check the inference result.

2 is a flowchart showing the execution flow of a query-based machine learning technique according to an embodiment of the present invention.

Referring to FIG. 2 , the query-based machine learning technology according to an embodiment of the present invention converts an ONNX format or a pre-learned model converted to the ONNX format into a QML format through a converter, and learns or An inference query is input, information is transmitted from the database 110 to QML, and learning and inference can be performed in QML. Then, when the results of learning or reasoning are stored in the database 110, the user terminal 130 can check the results stored in the database 110.

The user terminal 130 may input (Import) a learning model or receive an output (Export) from the database 110 (①).

In addition, when inputting or outputting a learning model, it can be converted to fit the schema structure of the database 110 through a model converter (②).

In addition, the database 110 may interpret the query and perform an appropriate task (③).

In addition, the deep learning framework 120 may perform a plug-in to the database 110 and perform learning and inference through information received from the database 110 (④).

In addition, the user terminal 130 may request learning or inference to the database 110 through a query (⑤).

In addition, the user terminal 130 may search the table of the database 110 for learning related information (⑥).

Then, the model data may be stored in the database 110 as a QML schema (⑦).

3 is a diagram illustrating an internal data flow of a database according to an embodiment of the present invention.

Referring to FIG. 3 , the database 110 according to an embodiment of the present invention stores data related to machine learning and provides functions necessary for machine learning as a procedure to perform machine learning at the request of a user. can do.

In the database 110, the table largely includes a dataset table, a network table, a project table, a job table, and a common table. can do.

A data set contains a data type, a network includes a network type and LeNet (one of CNN structures), and a project can copy information from the network to perform training or inference tasks.

The task table may include user information, project status, logs, and the like, and the common table may include a lookup table such as layer type and error code.

Network model information is stored in the network table, and project information for actual learning or inference copied from the network table may be stored in the project table. After the project is created, it has a separate configuration from the network table, so there is no effect even if the base network used in the project is modified. A large number of variable data (input/output data and weight information) is a BLOB (Binary Large Object) or text type, and a small number of variable data (each layer parameter, etc.) is divided into records and stored.

Procedures required for machine learning include Insert Network, Insert Layer, Make Project, Input Data Loader, Init Network, Train, and Model. It can include Save Model and Test.

The insert network can create a network including network name, network type, dataset name, optimizer type, optimizer parameters, learning rate, batch size, number of trainings, and output layer index.

The insert layer may register a layer including a network ID, a layer name, a layer type, a layer index, a layer parameter, and an input layer index.

A make project can create a project that includes the project name, dataset name, network name, training or inference flag, and number of GPUs.

The input data loader can input a query according to the network input selection (layer index, query type (2: learning table, 0: learning data, 4: verification table, 3: verification data)).

Network initialization can configure a network model.

The train can start learning, including the project ID, the number of learning generations, the batch size, whether or not to learn, the storage interval, the validation interval, and the GPU synchronization interval.

When saving a model, the network information of the project table can be copied to the network table (project name, network name).

A test can initiate an inference that includes a project ID and a flag whether or not to save the results of all layers.

4 is a flowchart illustrating an operation for explaining a query-based deep learning inference method according to an embodiment of the present invention.

Referring to FIG. 4 , a query-based deep learning inference system 100 according to an embodiment of the present invention is a query-based deep learning inference in a deep learning framework 120 that works with a user terminal 130 and a database 110. method can be executed.

The deep learning framework 120 receives a learning query (Call Train) or an inference query (Call Test) from the user terminal (S410).

At this time, the deep learning framework 120 uses a model converter for compatibility with an external framework, imports a pre-learned model of an existing framework, or uses a model converter when exporting information or data from a database to the outside. It can be converted to ONNX model format through

The model converter may convert the network structure and model data defined in the ONNX model format into the network model table format of the database, or conversely convert the network model of the database into the ONNX model format.

The deep learning framework 120 may execute network initialization (Init Network), network construction (Construct Network), and network update (Update Network) according to the learning query or inference query (S420).

The deep learning framework 120 may execute training or inference (test) after initializing all layers is performed (S430).

At this time, the deep learning framework 120 may acquire batch data (Get Batch Data) and store results and models (Store Result & Model) by repeating (Iteration) until learning ends.

In addition, the deep learning framework 120 may execute a test, obtain test data (Get Test Data), feed forward, and store inference results (Store Result).

Subsequently, the deep learning framework 120 may provide a learning result or reasoning result to the user terminal 130 when learning or reasoning is finished (S440).

The above-described learning or reasoning process may be separately described below.

5 is a diagram showing an execution flow of a learning procedure according to an embodiment of the present invention.

Referring to FIG. 5, the deep learning framework 120 according to an embodiment of the present invention executes the following steps for a user's learning request, and the database 110 calls the QML 120 to print data. and return the result.

The deep learning framework 120, when a learning query is input from the user terminal 130 and learning is called (Call Train) (S50), network initialization (Init Network) (S51), network construction (Construct Network) (S52) , network update (Update Network) (S53) can be executed.

The deep learning framework 120 executes layer initialization (Init Layer) (S55) until Initialize all layers is performed (S54-No), and initialized layer information (Initialized Layer Info) is obtained, and layer update (Update Layer) (S56) can be executed.

When all layers are initialized (S54-Yes), the deep learning framework 120 executes training (S57), acquires batch data until the end of training (S58-No), and obtains batch data (Get Batch Data). ) (S59), repeat (Iteration) (S60), store the result and model (Store Result & Model) (S610), and provide the learning result to the user terminal 130 at the end of learning (S58-Yes) Yes (S62).

6 is a diagram illustrating an execution flow of an inference procedure according to an embodiment of the present invention.

Referring to FIG. 6 , the deep learning framework 120 according to an embodiment of the present invention executes the following steps in response to a user's inference request, and the database 110 calls the QML 120 to input data. and return the result.

When an inference query is input from the user terminal 130 and inference is called (Call Test) (S63), the deep learning framework 120 performs network initialization (Init Network) (S64), network construction (Construct Network) (S65) , and network update (Update Network) (S66) can be executed.

The deep learning framework 120 executes layer initialization (Init Layer) (S68) and layer update (Update Layer) (S69) until all layers are initialized (S67-No). can

When all layers are initialized (S67-Yes), the deep learning framework 120 executes an inference test (S70), obtains inference data (Get Test Data) (S71), and feeds forward (feed forward) (S72), store the result (Store Result) (S73), and provide the inference result to the user terminal 130 (S74).

7 is a diagram illustrating a conversion operation of a model converter according to an embodiment of the present invention.

Referring to FIG. 7 , a network model stored in the database 110 according to an embodiment of the present invention may require a model converter for compatibility with external frameworks (tensorflow, pytorch, caffe, etc.). The ONNX (Open Neural Network Exchange) format can be used when importing pre-trained models from existing frameworks or exporting them out of the database.

Here, in addition to ONNX, Open Network Exchange (ONNX), Neural Network Exchange Format (NNEF), and model weight files including learning parameters and biases can be converted into a structured format.

The model converter may convert the network structure and model data (weight, bias) defined in the structured format including the ONNX model into the network model table format of the database 110 (a). Conversely, the network model of the database 110 may be converted into a structured format including the ONNX model (b).

Models learned from machine learning in existing frameworks (Caffe, tensorflow, pytorch, etc.) can be uploaded to the database 110 through a Converter (Import) function after the user converts them into a structured format including ONNX models.

The model learned in the QML 120 can be saved in a structured format including an ONNX model or a CVS file in the database 110 through a Converter (Export) function.

The ONNX model and structured format stored in the database 110 can be converted into a target framework desired by the user and used.

Meanwhile, QML is a deep learning framework 120 being developed in C language. It is connected to the database 110 through a User Defined Function (UDF) and can be executed by a call. Functions defined in the deep learning framework 120 may be registered in the database 110 through UDF, and the deep learning framework 120 may be executed through the registered UDF call. The types of parameter variables that can be used in UDF are defined as integer, real number, and string, and can be used in QML as follows. Integer is an integer value among essential parameters constituting a network model, and an address value of a structure memory defined inside QML. A real number is a real number among essential parameters constituting a network model. String is parameters whose number is variable and BLOB data (binary data).

In the QML framework, the NCHW (N: batch, C: channel, H: height, W: width) format, which is a channel-first data format, is followed. Layer types support layers used in ONNX, and parameters defined in each layer follow a structured format including the ONNX format.

The QML framework implements the backpropagation algorithm to learn the network model. The gradient calculation algorithm, which is an essential element of backpropagation, and the optimization algorithm for updating model parameters (weight, bias) are implemented. There are two ways to learn the network model, which can support Train from scratch and Fine tuning. Train from scratch trains the network model from scratch. Weights of each layer may be determined through a weight initialization algorithm. Fine tuning can read the weights of the pre-learned model (weights stored in the database through the import function or obtained through previous learning attempts), set the initial weights of the layer, and proceed with learning.

8 is a diagram schematically illustrating the internal structure of a QML framework according to an embodiment of the present invention.

Referring to FIG. 8 , in the deep learning framework 120 according to an embodiment of the present invention, a QML network t (qml_network_t) is composed of a plurality of QML layers t (qml_network_t), and one QML layer t (qmll_network_t) is It can be composed of a plurality of QML tensor t (qml_tensor_t).

Object qml_networks_t holds qml_network_t, and when learning a network model with Multi GPU, N qml_network_ts are included in qml_networks_t, and when inferring a network model, it holds one qml_network_t.

Object qml_network_t holds several qml_layer_t and network parameters.

Object qml_layer_t has input and output tensors (qml_tensor_t), Object qml_tensor_t is a 4-dimensional tensor composed of NCHW format, and includes dtype, qml_shape_t, data, name, etc.

Meanwhile, the query-based deep learning inference system 100 according to an embodiment of the present invention may manage clients, members, datasets, networks, learning, and learning execution as follows.

[Client]

The query-based deep learning inference system 100 according to an embodiment of the present invention may provide the user terminal 130 with a function to manage a dataset and a machine learning process and check the result.

[Member Management]

In addition, the query-based deep learning reasoning system 100 grants authority to create and modify data in the database 110 and network models through member management, and leaves a history of changes.

[Dataset management]

In addition, the query-based deep learning inference system 100 may create a new table to manage a dataset, and provide functions for searching, modifying, and uploading data. When you create a new dataset, you can automatically create a new table and upload the data. Accesses the database table to view data or displays the result of searching the database data through a query written by the user. Data can be modified according to authority. Data upload may be performed by receiving numerical data from the user or by reading one or more files. A function of labeling training data may be provided.

[Network management]

In addition, the query-based deep learning inference system 100 may provide a function for managing a network model as follows. New network models can be created by adding supported layers and adjusting layer parameters. A list of previously created network models can be queried. A new network model can be created by adding a new layer to an existing network model. A function to visualize and show the network model can be provided.

[Learning Management]

In addition, the query-based deep learning inference system 100 may provide a function for managing learning as follows. Learning can be created or modified by adjusting the network model, dataset, and learning parameters. The trained network model can be output through the converter function. You can check the resources of the server currently in use.

[Manage Learning Run]

In addition, the query-based deep learning inference system 100 may provide functions for performing learning and inference and checking the result as follows. You can check server resources. It informs the user whether learning and inference performance is possible. You can search the list of currently running or waiting learning plans. You can create a learning plan by setting the registered network model, dataset, and learning parameters. You can check the learning parameters of the currently running or waiting learning plan. You can check the middle and results of the currently running learning plan. You can stop the currently running learning plan. You can start a pending study plan. An inference plan can be created by setting the registered network model and dataset. You can check the results of the executed reasoning plan.

As described above, according to the present invention, the deep learning framework is connected to the information database in the form of a plug-in so that even a user without expert knowledge of deep learning can provide the user with necessary information without difficulty, A query-based deep learning inference system and method capable of inferring data corresponding to a query by learning data stored in an information database by a deep learning method can be realized.

9 is a block diagram of a machine tool predictive maintenance system using a query-based deep learning inference system according to an embodiment of the present invention. FIG. 10 is a block diagram of the data collector of FIG. 9 . See Figures 1 to 8.

9, a machine tool predictive maintenance system using a query-based deep learning inference system includes a data collector 200, a database (DB, 110), a deep learning framework (QML, 120), and a user terminal 130. can do. The database (DB, 110), the deep learning framework (QML, 120), and the user terminal 130 are components provided in the query-based deep learning inference system 100 of FIGS. 1 to 8, and a detailed description is given in FIG. 1 to Figure 8.

The data collector 200 may collect vibration, noise, power, and status information labels of the machine tool 300 as data sets.

The machine tool 300 is a machine that can make a desired shape by processing a material. The machine tool 300 is an assembly of numerous parts using electrical energy as a power source. The machine tool 300 may include rotation system components that rotate at a desired speed, and transport system components that can be transported to precise positions.

Types of the machine tool 300 include a machining center, a lathe, a milling machine, a drill machine, a polishing machine, a laser cutting machine, and a NCT punching machine (numerically controlled Turret), CNC Bending Machine, Slotter, Shaper, and the like.

The machine tool 300 may be a machine that is numerically controlled or computerized numerically controlled.

Various parts of the rotation system and feed system are subjected to mechanical movement, and thus parts of the machine tool 300 may fail. Failure or deterioration of parts may result in increased losses, resulting in a decrease in the output of the machine tool 300 .

In addition, failure or aging of parts of the machine tool 300 may cause safety accidents of managers or users.

Referring to FIG. 10 , the data collector 200 may include a collector control unit 210, a collector communication unit 220, and a collector detection unit 250.

The collector communication unit 220 may perform wired/wireless communication with an external device. External devices that communicate with the collector communication unit 220 may include the machine tool 300 , the user terminal 130 , and the database 110 . The external device is not limited thereto, and other servers managing the database 110 may also be applicable.

The collector communication unit 220 may communicate with various machine tools in addition to the specific machine tool 300 . Accordingly, the data collector 200 may collect data from a plurality of machine tools in a data set format. A data set collected from a plurality of machine tools enables the construction of learning network models for various types of machine tools, and enables construction of various learning network models for the same type of machine tools. Various learning network models for the same type of machine tool may be built with various models depending on the environment where the machine tool is installed or the amount of operation.

The collector communication unit 220 may include a short-distance communication module, a wireless Internet module, and a mobile communication module 127 for wireless communication.

Wired communication technologies may include power line communication (PLC), USB communication, Ethernet, serial communication, optical/coaxial cables, and the like.

The short-distance communication module refers to a module for short-distance communication. Short-range communication technologies include Bluetooth, RFID (Radio Frequency Identification), IrDA (Infrared Data Association), UWB (Ultra Wideband), ZigBee, Near Field Communication (NFC), and ultrasonic communication (Ultra Sound Communication: USC), Visible Light Communication (VLC), Wi-Fi, Wi-Fi Direct, and the like may be used.

The wireless Internet module refers to a module for wireless Internet access. Wireless Internet technologies include WLAN (Wireless LAN) (Wi-Fi), DLNA (Digital Living Network Alliance), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access), HSUPA ( High Speed Uplink Packet Access), IEEE 802.16, Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), Wireless Mobile Broadband Service (WMBS), etc. may be used. there is.

The mobile communication module may transmit/receive radio signals with at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the radio signal may include various types of data such as voice, video, photo, text, and combinations thereof.

The mobile communication module is GSM (Global System for Mobile communication), CDMA (Code Division Multi Access), CDMA2000 (Code Division Multi Access 2000), EV-DO (Enhanced Voice-Data Optimized or Enhanced Voice-Data Only), WCDMA (Wideband) CDMA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), and Long Term Evolution-Advanced (LTEA).

In addition, the collector communication unit 220 may include a low power wide area network module for IoT such as NB-IoT, LTE-M, LoRa, and Sigfox. The low-power wide-area network module may use the aforementioned short-distance/wireless Internet/mobile communication module.

The collector detector 250 may include a vibration sensor module 260 , a noise sensor module 270 , and a power amount measurement module 280 . The collector sensing unit 250 may include a plurality of each module as many as the number of machine tools connected to the data collector 200 .

The vibration sensor module 260 and the noise sensor module 270 each have a plurality of sensors and may be attached to various positions of the machine tool 300 . Data measured by the vibration sensor module 260 and the noise sensor module 270 may be used to predict failure or aging of the machine tool 300 or failure.

The vibration sensor module 260 may include a plurality of vibration sensors. It is preferable that the vibration sensor is capable of measuring vibration in the x, y, and z axes, respectively. A plurality of vibration sensors may be attached to a frame of the machine tool 300 or a part that undergoes mechanical movement.

The noise sensor module 270 may include a plurality of noise sensors. A plurality of noise sensors may be attached to a frame of the machine tool 300 or a part that undergoes mechanical movement.

The noise sensor module 270 may measure data for the collector control unit 210 to analyze the volume and frequency of sound generated from the machine tool 300 .

The power measurement module 280 may measure power consumption of the machine tool 300 . The measured power consumption is the basis for comparison with a normal power consumption pattern and/or whether there is an abnormal waveform in the measured power consumption signal, so that the machine tool 300 can help determine normal operation or abnormal operation. there is.

The amount of vibration measured by the vibration sensor module 260 of the collector detector 250, the amount of noise measured by the noise sensor module 270, and the amount of power measured by the power amount measuring module 280 may be defined as a diagnostic data set. .

The collector control unit 210 may control the overall operation of the data collector 200 by controlling the operation of each part.

The collector controller 210 uses the amount of vibration measured by the vibration sensor module 260 of the collector detector 250, the amount of noise measured by the noise sensor module 270, and the amount of power measured by the power measurement module 280 for diagnosis. It can be collected as a data set and transmitted to the database 110 .

The data collector 200 may further include a state recognition unit 230 for generating a learning network model for predicting failure or failure of the machine tool 300 .

The state recognition unit 230 may check a state information label indicating whether the machine tool 300 is out of order and where the machine tool 300 is out of order.

The state recognition unit 230 may receive state information labels for the transfer axis and the rotation axis through an encoder or a vision. The state detecting unit 230 may receive information about temperature, pressure, humidity, inclination, and position of the surroundings of the machine tool 300 through various sensors.

The state detecting unit 230 may receive an input of whether or not there is a failure and a failure part by a user.

The collector control unit 210 may collect a data set for diagnosis and a state information label as a data set for learning and transmit the collected data set to the database 110 .

The database 110 may store data sets collected by the data collector 200 . The database 110 may store a learning network model, learning parameters, and learning results.

The deep learning framework 120 may be connected to the database 110 in a plug-in manner. The deep learning framework 120 may check, correct, and add new data to information or data stored in the database 110 .

The user terminal 130 may input a query through the deep learning framework 120 . The user terminal 130 may receive an inference result corresponding to a query from the database 110 through the deep learning framework 120 .

Upon receiving a query, the deep learning framework 120 may check and/or modify the learning network model stored in the database 110, or create a learning network model for new learning. The deep learning framework 120 may execute machine learning by selecting information or data and a learning network model according to an input query and setting learning parameters. The deep learning framework 120 may provide an intermediate learning result and a final result according to the degree of machine learning.

The deep learning framework 120 selects a data set and a pre-learned learning network model through an input query, executes machine inference, and provides the inference result as failure diagnosis and/or failure prediction.

The database 110 may store a plurality of failure diagnosis prediction models 140 including first and second failure diagnosis prediction models 140-1 and 140-2. The failure diagnosis prediction model 140 may correspond to the aforementioned learning network model. For detailed description, refer to FIGS. 1 to 8 .

The failure diagnosis prediction model 140 may refer to a classification model learned to output a result classified as failure or failure prediction of the machine tool 300 . The failure diagnosis prediction model 140 may be generated through learning using one of various learning algorithms and one of a plurality of diagnostic data sets.

The data set for diagnosis may be collected from the data collector 200 having the state recognition unit 230 or obtained from a public database.

11 and 12 are flowcharts of a method for machine learning, fault diagnosis, and edge of a machine tool predictive maintenance system using a query-based deep learning inference system according to an embodiment of the present invention. See Figures 9 and 10.

Referring to FIG. 11 , the deep learning framework 120 may receive a learning query (call train) from the user terminal 130 (S510).

The data collector 200 obtains the state information from the state recognition unit 230 that checks the state information label of the machine tool 300 and the detection unit that measures the amount of vibration, noise, and power of the machine tool 300. A label, vibration amount, noise amount, and power amount may be collected as a learning data set and provided to the database 110 .

The deep learning framework 120 may designate at least a part of the training data set as a data set for suitability determination (S520).

The deep learning framework 120 may determine whether there is a model suitable for the suitability determination data set DS among the plurality of pre-stored learning network models 140 and whether a suitable model exists (S530).

The existence of a suitable model can be determined based on the correspondence between the data set of the failure diagnosis prediction model 140 and the suitability determination data set, and the degree of accuracy of failure diagnosis or prediction according to the suitability determination data set. there is.

When there is no suitable model as a result of determining whether a suitable model exists (S530), the deep learning framework 120 may generate a suitable failure diagnosis prediction model 140. To this end, the deep learning framework 120 initializes a network (NN; Nueral Network) according to the learning query (S540), configures the network (NN), and updates the network (NN) as necessary ( S550). The deep learning framework 120 may initialize all layers of the configured network NN (S560). Network initialization, configuration and update, and layer initialization refer to the description of FIG. 5 .

When all layers are initialized, the deep learning framework 120 may execute learning using the training data set (S580).

If there is a suitable model as a result of determining whether a suitable model exists (S530), the deep learning framework 120 checks the corresponding model, for example, the first failure diagnosis prediction model 140-1, and determines the first failure diagnosis prediction It may be determined whether the model 140-1 needs to be modified (S535). If the first failure diagnosis prediction model 140-1 needs to be modified, the deep learning framework 120 performs steps S540, S550, S560, and S580 to add the first failure diagnosis prediction model 140-1. can be learned by

A criterion for determining whether the first failure diagnosis prediction model 140-1 needs to be corrected may vary. For example, there may be examples such as the generation date of the first failure diagnosis prediction model 140-1 being older than a pre-set period or a tag indicating that correction is required.

Determination of whether or not to modify the first failure diagnosis prediction model 140-1 may be made simultaneously in a suitable model determination (S530). In this case, the network of the first failure diagnosis prediction model 140-1 may be initialized and configured, and the layer may be initialized. Accordingly, if correction is required, the deep learning framework 120 may directly execute learning (S580) on the corresponding first failure diagnosis prediction model 140-1.

When the learning execution (S580) ends, the deep learning framework 120 may provide the result to the user terminal 130 (S590) and store the completed learning network model in the database 110.

Referring to FIG. 12 , the deep learning framework 120 may receive an inference query (call test) from the user terminal 130 (S610).

The deep learning framework 120 may collect a diagnostic data set from the data collector 200 (S620).

The deep learning framework 120 may analyze the training data set and call an appropriate first failure diagnosis prediction model 140-1 among the plurality of failure diagnosis prediction models 140 (S640). The first failure diagnosis prediction model 140-1 may be selected and called by a user's input.

The deep learning framework 120 constructs a network NN according to the first failure diagnosis prediction model 140-1 (S650) and initializes all layers of the network NN (S660) for diagnosis. Inference can be executed based on the data set (S670).

When the inference execution (S670) ends, the deep learning framework 120 may diagnose and predict a failure of the machine tool 300 (S680).

The present invention may be implemented in hardware or software. In implementation, the present invention can also be implemented as computer readable codes on a computer readable recording medium. That is, it may be implemented in the form of a recording medium including instructions executable by a computer. Computer readable media includes all types of media in which data that can be read by a computer system is stored. Computer readable media may include computer storage media and communication storage media. Computer storage media includes all storable media implemented in any method or technology for storing information, such as computer readable instructions, data structures, program modules, and other data, and includes volatile/nonvolatile/hybrid memory. It is not limited to whether or not, separable/non-separable. Communication storage media includes modulated data signals or transport mechanisms such as carrier waves, any information delivery media, and the like. In addition, functional programs, codes, and code segments for implementing the present invention can be easily inferred by programmers in the art to which the present invention belongs.

In addition, although the preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention claimed in the claims. Of course, various modifications are possible by those skilled in the art, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

100: query-based deep learning inference system 110: database
120: deep learning framework 130: user terminal
140: failure diagnosis prognosis model 200: data collector
300: machine tool

Claims (8)

A machine tool predictive maintenance method using a query-based deep learning inference system of a deep learning framework that interworks with a user terminal and a database,
receiving a learning query from the user terminal;
At least some of the learning data sets provided by the data collector are used as suitability judgment data sets, and among the plurality of failure diagnosis prediction models stored in the database, there is a suitable model for determining whether there is a failure diagnosis prediction model suitable for the suitability judgment data set Existence determining whether or not;
a network initialization and configuration step of initializing a network according to the learning query and constructing the network when there is no suitable failure diagnosis prediction model as a result of determining whether the appropriate model exists;
a learning execution step of executing learning using the collected training data set when all layers of the configured network are initialized; and
A storage step of providing a result when the learning execution is terminated and storing the terminated failure diagnosis prediction model;
The suitability determination data set includes vibration of the machine tool, noise of the machine tool, power of the machine tool, and state information label of the machine tool,
The status information label indicates whether the machine tool is out of order and where the machine tool is out of order,
Whether or not the suitability model exists is determined based on whether a data set of the plurality of failure diagnosis prediction models corresponds to the suitability determination data set,
a model modification determination step of determining whether the first failure diagnosis prediction model needs to be modified, if there is a suitable first failure diagnosis prediction model among the plurality of failure diagnosis prediction models as a result of determining whether the appropriate model exists; and
If the first failure diagnosis prediction model needs to be modified, additionally training the first failure diagnosis prediction model by performing the network initialization and configuration step, the learning execution step, and the storage step; ,
Determining whether or not to modify the model is determined to require modification when the generation date of the first failure diagnosis prediction model is older than a predetermined period or when a tag indicating that modification is required is attached to the first failure diagnosis prediction model, Machine tool predictive maintenance method using query-based deep learning inference system. According to claim 1,
receiving an inference query from the user terminal;
collecting a diagnostic data set including vibration, noise, and power of the machine tool from the data collector;
analyzing the diagnosis data set and calling an appropriate failure diagnosis prediction model among the plurality of failure diagnosis prediction models;
an inference execution step of constructing a network according to the appropriate failure diagnosis predictive model and executing inference based on the diagnosis data set when all layers of the network are initialized; and
The machine tool predictive maintenance method using a query-based deep learning inference system, further comprising: diagnosing a failure of the machine tool and predicting a failure when the inference execution is terminated. According to claim 1,
The learning execution step acquires batch data until the end of the learning (Get Batch Data), repeats (Iteration), and stores the result and model (Store Result & Model),
A machine tool predictive maintenance method using a query-based deep learning inference system that converts a model weight file including ONNX, NNEF, and learning parameter biases into a structured format through a model converter when information or data is exported from the database. According to claim 2,
The database stores all input and output data used in machine learning and machine reasoning, stores models used in machine learning and machine reasoning, and provides a procedure corresponding to a user's query request,
The procedure includes Insert Network, Insert Layer, Make Project, Input Data Loader, Train, Save Model and Test ),
The deep learning framework is used for compatibility with external frameworks, and when a pre-learned model of an existing framework is imported or information or data is exported from the database, a model including ONNX, NNEF, and learning parameter bias A model converter that uses the weight file in a structured format; includes,
The model converter converts a network structure and model data defined in a model format using the model weight file including the ONNX, NNEF, and learning parameter bias in a structured format to the network model table of the database format, or conversely converting the failure diagnosis prediction model of the database into the structured model format,
The inference execution step executes an inference test (Test), obtains test data (Get Test Data), and feeds forward the machine tool predictive maintenance method using a query-based deep learning inference system. Collecting the vibration, noise, power, and status information labels as a learning data set from a sensing unit that measures the amount of vibration, noise, and power of the machine tool, and a state recognition unit that checks the state information of the machine tool a data collector;
a database for storing the learning data set, learning network model, learning parameters, and learning results;
a deep learning framework that is connected to the database in a plug-in manner and checks, corrects, and adds new data to information or data stored in the database; and
A user terminal that inputs a query through the deep learning framework and receives an inference result corresponding to the query from the database through the deep learning framework;
The status information label indicates whether the machine tool is out of order and where the machine tool is out of order,
The deep learning framework,
If the query is a learning query, determining whether there is a failure diagnosis prediction model suitable for the suitability judgment data set among a plurality of failure diagnosis prediction models stored in the database by using at least some of the training data sets as a suitability determination data set; ,
If there is no suitable failure diagnosis prediction model as a result of determining whether or not the suitable model exists, initializing the network according to the learning query and performing network initialization and configuration steps for constructing the network;
When all layers of the configured network are initialized, learning is performed using the collected learning data set;
When the learning execution is terminated, a result is provided and the terminated failure diagnosis prediction model is stored;
provide learning intermediate and final results;
Whether or not the suitability model exists is determined based on whether a data set of the plurality of failure diagnosis prediction models corresponds to the suitability determination data set,
The deep learning framework,
As a result of determining whether the appropriate model exists, if there is a suitable first failure diagnosis prediction model among the plurality of failure diagnosis prediction models, it is determined whether the first failure diagnosis prediction model needs to be modified;
If the first failure diagnosis prediction model needs to be modified, further training the first failure diagnosis prediction model by performing the network initialization and configuration steps, the learning execution, and the storage;
Determination of whether or not to modify the model is determined to require modification when the generation date of the first failure diagnosis prediction model is older than a predetermined period or when a tag indicating that modification is required is attached to the first failure diagnosis prediction model,
When the query is an inference query, the deep learning framework collects a diagnostic data set including vibration, noise, and power of the machine tool from the data collector, and analyzes the diagnostic data set to obtain the plurality of data sets. An appropriate failure diagnosis prediction model among failure diagnosis prediction models is called, a network is configured according to the appropriate failure diagnosis prediction model, and when all layers of the network are initialized, inference is executed based on the diagnosis data set, and the A machine tool predictive maintenance system using a query-based deep learning inference system that diagnoses and predicts a failure of the machine tool when inference execution is terminated. According to claim 5,
The failure diagnosis prediction model is a model used for machine learning, and is composed of input/output parameters, parameters defining the inside of the model, and parameters necessary for machine learning and inference, stored in the database in a relational data format, and through a converter, other deep learning models. A machine tool predictive maintenance system using a query-based deep learning inference system that can be converted into a learning framework. According to claim 6,
The database stores all input and output data used in machine learning and machine reasoning, stores models used in machine learning and machine reasoning, and provides a procedure corresponding to a user's query request,
The procedure includes Insert Network, Insert Layer, Make Project, Input Data Loader, Train, Save Model and Test ),
The deep learning framework is used for compatibility with external frameworks, and when a pre-learned model of an existing framework is imported or information or data is exported from the database, a model including ONNX, NNEF, and learning parameter bias A model converter that uses the weight file in a structured format; includes,
The model converter converts a network structure and model data defined in a model format using the model weight file including the ONNX, NNEF, and learning parameter bias in a structured format to the network model table of the database A machine tool predictive maintenance system using a query-based deep learning reasoning system that converts a failure diagnosis prediction model of the database into a format or, conversely, converts a failure diagnosis prediction model of the database into the structured model format. delete

Images