CN117215258A - Numerical control machine tool real-time state monitoring system and method based on Flink - Google Patents
Numerical control machine tool real-time state monitoring system and method based on Flink Download PDFInfo
- Publication number
- CN117215258A CN117215258A CN202311033378.7A CN202311033378A CN117215258A CN 117215258 A CN117215258 A CN 117215258A CN 202311033378 A CN202311033378 A CN 202311033378A CN 117215258 A CN117215258 A CN 117215258A
- Authority
- CN
- China
- Prior art keywords
- data
- state
- machine tool
- flink
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 238000012544 monitoring process Methods 0.000 title claims abstract description 44
- 238000012545 processing Methods 0.000 claims abstract description 96
- 230000002159 abnormal effect Effects 0.000 claims abstract description 34
- 238000013500 data storage Methods 0.000 claims abstract description 20
- 238000012549 training Methods 0.000 claims description 42
- 238000004458 analytical method Methods 0.000 claims description 31
- 239000011159 matrix material Substances 0.000 claims description 24
- 238000012423 maintenance Methods 0.000 claims description 22
- 230000008569 process Effects 0.000 claims description 19
- 238000012216 screening Methods 0.000 claims description 18
- 238000003860 storage Methods 0.000 claims description 16
- 238000006243 chemical reaction Methods 0.000 claims description 15
- 238000007781 pre-processing Methods 0.000 claims description 14
- 230000007246 mechanism Effects 0.000 claims description 11
- 238000010606 normalization Methods 0.000 claims description 11
- 238000001914 filtration Methods 0.000 claims description 10
- 238000004422 calculation algorithm Methods 0.000 claims description 9
- 238000013480 data collection Methods 0.000 claims description 9
- 239000013598 vector Substances 0.000 claims description 9
- 238000004140 cleaning Methods 0.000 claims description 8
- 238000004519 manufacturing process Methods 0.000 claims description 8
- 230000008859 change Effects 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 7
- 238000012360 testing method Methods 0.000 claims description 7
- 230000002776 aggregation Effects 0.000 claims description 6
- 238000004220 aggregation Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 6
- 230000005856 abnormality Effects 0.000 claims description 5
- 238000010801 machine learning Methods 0.000 claims description 5
- 230000000007 visual effect Effects 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000007621 cluster analysis Methods 0.000 claims description 3
- 238000009826 distribution Methods 0.000 claims description 3
- 238000011156 evaluation Methods 0.000 claims description 3
- 230000036541 health Effects 0.000 claims description 3
- 238000002360 preparation method Methods 0.000 claims description 3
- 230000008439 repair process Effects 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 2
- 230000010354 integration Effects 0.000 abstract description 3
- 230000006870 function Effects 0.000 description 7
- 238000013461 design Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 230000001133 acceleration Effects 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000013144 data compression Methods 0.000 description 2
- 238000013499 data model Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000003745 diagnosis Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000013467 fragmentation Methods 0.000 description 2
- 238000006062 fragmentation reaction Methods 0.000 description 2
- 238000003754 machining Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000013079 data visualisation Methods 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000005111 flow chemistry technique Methods 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000012731 temporal analysis Methods 0.000 description 1
- 238000000700 time series analysis Methods 0.000 description 1
Abstract
The invention provides a numerical control machine real-time state monitoring system and method based on a Flink, wherein the system comprises a data acquisition module, a Flink processing module, a data storage module and a user interface module, wherein the data acquisition module acquires various parameters and state data of the numerical control machine, the Flink processing module receives data from Kafka and cleans the received data to remove noise and abnormal values, then analyzes and preprocesses the cleaned data, calculates various statistics, identifies an abnormal mode, and further comprises a state prediction model for predicting possible machine faults according to historical data and current data. The invention has the characteristic of multi-data source integration, can monitor and early warn in real time, carry out accurate state prediction, adopts the Flink as a stream processing engine, realizes distributed and high-reliability data processing, and has extremely high expandability and customizable performance so as to meet the specific requirements of different users.
Description
Technical Field
The invention relates to the field of numerical control machine tools, in particular to a numerical control machine tool real-time state monitoring system and method based on a Flink.
Background
In recent years, with the development of technologies such as big data, cloud computing and internet of things, a numerical control machine tool state monitoring system based on real-time data processing and analysis is attracting more attention. As a key device in the manufacturing industry, state monitoring of a numerically controlled machine tool has been a focus of attention of a manufacturing enterprise.
The traditional numerical control machine state monitoring system has some problems. First, existing systems typically employ offline data processing and analysis, and are unable to monitor and respond in real time to changes in machine tool state. Secondly, the data acquisition and transmission efficiency is low, and the real-time requirements of high-speed and large-capacity data cannot be met. In addition, the existing system has weak capability of predicting the state of the machine tool and diagnosing faults, and cannot prevent faults and optimize maintenance plans in advance. The general fault diagnosis is carried out after faults occur, certain time delay exists, the algorithm of the fault diagnosis module also needs to run in a server of a user client, professional software is needed to support during running, the requirement on a computer of the client is high, the working state of a digital control machine tool cannot be monitored in real time through accessing other terminal equipment, and multi-terminal monitoring cannot be achieved.
Disclosure of Invention
The invention aims to provide a numerical control machine tool real-time state monitoring system and method based on a Flink, which can process and analyze a large amount of data accurately in real time, have an early warning function and can predict possible machine tool faults in advance.
In order to achieve the above purpose, the present invention proposes the following technical scheme: a numerical control machine real-time state monitoring system based on a Flink comprises:
the data acquisition module acquires various parameters and state data of the numerical control machine tool and caches the parameters and the state data through a Kafka message caching mechanism, and the data acquisition module sends the parameters and the state data to the Flink processing module in a data stream mode, and the data acquisition module sends the parameters and the state data to a remote processing module through a wireless network;
the Flink processing module is used for receiving data from Kafka, cleaning the received data to remove noise and abnormal values, analyzing and preprocessing the cleaned data, calculating various statistics and identifying abnormal modes, and further comprises a state prediction model for predicting possible machine tool faults according to historical data and current data;
The data storage module is used for storing the data processed by the Flink processing module;
and the user interface module is used for displaying the processed data to a user in a visual form.
Further, in the invention, the data acquisition module comprises an acoustic emission sensor, an accelerometer and an acquisition gateway, wherein the acoustic emission sensor acquires acoustic emission signals, the accelerometer acquires vibration signals of a main shaft and a cutter of the numerical control machine tool, the acquisition gateway receives various parameters and state data of the acquired machine tool, the parameters and the state data are converted into standard data formats, the data are uploaded through the 4G/5G/WiFi network module, and the data are cached through a Kafka message caching mechanism.
Further, in the present invention, the data processing by the link processing module includes the following procedures:
the method comprises the steps that a data stream is accessed, a Flink processing module is firstly connected to a Kafka message queue, corresponding topics are subscribed to obtain real-time data streams, corresponding topics and consumption groups are configured through a Kafka connector provided by the Flink, data are read from the Kafka and are converted into data streams of the Flink, and reliable access and distribution of the data are ensured;
After receiving the data stream, the Flink processing module performs data preprocessing, including data cleaning, format conversion, data aggregation and feature extraction, analyzes the data into objects or tuples in the Flink data stream by using a Jackson analyzer and a Apache Commons CSV library according to the received JSON and CSV format data, performs filtering operation on the data stream by using a Filter operator of the Flink according to service requirements, and screens out data meeting the requirements according to specified rules:
data range filtering: data with numeric fields within a specific range, such as data with temperatures between 60 ℃ and 70 ℃, are screened.
Outlier rejection: and eliminating possible abnormal values in the data according to the common business knowledge, for example, regarding the spindle multiplying power data exceeding 200 as abnormal values and eliminating.
Missing value processing: and screening out the data without the missing value, and ensuring the data integrity. For example, if a field in the data is missing, the piece of data is culled.
And (3) frequency filtering: and screening out the events with the occurrence times exceeding a certain threshold value in a certain time window according to the occurrence frequency of a certain event.
Threshold value judgment: based on the threshold value of some index, data which exceeds or falls below the threshold value is screened out, for example, data of which the spindle vibration acceleration is higher than 2.5g is screened out.
Keyword matching: according to the specific keyword or the field content, data containing the specific keyword is selected, for example, data containing keywords such as "dev-" "iot _" "box-" are selected from the data.
And (3) data format verification: and screening out data meeting the specific format according to the data format rule, for example, verifying the warning information of the numerical control machine tool, and only retaining the data meeting the format requirement.
Time window screening: and screening out data in a specific time window according to the time stamp field, for example, screening out the running data of the numerical control machine tool in the last hour.
Filtering invalid or abnormal data, screening out characteristic fields with important significance and relativity according to the characteristics and the prediction targets of the data, and performing numerical conversion, normalization, discretization and aggregation operation on the characteristics for subsequent state analysis and prediction models;
the method comprises the steps of performing state analysis, namely performing structured query calculation on data in HBase through Apache Phoenix, performing state analysis on the data by a Flink processing module after preprocessing, calculating various statistics, periodically analyzing and detecting abnormality to obtain the current state of the numerical control machine tool, detecting potential abnormal conditions, and using the result of the state analysis to monitor the health condition of the machine tool in real time and providing the result of the state analysis to a prediction model as an input characteristic;
The method comprises the steps of training a state prediction model, training the Flink processing module by using historical data and current state characteristics, obtaining a prediction model by training the Flink processing module, analyzing and training the prediction model by using TensorFlow and Keras, and predicting possible faults or abnormal conditions by using the prediction model according to the current machine tool state, so that measures are taken in advance to maintain and prevent the faults or abnormal conditions; and (3) constructing a prediction model:
data collection and preparation: historical data is collected, including machine tool state characteristics and corresponding fault or abnormal conditions. The data is cleaned, processed and feature extracted, and converted to csv format suitable for model training.
Dividing data: the collected data is divided into a training set and a test set. The training set is used for training the model, and the testing set is used for verifying the performance and generalization capability of the model.
Characteristic engineering: according to the characteristics of the problem, the data characteristics are further processed, including characteristic selection, conversion, standardization and the like. This may help the model better capture patterns and rules of the data.
Modeling a model: the real-time values of the sequence of the parameters corresponding to the machine tool state are independently analyzed, and the modeling thought is shown in fig. 6. Matrix of time length l (X n+1 ) m×l For a machine tool state matrix that needs prediction, (X) n ) m×l Is the current state matrix of the machine tool. From matrix X n Current sample time element x n Firstly, reversely moving the time window length l by a sliding unit w to obtain a state matrix general term represented by a kth time window, wherein the state matrix general term is as follows:
X k ＝[x(n-wk-l+1),x(n-wk-l+2),x(n-wk-l+3)...,x(n-wk)]
k=1, 2..represents the kth history state, w=1, 2..is the sliding interval length of the sliding time window. The window length l is greater than the sliding duration w.
Model training: the selected model is trained using a training set. The training process is to adjust parameters through a model so that the parameters can be better fit to actual conditions on training data.
The prediction method comprises the following steps:
pretreatment of data acquisition: the state characteristics of the current machine tool, including sensor data, operation records and the like, are collected in real time. The number of parameter types is m, n number of value points are obtained according to the sampling frequency of 1000ms, and all historical process data of the machine tool form a multi-dimensional time sequence matrix T with m rows and n columns m×n . The normalization process is carried out on each parameter mi by using the min-max normalization, and the normalization process is shown in a formula (1).
Multiple matching modeling: and (3) carrying out the same processing on the data acquired in real time according to a feature extraction method used in the model training stage to obtain feature vectors. The similarity analysis is used for solving a maximum similarity set, and the number of elements in the similarity set is used for solving an optimal sliding unit w and an optimal prediction duration l; performing cluster analysis on similar concentrated elements through a DBSCAN clustering algorithm to complete accurate matching of states, and determining a sequence matrix X which is most similar to the current state change trend in a historical state k And will then be of length lAs a final prediction result.
Model loading: and loading the trained prediction model into a Flink processing module.
Feature conversion: and performing feature conversion identical to the training data on the feature vectors acquired in real time, and ensuring that the input data accords with a data format expected by the model.
And (3) predicting: and predicting the feature vector by using the loaded prediction model.
Interpretation and processing of results: and judging whether the state of the current machine tool is likely to be faulty or abnormal according to the prediction result of the model. Based on the prediction, an alarm, a maintenance request, etc. may be triggered.
Continuously monitoring and updating: the steps are repeated regularly to monitor the state of the machine tool in real time and update the prediction model, so that the accuracy and the practicability of the model are ensured.
And the Flink processing module sends the processed data and the prediction result to the data storage module, and generates real-time alarm information which can be displayed through the user interface module so that a user can timely obtain the change of the machine tool state and possible fault early warning.
Further, in the present invention, the subject matter of Kafka includes:
the machine_data theme is used for collecting and transmitting original data of the machine tool, the original data comprise sensor data and accelerometer data, a publishing end of the theme is a data acquisition module, the machine tool data is published to the theme in a JSON format, and a subscribing end is a link processing module which is responsible for processing and converting the data;
The alarm event issuing system comprises an alarm event issuing module, a feedback processing module, a data storage module and a data storage module, wherein the alarm event issuing module is used for issuing an alarm event of a machine tool, including faults and anomalies, the alarm event issuing module judges the working state of the current machine tool according to alarm information and then issues the alarm event to the alarm event issuing module, and the subscription end is responsible for storing the alarm event to a database or other storage media;
the status_updates theme is used for publishing updated information of the machine tool state, including the running state and the production progress of the machine tool, the Flink processing module performs state analysis and processing according to the collected machine tool data, and publishes the updated state information to the theme, and the subscribing end is a user interface module and is used for displaying and monitoring the state of the machine tool in real time;
the prediction_results theme is used for issuing machine tool state prediction results, including prediction and evaluation of future states of the machine tool, the state analysis and processing can train and predict historical data by using a machine learning algorithm, the prediction results are issued to the theme, and the subscribing end is a data storage module and is responsible for storing the prediction results in a database or other storage media;
the main_requests theme is used for issuing maintenance requests, including the requirements or repair requests of machine tool maintenance submitted by users, the user interface module receives the maintenance requests of the users and issues the maintenance requests to the theme, and the Flink processing module subscribes to the theme and is responsible for processing and responding to the maintenance requests.
A method for monitoring the real-time state of a numerical control machine tool based on a Flink comprises the following steps:
the method comprises the steps that various parameters and state data of a numerical control machine tool are collected through a data collection module, and are cached through a Kafka message caching mechanism, the data collection module sends the parameters and the state data to a Flink processing module in a data stream mode, and the data collection module sends the parameters and the state data to a remote processing module through a wireless network;
the method comprises the steps of receiving data from Kafka through a Flink processing module, cleaning the received data to remove noise and abnormal values, analyzing and preprocessing the cleaned data, calculating various statistics, identifying abnormal modes, analyzing and predicting the data by using a pre-trained state prediction model to identify the state and faults of a machine tool, monitoring and analyzing real-time data by using the trained state prediction model, judging the current state of the numerical control machine tool according to the prediction result of the state prediction model, including normal operation, abnormal state or possible fault state, and if abnormality or fault is detected, triggering a corresponding early warning mechanism by a system, and sending an alarm notification to an operator or related maintenance personnel;
The data storage module is used for storing the data processed by the Flink processing module;
and the user interface module is used for displaying the processed data to a user in a visual form.
An electronic device comprising a processor, a memory and a computer program stored in the memory, the computer program being configured to perform the above-described method when run by the processor.
A computer readable storage medium having stored thereon a computer program for performing the method described above.
The beneficial effects are that the technical scheme of the application has the following technical effects:
(1) Real-time monitoring and early warning: the system can monitor the state of the numerical control machine tool in real time, timely capture abnormal conditions and faults in the running process of the machine tool, and early warn in advance, so that production progress is prevented from being delayed or the machine tool is prevented from being damaged due to the faults.
(2) Accurate state prediction: by using a Flink flow processing framework and combining a machine learning algorithm, preprocessing and analyzing the real-time data of the numerical control machine tool, and establishing an accurate state prediction model. The working state of the machine tool can be predicted in advance, the downtime caused by machine tool faults is reduced, and the production efficiency is improved.
(3) Multiple data source integration: the system can flexibly integrate different data sources, such as acoustic emission sensors, accelerometers and the like, so as to realize comprehensive monitoring of all aspects of the machine tool. By integrating a plurality of data sources, the system can comprehensively acquire the running state and performance index of the machine tool and provide more comprehensive monitoring and analysis functions.
(4) Distributed stream processing: and the Flink is adopted as a stream processing engine to realize distributed and high-reliability data processing. The Flink has the characteristics of excellent fault tolerance mechanism and high throughput, can effectively process large-scale real-time data, and ensures the accuracy and stability of data processing.
(5) Flexible data storage and querying: the system adopts two storage modes of the relational MySQL and the non-relational HBase, and stores according to different data characteristics. The relational database is adapted to store structured data and the non-relational database is adapted to store large-scale unstructured data. Thus, the proper data storage mode can be flexibly selected, and the data can be conveniently inquired and analyzed.
(6) Extensibility and customizable: the system adopts a modularized architecture design, and all modules are decoupled, so that the expansion and customization are facilitated. The modules can be added, replaced or custom developed according to the actual requirements so as to meet the specific requirements of different users.
It should be understood that all combinations of the foregoing concepts, as well as additional concepts described in more detail below, may be considered a part of the inventive subject matter of the present disclosure as long as such concepts are not mutually inconsistent.
The foregoing and other aspects, embodiments, and features of the present teachings will be more fully understood from the following description, taken together with the accompanying drawings. Other additional aspects of the invention, such as features and/or advantages of the exemplary embodiments, will be apparent from the description which follows, or may be learned by practice of the embodiments according to the teachings of the invention.
Drawings
The drawings are not intended to be drawn to scale. In the drawings, each identical or nearly identical component that is illustrated in various figures may be represented by a like numeral. For purposes of clarity, not every component may be labeled in every drawing. Embodiments of various aspects of the invention will now be described, by way of example, with reference to the accompanying drawings, in which:
FIG. 1 is a diagram of a hardware environment set up in accordance with the present invention.
Fig. 2 is a diagram of the overall architecture of the system of the present invention.
FIG. 3 is a diagram showing the system module requirements according to the present invention.
Fig. 4 is a flow chart of the system of the present invention.
Fig. 5 is a Kafka channel theme service storage design diagram of the present invention.
FIG. 6 is a diagram of a modeling concept of the present invention.
Detailed Description
For a better understanding of the technical content of the present invention, specific examples are set forth below, along with the accompanying drawings. Aspects of the invention are described in this disclosure with reference to the drawings, in which are shown a number of illustrative embodiments. The embodiments of the present disclosure need not be defined to include all aspects of the present invention. It should be understood that the various concepts and embodiments described above, as well as those described in more detail below, may be implemented in any of a number of ways, as the disclosed concepts and embodiments are not limited to any implementation. Additionally, some aspects of the disclosure may be used alone or in any suitable combination with other aspects of the disclosure.
1-6, the embodiment provides a numerical control machine tool real-time state monitoring system based on a Flink, which can process and analyze a large amount of data accurately in real time and has an early warning function, and possible machine tool faults can be predicted in advance.
The system comprises the following modules:
and a data acquisition module: the module adopts high-precision sensors and data acquisition equipment to collect various parameters and state data such as temperature, pressure, vibration, current, voltage and the like from the numerical control machine tool, and sends the data to the Flink processing module in a data stream form. The module also has a network communication function, and transmits data to a remote processing module through a network.
In the embodiment, the 8152C acoustic emission sensor is adopted, so that acoustic emission signals of the numerical control machine tool can be monitored very accurately. Acoustic emission is a powerful monitoring technique that can detect potential faults and problems in advance by monitoring tiny acoustic waves generated by the material. For example, if the tool wears or the workpiece is defective, the characteristics of the acoustic emission signal may be altered. By real-time monitoring and analysis of these signals, these problems can be found and resolved in advance.
In this embodiment, a 8763b100bb accelerometer is used to accurately monitor vibration of the spindle and tool of the numerically controlled machine tool. Vibration of the machine tool is an important status signal that reflects the operating condition of the machine tool. For example, if the movement of the spindle during machining and the wear caused by machining of the tool change its vibration characteristics. By real-time monitoring and analysis of the vibration signal, these problems can be found and solved in advance.
In addition to the above-described sensors, the data acquisition module also includes a numerically controlled machine tool acquisition gateway. The gateway can be communicated with a numerical control system of the machine tool, receives various control signals and state signals of the machine tool, such as main shaft state, position, rotating speed, load, servo shaft feeding speed, current, voltage and the like, converts the data into a standard data format, uploads the acquired data through a 4G/5G/WiFi network module, and caches the acquired data through a Kafka message caching mechanism.
A Flink processing module: the module is the core of the system and can process and analyze large-scale data streams in real time based on the Flink platform. The module first receives data from Kafka and cleans the received data to remove noise and outliers, then analyzes and pre-processes the cleaned data, calculates various statistics, identifies possible outlier patterns, and the like. In addition, the module also comprises an early warning model, and the model predicts possible machine tool faults according to historical data and current data.
The Flink processing module processes data, which comprises the following steps:
1. and (3) data stream access: the Flink processing module is firstly connected to the Kafka message queue and subscribes to the corresponding theme to acquire the real-time data stream. And (3) configuring corresponding topics and consumption groups through a Kafka connector provided by the Flink, reading data from the Kafka, converting the data into a data stream of the Flink, and ensuring reliable access and distribution of the data. The system designs the following topics:
the machine_data theme is used to collect and transmit raw data of the machine tool, including sensor data, accelerometer data, and the like. The publishing end of the theme is a data acquisition module which publishes machine tool data to the theme in a JSON format, and the subscribing end is a Flink processing module which is responsible for processing and converting the data.
The alarm events topic is used to issue alarm events for machine tools, such as faults, anomalies, etc. The Flink processing module judges the working state of the current machine tool according to the alarm information, then the current machine tool is published to the theme, and the subscribing terminal is a data storage module and is responsible for storing alarm events into a database or other storage media.
The status_updates topic is used to issue updated information on the machine tool status, such as the machine tool's running status, production schedule, etc. The Flink processing module performs state analysis and processing according to the collected machine tool data and issues updated state information to the subject. The subscription terminal is a user interface module and is used for displaying and monitoring the state of the machine tool in real time.
The prediction_results topic is used to issue machine tool state prediction results, such as predictions and evaluations of future machine tool states. The link processing module will use machine learning algorithms to train and predict historical data and issue the prediction results to the topic. The subscription terminal is in charge of storing the prediction result into a database or other storage media.
The main_requests topic is used to issue maintenance requests, such as requests for machine tool maintenance or repair requests submitted by users. The user interface module receives the maintenance request from the user and issues it to the topic. The flank processing module subscribes to the topic and is responsible for processing and responding to these maintenance requests.
2. Data preprocessing: after receiving the data stream, the Flink processing module performs data preprocessing including data cleaning, format conversion, data aggregation and feature extraction. Based on the received JSON, CSV format data, the data is parsed into objects or tuples in the Flink data stream using a JackSon parser and Apache Commons CSV library. And according to the service requirement, using a Filter operator of the Flink to Filter the data stream. And screening out data meeting the requirements according to the specified rules, and filtering out invalid or abnormal data. And screening out characteristic fields with important significance and relevance according to the characteristics of the data and the prediction targets. The features are subjected to numerical conversion, normalization, discretization and aggregation operations for subsequent state analysis and prediction models.
Screening out data meeting the requirements according to the specified rules comprises the following steps:
data range filtering: data with numeric fields within a specific range, such as data with temperatures between 60 ℃ and 70 ℃, are screened.
Outlier rejection: and eliminating possible abnormal values in the data according to the common business knowledge, for example, regarding the spindle multiplying power data exceeding 200 as abnormal values and eliminating.
Missing value processing: and screening out the data without the missing value, and ensuring the data integrity. For example, if a field in the data is missing, the piece of data is culled.
And (3) frequency filtering: and screening out the events with the occurrence times exceeding a certain threshold value in a certain time window according to the occurrence frequency of a certain event.
Threshold value judgment: based on the threshold value of some index, data which exceeds or falls below the threshold value is screened out, for example, data of which the spindle vibration acceleration is higher than 2.5g is screened out.
Keyword matching: according to the specific keyword or the field content, data containing the specific keyword is selected, for example, data containing keywords such as "dev-" "iot _" "box-" are selected from the data.
And (3) data format verification: and screening out data meeting the specific format according to the data format rule, for example, verifying the warning information of the numerical control machine tool, and only retaining the data meeting the format requirement.
Time window screening: and screening out data in a specific time window according to the time stamp field, for example, screening out the running data of the numerical control machine tool in the last hour.
3. And (3) state analysis: structured query computation of data in HBase was performed by Apache Phoenix. After preprocessing, the Flink processing module performs state analysis on the data, and calculates various statistics, periodicity analysis, anomaly detection and the like. These analyses can help to understand the current state of the numerically controlled machine tool and detect potential anomalies. The results of the state analysis are used to monitor the health of the machine tool in real time and are provided to the predictive model as input features.
4. Training a state prediction model: the Flink processing module also includes a state prediction model training module. Using the historical data and current state characteristics, the Flink processing module may train a predictive model, such as a machine learning model or a time series analysis model. And analyzing and training a prediction model by using TensorFlow and Keras. And predicting possible faults or abnormal conditions by using a prediction model according to the current machine tool state, so that measures are taken in advance to maintain and prevent.
Wherein, the construction step of the prediction model:
data collection and preparation: historical data is collected, including machine tool state characteristics and corresponding fault or abnormal conditions. The data is cleaned, processed and feature extracted, and converted to csv format suitable for model training.
Dividing data: the collected data is divided into a training set and a test set. The training set is used for training the model, and the testing set is used for verifying the performance and generalization capability of the model.
Characteristic engineering: according to the characteristics of the problem, the data characteristics are further processed, including characteristic selection, conversion, standardization and the like. This may help the model better capture patterns and rules of the data.
Modeling a model: the real-time values of the sequence of the parameters corresponding to the machine tool state are independently analyzed, and the modeling thought is shown in fig. 6. Matrix of time length l (X n+1 ) m×l For a machine tool state matrix that needs prediction, (X) n ) m×l Is the current state matrix of the machine tool. From matrix X n Current sample time element x n Firstly, reversely moving the time window length l by a sliding unit w to obtain a state matrix general term represented by a kth time window, wherein the state matrix general term is as follows:
X k ＝[x(n-wk-l+1),x(n-wk-l+2),x(n-wk-l+3)...,x(n-wk)]
k=1, 2..represents the kth history state, w=1, 2..is the sliding interval length of the sliding time window. The window length l is greater than the sliding duration w.
Model training: the selected model is trained using a training set. The training process is to adjust parameters through a model so that the parameters can be better fit to actual conditions on training data.
The prediction method comprises the following steps:
pretreatment of data acquisition: the state characteristics of the current machine tool, including sensor data, operation records and the like, are collected in real time. The number of parameter types is m, n number of value points are obtained according to the sampling frequency of 1000ms, and all historical process data of the machine tool form a multi-dimensional time sequence matrix T with m rows and n columns m×n . The normalization process is carried out on each parameter mi by using the min-max normalization, and the normalization process is shown in a formula (1).
Multiple matching modeling: and (3) carrying out the same processing on the data acquired in real time according to a feature extraction method used in the model training stage to obtain feature vectors.The similarity analysis is used for solving a maximum similarity set, and the number of elements in the similarity set is used for solving an optimal sliding unit w and an optimal prediction duration l; performing cluster analysis on similar concentrated elements through a DBSCAN clustering algorithm to complete accurate matching of states, and determining a sequence matrix X which is most similar to the current state change trend in a historical state k And will then be of length lAs a final prediction result.
Model loading: and loading the trained prediction model into a Flink processing module.
Feature conversion: and performing feature conversion identical to the training data on the feature vectors acquired in real time, and ensuring that the input data accords with a data format expected by the model.
And (3) predicting: and predicting the feature vector by using the loaded prediction model.
Interpretation and processing of results: and judging whether the state of the current machine tool is likely to be faulty or abnormal according to the prediction result of the model. Based on the prediction, an alarm, a maintenance request, etc. may be triggered.
Continuously monitoring and updating: the steps are repeated regularly to monitor the state of the machine tool in real time and update the prediction model, so that the accuracy and the practicability of the model are ensured.
5. Outputting and alarming: the processing module sends the processed data and the prediction result to the data storage module and generates real-time alarm information. The alarm information can be displayed through the user interface module so that a user can timely acquire the change of the state of the machine tool and possible fault early warning.
And a data storage module: the module is responsible for storing the data processed by the Flink processing module for subsequent query and analysis. In order to meet different data storage requirements, the system adopts a relational database MySQL, a non-relational database HBase and an in-memory database Redis as a storage solution.
MySQL is a common relational database management system with stability, reliability and high efficiency. For some structured data, such as basic information of the machine tool, running logs, etc., mySQL is used for storage. When the database table structure is designed, corresponding tables, fields and indexes are created according to specific requirements and data characteristics so as to support efficient data insertion, query and update operations. In addition, proper data backup and recovery strategies are set, and data security is guaranteed.
HBase is an open-source distributed non-relational database with high expansibility and high availability. HBase is used for storage of some semi-structured or unstructured data, e.g. sensor data of a machine tool, real-time monitoring data, etc. The data model of HBase is based on a column family key value storage structure, and can flexibly store and inquire large-scale data. When the HBase data model is designed, proper column families and column qualifiers are defined according to the characteristics of different data so as to support efficient data writing and query operation. Meanwhile, proper data compression and data fragmentation strategies are set, and storage efficiency and query performance are improved.
In addition, the server caches the frequently accessed data in the memory database Redis, so that the access speed and the calculation efficiency are improved.
A user interface module: the module is responsible for displaying the processed data to a user in a visual form, so that the user can understand and operate conveniently. The user can check the real-time state of the machine tool and the early warning information through the module. The interface design of the module is simple and easy to use, and a user can quickly master the operation method.
Through the technical scheme, the invention realizes the real-time state monitoring of the digital machine tool, improves the accuracy and the real-time performance of the monitoring, has an early warning function, can help enterprises to prevent machine tool faults in advance, reduces the maintenance cost of equipment and improves the production efficiency.
The embodiment also provides a method for monitoring the real-time state of the numerical control machine tool based on the Flink, which comprises the following steps:
building a hardware environment: firstly, a hardware environment required by a real-time state monitoring system of a numerical control machine tool is built. The method comprises the steps of installing and configuring a numerical control machine tool acquisition gateway, connecting 8152C acoustic emission sensor, 8763b100bb accelerometer and other sensor equipment to the numerical control machine tool, and ensuring that the sensor equipment can work normally.
And (3) configuring a data acquisition module: and connecting a data interface of the 8152C acoustic emission sensor and the 8763b100bb accelerometer with a numerical control machine tool acquisition gateway, and carrying out corresponding configuration and calibration. The sensor equipment can accurately collect acoustic emission signals and acceleration data of the machine tool and send the acoustic emission signals and the acceleration data to the data collecting module. The acquisition gateway is connected with the numerical control machine tool, configures parameters of acquisition equipment and reads corresponding acquisition indexes of the numerical control system.
Data transmission and processing: and creating a Kafka message subscription topic, defining a topic name, the number of copies and the number of partitions to realize data load balancing, and improving the real-time processing efficiency. Data is sent from the data acquisition module to the Kafka message queue via the MQTT protocol. And receiving the data stream accessed from the Kafka by using a Flink processing module, and performing real-time data processing, preprocessing and feature extraction. And filtering and noise elimination are carried out on the acoustic emission signals by applying a signal processing algorithm, and characteristic values of vibration data are extracted. At the same time, the data is analyzed and predicted using a pre-trained state prediction model to identify the state of the machine tool and possible faults. And monitoring and analyzing the real-time data by using the trained state prediction model. And judging the current state of the numerical control machine tool, such as normal operation, abnormal state or possible fault state, according to the prediction result of the model. If an abnormality or failure is detected, the system will trigger a corresponding pre-warning mechanism, sending an alert notification to the operator or related maintenance personnel.
Data storage and management: and storing the data processed by the data transmission and processing module into a relational database MySQL and a non-relational database HBase. And proper data compression and fragmentation strategies are used, so that the storage efficiency and the query performance are improved. And the numerical control machine tool data of different types are respectively stored in the corresponding table structures, so that the subsequent data query and analysis are convenient.
User interface design: the user interface module is designed, and an intuitive and easy-to-use user interface is created by using a Web development technology. The interface provides functions of real-time machine tool state display, alarm information prompt, data visualization chart, user authority management and the like. Meanwhile, through interaction with a data interface at the rear end, monitoring, inquiring and configuring of the machine tool state by a user are realized.
System integration and deployment: and integrating all the modules to ensure the smoothness of the data flow and the cooperation of functions. And (5) performing system testing and debugging, and verifying the stability and performance of the whole system. And finally, delivering the deployed numerical control machine tool real-time state monitoring system to a user for use.
While the invention has been described with reference to preferred embodiments, it is not intended to be limiting. Those skilled in the art will appreciate that various modifications and adaptations can be made without departing from the spirit and scope of the present invention. Accordingly, the scope of the invention is defined by the appended claims.
Claims (9)
1. A numerical control machine tool real-time state monitoring system based on a Flink is characterized in that: comprising the following steps:
the data acquisition module acquires various parameters and state data of the numerical control machine tool and caches the parameters and the state data through a Kafka message caching mechanism, and the data acquisition module sends the parameters and the state data to the Flink processing module in a data stream mode, and the data acquisition module sends the parameters and the state data to a remote processing module through a wireless network;
the Flink processing module is used for receiving data from Kafka, cleaning the received data to remove noise and abnormal values, analyzing and preprocessing the cleaned data, calculating various statistics and identifying abnormal modes, and further comprises a state prediction model for predicting possible machine tool faults according to historical data and current data;
the data storage module is used for storing the data processed by the Flink processing module;
and the user interface module is used for displaying the processed data to a user in a visual form.
2. The intelligent real-time status monitoring system for a numerically-controlled machine tool based on a link according to claim 1, wherein the intelligent real-time status monitoring system is characterized in that: the data acquisition module comprises an acoustic emission sensor, an accelerometer and an acquisition gateway, wherein the acoustic emission sensor acquires acoustic emission signals, the accelerometer acquires vibration signals of a main shaft and a cutter of the numerical control machine tool, the acquisition gateway receives various parameters and state data of the acquired machine tool, the parameters and the state data are converted into a standard data format, the data are uploaded through the 4G/5G/WiFi network module, and the data are cached through a Kafka message caching mechanism.
3. The intelligent real-time status monitoring system for a numerically-controlled machine tool based on a link according to claim 1, wherein the intelligent real-time status monitoring system is characterized in that: the Flink processing module processes data, and comprises the following processes:
the method comprises the steps that a data stream is accessed, a Flink processing module is firstly connected to a Kafka message queue, corresponding topics are subscribed to obtain real-time data streams, corresponding topics and consumption groups are configured through a Kafka connector provided by the Flink, data are read from the Kafka and are converted into data streams of the Flink, and reliable access and distribution of the data are ensured;
the method comprises the steps of preprocessing data, after receiving a data stream, performing data preprocessing by a Flink processing module, including data cleaning, format conversion, data aggregation and feature extraction, analyzing the data into objects or tuples in the Flink data stream by using a Jackson analyzer and a Apache Commons CSV library according to received JSON and CSV format data, performing filtering operation on the data stream by using a Filter operator of the Flink according to service requirements, screening out data meeting the requirements according to specified rules, filtering out invalid or abnormal data, screening out feature fields with important significance and relevance according to the characteristics and prediction targets of the data, and performing numerical conversion, normalization, discretization and aggregation operation on the features for the use of subsequent state analysis and prediction models;
The method comprises the steps of performing state analysis, namely performing structured query calculation on data in HBase through Apache Phoenix, performing state analysis on the data by a Flink processing module after preprocessing, calculating various statistics, periodically analyzing and detecting abnormality to obtain the current state of the numerical control machine tool, detecting potential abnormal conditions, and using the result of the state analysis to monitor the health condition of the machine tool in real time and providing the result of the state analysis to a prediction model as an input characteristic;
the method comprises the steps of training a state prediction model, training the Flink processing module by using historical data and current state characteristics, obtaining a prediction model by training the Flink processing module, analyzing and training the prediction model by using TensorFlow and Keras, and predicting possible faults or abnormal conditions by using the prediction model according to the current machine tool state, so that measures are taken in advance to maintain and prevent the faults or abnormal conditions;
and the Flink processing module sends the processed data and the prediction result to the data storage module, and generates real-time alarm information which can be displayed through the user interface module so that a user can timely obtain the change of the machine tool state and possible fault early warning.
4. A link-based real-time status monitoring system for a numerically-controlled machine tool as set forth in claim 3, wherein: the construction of the prediction model comprises the following steps:
data collection and preparation: collecting historical data, including machine tool state characteristics and corresponding faults or abnormal conditions, cleaning, processing and extracting the data, and converting the data into a csv format suitable for model training;
dividing data: dividing the collected data into a training set and a testing set, wherein the training set is used for training a model, and the testing set is used for verifying the performance and generalization capability of the model;
characteristic engineering: according to the characteristics of the problems, further processing the data characteristics, including characteristic selection, conversion and standardization;
modeling a model: independently analyzing the sequence real-time value of the parameters corresponding to the machine tool state, and obtaining a matrix (X n+1 ) m×l For a machine tool state matrix that needs prediction, (X) n ) m×l For the current state matrix of the machine tool, the slave matrix X n Current sample time element x n Firstly, reversely moving the time window length l by a sliding unit w to obtain a state matrix general term represented by a kth time window, wherein the state matrix general term is as follows:
X k ＝[x(n-wk-l+1),x(n-wk-l+2),x(n-wk-l+3)...,x(n-wk)]
k=1, 2..represents the kth history state, w=1, 2..is the sliding interval length of the sliding time window, the window length l is greater than the sliding time length w;
Model training: the selected model is trained by using a training set, and parameters are adjusted through the model in the training process, so that the training set can better fit actual conditions on training data.
5. A link-based real-time status monitoring system for a numerically-controlled machine tool as set forth in claim 3, wherein: the prediction method comprises the following steps:
pretreatment of data acquisition: the method comprises the steps of collecting the state characteristics of a current machine tool in real time, wherein the state characteristics comprise sensor data and operation records, the parameters are m in types, n number of value points are obtained according to the sampling frequency of 1000ms, and all historical process data of the machine tool form a multi-dimensional time sequence matrix T with m rows and n columns m×n The normalization processing is carried out on each parameter mi by using the min-max normalization, the formula (1),
multiple matching modeling: according to a feature extraction method used in a model training stage, carrying out the same processing on data acquired in real time to obtain feature vectors, obtaining a maximum similarity set through similarity analysis, and obtaining an optimal sliding unit w and an optimal prediction duration l by using the number of elements in the similarity set; performing cluster analysis on similar concentrated elements through a DBSCAN clustering algorithm to complete accurate matching of states, and determining a sequence matrix X which is most similar to the current state change trend in a historical state k And X with length l is taken as follows k * As the most important partFinal prediction results;
model loading: loading the trained prediction model into a Flink processing module;
feature conversion: performing feature conversion identical to the training data on the feature vectors acquired in real time, and ensuring that the input data accords with a data format expected by the model;
and (3) predicting: predicting the feature vector by using the loaded prediction model;
interpretation and processing of results: judging whether the state of the current machine tool is likely to be faulty or abnormal according to the prediction result of the model, and triggering operations including alarm and maintenance request according to the prediction result;
continuously monitoring and updating: the steps are repeated regularly to monitor the state of the machine tool in real time and update the prediction model, so that the accuracy and the practicability of the model are ensured.
6. A link-based real-time status monitoring system for a numerically-controlled machine tool as set forth in claim 3, wherein: the subject matter of Kafka includes:
the machine_data theme is used for collecting and transmitting original data of the machine tool, the original data comprise sensor data and accelerometer data, a publishing end of the theme is a data acquisition module, the machine tool data is published to the theme in a JSON format, and a subscribing end is a link processing module which is responsible for processing and converting the data;
The alarm event issuing system comprises an alarm event issuing module, a feedback processing module, a data storage module and a data storage module, wherein the alarm event issuing module is used for issuing an alarm event of a machine tool, including faults and anomalies, the alarm event issuing module judges the working state of the current machine tool according to alarm information and then issues the alarm event to the alarm event issuing module, and the subscription end is responsible for storing the alarm event to a database or other storage media;
the status_updates theme is used for publishing updated information of the machine tool state, including the running state and the production progress of the machine tool, the Flink processing module performs state analysis and processing according to the collected machine tool data, and publishes the updated state information to the theme, and the subscribing end is a user interface module and is used for displaying and monitoring the state of the machine tool in real time;
the prediction_results theme is used for issuing machine tool state prediction results, including prediction and evaluation of future states of the machine tool, the state analysis and processing can train and predict historical data by using a machine learning algorithm, the prediction results are issued to the theme, and the subscribing end is a data storage module and is responsible for storing the prediction results in a database or other storage media;
the main_requests theme is used for issuing maintenance requests, including the requirements or repair requests of machine tool maintenance submitted by users, the user interface module receives the maintenance requests of the users and issues the maintenance requests to the theme, and the Flink processing module subscribes to the theme and is responsible for processing and responding to the maintenance requests.
7. A method for monitoring the real-time state of a numerical control machine tool based on a Flink is characterized by comprising the following steps: the method comprises the following steps:
the method comprises the steps that various parameters and state data of a numerical control machine tool are collected through a data collection module, and are cached through a Kafka message caching mechanism, the data collection module sends the parameters and the state data to a Flink processing module in a data stream mode, and the data collection module sends the parameters and the state data to a remote processing module through a wireless network;
the method comprises the steps of receiving data from Kafka through a Flink processing module, cleaning the received data to remove noise and abnormal values, analyzing and preprocessing the cleaned data, calculating various statistics, identifying abnormal modes, analyzing and predicting the data by using a pre-trained state prediction model to identify the state and faults of a machine tool, monitoring and analyzing real-time data by using the trained state prediction model, judging the current state of the numerical control machine tool according to the prediction result of the state prediction model, including normal operation, abnormal state or possible fault state, and if abnormality or fault is detected, triggering a corresponding early warning mechanism by a system, and sending an alarm notification to an operator or related maintenance personnel;
The data storage module is used for storing the data processed by the Flink processing module;
and the user interface module is used for displaying the processed data to a user in a visual form.
8. An electronic device, characterized in that: comprising a processor, a memory and a computer program stored in the memory, which computer program is configured to perform the method of claim 7 when run by the processor.
9. A computer-readable storage medium, characterized by: on which a computer program is stored for performing the method of claim 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311033378.7A CN117215258A (en) | 2023-08-16 | 2023-08-16 | Numerical control machine tool real-time state monitoring system and method based on Flink |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311033378.7A CN117215258A (en) | 2023-08-16 | 2023-08-16 | Numerical control machine tool real-time state monitoring system and method based on Flink |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117215258A true CN117215258A (en) | 2023-12-12 |
Family
ID=89045217
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311033378.7A Pending CN117215258A (en) | 2023-08-16 | 2023-08-16 | Numerical control machine tool real-time state monitoring system and method based on Flink |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117215258A (en) |
-
2023
- 2023-08-16 CN CN202311033378.7A patent/CN117215258A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110347116B (en) | Machine tool state monitoring system and monitoring method based on operation data flow | |
| CN110456774B (en) | Fault diagnosis and early warning device and method for fast freight locomotive | |
| CN107577588B (en) | Intelligent operation and maintenance system for mass log data | |
| CN114282434A (en) | Industrial equipment health management system and method | |
| JP2019108117A (en) | Real time streaming analytics for flight data processing | |
| CN110806743A (en) | Equipment fault detection and early warning system and method based on artificial intelligence | |
| CN110728443A (en) | Motor full life cycle management and control system | |
| US20160171037A1 (en) | Model change boundary on time series data | |
| JP2018128855A (en) | Event analysis device, event analysis system, event analysis method, event analysis program, and recording medium | |
| KR20180108446A (en) | System and method for management of ict infra | |
| CN108229538B (en) | Vehicle system prediction apparatus and method | |
| WO2001050210A1 (en) | A method and system for sorting incident log data from a plurality of machines | |
| US10810225B2 (en) | System and method for large scale data processing of source data | |
| EP3553044A1 (en) | System and method of remote object monitoring | |
| CN115186883A (en) | Industrial equipment health state monitoring system and method based on Bian Yun collaborative computing | |
| CN112428872A (en) | Vehicle battery management system, method, storage medium, and server system | |
| CN113196311A (en) | System and method for identifying and predicting abnormal sensing behavior patterns of a machine | |
| CN117194919A (en) | Production data analysis system | |
| CN106951360B (en) | Data statistical integrity calculation method and system | |
| CN113697424B (en) | Belt conveyor monitoring and fault diagnosis system and method based on cloud technology | |
| CN116755992B (en) | Log analysis method and system based on OpenStack cloud computing | |
| CN117392824A (en) | Steel structure monitoring and early warning platform of offshore wind power booster station | |
| CN117172751A (en) | Construction method of intelligent operation and maintenance information analysis model | |
| CN117215258A (en) | Numerical control machine tool real-time state monitoring system and method based on Flink | |
| CN110794799A (en) | Big data system with fault diagnosis function applied to industrial production |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |