CN117390507A - Spacecraft telemetry data detection system based on deep learning - Google Patents
Spacecraft telemetry data detection system based on deep learning Download PDFInfo
- Publication number
- CN117390507A CN117390507A CN202311234601.4A CN202311234601A CN117390507A CN 117390507 A CN117390507 A CN 117390507A CN 202311234601 A CN202311234601 A CN 202311234601A CN 117390507 A CN117390507 A CN 117390507A
- Authority
- CN
- China
- Prior art keywords
- data
- algorithm
- module
- task
- telemetry
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 79
- 238000013135 deep learning Methods 0.000 title claims abstract description 18
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 163
- 238000012545 processing Methods 0.000 claims abstract description 45
- 238000007781 pre-processing Methods 0.000 claims abstract description 21
- 238000004458 analytical method Methods 0.000 claims abstract description 14
- 238000013079 data visualisation Methods 0.000 claims abstract description 13
- 238000000034 method Methods 0.000 claims description 48
- 230000006870 function Effects 0.000 claims description 20
- 238000000605 extraction Methods 0.000 claims description 16
- 230000007246 mechanism Effects 0.000 claims description 15
- 230000002159 abnormal effect Effects 0.000 claims description 14
- 238000007619 statistical method Methods 0.000 claims description 14
- 238000012549 training Methods 0.000 claims description 14
- 239000012634 fragment Substances 0.000 claims description 13
- 230000000007 visual effect Effects 0.000 claims description 11
- 230000008569 process Effects 0.000 claims description 10
- 230000011218 segmentation Effects 0.000 claims description 9
- 238000010219 correlation analysis Methods 0.000 claims description 5
- 239000000203 mixture Substances 0.000 claims description 5
- 230000002547 anomalous effect Effects 0.000 claims description 4
- 238000009472 formulation Methods 0.000 claims description 4
- 238000012847 principal component analysis method Methods 0.000 claims description 4
- 230000001502 supplementing effect Effects 0.000 claims description 4
- 238000010998 test method Methods 0.000 claims description 4
- 230000004913 activation Effects 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 2
- 230000005856 abnormality Effects 0.000 abstract description 11
- 230000003993 interaction Effects 0.000 abstract description 3
- 238000005259 measurement Methods 0.000 abstract description 2
- 230000000875 corresponding effect Effects 0.000 description 16
- 238000007405 data analysis Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000007418 data mining Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000004451 qualitative analysis Methods 0.000 description 2
- 238000004445 quantitative analysis Methods 0.000 description 2
- 238000001228 spectrum Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000001276 controlling effect Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000036316 preload Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 238000000611 regression analysis Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/10—Pre-processing; Data cleansing
- G06F18/15—Statistical pre-processing, e.g. techniques for normalisation or restoring missing data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
- G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Molecular Biology (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Navigation (AREA)
Abstract
The invention provides a spacecraft telemetry data detection system based on deep learning, which relates to the telemetry technical field, and comprises a data acquisition and preprocessing module, a remote sensing data processing module, an algorithm management module, a task customization module and a data visualization module. The system can enrich the abnormality detection means, solve the problem of the prolongation of the time sequence data instruction of the spacecraft, realize the statistics and analysis of data according to the needs of users, personally define algorithms and algorithm flows, promote online high-performance real-time interaction and realize the high-efficiency remote measurement data real-time abnormality detection.
Description
Technical Field
The invention relates to the technical field of telemetry, in particular to a spacecraft telemetry data detection system based on deep learning.
Background
With the gradual development of the technology level, the requirements and functions of the spacecraft are more and more complex, and more complex correlation between telemetry data also appears. The spacecraft data anomaly detection device is an important means for guaranteeing the operation safety and maintenance flight stability of the spacecraft. Spacecraft can generate a lot of data when performing tasks, including parameters of temperature, pressure, speed, attitude and the like. The change and the abnormality of the parameters can possibly influence the running state of the spacecraft, so that a set of remote measurement data of the spacecraft can be monitored and analyzed in real time, abnormal conditions can be found in time, and a system for guaranteeing the safe running of the spacecraft has important significance for analysis-related research of the abnormal data of the spacecraft. The telemetry data anomaly detection method in the prior art comprises the following steps:
1. the anomaly detection method based on manual interpretation and threshold detection is a traditional spacecraft anomaly detection means, and is realized by means of real-time observation of spacecraft telemetry data by ground technicians, setting of threshold values at key nodes and investigation through physical quantities and other indexes. Although the method has the advantages of easy implementation in practice, because the manual workload is too great and the experience and technical requirements on technicians are high, and some special abnormal situations which do not occur cannot be detected simply by manual work, many abnormal situations cannot be found through simple threshold judgment;
2. the spacecraft anomaly detection method based on the rules is based on the original expert experience, and an expert system for enhancing the anomaly detection effect on a spacecraft control system is designed. The method can convert the corresponding rule into a program language through controlling the spacecraft operation rule, and can predict and check the spacecraft operation state through forward or backward reasoning construction. But requires sufficient a priori knowledge if a sufficiently powerful rule-based expert system is to be built. Meanwhile, when the system is changed, the old expert system also needs to be updated in time to keep up with the change of the spacecraft, so that the method has poor self-adaptability;
3. the spacecraft anomaly detection method based on the model is a method for automatically detecting anomalies by modeling internal information, structures and behaviors of a spacecraft system. The model-based method is mainly divided into three types of qualitative analysis, quantitative analysis and analytical analysis. The method overcomes the difficulties of high labor cost, high prior knowledge requirement and the like in the prior art due to the automatic method. However, for qualitative analysis models, such models are relatively simple and cannot be used to describe the system dynamics; if the quantitative analysis process is to realize more accurate simulation, the calculation process has too great requirement on mathematical calculation amount; for the analytical model, the method has the defects of higher implementation threshold and difficulty of some specific methods;
4. the telemetry data mining and anomaly detection method based on machine learning is a method which jumps out of the original detection method based on professional field knowledge and experience and uses a detection means based on data instead at the present time of data mining and artificial intelligence technology development. While the machine learning based approach overcomes the previous problems, it is limited by the algorithm itself, which still fails to perform context-dependent analysis of long-time series of spacecraft data, and thus performs poorly in some collective anomalies and context anomalies in long-time series.
Therefore, the methods based on machine learning and statistical learning in the prior art have not yet been well adapted to exhibit more desirable effects and practicality in long time series and high dimensional data. On the basis of data driving, developing a method integrated system capable of automatically learning the characteristics and rules of data and processing high-dimensional, nonlinear and complex telemetry data is a problem to be solved.
Disclosure of Invention
In view of the above, the embodiment of the invention provides a spacecraft telemetry data detection system based on deep learning to solve the above technical problems. The system comprises:
the data acquisition and preprocessing module is used for acquiring telemetry data and preprocessing the telemetry data in a wild picking, slicing, time synchronization and standardization way;
the remote sensing data processing module is used for carrying out integrated data statistical analysis, feature extraction and anomaly detection on the preprocessed remote sensing data and carrying out visual display on a data processing result;
the algorithm management module is used for uploading, modifying, deleting and configuring the uploading interface for each data processing algorithm in the remote sensing data processing module;
the task customization module is used for customizing the flow of the operation task according to the algorithm and the data in the data acquisition and preprocessing module and the remote sensing data processing module, so as to realize the flow operation;
and the data visualization module is used for realizing data visualization display of data processing results of the data acquisition and preprocessing module, the remote sensing data processing module, the algorithm management module and the task customization module.
Further, the data acquisition and preprocessing module comprises a wild rejection algorithm, a segmentation algorithm, a time synchronization algorithm and a data standardization algorithm.
Further, the remote sensing data processing module comprises a statistical analysis algorithm, a feature extraction algorithm and an anomaly detection algorithm.
Further, the feature extraction algorithm includes a correlation analysis method, a moving average algorithm, a stationarity test method, an EMD trend analysis method, and a principal component analysis method.
Further, the anomaly detection algorithm is a TPA-LSTM method that uses an improved version of the attention mechanism on the basis of the LSTM network.
Furthermore, the system can also realize offline task scheduling, and the method for realizing the offline task scheduling comprises the following steps:
step one, a user enters a flow chart of drawing execution tasks of an offline control page provided by the system, a page is formulated through a front-end visual flow provided by the data visualization module, an algorithm needing to be customized and required data are selected, parameters of each algorithm are designated, a task flow is formulated through the front-end visual flow in a dragging mode, the flow chart is stored after the task formulation is finished, and the task flow can be executed on an offline panel or an online panel according to the flow chart;
analyzing the key path of the flow chart by the task customization module through a topological ordering method;
and thirdly, asynchronously executing the user-defined flow chart by the system, adding the flow chart tasks into a thread pool for concurrent execution, feeding back that the tasks are started by the user, and acquiring a task execution result by clicking a view button by the user.
Further, the system obtains the telemetry data from an external database or Redis or bus.
Furthermore, the system can also realize high concurrency online task scheduling, the high concurrency online task scheduling is completed through a data request thread, and the realization method of the high concurrency online task scheduling comprises the following steps:
carrying out algorithm thread registration through the data request thread to obtain a data source, a data interface and a callback function of a required algorithm;
the data request thread adds the data interface of the required algorithm into a waiting queue, when the data interface has data access, the data interface is transferred from the waiting queue to a linked list, and then the interface in the linked list is polled;
and when new data are generated on the data interface, calling a callback function corresponding to the data interface, executing an algorithm thread corresponding to the required algorithm, and completing online task scheduling of the required algorithm, wherein a single thread is adopted to process data access, a corresponding algorithm module is dormant when no data is accessed, and the corresponding algorithm thread is awakened in a callback mode after the data is accessed.
Further, the anomaly detection algorithm includes an instruction classification module and a data anomaly detection module, and the implementation flow of the anomaly detection algorithm includes:
classifying the occurrence position and the occurrence range of abnormal data in the telemetry data and supplementing instructions to obtain marked instruction-telemetry time sequence data;
carrying out data processing on marked fragments of the instruction-telemetry data, processing all fragments into subsequences with the length of w in a sliding window segmentation mode, and carrying out sliding window slicing with different step sizes on each fragment;
training a classifier structure, obtaining the sliding window slice with the length w of the telemetry data to be detected according to the classifier after training, classifying the sliding window slice, and marking the output value of the classification result below the corresponding time dimension;
obtaining the output values of a plurality of classification results in each time step after prediction is finished, selecting the maximum output value in each category to carry out category voting, and carrying out instruction continuation by adopting a classification strategy;
after instruction continuation is finished, obtaining n-dimensional instruction-telemetry data, inputting the n-dimensional instruction-telemetry data into a training model of the anomaly detection module for prediction, wherein a prediction result is a result of whether the instruction-telemetry data is anomalous or not.
Further, the instruction classification module comprises a classification network and a decision module, wherein the classification network is a ResNet network, each substructure of the ResNet network comprises 3 layers of one-dimensional convolutional networks with the sizes of 8, 5 and 3 respectively, and each layer of convolutional network is followed by a BN layer and an activation function ReLU; the inputs and outputs of each substructure of the ResNet network are summed; the instruction classification module classifies by softmax.
Compared with the prior art, the beneficial effects that above-mentioned at least one technical scheme that this description embodiment adopted can reach include at least: the invention provides a spacecraft telemetry data detection system based on deep learning, which can enrich anomaly detection means, solve the problem of spacecraft time sequence data instruction prolongation, realize statistical analysis of data according to requirements by users, personally define algorithms and algorithm flows and promote online high-performance real-time interaction.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of an algorithm for implementing offline task scheduling by a system provided by an embodiment of the present invention;
fig. 2 is an algorithm schematic diagram of a system for implementing online task real-time scheduling according to an embodiment of the present invention.
Detailed Description
Embodiments of the present application are described in detail below with reference to the accompanying drawings.
Other advantages and effects of the present application will become apparent to those skilled in the art from the present disclosure, when the following description of the embodiments is taken in conjunction with the accompanying drawings. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the present application. The present application may be embodied or carried out in other specific embodiments, and the details of the present application may be modified or changed from various points of view and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments may be combined with each other without conflict. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.
The invention provides a spacecraft telemetry data detection system based on deep learning, which can enrich anomaly detection means, solve the problem of spacecraft time sequence data instruction prolongation, realize statistical analysis of data according to requirements by users, personally define algorithms and algorithm flows and promote online high-performance real-time interaction.
In an embodiment of the present invention, as shown in fig. 1, a spacecraft telemetry data detection system based on deep learning includes: the data acquisition and preprocessing module is used for acquiring telemetry data and preprocessing the telemetry data in a wild picking, slicing, time synchronization and standardization way; the remote sensing data processing module is used for carrying out integrated data statistical analysis, feature extraction and anomaly detection on the preprocessed remote sensing data and carrying out visual display on a data processing result; the algorithm management module is used for uploading, modifying, deleting and configuring the uploading interface for each data processing algorithm in the remote sensing data processing module; the task customization module is used for customizing the flow of the operation task according to the algorithm and the data in the data acquisition and preprocessing module and the remote sensing data processing module, so as to realize the flow operation; and the data visualization module is used for realizing data visualization display of data processing results of the data acquisition and preprocessing module, the remote sensing data processing module, the algorithm management module and the task customization module.
Specifically, the system is oriented to a real scene, integrates various processing methods applied to time sequence data of the spacecraft, and provides functions of uploading and customizing task flows for users. The spacecraft anomaly detection system is used as a platform for algorithm implementation and is used for testing various algorithms, and support is provided for spacecraft time sequence data analysis and anomaly detection. The whole system is composed of five functional modules, including a data acquisition and preprocessing module, a remote sensing data processing module, an algorithm management module, a task customization module and a data visualization module.
Further, the data acquisition and preprocessing module realizes operations of field picking, segmentation, time synchronization and standardization aiming at the data and supports the inquiry of processing analysis results; the remote sensing data processing module realizes the realization and processing of a plurality of integrated data statistical analysis, feature extraction and anomaly detection algorithms, and the result is visually displayed through the data visualization module; the algorithm management module provides a user-defined uploading algorithm, modifies the algorithm, deletes the algorithm, configures an algorithm uploading interface, and realizes a user-friendly integrated algorithm calling interface; the task customization module provides a visual drag window, various algorithms and various data can be used for customizing task flows, and the customized flows can realize flow operation; the data visualization module realizes the front end display of the functions of data acquisition and preprocessing, basic algorithm processing, algorithm management and task formulation through the visualization page.
Further, the data acquisition and preprocessing module comprises a wild rejection algorithm, a segmentation algorithm, a time synchronization algorithm and a data standardization algorithm.
Further, the system obtains the telemetry data from an external database or Redis or bus.
Specifically, the data acquisition and preprocessing module performs processes such as cleaning, conversion, integration, normalization and the like on the original data before data analysis or modeling. The data preprocessing algorithm in the module mainly comprises a data wild rejection algorithm, a data segmentation algorithm, a data time synchronization algorithm and a data standardization algorithm. In the aspect of specific processing of spacecraft data, the system respectively applies a data outlier rejection algorithm which can reject some outliers or abnormal values in original data and does not cause the outliers to excessively influence the analysis and modeling of the follow-up algorithm when the follow-up algorithm is carried out, a data segmentation algorithm which can analyze data for different time periods and a time synchronization algorithm which can process data time by interpolating or deleting some existing values.
Further, the remote sensing data processing module comprises a statistical analysis algorithm, a feature extraction algorithm and an anomaly detection algorithm.
In particular, statistical analysis algorithm modules are a class of algorithms that provide decision support and predictive capabilities by analyzing and interpreting the collected data to discover patterns, trends, associations, and laws in the data. The system-integrated statistical analysis algorithm comprises a minimum value, a maximum value, a probability, a mean value, a median value, a variance, a standard deviation, a mean square, a root mean square, a peak value, a peak factor, a skewness, a kurtosis, a margin, a waveform factor, a regression analysis, a weighted moving average, a center of gravity frequency, a power spectrum variance, a mean square frequency and a spectrum entropy.
Further, the feature extraction algorithm includes a correlation analysis method, a moving average algorithm, a stationarity test method, an EMD trend analysis method, and a principal component analysis method.
Specifically, the feature extraction algorithm analyzes the spacecraft time sequence to extract some feature information about the time sequence, such as period, trend change, correlation of different time sequences and the like. Aiming at the analysis requirement of spacecraft data, the integrated feature extraction algorithm comprises the following steps: correlation analysis method, moving average algorithm, stationarity test method, EMD trend analysis method and principal component analysis method. The correlation analysis method is mainly used for measuring whether correlation exists between different variables; the moving average method is used for eliminating noise, reducing data fluctuation and identifying trend; the stationarity test refers to a statistical method for checking time series data to determine whether the stationarity is satisfied.
Further, the anomaly detection algorithm is a TPA-LSTM method that uses an improved version of the attention mechanism on the basis of the LSTM network.
Specifically, the anomaly detection algorithm is a TPA-LSTM method designed for multi-dimensional spacecraft telemetry data that is strongly correlated to instructions, which uses an improved version of the attention mechanism on the basis of the LSTM network, changes the attention mechanism from weighting scoring of one time step to scoring of each dimension at each time step, and simultaneously uses lateral convolution within one data dimension to extract timing features, such a combination of methods greatly improves the performance of the attention mechanism on multi-dimensional timing data.
Furthermore, the system can also realize offline task scheduling, and the method for realizing the offline task scheduling comprises the following steps:
step one, a user enters a flow chart of drawing execution tasks of an offline control page provided by the system, a page is formulated through a front-end visual flow provided by the data visualization module, an algorithm needing to be customized and required data are selected, parameters of each algorithm are designated, a task flow is formulated through the front-end visual flow in a dragging mode, the flow chart is stored after the task formulation is finished, and the task flow can be executed on an offline panel or an online panel according to the flow chart;
specifically, the user enters the offline control page shown in fig. 1 through the function provided by the system and capable of designating the task flow by the user, draws the flow chart for executing the task, and the function can realize serial or parallel operation of a plurality of algorithms. The user sets up the page through the visual flow of front end that this module provides, choose algorithm and data packet/data source that need to go on and need, appoint the parameter of each algorithm, arrange the task flow through the way of dragging, save this flow chart after finishing, can carry out this task flow on the off-line/on-line panel finally.
Analyzing the key path of the flow chart by the task customization module through a topological ordering method;
specifically, the system analyzes the critical path of the flow chart by a topological ordering method, and the analysis is completed in a flow chart analysis module. The topology ordering algorithm is an algorithm that orders the directed acyclic graph (Directed Acyclic Graph, DAG) that orders all nodes in the flow chart in a certain order. In the topology ordering algorithm, if a ring exists in the graph, the node with the ring can never become a node with an ingress of 0, and therefore can never traverse. The task flow graph should therefore be a directed acyclic graph to execute, so the algorithm flow graph needs to satisfy the topological ordering to check the rationality of the algorithm flow.
And thirdly, asynchronously executing the user-defined flow chart by the system, adding the flow chart tasks into a thread pool for concurrent execution, feeding back that the tasks are started by the user, and acquiring a task execution result by clicking a view button by the user.
In particular, the system will execute the user-defined algorithm flow diagram asynchronously in order to make the system's performance better. After the user starts the task, the background adds the task of the flow chart into the thread pool for concurrent execution, the user can obtain feedback that the task has started, and when the user wants to acquire the result, the user can click a view button to directly view the result which has been calculated. All the processes of the system off-line processing algorithm are completed.
Specifically, as shown in fig. 1, in order to cope with the log generation requirements of web application programs in different service scenes, the invention encodes the functional composition of each layer of the application program based on the software architecture modeling thought and adopts an automatic mapping mode to replace log information defined in an application system. User information corresponding to the corresponding operation behavior is intercepted, log content is generated together with other necessary system description information, and the log content is stored in the form of log files or database files. Meanwhile, the collection work of the log information can be started or stopped at will in a mode of modifying the configuration file, and the existing log information can be expanded or deleted according to real-time requirements.
In the task flow function, the real-time algorithm operation is an important component, the partial algorithm is mainly oriented to spacecraft real-time telemetry, real-time control of researchers on the spacecraft operation state is met by carrying out statistical analysis, feature extraction and anomaly detection on real-time data, and manual intervention can be timely carried out when key nodes or anomalies occur, so that accidents are avoided.
Further, as shown in fig. 2, the system can also implement high concurrency online task scheduling, where the high concurrency online task scheduling is completed by a data request thread, and the implementation method of the high concurrency online task scheduling includes:
step one, carrying out algorithm thread registration through the data request thread to obtain a data source, a data interface and a callback function of a required algorithm;
step two, the data request thread adds the data interface of the required algorithm into a waiting queue, when the data interface has data access, the data interface is transferred from the waiting queue to a linked list, and then the interfaces in the linked list are polled;
specifically, in the aspect of data pulling of spacecraft data real-time monitoring, the system supports three modes of reading from an external database, redis and a bus. When the result is requested, the result is preferentially read from the Redis, so that the hard disk reading consumption is reduced, the Redis is used as a perpetual method, and the perpetual operation is carried out in idle time to store the operation result. When the real-time algorithm starts, firstly, registration is carried out in the data request module, and the registration is mainly completed through a data request thread of a single instance mode statement so as to obtain a data source, a data interface and a callback function which indicate the use of the data source, the data interface and the callback function. After the execution is started, the data request thread adds the interface to be queried into the waiting queue, when a certain data interface has data access, the interface is transferred from the waiting queue to the linked list, then the interface in the linked list is polled, when new data is generated, a callback function corresponding to the interface is called, and the algorithm thread is executed.
And thirdly, when new data are generated on the data interface, calling a callback function corresponding to the data interface, executing an algorithm thread corresponding to the required algorithm, and completing online task scheduling of the required algorithm, wherein a single thread is adopted to process data access, a corresponding algorithm module is dormant when no data is accessed, and the corresponding algorithm thread is awakened in a callback mode after the data is accessed.
Specifically, in order to meet the real-time requirement, and the online architecture needs to have good enough concurrency and running efficiency, the online module part of the system refers to the implementation of the IO multiplexing mechanism epoll. The method has the advantages that a single thread (data request thread) is adopted to process data access, an algorithm module is dormant when no data access exists, and a corresponding algorithm thread is awakened in a callback mode after the data access, so that CPU consumption caused by thread switching is reduced as much as possible. In order to cope with the possibly facing situation of the complexity of the online system, online data is not inputted into the algorithm at one time, and it is necessary to provide the online algorithm with data stepwise according to the lapse of time. And decoupling the data interface and the algorithm interface between the data request thread and the concurrently working algorithm execution thread, and establishing a connection between the data interface and the algorithm interface through a data channel. For the algorithm interface, it is required to ensure that the algorithm is preloaded into the memory and kept in a standby state, and the algorithm is executed when the data channel transmits online data, so that the online interface is divided into a pre-start interface and an algorithm execution interface. The pre-boot interface will pre-load the content needed by the algorithm into the memory, such as the computation graph of the neural network, and then the algorithm execution interface waits for the information of the data channel to wake up the algorithm and execute a computation. The arrangement can ensure that the online algorithm does not waste time in the action of repeatedly loading the content into the memory, thereby greatly improving the response speed of the online system.
Further, the anomaly detection algorithm includes an instruction classification module and a data anomaly detection module, and the implementation flow of the anomaly detection algorithm includes:
classifying the occurrence positions and the occurrence ranges of abnormal data in the telemetry data and supplementing commands to obtain marked command-telemetry time sequence data;
specifically, the telemetry data of each index of the spacecraft processed by the system is characterized in that only the position and the range of occurrence of the abnormality are marked in the long time sequence data, so that the abnormality detection function of the model is judged, but the action length of instruction information is not marked. Based on the nature of this instruction-telemetry data, the data requires feature processing and continuation of the instruction prior to anomaly detection. The continuation of the instruction is realized through a multi-classification model, so the anomaly detection algorithm is subdivided into an instruction classification module and a data anomaly detection module.
Further, the instruction classification module comprises a classification network and a decision module, wherein the classification network is a ResNet network, each substructure of the ResNet network comprises 3 layers of one-dimensional convolutional networks with the sizes of 8, 5 and 3 respectively, and each layer of convolutional network is followed by a BN layer and an activation function ReLU; the inputs and outputs of each substructure of the ResNet network are summed; the instruction classification module classifies by softmax.
The data anomaly detection module is an LSTM network loaded with a transverse attention mechanism and used for carrying out transverse convolution to extract time sequence characteristics, and the input of the module is instruction-telemetry time sequence data of the classifying and supplementing instruction at the last stage. Considering the data specificity of the system application, our attention mechanism will not only link the instructions related to telemetry data, but also link the unrelated instructions together, because the information in one time step is not separable, thus the weighting mode needs to be changed, and attention is paid to the information in one time dimension.
Step two, carrying out data processing on marked fragments of the instruction-telemetry data, processing all fragments into subsequences with the length of w in a sliding window segmentation mode, and carrying out sliding window slicing with different step sizes on each fragment;
specifically, data processing is performed on marked command-telemetry data fragments, all fragments are processed into subsequences with length w in a sliding window segmentation mode, because marked command-telemetry data fragments are unequal in length, if the same time window moving step is adopted, different types of telemetry data in a final result are unbalanced, in order to balance the quantity of each type of training data in a final training set, asynchronous long sliding window slicing is performed on each fragment, and the fact that the original data of each type of sliding window slices are equal in quantity is guaranteed.
Training a classifier structure, obtaining the sliding window slice with the length w of the telemetry data to be detected according to the classifier after training, classifying the sliding window slice, and marking the output value of the classification result below the corresponding time dimension;
specifically, training of the classifier structure based on the ResNet classification network is completed, and after the classifier model is trained, the classifier model can be used for classifying later. The classifier selects a sliding window slice of w length for telemetry data to be measured, classifies the slice, and marks the classified softmax output value below each time dimension. After the prediction is finished, an equal number of softmax output values can be obtained under each time step (except the time steps with the length of the head and the tail w), the maximum value under each category is selected for category voting, and then a classification strategy is adopted for instruction continuation.
Step four, obtaining the output values of a plurality of classification results in each time step after the prediction is finished, selecting the maximum output value in each category to carry out category voting, and carrying out instruction continuation by adopting a classification strategy;
and fifthly, after instruction continuation is finished, acquiring n-dimensional instruction-telemetry data, and inputting the n-dimensional instruction-telemetry data into a training model of the anomaly detection module for prediction, wherein a prediction result is a result of whether the instruction-telemetry data is anomalous or not.
Specifically, after instruction continuation is completed, n-dimensional instruction-telemetry data is obtained. And in the next step, the model for abnormality detection needs to be unfolded and trained, and the training model for abnormality detection adopts two models, namely TPA-LSTM and TranAD, which use an attention mechanism as a model for modeling and guiding a time sequence mode. And finally, after training the model, performing model test by using telemetry data with a certain proportion of anomalies, and inputting the data into the model for prediction after instruction prolongation. And judging whether the real data exceeds a reasonable boundary of the predicted value or not through the specified threshold value, so as to judge whether the real data is abnormal or not.
The system integrates common algorithms of tasks such as data analysis, feature extraction and anomaly detection, supports definition of algorithm flows and personalized task arrangement, and simultaneously realizes high-performance online real-time detection requirements through an epoll mechanism of asynchronous IO communication. In the aspect of specific multidimensional telemetry data anomaly detection with strong correlation to instructions, the detection method is realized that the time sequence data characteristics are extracted through an LSTM network and a convolution means, and whether the data is anomalous is detected by calculating the correlation degree through an attention mechanism.
In summary, the following technical effects are achieved by the embodiment of the invention:
1. after the characteristics of the telemetry data of the spacecraft and the requirement for carrying out anomaly detection on the data are analyzed, various algorithms including a data processing algorithm, a statistical analysis algorithm, a feature extraction algorithm and an anomaly detection algorithm are realized and integrated, so that the requirements for analyzing and processing the telemetry data of the spacecraft are fully met;
2. the system provided by the invention integrates the common algorithm of spacecraft data analysis and simultaneously provides an algorithm uploading interface and an algorithm flow making interface. The user can realize tasks such as spacecraft data statistics analysis, feature extraction and anomaly detection according to own requirements. Meanwhile, the function of arranging personalized tasks through an algorithm uploading interface and an algorithm flow making interface is provided for a user conveniently;
3. in the aspect of considering the real-time performance of the requirement, in order to solve the requirement of online real-time abnormality detection, the system provided by the patent refers to a high-performance asynchronous IO communication epoll mechanism, and processes data requests of different algorithms through a single data processing thread, so that the performance loss caused by thread switching is reduced as much as possible, and the high-efficiency real-time abnormality detection is realized;
4. the invention solves the problem of sequential data instruction prolongation of the spacecraft by a multi-classification method. In order to solve the problem that in the task of detecting the abnormal state of the spacecraft, the telemetry data with instructions need to be classified in time series, but the instructions cannot cover the corresponding telemetry information, the invention fills the length occupied by the instruction information in the time series according to the classification result by a multi-classification method for the telemetry data, and performs characteristic processing for the abnormal state detection work;
5. the invention provides a time sequence abnormality detection algorithm based on an attention mechanism to perform spacecraft abnormality detection work. Aiming at multi-dimensional spacecraft telemetry data strongly related to instructions, the algorithm uses an LSTM network and a convolution means to extract time sequence data characteristics within a period of time, builds association between instruction information and telemetry data through an attention mechanism, predicts the next time sequence data through a regression method, and finally judges whether real data is abnormal or not through threshold constraint.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, and various modifications and variations can be made to the embodiments of the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A deep learning-based spacecraft telemetry data detection system, the system comprising:
the data acquisition and preprocessing module is used for acquiring telemetry data and preprocessing the telemetry data in a wild picking, slicing, time synchronization and standardization way;
the remote sensing data processing module is used for carrying out integrated data statistical analysis, feature extraction and anomaly detection on the preprocessed remote sensing data and carrying out visual display on a data processing result;
the algorithm management module is used for uploading, modifying, deleting and configuring the uploading interface for each data processing algorithm in the remote sensing data processing module;
the task customization module is used for customizing the flow of the operation task according to the algorithm and the data in the data acquisition and preprocessing module and the remote sensing data processing module, so as to realize the flow operation;
and the data visualization module is used for realizing data visualization display of data processing results of the data acquisition and preprocessing module, the remote sensing data processing module, the algorithm management module and the task customization module.
2. The deep learning based spacecraft telemetry data detection system of claim 1, wherein said data acquisition and preprocessing module comprises a outlier algorithm, a slicing algorithm, a time synchronization algorithm, and a data normalization algorithm.
3. The deep learning-based spacecraft telemetry data detection system of claim 1, wherein the telemetry data processing module comprises a statistical analysis algorithm, a feature extraction algorithm, and an anomaly detection algorithm.
4. A spacecraft telemetry data detection system based on deep learning as claimed in claim 3, wherein the feature extraction algorithm comprises a correlation analysis method, a moving average algorithm, a stationarity test method, an EMD trend analysis method and a principal component analysis method.
5. A deep learning based spacecraft telemetry data detection system according to claim 3, wherein the anomaly detection algorithm is a TPA-LSTM method using an improved version of the attention mechanism on the basis of LSTM networks.
6. The deep learning-based spacecraft telemetry data detection system of claim 1, wherein said system is further capable of implementing offline task scheduling, the method of implementing said offline task scheduling comprising:
a user enters a flow chart of an offline control page drawing execution task provided by the system, a page is formulated through a front-end visual flow provided by the data visualization module, an algorithm needing to be customized and required data are selected, parameters of each algorithm are designated, a page scheduling task flow is formulated in the front-end visual flow in a dragging mode, the flow chart is stored after the task formulation is finished, and the task flow can be executed on an offline panel or an online panel according to the flow chart;
the task customization module analyzes the key path of the flow chart by a topological ordering method;
and the system asynchronously executes the user-defined flow chart, the tasks of the flow chart are added into a thread pool to be concurrently executed, the user obtains feedback that the tasks are started, and the user obtains the task execution result by clicking a view button.
7. A deep learning based spacecraft telemetry data detection system according to claim 1, wherein said system obtains said telemetry data from an external database or dis or bus.
8. The deep learning-based spacecraft telemetry data detection system of claim 7, wherein the system is further capable of implementing high concurrency online task scheduling, the high concurrency online task scheduling being accomplished by a data request thread, the method for implementing the high concurrency online task scheduling comprising:
carrying out algorithm thread registration through the data request thread to obtain a data source, a data interface and a callback function of a required algorithm;
the data request thread adds the data interface of the required algorithm into a waiting queue, when the data interface has data access, the data interface is transferred from the waiting queue to a linked list, and then the interface in the linked list is polled;
and when new data are generated on the data interface, calling a callback function corresponding to the data interface, executing an algorithm thread corresponding to the required algorithm, and completing online task scheduling of the required algorithm, wherein a single thread is adopted to process data access, a corresponding algorithm module is dormant when no data is accessed, and the corresponding algorithm thread is awakened in a callback mode after the data is accessed.
9. A spacecraft telemetry data detection system based on deep learning as claimed in claim 3, wherein the implementation flow of the anomaly detection algorithm comprises:
classifying the occurrence position and the occurrence range of abnormal data in the telemetry data and supplementing instructions to obtain marked instruction-telemetry time sequence data;
carrying out data processing on marked fragments of the instruction-telemetry data, processing all fragments into subsequences with the length of w in a sliding window segmentation mode, and carrying out sliding window slicing with different step sizes on each fragment;
training a classifier structure, obtaining the sliding window slice with the length w of the telemetry data to be detected according to the classifier after training, classifying the sliding window slice, and marking the output value of the classification result below the corresponding time dimension;
acquiring the output values of a plurality of classification results in each time step after the abnormal value detection is finished through a data abnormal detection module, selecting the maximum output value in each category to carry out category voting, and adopting a classification strategy to carry out instruction continuation;
after instruction continuation is finished, obtaining n-dimensional instruction-telemetry data, inputting the n-dimensional instruction-telemetry data into a training model of the anomaly detection module for prediction, wherein a prediction result is a result of whether the instruction-telemetry data is anomalous or not.
10. The deep learning based spacecraft telemetry data detection system of claim 9, wherein said anomaly detection algorithm comprises an instruction classification module and a data anomaly detection module, said instruction classification module comprising a classification network and a decision module, wherein the classification network is a res net network, each substructure of said res net network comprises 3 layers of one-dimensional convolutional networks of 8, 5, 3 sizes, respectively, each layer of said convolutional network being followed by a BN layer and an activation function ReLU; the inputs and outputs of each substructure of the ResNet network are summed; the instruction classification module classifies by softmax.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311234601.4A CN117390507A (en) | 2023-09-22 | 2023-09-22 | Spacecraft telemetry data detection system based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311234601.4A CN117390507A (en) | 2023-09-22 | 2023-09-22 | Spacecraft telemetry data detection system based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117390507A true CN117390507A (en) | 2024-01-12 |
Family
ID=89438173
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311234601.4A Pending CN117390507A (en) | 2023-09-22 | 2023-09-22 | Spacecraft telemetry data detection system based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117390507A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117828517A (en) * | 2024-03-06 | 2024-04-05 | 北京开运联合信息技术集团股份有限公司 | Spacecraft on-orbit running state evaluation method based on data mining |
-
2023
- 2023-09-22 CN CN202311234601.4A patent/CN117390507A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117828517A (en) * | 2024-03-06 | 2024-04-05 | 北京开运联合信息技术集团股份有限公司 | Spacecraft on-orbit running state evaluation method based on data mining |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220024607A1 (en) | Predictive maintenance model design system | |
| JP7009438B2 (en) | Computer systems and methods for monitoring key performance indicators (KPIs) using time series pattern models | |
| US9047559B2 (en) | Computer-implemented systems and methods for testing large scale automatic forecast combinations | |
| CN111861020A (en) | Model deployment method, device, equipment and storage medium | |
| CN113950665A (en) | Process understanding of Robotic Process Automation (RPA) using sequence extraction | |
| CN103597417A (en) | Method and device for condition monitoring | |
| CN117390507A (en) | Spacecraft telemetry data detection system based on deep learning | |
| CN116485576A (en) | Intelligent manufacturing management platform for brain data with known source in aviation manufacturing industry | |
| CN111260073A (en) | Data processing method, device and computer readable storage medium | |
| KR102543064B1 (en) | System for providing manufacturing environment monitoring service based on robotic process automation | |
| WO2022030041A1 (en) | Prediction system, information processing device, and information processing program | |
| Giobergia et al. | Mining sensor data for predictive maintenance in the automotive industry | |
| Du et al. | An integrated system for on-line intelligent monitoring and identifying process variability and its application | |
| Silva et al. | A multi-criteria decision model for the selection of a more suitable Internet-of-Things device | |
| CN117709185A (en) | Digital twin architecture design method for process industry | |
| CN110727670A (en) | Data structure prediction transfer and automatic data processing method based on flow chart | |
| CN117710156A (en) | Highway construction optimization method and system based on big data | |
| KR20200002433A (en) | Statistical quality control system and method using big data analysis | |
| CN114154816A (en) | Enterprise management system and execution method thereof | |
| CN113177040A (en) | Full-process big data cleaning and analyzing method for aluminum/copper plate strip production | |
| Pan et al. | Sequential design command prediction using BIM event logs | |
| Lei et al. | Application of distributed machine learning model in fault diagnosis of air preheater | |
| CN116205354B (en) | Method, device and storage medium for predicting flying heavy landing | |
| CN117370870B (en) | Knowledge and data compound driven equipment multi-working condition identification and performance prediction method | |
| Liu et al. | AutoSlicer: Scalable Automated Data Slicing for ML Model Analysis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |