CN111191710A - Abnormal flight identification method based on big data - Google Patents
Abnormal flight identification method based on big data Download PDFInfo
- Publication number
- CN111191710A CN111191710A CN201911361959.7A CN201911361959A CN111191710A CN 111191710 A CN111191710 A CN 111191710A CN 201911361959 A CN201911361959 A CN 201911361959A CN 111191710 A CN111191710 A CN 111191710A
- Authority
- CN
- China
- Prior art keywords
- data
- attribute
- flight
- abnormal
- metadata
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000002159 abnormal effect Effects 0.000 title claims abstract description 56
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000012545 processing Methods 0.000 claims abstract description 37
- 238000001514 detection method Methods 0.000 claims abstract description 28
- 230000005856 abnormality Effects 0.000 claims abstract description 27
- 238000004364 calculation method Methods 0.000 claims abstract description 14
- 238000004140 cleaning Methods 0.000 claims abstract description 13
- 238000012549 training Methods 0.000 claims description 6
- 238000012795 verification Methods 0.000 claims description 5
- 238000012937 correction Methods 0.000 claims description 4
- 238000004458 analytical method Methods 0.000 claims description 2
- 230000010006 flight Effects 0.000 abstract description 3
- 238000011161 development Methods 0.000 description 3
- 238000012544 monitoring process Methods 0.000 description 3
- 238000013480 data collection Methods 0.000 description 2
- 150000001875 compounds Chemical class 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/24323—Tree-organised classifiers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
- G06F16/90335—Query processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Databases & Information Systems (AREA)
- Computational Linguistics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The abnormal flight identification method based on the big data is provided with a data acquisition module, a data processing module and an abnormality detection module, wherein the data acquisition module is connected with the data processing module, the data processing module is connected with the abnormality detection module, the data acquisition module is used for acquiring flight historical booking metadata and sending the acquired historical booking metadata to the data processing module, the data processing module is used for sequentially carrying out data cleaning according to the historical booking metadata to obtain first data and sending the first data to the abnormality detection module, and the abnormality detection module is used for obtaining a data abnormality index and judging whether the first data is abnormal or not through an isolated forest algorithm calculation step according to the first data. The abnormal flight identification method based on the big data can automatically judge and calculate whether the flight data is abnormal or not after cleaning and sorting according to the historical seat booking metadata of the flights, can calculate a large amount of flight metadata, saves manpower and material resources, and improves the precision of flight prediction.
Description
Technical Field
The invention relates to the technical field of aviation flight data monitoring, in particular to an abnormal flight identification method based on big data.
Background
With the development of social productivity, the continuous progress of science and technology and the continuous development of aviation business in China, aviation data needs to be organized to manage the aviation business while the aviation business is continuously progressed, and the core of the aviation data acquires the potential value of the aviation data from a large amount of collected data, so that the benign development of the aviation business is continuously realized.
However, in the existing data, due to the existence of abnormal information in the aviation data, the accuracy of the collected data is affected, and a large amount of labor cost and monitoring cost are consumed.
Therefore, it is necessary to provide a big data-based abnormal flight identification method to overcome the deficiencies of the prior art.
Disclosure of Invention
The invention aims to avoid the defects of the prior art and provides the abnormal flight identification method based on the big data, which can automatically identify the abnormal state in the aviation data, can calculate a large amount of aviation data simultaneously and reduce the data monitoring cost.
The above object of the present invention is achieved by the following technical means.
Provides an abnormal flight identification method based on big data, which is provided with a data acquisition module, a data processing module and an abnormal detection module, wherein the data acquisition module is connected with the data processing module, the data processing module is connected with the abnormal detection module,
preferably, the data acquisition module is used for acquiring flight historical booking metadata and sending the acquired historical booking metadata to the data processing module,
preferably, the data processing module performs data cleaning on the historical booking metadata to obtain first data, sends the first data to the abnormality detection module,
preferably, the anomaly detection module obtains a data anomaly index through an isolated forest algorithm calculation step according to the first data and judges whether the first data is abnormal or not according to the calculated data anomaly index.
Preferably, the data cleaning specifically comprises; s1, classifying the flight historical booking metadata file through Python to obtain classified data;
s2, removing invalid repeated wrong flight historical booking metadata in the classification data in the step S1 to obtain valid flight historical booking metadata;
s3, formatting and arranging the effective flight historical booking metadata to obtain first data;
s4, performing relevance verification on the first data in the step S3.
Preferably, the isolated forest algorithm calculation process comprises a model data training stage and a prediction anomaly judgment stage.
Preferably, the model data training stage comprises establishing t isolated trees, wherein t is more than or equal to 1, t is the number of the isolated trees, and t is a positive integer.
Preferably, the process of establishing each isolated tree is as follows:
step a1-1, randomly and unreleased selecting psi data as root nodes of a subsample put tree from first data to establish the root nodes as isolated tree root nodes, psi is larger than or equal to 1, psi is a positive integer, and the data with the most attributes in the psi data has T attributes;
step a1-2, selecting any one attribute in the first data as a first fixed attribute, cutting the psi data according to the first fixed attribute, and forming Q by the first data with the first fixed attribute11The data set, first data without first fixed attributes, constitutes Q12Data set, Q by cutting11Data set, Q12Establishing a data set as a first layer of an isolated tree;
step a1-3, defining K as the cutting times, K as a natural number and K not more than psi-1; let K be 2, go to step a1-4,
step a1-4, randomly assigning Q(K-1)1Any unused attribute of data in the dataset is taken as a Kth attribute, and Q is paired on a node at the K-1 th level of the previous orphan tree(K-1)1The data in the data set is cut by the Kth attribute, and the data with the Kth attribute is divided into QK1Data set, data without Kth Attribute split to QK2Data set, Q by cuttingK1Data set, QK2The data set is established as isolatedThe K layer of the tree;
step a1-5, judging QK1Whether the data in the data set meets the defined cutting condition, if so, entering step a 1-7; if not, go to step a 1-6;
step a1-6, enabling K to be K +1, and returning to step a 1-4;
step a1-7, orphan tree building is complete.
Preferably, the condition for limiting cutting is at least one of that only one data exists in the formatted data of the current node, that all the formatted data are the same, or that the children nodes reach the limited height of the isolated tree.
Preferably, the abnormal stage is predicted by calculating the abnormal value score of the first data according to the formula (I) in an isolated tree,
where s (x, ψ) is an anomaly score, E (h (x)) is an average value representing the height of record x per isolated tree, c (ψ) is an average value of path lengths when the given number of samples is ψ, x is arbitrarily designated data in the first data, and h (x) is a corrected path length of data x.
h(X)＝e+C(n)
… … formula (II);
H(n-1)＝ln(n-1)+0.5772156649
… … formula (IV);
c (n) is a correction value representing the average path length of a binary tree constructed by using n pieces of sample data;
e represents the number of edges that data x passes through in going from the root node to the leaf nodes of the isolated tree.
When s (x, psi) is more than or equal to 0.8 and less than or equal to 1, the aviation data is defined to be very abnormal;
when s (x, psi) is more than 0.6 and less than or equal to 0.8, defining that the aviation data is generally abnormal;
and when the s (x, psi) is more than 0.4 and less than or equal to 0.6, the aviation data is defined to be abnormal.
Preferably, the first fixed attribute is any one of a DayOfWeek time attribute, a cabin space attribute, a collection date attribute, a departure attribute, an arrival attribute and a takeoff time;
preferably, the formatting process includes at least one of formatting content, date type formatting, week formatting and data calculation of sales volume of the slots.
Preferably, the limited cutting condition is at least one of that only one data exists in the formatted data of the current node, all formatted data are the same, or the child nodes reach the limited height of the isolated tree;
preferably, the data cleansing further includes step S4, and the first data in step S3 is subjected to relevance verification.
Preferably, the remaining required data is at least one of collection date, flight number, flight segment, departure, arrival, departure date, departure time, arrival time, cabin space and sales quantity;
preferably, the data acquisition module sends the historical booking metadata to the data processing module through the ETL tool;
preferably, the historical subscription metadata comprises multi-dimensional category data of BFG, BLB, BLC, BLG, BSB, BSG, CFD and SCH;
preferably, the real-time response is processed by streaming when the multidimensional category data is real-time data, and preferably, the analysis and calculation are performed based on data of the HDFS distributed file system when the multidimensional category data is offline data.
Preferably, the first fixed attribute is a DayOfWeek time attribute,
preferably, the abnormality detection module is further provided with an alarm display unit.
Preferably, the alarm display unit displays the current first data and issues an alarm when s (x, ψ) > 0.4.
The abnormal flight identification method based on the big data is provided with a data acquisition module, a data processing module and an abnormality detection module, wherein the data acquisition module is connected with the data processing module, the data processing module is connected with the abnormality detection module, the data acquisition module is used for acquiring flight historical booking metadata and sending the acquired historical booking metadata to the data processing module, the data processing module is used for sequentially carrying out data cleaning according to the historical booking metadata to obtain first data and sending the first data to the abnormality detection module, and the abnormality detection module is used for obtaining a data abnormality index through an isolated forest algorithm calculation step according to the first data and judging whether the first data is abnormal or not. The abnormal flight identification method based on the big data can automatically judge and calculate whether the flight data is abnormal or not after cleaning and sorting according to the historical seat booking metadata of the flights, can calculate a large amount of flight metadata, saves manpower and material resources, and improves the precision of flight prediction.
Drawings
The invention is further illustrated by means of the attached drawings, the content of which is not in any way limiting.
Fig. 1 is a schematic structural view of embodiment 1 of the present invention.
Fig. 2 is a schematic structural diagram of embodiment 3 of the present invention.
In fig. 1 to 2, the method includes:
the system comprises a data acquisition module 100, a data processing module 200, an abnormality detection module 300 and an alarm display unit 301.
Detailed Description
The invention is further described with reference to the following examples.
Example 1.
An abnormal flight identification method based on big data is provided, as shown in fig. 1, with a data acquisition module 100, a data processing module 200 and an abnormality detection module 300, wherein the data acquisition module 100 is connected with the data processing module 200, the data processing module 200 is connected with the abnormality detection module 300,
specifically, the data collection module 100 is configured to collect flight historical booking metadata and send the collected historical booking metadata to the data processing module 200,
specifically, the data processing module 200 performs data cleaning on the historical booking metadata to obtain first data, and sends the first data to the anomaly detection module 300,
specifically, the anomaly detection module 300 obtains a data anomaly index through an isolated forest algorithm calculation step according to the first data and judges whether the first data is abnormal according to the calculated data anomaly index.
Specifically, the data cleaning specifically comprises; s1, classifying the flight historical booking metadata file through Python;
s2, removing invalid repeated wrong flight historical booking metadata in the classification data in the step S1 to obtain valid flight historical booking metadata;
and S3, formatting the effective flight historical booking metadata to obtain first data.
S4, performing relevance verification on the first data in the step S3.
The purpose of the correlation verification is to ensure that the current first data is correct and valid flight history booking metadata, and details are not repeated as a specific principle of common general knowledge of those skilled in the art.
Specifically, the isolated forest algorithm calculation process comprises a model data training stage and a prediction abnormity judgment stage.
Specifically, t isolated trees are established in the model data training stage, t is more than or equal to 1, t is the number of the isolated trees, and t is a positive integer.
Specifically, the process of establishing each isolated tree is as follows:
step a1-1, randomly and unreleased selecting psi data as root nodes of a subsample put tree from first data to establish isolated tree root nodes, wherein psi is larger than or equal to 1, psi is a positive integer, and the data with the most attributes in the psi data has T attributes;
step a1-2, selecting any one attribute in the first data as a first fixed attribute, cutting the psi data according to the first fixed attribute, and forming Q by the first data with the first fixed attribute11The data set, first data without first fixed attributes, constitutes Q12Data set, Q by cutting11Data set, Q12Establishing a data set as a first layer of an isolated tree;
step a1-3, defining K as the cutting times, K as a natural number and K not more than psi-1; let K be 2, go to step a1-4,
step a1-4, randomly assigning Q(K-1)1Any unused attribute of data in the dataset is taken as a Kth attribute, and Q is paired on a node at the K-1 th level of the previous orphan tree(K-1)1The data in the data set is cut by the Kth attribute, and the data with the Kth attribute is divided into QK1Data set, data without Kth Attribute split to QK2Data set, Q by cuttingK1Data set, QK2Establishing a data set as a K-th layer of an isolated tree;
step a1-5, judging QK1Whether the data in the data set meets the defined cutting condition, if so, entering step a 1-7; if not, go to step a 1-6;
step a1-6, enabling K to be K +1, and returning to step a 1-4;
step a1-7, orphan tree building is complete.
Specifically, the limited cutting condition is at least one of that only one data exists in the formatted data of the current node, all formatted data are the same, or the child nodes reach the limited height of the isolated tree.
Specifically, the abnormal stage prediction method specifically comprises the steps of performing abnormal value score calculation on first data according to an isolated tree by using a formula (I),
wherein s (x, ψ) is an anomaly score, E (h (x)) is an average value representing the height of record x per isolated tree, c (ψ) is an average value of path lengths when the given number of samples is ψ, x is arbitrarily designated data in the first data, and h (x) is a corrected path length of data x;
h(X)＝e+C(n)
… … formula (II);
H(n-1)＝ln(n-1)+0.5772156649
… … formula (IV);
c (n) is a correction value representing the average path length of a binary tree constructed by using n pieces of sample data;
e represents the number of edges that data x passes through in going from the root node to the leaf nodes of the isolated tree.
In the formula, e represents the number of edges that data x passes from the root node to the leaf node of the isolated tree, i.e., the so-called path length, and c (n) is a correction value representing the average path length in a binary tree constructed by n pieces of sample data.
The calculation formula of C (n) is as follows:
h (n-1) ═ ln (n-1) +0.5772156649 … … formula (vii);
0.5772156649 is the Euler constant.
When the average path length of the data x in the isolated trees is shorter, the score is closer to 1, and the more abnormal the data x is; if the average path length of the data x in the isolated trees is longer, the score is closer to 0, and the data x is more normal; if the credential length of data x in the number of orphan trees is close to the overall mean, the score will be close to 0.5.
When s (x, psi) is more than or equal to 0.8 and less than or equal to 1, the aviation data is defined to be very abnormal;
when s (x, psi) is more than 0.6 and less than or equal to 0.8, defining that the aviation data is generally abnormal;
and when the s (x, psi) is more than 0.4 and less than or equal to 0.6, the aviation data is defined to be abnormal.
Specifically, the first fixed attribute is any one of a DayOfWeek time attribute, a cabin space attribute, a collection date attribute, a departure attribute, an arrival attribute and a takeoff time; specifically, in this embodiment, the first fixed attribute is a DayOfWeek time attribute.
Specifically, the formatting process includes at least one of formatting content, date type formatting, week formatting and data calculation of sales volume of the slots.
Specifically, the remaining required data is at least one of collection date, flight number, flight segment, departure, arrival, departure date, departure time, arrival time, cabin space and sales volume;
specifically, the data collection module 100 sends the historical booking metadata to the data processing module 200 through the ETL tool;
specifically, the historical subscription metadata includes multi-dimensional category data of BFG, BLB, BLC, BLG, BSB, BSG, CFD, and SCH;
the historical booking metadata is data obtained according to the actual flight metadata file type, wherein the SCH is flight time data and the CFD is check-in flight data, and the rest is the specific information data of the flight: departure, arrival, slot, time of day, leg, number of sales, etc. BFG, BLB, BLC, BLG, BSB, BSG, CFD, SCH are common knowledge of those skilled in the art, and details thereof are not repeated.
The abnormal flight identification method based on the big data is provided with a data acquisition module, a data processing module and an abnormality detection module, wherein the data acquisition module is connected with the data processing module, the data processing module is connected with the abnormality detection module, the data acquisition module is used for acquiring flight historical booking metadata and sending the acquired historical booking metadata to the data processing module, the data processing module is used for sequentially carrying out data cleaning according to the historical booking metadata to obtain first data and sending the first data to the abnormality detection module, and the abnormality detection module is used for obtaining a data abnormality index and judging whether the first data is abnormal or not through an isolated forest algorithm calculation step according to the first data. The abnormal flight identification method based on the big data can automatically judge and calculate whether the flight data is abnormal or not after cleaning and sorting according to the historical seat booking metadata of the flights, can calculate a large amount of flight metadata, saves manpower and material resources, and improves the precision of flight prediction.
Example 2.
The other structures of the method are the same as those of embodiment 1, but the method is different in that real-time response is performed through streaming processing when multidimensional category data is real-time data.
And when the multidimensional category data are offline data, analyzing and calculating the data based on the HDFS distributed file system.
When the data is real-time data, whether the data is abnormal can be detected in time by processing the multidimensional category data in a streaming mode, and when the data is off-line data, the data of the unstructured compound semi-structured can be effectively processed in time, so that various types of mass data can be flexibly processed.
Example 3.
The other structure of the abnormal flight identification method based on big data is the same as that of the embodiment 1 or 2, and the difference is that the abnormal detection module 300 is further provided with an alarm display unit 301.
Specifically, the alarm display unit 301 displays the current first data and issues an alarm when s (x, ψ) > 0.4.
The abnormal state of the user connection data can be conveniently realized by arranging the alarm display unit.
Finally, it should be noted that the above embodiments are only used for illustrating the technical solutions of the present invention and not for limiting the protection scope of the present invention, and although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions can be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims (10)
1. An abnormal flight identification method based on big data is characterized in that: the system is provided with a data acquisition module, a data processing module and an abnormality detection module, wherein the data acquisition module is connected with the data processing module, and the data processing module is connected with the abnormality detection module;
the data acquisition module acquires flight historical booking metadata and sends the acquired historical booking metadata to the data processing module,
the data processing module carries out data cleaning on the historical booking metadata to obtain first data and sends the first data to the abnormality detection module,
and the abnormality detection module calculates a data abnormality index according to the first data through an isolated forest algorithm and judges whether the first data is abnormal according to the calculated data abnormality index.
2. The big-data-based abnormal flight identification method according to claim 1, wherein: the data cleaning specifically comprises the following steps:
s1, classifying the flight historical booking metadata file through Python to obtain classified data;
s2, removing invalid repeated wrong flight historical booking metadata in the classification data in the step S1 to obtain valid flight historical booking metadata;
and S3, formatting the effective flight historical booking metadata to obtain first data.
3. The big-data-based abnormal flight identification method according to claim 2, wherein: the isolated forest algorithm calculation process comprises a model data training stage and a prediction abnormity judgment stage.
4. The big-data-based abnormal flight identification method according to claim 3, wherein: in the model data training stage, t isolated trees are established, t is more than or equal to 1, t is the number of the isolated trees, and t is a positive integer.
5. The big-data-based abnormal flight identification method according to claim 4, wherein: the process of establishing each isolated tree is as follows:
step a1-1, randomly and unreleased selecting psi data as root nodes of a subsample put tree from first data to establish isolated tree root nodes, wherein psi is larger than or equal to 1, psi is a positive integer, and the data with the most attributes in the psi data has T attributes;
step a1-2, selecting any one attribute in the first data as a first fixed attribute, cutting the psi data according to the first fixed attribute, and forming Q by the first data with the first fixed attribute11The data set, first data without first fixed attributes, constitutes Q12Data set, Q by cutting11Data set, Q12Establishing a data set as a first layer of an isolated tree;
step a1-3, defining K as the cutting times, K as a natural number and K not more than psi-1; let K be 2, go to step a1-4,
step a1-4, randomly assigning Q(K-1)1Any unused attribute of data in the dataset is taken as a Kth attribute, and Q is paired on a node at the K-1 th level of the previous orphan tree(K-1)1The data in the data set is cut by the Kth attribute, and the data with the Kth attribute is divided into QK1Data set, data without Kth Attribute split to QK2Data set, Q by cuttingK1Data set, QK2Establishing a data set as a K-th layer of an isolated tree;
step a1-5, judging QK1Whether the data in the data set meets the defined cutting condition, if so, entering step a 1-7; if not, go to step a 1-6;
step a1-6, enabling K to be K +1, and returning to step a 1-4;
step a1-7, orphan tree building is complete.
6. The big-data-based abnormal flight identification method according to claim 5, wherein: the limiting cutting condition is at least one of that only one data exists in the formatted data of the current node, all the formatted data are the same or the children nodes reach the limiting height of the isolated tree.
7. The big-data-based abnormal flight identification method according to claim 6, wherein:
the abnormal prediction stage is to calculate the abnormal value score of the first data according to an isolated tree by formula (I),
where s (x, ψ) is an anomaly score, E (h (x)) represents the height average of x in each isolated tree, C (ψ) is the average of path lengths given a sample number of ψ, x is arbitrarily designated data in the first data, and h (x) is the corrected path length of data x;
h(X)＝e+C(n)
… … formula (II);
H(n-1)＝ln(n-1)+0.5772156649
… … formula (IV);
c (n) is a correction value representing the average path length of a binary tree constructed by using n pieces of sample data;
e represents the number of edges that data x passes through in going from the root node to the leaf nodes of the isolated tree.
8. The big-data-based abnormal flight identification method according to claim 7, wherein:
when s (x, psi) is more than or equal to 0.8 and less than or equal to 1, judging that the aviation data is abnormal;
when s (x, psi) is more than 0.6 and less than or equal to 0.8, judging that the aviation data is generally abnormal;
and when s (x, psi) is more than 0.4 and less than or equal to 0.6, judging that the aviation data is abnormal.
9. The big-data-based abnormal flight identification method according to claim 8, wherein: the data cleaning further comprises a step S4 of performing relevance verification on the first data in the step S3;
the first fixed attribute is any one of a time attribute, a cabin position attribute, an acquisition date attribute, a departure attribute, an arrival attribute and a takeoff time;
the formatting treatment specifically comprises at least one of format content arrangement, date type formatting, week formatting and cabin sales quantity data operation;
the rest required data is at least one of data of collection date, flight number, flight segment, departure, arrival, departure date, departure time, arrival time, cabin space and sales quantity;
the data acquisition module sends the historical booking metadata to the data processing module through an ETL tool;
the historical booking metadata comprises multi-dimensional category data of BFG, BLB, BLC, BLG, BSB, BSG, CFD and SCH;
and when the multi-dimensional category data is real-time data, real-time response is carried out through streaming processing, and when the multi-dimensional category data is offline data, analysis and calculation are carried out on the data based on the HDFS distributed file system.
10. The big-data-based abnormal flight identification method according to claim 9, wherein: the first fixed attribute is a time attribute;
the abnormality detection module is also provided with an alarm display unit which displays the current first data and gives an alarm when s (x, ψ) > 0.4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911361959.7A CN111191710A (en) | 2019-12-26 | 2019-12-26 | Abnormal flight identification method based on big data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911361959.7A CN111191710A (en) | 2019-12-26 | 2019-12-26 | Abnormal flight identification method based on big data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111191710A true CN111191710A (en) | 2020-05-22 |
Family
ID=70709394
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911361959.7A Pending CN111191710A (en) | 2019-12-26 | 2019-12-26 | Abnormal flight identification method based on big data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111191710A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111784966A (en) * | 2020-06-15 | 2020-10-16 | 武汉烽火众智数字技术有限责任公司 | Personnel management and control method and system based on machine learning |
| CN114066038A (en) * | 2021-11-10 | 2022-02-18 | 上海市大数据股份有限公司 | Subway passenger flow prediction method and system |
| CN115168456A (en) * | 2022-09-07 | 2022-10-11 | 中国民航信息网络股份有限公司 | Flight sales process feature acquisition method and device, storage medium and electronic equipment |
| CN116757728A (en) * | 2023-08-16 | 2023-09-15 | 中国民航信息网络股份有限公司 | Method and related device for processing sales data of historical flights |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109345137A (en) * | 2018-10-22 | 2019-02-15 | 广东精点数据科技股份有限公司 | A kind of rejecting outliers method based on agriculture big data |
-
2019
- 2019-12-26 CN CN201911361959.7A patent/CN111191710A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109345137A (en) * | 2018-10-22 | 2019-02-15 | 广东精点数据科技股份有限公司 | A kind of rejecting outliers method based on agriculture big data |
Non-Patent Citations (2)
| Title |
|---|
| 李倩: "多维数据异常检测方法的研究与应用", 《中国优秀硕士学位论文全文数据库（电子期刊）信息科技辑》 * |
| 梁丽琴: "虚占时刻航班异常延误行为研究", 《中国优秀硕士学位论文全文数据库（电子期刊）经济与管理科学辑》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111784966A (en) * | 2020-06-15 | 2020-10-16 | 武汉烽火众智数字技术有限责任公司 | Personnel management and control method and system based on machine learning |
| CN114066038A (en) * | 2021-11-10 | 2022-02-18 | 上海市大数据股份有限公司 | Subway passenger flow prediction method and system |
| CN115168456A (en) * | 2022-09-07 | 2022-10-11 | 中国民航信息网络股份有限公司 | Flight sales process feature acquisition method and device, storage medium and electronic equipment |
| CN115168456B (en) * | 2022-09-07 | 2022-11-25 | 中国民航信息网络股份有限公司 | Flight sales process feature acquisition method and device, storage medium and electronic equipment |
| CN116757728A (en) * | 2023-08-16 | 2023-09-15 | 中国民航信息网络股份有限公司 | Method and related device for processing sales data of historical flights |
| CN116757728B (en) * | 2023-08-16 | 2023-10-24 | 中国民航信息网络股份有限公司 | Method and related device for processing sales data of historical flights |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111191710A (en) | Abnormal flight identification method based on big data | |
| Wang et al. | An IoT application for fault diagnosis and prediction | |
| CN110399935A (en) | The real-time method for monitoring abnormality of robot and system based on isolated forest machine learning | |
| US20210160266A1 (en) | Computer-implemented method and arrangement for classifying anomalies | |
| JP2019023937A5 (en) | ||
| JP5933463B2 (en) | Log occurrence abnormality detection device and method | |
| CN109190051B (en) | User behavior analysis method and resource recommendation method based on analysis method | |
| KR20150057322A (en) | System and Method for Deducting about Weak Signal Using Big Data Analysis | |
| CN113419492A (en) | NB-IoT industrial pollutant monitoring system based on multi-source heterogeneous sensor data fusion | |
| CN109299740A (en) | A kind of construction method and system of Logic Regression Models | |
| CN104008633A (en) | Early-warning method and system of facility spinach diseases | |
| CN106780273A (en) | Passenger flight requirement analysis method and system | |
| CN112116168B (en) | User behavior prediction method and device and electronic equipment | |
| CN106682206A (en) | Method and system for big data processing | |
| CN112686521A (en) | Wind control rule tuning method and system | |
| CN106644035B (en) | Vibration source identification method and system based on time-frequency transformation characteristics | |
| CN116541782A (en) | Power marketing data anomaly identification method | |
| CN118296164A (en) | Automatic agricultural product information acquisition and updating method and system based on knowledge graph | |
| CN117009180A (en) | Log and abnormal alarm information processing method and device | |
| CN116317103A (en) | Power distribution network voltage data processing method | |
| CN112765553B (en) | Engineering project management system based on big data | |
| CN115470251A (en) | Big data analysis display device | |
| CN115767601A (en) | 5GC network element automatic nanotube method and device based on multidimensional data | |
| CN110968570A (en) | Distributed big data mining system facing E-commerce platform | |
| CN114579519A (en) | File system abnormality detection method and device, storage medium and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200522 |