CN117544525A - Wireless AP log analysis method and system for subway - Google Patents
Wireless AP log analysis method and system for subway Download PDFInfo
- Publication number
- CN117544525A CN117544525A CN202311750917.9A CN202311750917A CN117544525A CN 117544525 A CN117544525 A CN 117544525A CN 202311750917 A CN202311750917 A CN 202311750917A CN 117544525 A CN117544525 A CN 117544525A
- Authority
- CN
- China
- Prior art keywords
- data
- subway
- log
- model
- effective
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 79
- 238000000034 method Methods 0.000 claims abstract description 36
- 230000005856 abnormality Effects 0.000 claims abstract description 17
- 238000005311 autocorrelation function Methods 0.000 claims description 18
- 238000005065 mining Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 11
- 238000012549 training Methods 0.000 claims description 11
- 230000006870 function Effects 0.000 claims description 10
- 238000012360 testing method Methods 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 8
- 230000002159 abnormal effect Effects 0.000 claims description 7
- 238000010801 machine learning Methods 0.000 claims description 7
- 238000005070 sampling Methods 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 5
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 238000011156 evaluation Methods 0.000 claims description 4
- 238000001914 filtration Methods 0.000 claims description 4
- 230000001960 triggered effect Effects 0.000 claims description 4
- 238000004422 calculation algorithm Methods 0.000 claims description 3
- 238000004140 cleaning Methods 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 3
- 238000012423 maintenance Methods 0.000 abstract 1
- 238000004891 communication Methods 0.000 description 8
- 238000007667 floating Methods 0.000 description 8
- 230000008901 benefit Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- YHXISWVBGDMDLQ-UHFFFAOYSA-N moclobemide Chemical compound C1=CC(Cl)=CC=C1C(=O)NCCN1CCOCC1 YHXISWVBGDMDLQ-UHFFFAOYSA-N 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 description 1
- 238000012952 Resampling Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013075 data extraction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 229910021389 graphene Inorganic materials 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/147—Network analysis or design for predicting network behaviour
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/145—Network analysis or design involving simulating, designing, planning or modelling of a network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/149—Network analysis or design for prediction of maintenance
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/16—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks using machine learning or artificial intelligence
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Mobile Radio Communication Systems (AREA)
Abstract
The invention discloses a method and a system for analyzing subway wireless AP logs. And constructing a data node model by using mechanical learning, calculating node positions to optimize data downsampling, and generating hour-level time series data. The system classifies the valid data by feature recognition and analyzes the links between the data, and builds and trains an initial analysis model to eliminate interference. According to the method, the node positions are predicted, key data are extracted from the nodes, the operation amount is obviously reduced, and a log analysis model is quickly built. Through the model, subway operation safety can be timely evaluated, potential hidden danger is identified in advance, instant discovery capability of AP equipment abnormality is enhanced, and prediction accuracy and operation and maintenance efficiency are improved.
Description
Technical Field
The invention relates to the technical field of subway operation safety, in particular to a wireless AP log analysis method and system for subways.
Background
When the subway arrives at the station and leaves the station, the subway vehicle-mounted MR equipment communicates with the AP equipment beside the track in the station to generate subway AP log data. When the subway runs on a line, tens of thousands of magnitude of subway AP log data can be generated in a system running at any time, the subway AP log data contains a lot of potential information, hidden fault information behind the deep mining data and internal and external connection of fault generation reasons can be better used for helping good and healthy running of all equipment, and riding safety is further guaranteed.
Communication connection faults of trains, tracksides or shielding networks and the like, which occur in the automatic operation vehicle-mounted mode of the subway, are related to communication faults of the AP connected with the subway, but communication abnormality in the AP equipment is hidden deeply, and only when the AP data of all stations have problems, the problems can be found; on one hand, the problems of all the site AP data can be found, namely, the problems are difficult to find in time when only single AP data are in problem, communication faults are happened, and serious loss is easy to cause to the operation of the subway; in addition, the subway AP log data comprises effective data and invalid data, wherein the volume of the invalid data is larger, and when all data are read, more data reading time is required, and the quick acquisition of the subway AP log data is not facilitated.
Disclosure of Invention
The present invention has been made in view of the above-described problems.
Therefore, the technical problems solved by the invention are as follows: the method has the advantages that effective data in the subway AP log data is slow in reading speed and lack of early warning.
In order to solve the technical problems, the invention provides the following technical scheme: a method for subway wireless AP log analysis, comprising: mining subway AP log data, extracting effective data in associated data through keywords, storing the effective data, and deleting ineffective data in the associated data;
segmenting different text paragraphs in the subway AP log data by using keywords, counting nodes where effective data in the different text paragraphs are located, and analyzing the relation between the effective data in the text paragraphs and the nodes by using mechanical learning to construct an effective data node model;
calculating nodes where the effective data in the residual text paragraphs are located through the effective data node model, and extracting the residual effective data in the residual text paragraphs through keywords;
identifying and classifying all the effective data according to the characteristics and the attributes of the effective data, analyzing the relation among the effective data in different categories by using a machine learning method, and establishing an initial analysis model;
performing error checking on the interference data, incorporating the interference data without abnormality into the initial analysis model for training and testing until the effective data completely pass through the initial analysis model to obtain an AP log analysis model, analyzing and predicting the AP log data, and evaluating the safety of subway AP equipment.
As a preferable scheme of the method for analyzing the subway wireless AP log, the present invention comprises: mining the subway AP log data comprises mining the subway AP log data saved in the past, performing data cleaning, data conversion and data merging on the AP log data by utilizing a data processing library of python language, separating the subway AP log data into different text paragraphs, and extracting associated data in the subway AP log data by a keyword extraction technology.
As a preferred scheme of the method for analyzing the subway wireless AP log, nodes where the effective data in the different text paragraphs are located comprise nodes where the different text paragraphs are located in the subway AP log data are segmented through keywords, the nodes where the effective data in the different text paragraphs are located are counted, a plurality of text data are loaded into a memory to form a perfect data set, filtering is carried out by taking an AP node as a unit, and the filtered data are time ordered to form an effective data set to be analyzed with a second sampling frequency and are used for constructing an effective data node model;
constructing the effective data node model comprises the step of adopting an SVM algorithm, wherein an objective function of the SVM is as follows:
wherein w represents a weight vector, w S Representing the weight of signal strength in the model, w D Representing the weight of the connection duration in the model, w U Representing the weight of the user traffic in the model, b representing the bias term for determining the position of the decision boundary in the feature space; c represents a penalty parameter, controlling the severity of the error classification; n represents the total number of data points, ζ i As a relaxation variable, representing the degree to which the ith data point violates classification;
the kernel function formula is:
K(x i ，x j )＝exp(-γ(||S i -S j || 2 +||D i -D j || 2 +||U i -U j || 2 ))
wherein K represents a kernel function for calculating the data point x i And x j Similarity in the new feature space; s is S i Represents the signal intensity of the ith data point, D i Indicating the connection time of the ith data point, U i User traffic representing the ith data point; gamma denotes a kernel parameter, controlling the distribution relation of different data points in a high-dimensional space.
As a preferable scheme of the method for analyzing the subway wireless AP log, the present invention comprises: the node position calculation comprises the steps of calculating the node positions of the rest text segments in the subway AP log data by applying the effective data node model to form a contour range, extracting effective data through keywords in the range, and analyzing the effective data;
when the head and the tail are not missing, the authenticity of the effective data is reliable;
when the tail of the effective data is missing, the head of the effective data is problematic, at the moment, the data is extended to the next node, one node position is extended at a time until the tail of the effective data is found, and meanwhile, the part exceeding the byte length is deleted;
when the head of the effective data is missing, the tail of the effective data is indicated to have a problem, at the moment, the data is extended to the previous node, and the node position is extended once until the head of the effective data is found, and meanwhile, the part exceeding the byte length is deleted.
As a preferable scheme of the method for analyzing the subway wireless AP log, the method comprises the following steps: the initial analysis model is built, wherein the initial analysis model comprises the steps of analyzing log data by using an ARIAM model, and analyzing the time series data by using an auto-correlation function ACF and a partial auto-correlation function PACF;
the autocorrelation function ACF is:
wherein k represents the number of hysteresis periods,represents the autocorrelation coefficient of the kth lag phase, coy (y t ，y t-k ) Representing the covariance between the values of time points t and t-k in the time series, var (y t ) Representing the variance of the values of the time series at the point in time t.
As a preferable scheme of the method for analyzing the subway wireless AP log, the method comprises the following steps: the AP log analysis model comprises the steps of performing error checking on interference data, determining that no abnormality exists in the interference data as normal data, and determining that the abnormality exists in the interference data as error data;
and re-importing the normal data into the initial analysis model for data training and testing, correcting the initial analysis model to enable the initial analysis model to conform to the normal data, and exporting the interference data which cannot pass through the initial analysis model for error checking, and sequentially reciprocating until the effective data completely pass through the initial analysis model.
As a preferable scheme of the method for analyzing the subway wireless AP log, the method comprises the following steps: the method comprises the steps of evaluating the safety of subway AP equipment, wherein whether the AP equipment is likely to fail or not is judged by comparing a prediction result with actual log data so as to trigger early warning;
after the model predicts the result, the staff sets the allowable error range to obtain a prediction interval;
if the true value is in the prediction interval, not triggering an alarm;
if the true value exceeds the predicted interval, an alarm is triggered uniformly.
The utility model provides a wireless AP log analysis system for subway which characterized in that: comprising the steps of (a) a step of,
and a data acquisition module: mining subway AP log data, extracting associated data through keywords, deleting invalid data, and storing the valid data;
and a data processing module: segmenting data by using keywords, counting effective data nodes, and analyzing the relation between the data and the nodes by applying mechanical learning to construct an effective data node model;
model construction module: the node position calculation is carried out by applying the effective data node model, then the residual effective data is extracted through the key words, the data downsampling is carried out, the data sampling frequency is reduced to the hour level, and the time sequence data is generated;
an analysis model module: identifying and classifying all effective data according to the characteristics and the attributes, analyzing the relation between the data by using a machine learning method, and establishing an initial analysis model;
and a security evaluation module: and performing error checking on the interference data, incorporating the non-abnormal data into the initial analysis model, training and testing until the effective data completely pass through the initial analysis model, acquiring an AP log analysis model, analyzing and predicting the AP log data, and evaluating the safety of subway AP equipment.
A computer device comprising a memory storing a computer program and a processor implementing the steps of the method as described above when the processor executes the computer program.
A computer readable storage medium having stored thereon a computer program which when executed by a processor realizes the steps of the method as described above.
The invention has the beneficial effects that: the relation between the effective data in the text paragraphs and the nodes is analyzed through mechanical learning, the nodes where the effective data in the text paragraphs with different subway AP log data are located are predicted, and then the keyword extraction is carried out at the positions where the nodes are located, so that the data operand can be reduced, the effective data can be extracted rapidly, and the AP log analysis model can be built rapidly;
after the vehicle-mounted MR and the subway AP equipment are in contact, the effective data in the subway AP log data can be quickly obtained through the effective data-node model, and the safety of subway operation is timely evaluated through the effective data, so that whether hidden danger exists in the subway operation is found in advance;
by analyzing the links existing among the effective data in different categories, the abnormal change of the effective data can be found in advance, and the abnormality of the AP equipment can be found through calculation when the abnormality occurs to the AP data of a single site, so that communication faults are avoided.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. Wherein:
fig. 1 is an overall flowchart of a method for analyzing a subway wireless AP log according to a first embodiment of the present invention;
fig. 2 is a specific flowchart of a method for analyzing a subway wireless AP log according to a first embodiment of the present invention;
fig. 3 is a related parameter diagram of ARIMA model for subway wireless AP log analysis method according to the first embodiment of the present invention;
fig. 4 is a first-order differential schematic diagram of a method for analyzing a subway wireless AP log according to a first embodiment of the present invention;
fig. 5 is a prediction early warning range diagram for a subway wireless AP log analysis method according to a first embodiment of the present invention.
Detailed Description
So that the manner in which the above recited objects, features and advantages of the present invention can be understood in detail, a more particular description of the invention, briefly summarized above, may be had by reference to the embodiments, some of which are illustrated in the appended drawings. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
Example 1
Referring to fig. 1 to 5, for one embodiment of the present invention, there is provided a wireless AP log analysis method for a subway, including:
s1: mining subway AP log data, extracting effective data in the associated data through keywords, storing the effective data, and deleting ineffective data in the associated data.
And mining the past saved subway AP log data, and performing data cleaning, data conversion and data merging on the AP log data by using a data processing library of python language to separate the subway AP log data into different text paragraphs.
Extracting associated data in the subway AP log data by a keyword extraction technology, wherein the subway AP log data contains a large amount of effective data and ineffective data, and the extracted associated data also contains the ineffective data and the effective data.
And sorting the separated different text paragraphs according to the extracted data volume, and respectively collecting effective data from the different text paragraphs according to the sorting of the different text paragraphs when extracting the associated data.
S2: and segmenting different text paragraphs in the subway AP log data by using keywords, counting nodes where effective data in the different text paragraphs are located, and analyzing the relation between the effective data in the text paragraphs and the nodes by using mechanical learning to construct an effective data node model.
Because the keywords are repeated, the collected data can contain invalid data in the extracted associated data because the keywords are the invalid data, so that the invalid data needs to be deleted by identifying the valid data after the associated data are extracted, the keywords corresponding to the invalid data also belong to the invalid keywords, and a large amount of subway AP log data needs to be read by identifying the keywords.
After the associated data is extracted, the associated data is required to be read, and the extraction efficiency of the effective data can be delayed when a large amount of data is read.
A plurality of paragraphs are separated from one text paragraph of the subway AP log data, different text paragraphs in the subway AP log data are segmented through keywords, the position of each keyword is a node, then nodes where effective data in different text paragraphs are located are counted, then the relation between the effective data and the nodes in the text paragraphs is analyzed through mechanical learning, the position relation between the nodes and the effective data is established, and an effective data-node model is constructed.
Further, constructing the effective data node model comprises adopting an SVM algorithm, wherein an objective function of the SVM is as follows:
wherein,w represents a weight vector, w S Representing the weight of signal strength in the model, w D Representing the weight of the connection duration in the model, w U Representing the weight of the user traffic in the model, b representing the bias term for determining the position of the decision boundary in the feature space; c represents a penalty parameter, controlling the severity of the error classification; n represents the total number of data points, ζ i As a relaxation variable, representing the degree to which the ith data point violates classification;
the kernel function formula is:
K(x i ，x j )＝exp(-γ(||S i -S j || 2 +||D i -D j || 2 +||U i -U j || 2 ))
wherein K represents a kernel function for calculating the data point x i And x j Similarity in the new feature space; s is S i Represents the signal intensity of the ith data point, D i Indicating the connection time of the ith data point, U i User traffic representing the ith data point; gamma denotes a kernel parameter, controlling the distribution relation of different data points in a high-dimensional space.
It should be noted that, loading a plurality of text data into the memory makes it become a more perfect data set, filters with the AP node as the unit, sorts the data after filtering according to time, and the data set of this moment data is the effective data set of waiting to analyze, and the sampling frequency of data set is the second level this moment, makes things convenient for later stage to carry out further resampling (downsampling) to the data set.
Part of the valid data is shown below:
s3: and calculating nodes where the effective data in the residual text paragraphs are located through the effective data node model, and extracting the residual effective data in the residual text paragraphs through keywords.
And carrying out downsampling and data statistics on the effective data, so that the sampling evaluation rate of the data is reduced from the second level to the hour, and the time series data with stronger applicability is obtained.
The node position in the rest text paragraph in the subway AP log data is calculated through the effective data-node model, the calculated node position is a contour range, then data extraction is carried out in the contour range through keywords, the content of invalid data in the extracted associated data is low, even no, only when the AP equipment is abnormal, the invalid data is acquired when the calculated node position is located in the subway AP log data and deviates, therefore, after the rest effective data is extracted in the rest text paragraph through the keywords, the effective data also needs to be read to ensure the authenticity of the effective data, and the format of the position of the head and the tail of the effective data and text coding can be analyzed through data analysis of the head and the tail of the effective data in the step one.
After the effective data is obtained through node calculation, the format and text codes of the positions of the head and the tail of the effective data are identified, and whether the head and the tail of the effective data are missing or not is confirmed; when the head and the tail are not missing, the authenticity of the effective data is reliable; when the tail of the effective data is missing, the head of the effective data is indicated to exist, at the moment, the data is extended to the next node, one node position is extended at a time until the tail of the effective data is found, and meanwhile, the part exceeding the byte length is deleted; when the head of the effective data is missing, the existence of the tail of the effective data is indicated, at the moment, the data extension is carried out on the upper node, one node position is extended at a time until the head of the effective data is found, and meanwhile, the part exceeding the byte length is deleted; the byte length of the effective data is stable, and before the authenticity of the effective data is detected, the byte length of the effective data is counted, so that redundant data can be deleted conveniently when the byte length of the effective data is overlong.
S4: and identifying and classifying all the effective data according to the characteristics and the attributes of the effective data, analyzing the relation existing between the effective data in different categories by using a machine learning method, and establishing an initial analysis model.
Analyzing the log data by using an ARIAM model, and analyzing the time series data by using an autocorrelation function ACF and a partial autocorrelation function PACF;
the autocorrelation function ACF is:
wherein k represents the number of hysteresis periods,represents the autocorrelation coefficient of the kth lag phase, coy (y t ，y t-k ) Representing the covariance between the values of time points t and t-k in the time series, var (y t ) Representing the variance of the values of the time series at the point in time t.
The 10 th order autocorrelation/partial autocorrelation function images of ACF and PACF were plotted and the correlation parameters of ARIMA model were determined by fig. 3.
Further, all the effective data are identified and classified according to the characteristics and the attributes of the effective data, certain relations exist in the classified data, the relations among the effective data in different categories are analyzed by using a machine learning method, an initial analysis model is built by using the relations among the effective data in different categories, then the initial analysis model is subjected to data training and testing by more effective data, the effective data which do not accord with the initial analysis model are set as interference data, and the interference data are extracted.
It should be noted that after classifying the effective data, deep learning is performed on the effective data of different categories, the links existing between the effective data of the same category are analyzed, an effective data fluctuation model is established, and the prediction result of the AP log analysis model is verified by using the effective data fluctuation model.
S5: performing error checking on the interference data, incorporating the interference data without abnormality into the initial analysis model for training and testing until the effective data completely pass through the initial analysis model to obtain an AP log analysis model, analyzing and predicting the AP log data, and evaluating the safety of subway AP equipment.
If the data sets participating in ACF and PACF contain abnormal data or have insignificant correlation, the data sets need to be processed, such as filtering abnormal values and differentiating the data to further stabilize the data, and then the processed stable data is brought into an ARIMA model for training. As shown in fig. 4, the original time series is made smoother by the first order difference.
Performing error checking on the interference data, determining that no abnormality exists in the interference data as normal data, and determining that abnormality exists in the interference data as error data; then, normal data are reintroduced into the initial analysis model for data training and testing, at the moment, the initial analysis model is corrected, so that the initial analysis model can conform to the normal data, error checking is conducted on the interference data which cannot pass through the initial analysis model, and the data are sequentially reciprocated until the effective data completely pass through the initial analysis model, and an AP log analysis model is obtained; the error data are divided into floating data and error reporting data, wherein the floating data refer to data abnormality caused by some unexpected events, and the interference data still have certain referential property; error reporting data refers to data errors caused by abnormality of subway AP equipment, and the error data does not have referential property; when error data occurs, counting the reasons of the subway AP equipment abnormality when the error data occurs, and establishing an error reporting table according to the relation between the error reporting data and the subway AP equipment abnormality; performing mechanical learning on floating data, analyzing the relation among the floating data, establishing a floating data interference model, and correcting a prediction result of an AP log analysis model by using the floating data interference model; when the subway AP log data generates the data condition in the floating data interference model again, replacing the analysis and prediction result of the AP log analysis model through the error correction data generated by the floating data interference model.
Furthermore, after the vehicle-mounted MR and the subway AP equipment are connected, the effective data in the subway AP log data are obtained through the effective data-node model, the effective data are analyzed and predicted through the AP log analysis model, and the safety of subway operation is evaluated according to the prediction result.
Furthermore, the autocorrelation analysis of the data set and the determination of relevant parameters of the MARIA model are realized through the previous steps, then an analysis model can be built through effective time series data, the prediction function of the past and future time series is realized through the model, and whether the AP equipment is likely to be in fault or not is judged through the comparison of the prediction result and the actual log data so as to trigger early warning.
After the model predicts the result, the staff sets the allowable error range to obtain a prediction interval, and if the true value exceeds the prediction interval, an alarm is triggered uniformly. As shown in fig. 5, the dashed line is the original data set, implemented as a model for a predicted value of a certain period of time in the future, the hatched portion is the fault tolerance interval after the fault tolerance value is set, and an alarm is triggered if the actual value exceeds the fault tolerance interval.
It should be noted that, after the valid data in the subway AP log data is extracted, the valid data at this time is not suitable for building a model, so the text in the valid data is encoded so that the text is suitable for building a model, thereby reducing the data operand; in the sixth step, after the AP log analysis model analyzes and predicts the subway AP log data, the generated coded conversion text data is broadcasted.
The above embodiment further includes a system for analyzing subway wireless AP logs, specifically:
and a data acquisition module: mining subway AP log data, extracting associated data through keywords, deleting invalid data, and storing the valid data;
and a data processing module: segmenting data by using keywords, counting effective data nodes, and analyzing the relation between the data and the nodes by applying mechanical learning to construct an effective data node model;
model construction module: the node position calculation is carried out by applying the effective data node model, then the residual effective data is extracted through the key words, the data downsampling is carried out, the data sampling frequency is reduced to the hour level, and the time sequence data is generated;
an analysis model module: identifying and classifying all effective data according to the characteristics and the attributes, analyzing the relation between the data by using a machine learning method, and establishing an initial analysis model;
and a security evaluation module: and performing error checking on the interference data, incorporating the non-abnormal data into the initial analysis model, training and testing until the effective data completely pass through the initial analysis model, acquiring an AP log analysis model, analyzing and predicting the AP log data, and evaluating the safety of subway AP equipment.
The computer device may be a server. The computer device includes a processor, a memory, an Input/Output interface (I/O) and a communication interface. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface is connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is used for storing data cluster data of the power monitoring system. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for communicating with an external terminal through a network connection. The computer program, when executed by a processor, implements a method for subway wireless AP log analysis.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the various embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile memory may include Read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high density embedded nonvolatile memory, resistive random access memory (ReRAM), magnetic random access memory (MagnetoresistiveRandomAccessMemory, MRAM), ferroelectric memory (FerroelectricRandomAccessMemory, FRAM), phase change memory (PhaseChangeMemory, PCM), graphene memory, and the like. Volatile memory may include random access memory (RandomAccessMemory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can take many forms, such as static random access memory (StaticRandomAccessMemory, SRAM) or dynamic random access memory (DynamicRandomAccessMemory, DRAM), among others. The databases referred to in the various embodiments provided herein may include at least one of relational databases and non-relational databases. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic units, quantum computing-based data processing logic units, etc., without being limited thereto.
Example 2
For one embodiment of the invention, a method for analyzing the wireless AP log of the subway is provided, and in order to verify the beneficial effects of the method, scientific demonstration is carried out through economic benefit calculation and simulation/comparison experiments. Reference may be made to table 1:
table 1 experimental data reference table
Compared with the prior art, the method has the advantages that the accuracy is improved by 10.1%, the treatment time is reduced by 36.9%, and higher efficiency is shown. Memory usage was reduced by 22.7%, indicating that resource management was optimized. The false alarm rate is reduced by 81.4%, and the reliability of the system is enhanced. The prediction time interval is reduced by 50.8%, and the response speed of the system is improved.
It should be noted that the above embodiments are only for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention may be modified or substituted without departing from the spirit and scope of the technical solution of the present invention, which is intended to be covered in the scope of the claims of the present invention.
Claims (10)
1. A method for analyzing a subway wireless AP log, comprising:
mining subway AP log data, extracting effective data in associated data through keywords, storing the effective data, and deleting ineffective data in the associated data;
segmenting different text paragraphs in the subway AP log data by using keywords, counting nodes where effective data in the different text paragraphs are located, and analyzing the relation between the effective data in the text paragraphs and the nodes by using mechanical learning to construct an effective data node model;
calculating nodes where the effective data in the residual text paragraphs are located through the effective data node model, and extracting the residual effective data in the residual text paragraphs through keywords;
identifying and classifying all the effective data according to the characteristics and the attributes of the effective data, analyzing the relation among the effective data in different categories by using a machine learning method, and establishing an initial analysis model;
performing error checking on the interference data, incorporating the interference data without abnormality into the initial analysis model for training and testing until the effective data completely pass through the initial analysis model to obtain an AP log analysis model, analyzing and predicting the AP log data, and evaluating the safety of subway AP equipment.
2. The method for subway wireless AP log analysis according to claim 1, wherein: mining the subway AP log data comprises mining the subway AP log data saved in the past, performing data cleaning, data conversion and data merging on the AP log data by utilizing a data processing library of python language, separating the subway AP log data into different text paragraphs, and extracting associated data in the subway AP log data by a keyword extraction technology.
3. The method for subway wireless AP log analysis according to claim 2, wherein: the nodes where the effective data in the different text paragraphs are located comprise nodes where the different text paragraphs are located in the subway AP log data are segmented through keywords, the nodes where the effective data in the different text paragraphs are located are counted, a plurality of text data are loaded into a memory to form a perfect data set, filtering is carried out by taking the AP nodes as units, and the filtered data are ordered in time to form an effective data set to be analyzed with a second sampling frequency and are used for constructing the effective data node model;
constructing the effective data node model comprises the step of adopting an SVM algorithm, wherein an objective function of the SVM is as follows:
wherein w represents a weight vector, w S Representing the weight of signal strength in the model, w D Representing the weight of the connection duration in the model, w U Representing the weight of the user traffic in the model, b representing the bias term for determining the position of the decision boundary in the feature space; c represents a penalty parameter, controlling the severity of the error classification; n represents the total number of data points, ζ i As a relaxation variable, representing the degree to which the ith data point violates classification;
the kernel function formula is:
K(x i ,x j )＝exp(-γ(||S i -S j || 2 +||D i -i|| 2 +||U i -U j || 2 ))
wherein K represents a kernel function for calculating the data point x i And x j Similarity in the new feature space; s is S i Represents the signal intensity of the ith data point, D i Indicating the connection time of the ith data point, U i User traffic representing the ith data point; gamma denotes a kernel parameter, controlling the distribution relation of different data points in a high-dimensional space.
4. The method for subway wireless AP log analysis according to claim 5, wherein: the node position calculation comprises the steps of calculating the node positions of the rest text segments in the subway AP log data by applying the effective data node model to form a contour range, extracting effective data through keywords in the range, and analyzing the effective data;
when the head and the tail are not missing, the authenticity of the effective data is reliable;
when the tail of the effective data is missing, the head of the effective data is problematic, at the moment, the data is extended to the next node, one node position is extended at a time until the tail of the effective data is found, and meanwhile, the part exceeding the byte length is deleted;
when the head of the effective data is missing, the tail of the effective data is indicated to have a problem, at the moment, the data is extended to the previous node, and the node position is extended once until the head of the effective data is found, and meanwhile, the part exceeding the byte length is deleted.
5. The method for subway wireless AP log analysis according to claim 4, wherein: the initial analysis model is built, wherein the initial analysis model comprises the steps of analyzing log data by using an ARIAM model, and analyzing the time series data by using an auto-correlation function ACF and a partial auto-correlation function PACF;
the autocorrelation function ACF is:
where k represents the hysteresis number ρ k Represents the autocorrelation coefficient of the kth lag phase, cov (y t ,y t-k ) Representing the covariance between the values of time points t and t-k in the time series, var (y t ) Representing the variance of the values of the time series at the point in time t.
6. The method for subway wireless AP log analysis according to claim 5, wherein: the AP log analysis model comprises the steps of performing error checking on interference data, determining that no abnormality exists in the interference data as normal data, and determining that the abnormality exists in the interference data as error data;
and re-importing the normal data into the initial analysis model for data training and testing, correcting the initial analysis model to enable the initial analysis model to conform to the normal data, and exporting the interference data which cannot pass through the initial analysis model for error checking, and sequentially reciprocating until the effective data completely pass through the initial analysis model.
7. The method for subway wireless AP log analysis according to claim 6, wherein: the method comprises the steps of evaluating the safety of subway AP equipment, wherein whether the AP equipment is likely to fail or not is judged by comparing a prediction result with actual log data so as to trigger early warning;
after the model predicts the result, the staff sets the allowable error range to obtain a prediction interval;
if the true value is in the prediction interval, not triggering an alarm;
if the true value exceeds the predicted interval, an alarm is triggered uniformly.
8. A system for wireless AP log analysis of subways using the method of any one of claims 1-7, characterized in that: comprising the steps of (a) a step of,
and a data acquisition module: mining subway AP log data, extracting associated data through keywords, deleting invalid data, and storing the valid data;
and a data processing module: segmenting data by using keywords, counting effective data nodes, and analyzing the relation between the data and the nodes by applying mechanical learning to construct an effective data node model;
model construction module: the node position calculation is carried out by applying the effective data node model, then the residual effective data is extracted through the key words, the data downsampling is carried out, the data sampling frequency is reduced to the hour level, and the time sequence data is generated;
an analysis model module: identifying and classifying all effective data according to the characteristics and the attributes, analyzing the relation between the data by using a machine learning method, and establishing an initial analysis model;
and a security evaluation module: and performing error checking on the interference data, incorporating the non-abnormal data into the initial analysis model, training and testing until the effective data completely pass through the initial analysis model, acquiring an AP log analysis model, analyzing and predicting the AP log data, and evaluating the safety of subway AP equipment.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 7 when the computer program is executed.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311750917.9A CN117544525A (en) | 2023-12-19 | 2023-12-19 | Wireless AP log analysis method and system for subway |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311750917.9A CN117544525A (en) | 2023-12-19 | 2023-12-19 | Wireless AP log analysis method and system for subway |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117544525A true CN117544525A (en) | 2024-02-09 |
Family
ID=89795982
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311750917.9A Pending CN117544525A (en) | 2023-12-19 | 2023-12-19 | Wireless AP log analysis method and system for subway |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117544525A (en) |
-
2023
- 2023-12-19 CN CN202311750917.9A patent/CN117544525A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111914873B (en) | Two-stage cloud server unsupervised anomaly prediction method | |
| CN107294993B (en) | WEB abnormal traffic monitoring method based on ensemble learning | |
| CN108038049B (en) | Real-time log control system and control method, cloud computing system and server | |
| CN111507376B (en) | Single-index anomaly detection method based on fusion of multiple non-supervision methods | |
| CN113962259B (en) | Multi-mode double-layer fault diagnosis method for fuel cell system | |
| CN108304567B (en) | Method and system for identifying working condition mode and classifying data of high-voltage transformer | |
| CN113887616A (en) | Real-time abnormity detection system and method for EPG (electronic program guide) connection number | |
| CN115981984A (en) | Equipment fault detection method, device, equipment and storage medium | |
| CN115269314A (en) | Transaction abnormity detection method based on log | |
| CN112069039A (en) | Monitoring and predicting alarm method and device for artificial intelligence development platform and storage medium | |
| CN114818313A (en) | Hidden electricity stealing behavior identification method based on synthesized minority oversampling technology | |
| CN113268370A (en) | Root cause alarm analysis method, system, equipment and storage medium | |
| CN115905959A (en) | Method and device for analyzing relevance fault of power circuit breaker based on defect factor | |
| CN115859191A (en) | Fault diagnosis method and device, computer readable storage medium and computer equipment | |
| CN116739829A (en) | Big data-based power data analysis method, system and medium | |
| CN111666978A (en) | Intelligent fault early warning system for IT system operation and maintenance big data | |
| CN115858606A (en) | Method, device and equipment for detecting abnormity of time series data and storage medium | |
| CN113778766B (en) | Hard disk fault prediction model establishment method based on multidimensional characteristics and application thereof | |
| CN112733897B (en) | Method and apparatus for determining abnormality cause of multi-dimensional sample data | |
| CN114021425A (en) | Power system operation data modeling and feature selection method and device, electronic equipment and storage medium | |
| CN115438239A (en) | Abnormity detection method and device for automatic abnormal sample screening | |
| CN115169650B (en) | Equipment health prediction method for big data analysis | |
| CN116400168A (en) | Power grid fault diagnosis method and system based on depth feature clustering | |
| CN111209955A (en) | Airplane power supply system fault identification method based on deep neural network and random forest | |
| CN113469247B (en) | Network asset abnormity detection method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |