CN114385494A

CN114385494A - Data analysis method, device, equipment and medium

Info

Publication number: CN114385494A
Application number: CN202210002503.7A
Authority: CN
Inventors: 邓一兵; 崔岩; 彭栋; 韩新宇
Original assignee: CRSC Research and Design Institute Group Co Ltd
Current assignee: CRSC Research and Design Institute Group Co Ltd
Priority date: 2022-01-04
Filing date: 2022-01-04
Publication date: 2022-04-22

Abstract

The embodiment of the invention discloses a data analysis method, a data analysis device, data analysis equipment and a data analysis medium. The method comprises the following steps: determining a target analysis file from the candidate analysis files according to the selection operation of the user; analyzing the data to be analyzed according to the candidate structural body in the target analysis file and the attribute information of the candidate data domain in the candidate structural body; wherein the attribute information includes data length and paraphrase information. The embodiment of the invention realizes the effects of improving the data analysis efficiency and increasing the data analysis flexibility.

Description

Data analysis method, device, equipment and medium

Technical Field

The embodiment of the invention relates to the technical field of data processing, in particular to a data analysis method, a data analysis device, data analysis equipment and a data analysis medium.

Background

The CTCS (Chinese Train operation Control System) is a Train operation Control System established in China to achieve the digitization, networking, intelligence and integrated directional development of a railway signal System and achieve a high-speed rail strategy. The CTCS performs data interaction with the train to realize the control of the train.

The CTCS needs to be tested before the CTCS is put into official delivery, that is, a technician judges the operation state of the CTCS based on the analyzed interactive data. At present, interactive data are mostly analyzed through a logic structure in a code, and the analysis efficiency is low and the flexibility is poor.

Disclosure of Invention

The embodiment of the application discloses a data analysis method, a data analysis device, data analysis equipment and a data analysis medium, and aims to solve the problem that the analysis efficiency is low when data is analyzed at present.

In a first aspect, an embodiment of the present invention provides a data parsing method, where the method includes:

determining a target analysis file from the candidate analysis files according to the selection operation of the user;

analyzing the data to be analyzed according to the candidate structural body in the target analysis file and the attribute information of the candidate data domain in the candidate structural body; wherein the attribute information includes data length and paraphrase information.

In a second aspect, an embodiment of the present invention provides a data parsing apparatus, where the apparatus includes:

the target analysis file determining module is used for determining a target analysis file from the candidate analysis files according to the selection operation of the user;

the data analysis module is used for analyzing the data to be analyzed according to the candidate structural body in the target analysis file and the attribute information of the candidate data domain in the candidate structural body; wherein the attribute information includes data length and paraphrase information.

In a third aspect, an embodiment of the present invention provides an apparatus, where the apparatus includes:

one or more processors;

a storage device for storing one or more programs,

when executed by the one or more processors, cause the one or more processors to implement a data parsing method as in any one of the embodiments of the invention.

In a fourth aspect, an embodiment of the present invention provides a computer-readable medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the data parsing method according to any one of the embodiments of the present invention.

The embodiment of the invention realizes the effects of improving the data analysis efficiency and increasing the data analysis flexibility.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.

Fig. 1A is a flowchart of a data parsing method according to an embodiment of the present invention;

fig. 1B is a schematic diagram of data transmission according to an embodiment of the present invention;

fig. 1C is a schematic diagram of acquiring data to be analyzed according to an embodiment of the present invention;

fig. 2A is a flowchart of a data parsing method according to a second embodiment of the present invention;

fig. 2B is a schematic view of an orientation of a structure according to a second embodiment of the present invention;

fig. 2C is a schematic diagram of data parsing according to a second embodiment of the present invention;

fig. 3 is a schematic structural diagram of a data analysis apparatus according to a third embodiment of the present invention;

fig. 4 is a schematic structural diagram of an apparatus according to a fourth embodiment of the present invention.

Detailed Description

The embodiments of the present invention will be described in further detail with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the embodiments of the invention and that no limitation of the invention is intended. It should be further noted that, for convenience of description, only the structures related to the embodiments of the present invention are shown in the drawings, not all the structures.

The applicant finds that the CTCS can be divided into 5 levels according to functions in the technical specification, wherein the CTCS-3 level Train Control System is a Train Control System standard with independent intellectual property rights formulated by China by introducing, digesting and absorbing ETCS (European Train Control System) and combining the national conditions of the country. The RBC (Radio Block Center) is the most critical device in the ground devices of the CTCS-3 level train control system, generates a safety train control command according to line information and train position information obtained by interacting with other external ground device systems, calculates MA (Movement Authority) and transmits the MA (Movement Authority) to a train through a GSM-R (GSM For Rail) wireless network, and realizes that the RBC controls the train to safely and reliably run in the coverage range of the train.

As a safety device with SIL (safety Integrity Level) at SIL4 Level, RBCs need to be strictly tested before they are normally delivered for normal use, and a large number of test cases cover all usage scenarios during the test process. For the result judgment of each test case, a tester judges according to DMI (Driver Machine Interface) display and the actual running condition of the train, and when the test result is inconsistent with the expectation, the tester further analyzes the phenomenon.

The DMI displays analysis data obtained by analyzing interactive data between the RBC and an ATP (Automatic Train Protection) Train, and a tester judges the operation state of the RBC according to the analysis data. At present, most of interactive data analysis is realized through a logic structure in a code, the analysis efficiency is low, and the flexibility is poor, so that the test efficiency of the whole test flow is indirectly influenced.

Example one

Fig. 1A is a flowchart of a data parsing method according to an embodiment of the present invention. The present embodiment is applicable to the case of parsing the interaction data between the RBC and the ATP, and the method may be performed by a data parsing apparatus provided in the embodiment of the present invention, where the apparatus may be implemented by software and/or hardware. As shown in fig. 1A, the method may include:

s101, determining a target analysis file from the candidate analysis files according to selection operation of a user.

The candidate parsing file is written by a relevant technician according to a special format, and is a configuration file for performing data parsing on data to be parsed, which is usually a file in an xml format. And aiming at different communication protocols adopted in different scenes, a corresponding candidate analysis file is compiled according to a data transmission format specified by each communication protocol, and a user can determine a target analysis file conforming to the communication protocol from the candidate analysis files according to the communication protocol adopted in the current scene.

In one embodiment, a user selects at least one candidate parsing file included in a parsing file folder through a front-end interface according to a communication protocol adopted by data to be parsed, wherein each candidate parsing file is distinguished by taking a file name as an identifier. And according to the selection operation of the user on any candidate analysis file, taking the candidate analysis file as a target analysis file. In this embodiment, the data to be parsed may be a data frame in any scene, and the data content of the data to be parsed is not limited in any way in this embodiment. Optionally, the data to be analyzed includes interaction data between the RBC and the ATP, and for convenience of understanding of the scheme, the data to be analyzed is regarded as the interaction data between the RBC and the ATP by subsequent default.

A communication module, such as a GSM-R unit, is provided in the RBC device for wireless communication with the ATP in the train. The communication unit is connected and communicated with a communication unit, such as a C machine, in an RBC host of the RBC device, and then the communication unit transmits the received data to an arithmetic unit, such as an A/B machine, in the RBC host, so as to perform logic processing based on the received data.

Fig. 1B is a schematic diagram of data transmission according to an embodiment of the present invention, as shown in fig. 1B, where data interaction is performed between an ATP101 in a train 100 and a GSM-R unit 102 in an RBC device, the GSM-R unit 102 sends received data to a C unit 104 in the RBC host 103, and the C unit 104 transmits the data to an a/B unit 105 for logic processing. In this embodiment, only the GSM-R unit 102 is taken as a communication module, the C machine 104 is taken as a communication unit, and the a/B machine 105 is taken as an operation unit for explanation, which does not limit the specific types of the communication module, the communication unit, and the operation unit, and all the modules/units that can perform the same function are within the scope of protection of this embodiment.

Because the RBC device only uses the arithmetic unit for data processing, the communication unit and the communication module are only responsible for data transmission, and data is not changed in the period, the embodiment uses the data transmitted between the communication module and the arithmetic unit as data to be analyzed. In other words, the data to be analyzed is train operation data sent by the ATP to the RBC device, or train control data sent by the RBC device to the ATP.

S102, analyzing data to be analyzed according to the candidate structural body in the target analysis file and the attribute information of the candidate data domain in the candidate structural body; wherein the attribute information includes data length and paraphrase information.

And any candidate parsing file comprises at least one candidate structure body, and each candidate structure body comprises at least one candidate data field. The candidate structure body is used for bearing the candidate data fields and making an aggregated organization structure for each candidate data field, so that the readability, the logicality and the reusability of the candidate analysis file can be improved. Each candidate structure has a unique identifier for indexing, for example, a candidate structure identified as "inherent variable in message header" may include two candidate data fields, i.e., "vehicle-mounted timestamp" and "vehicle-mounted device identification number". The candidate data field is used for data interpretation, and is given with attribute information of "data length" and "paraphrase information" in advance. The data length is used for calibrating the data length of the corresponding candidate data field in the data to be analyzed so as to intercept the data according to the data length, and the paraphrasing information is used for marking the intercepted data so as to ensure that a user knows the meaning represented by each intercepted data.

In one embodiment, a predetermined starting structure body is determined from the target analysis file, and the data to be analyzed is analyzed according to the arrangement order of each candidate data field in the starting structure body and the orientation relation of each candidate data field to other candidate structure bodies.

Specifically, whether a first order candidate data field in the starting structure body points to any other candidate structure body is determined, if not, data interception is carried out from the head of the data to be analyzed according to the data length of the first order candidate data field, and the intercepted data in the range is marked according to paraphrase information of the first order candidate data field, so that analysis data corresponding to the first order candidate data field is generated.

And if the data points to any other candidate structure, analyzing the data to be analyzed according to the arrangement order of each candidate data field in the other candidate structure and the data length and the paraphrase information respectively corresponding to each candidate data field.

And returning to the initial structure body after the analysis is finished, determining whether the second-order candidate data field in the initial structure body points to any other candidate structure body, if not, continuously intercepting data from the head of the rest data of the data to be analyzed according to the data length of the second-order candidate data field, marking the intercepted data in the range according to the paraphrasing information of the second-order candidate data field, and generating the analysis data corresponding to the second-order candidate data field.

By analogy, according to the above embodiment, all the candidate data fields in the starting structure are sequentially traversed, thereby completing the analysis of the data to be analyzed.

Illustratively, assume that the starting structure includes a candidate data field 1, a candidate data field 2, and a candidate data field 3, the candidate data field 2 points to the other structure 1, and the other structure 1 includes a candidate data field 4 and a candidate data field 5. The "data length" of the candidate data field 1 is "1 bits", and the "paraphrase information" is a "time stamp"; the "data length" of the candidate data field 3 is "2 bits", and the "paraphrasing information" is "line description"; the "data length" of the candidate data field 4 is "1 bits", and the "paraphrase information" is "device identification number"; the "data length" of the candidate data field 5 is "2 bits", and the "paraphrase information" is "message type".

The analyzing process of the data to be analyzed is as follows: and if the candidate data field 1 does not point to any other structural body, intercepting the data of the data to be analyzed, namely 0-1 bits, and taking the 'timestamp' as the mark of the intercepted data to generate the analysis data corresponding to the candidate data field 1. And if the candidate data field 2 points to other structural bodies 1, intercepting data of 1 to 2bits of the data to be analyzed, generating analysis data corresponding to the candidate data field 4 by taking the 'equipment identification number' as the label of the intercepted data, continuously intercepting the data of 2 to 4bits of the data to be analyzed, and generating analysis data corresponding to the candidate data field 5 by taking the 'message type' as the label of the intercepted data. If the candidate data field 3 does not point to any other structural body, intercepting data of 4 bits-6 bits of data to be analyzed, and taking the 'line description' as the label of the intercepted data to generate the analysis data corresponding to the candidate data field 3. In the above, the data analysis of the data to be analyzed is completed.

The interactive data between the RBC equipment and the ATP are grouped according to a proprietary communication protocol, the communication protocols adopted in the direction from ground to vehicle (the ATP sends data to the RBC equipment) and the direction from vehicle to ground (the RBC equipment sends data to the ATP) are different, and in the prior art, data analysis is carried out through a logic structure in codes, so that the analysis efficiency is low and the flexibility is poor. According to the technical scheme provided by the embodiment of the invention, the target analysis file is determined from the candidate analysis files through the selection operation of the user, and the data to be analyzed is analyzed according to the candidate structure body in the target analysis file and the attribute information of the candidate data domain in the candidate structure body; the attribute information comprises data length and paraphrase information, and the data length and the paraphrase information are configured for the candidate data field of each candidate structure body in the candidate analysis file in advance, so that after a user selects the target analysis file according to requirements, the file to be analyzed is directly analyzed according to the data length and the paraphrase information of the candidate data field of each candidate structure body in the target analysis file, and the data analysis efficiency is greatly improved; in addition, the user can select different target analysis files to analyze the file to be analyzed according to the requirement, and the flexibility of data analysis is improved.

The embodiment also provides an implementation manner for obtaining data to be analyzed, where data transmission between a communication module and a communication unit in the RBC device is usually implemented based on a switch, and taking the communication module as a GSM-R unit, the communication unit as a C unit, and the operation unit as an a/B unit as an example, because the a/B unit and the C unit are in a board card structure, and are connected by a backplane inside a cage therebetween, there is no external interface, and a hardware interface of the GSM-R unit and the C unit is an RJ-45 ethernet interface, because the C unit is used as a communication unit, data communicated with a host in the whole RBC device is transmitted to the a/B unit through the C unit, and therefore, the GSM-R unit and the C unit are not in a direct connection relationship, but are in data communication with the C unit through the switch as with other modules.

Based on the above connection relationship, in this embodiment, a to-be-analyzed data between the GSM-R unit and the C machine is captured by using an exchange mirror image, fig. 1C is a schematic diagram for obtaining the to-be-analyzed data according to an embodiment of the present invention, as shown in fig. 1C, the GSM-R unit 106 sends the data to the C machine 108 through the exchange 107, and other units 109 in the RBC device, in this embodiment, the GSM-R unit 106 is used as a mirror image source, and the data of the GSM-R unit 106 is sent to a mirror image destination interface 110 through the exchange 107, so as to obtain the to-be-analyzed data.

The data to be analyzed is acquired by adopting a mode of switch mirroring, the hardware environment of the conventional RBC equipment is not damaged, external information is not injected, and the function of the RBC equipment is not influenced.

Example two

Fig. 2A is a flowchart of a data parsing method according to a second embodiment of the present invention. The present embodiment is optimized based on the above optional embodiments, as shown in fig. 2A, the method may include:

s201, removing network layer data and link layer data in the original data to obtain the safety data.

The original data is a set of all data transmitted in the data transmission process.

In an embodiment, since the data to be parsed is the application layer data in the original data, the non-application data that does not have practical meaning in the original data needs to be removed before parsing the data to be parsed. After receiving the original data, determining whether the original data contains application layer data, and if the original data does not contain the application layer data, directly discarding the original data, for example, if the original data is an ACK (Acknowledgement), directly discarding the original data; and if the original data contains application layer data, removing network layer data and link layer data in the original data, and using the residual data as safety data.

S202, removing safety layer data in the safety data under the condition that the safety data accord with a preset safety protocol, and taking the residual data as data to be analyzed.

The preset safety protocol is a protocol for ensuring communication safety, and taking communication between the RBC device and the ATP as an example, the preset safety protocol optionally includes an RSSP-II (railway safety communication protocol-II) protocol.

In one embodiment, the protocol of the safety data is checked, whether the safety data conforms to a preset safety protocol is determined, and if the safety data does not conform to the preset safety protocol, the safety data is directly discarded. Because the security protocol ensures communication security by covering the application layer data with the security layer data, if the security data is determined to conform to the preset security protocol, the security layer data in the security data is removed, and the remaining application layer data is used as the data to be analyzed.

And S203, determining a target analysis file from the candidate analysis files according to the selection operation of the user.

S204, determining a first structure body from candidate structure bodies in a target analysis file, taking each candidate data field in the first structure body as a first data field, and analyzing the data to be analyzed in sequence according to the sequence of each first data field and the attribute information of each first data field.

Wherein the first structure represents a predetermined initial structure.

In one embodiment, the data to be analyzed is sequentially intercepted according to the data length of each first data field and the sequence of each first data field, and the intercepted data is labeled according to the paraphrase information of each first data field, so as to obtain the analysis data corresponding to each first data field.

For example, it is assumed that the first structure body includes a first data field a, a first data field B, and a first data field C, where "data length" of the first data field a is "1 bits", and "definition information" is "timestamp"; the "data length" of the first data field B is "2 bits", and the "paraphrasing information" is "line description"; the "data length" of the first data field C is "1 bits", and the "paraphrasing information" is "device identification number". Intercepting data of 0-1 bits of data to be analyzed, and generating analysis data corresponding to the first data domain A by using the 'timestamp' as a mark of the intercepted data. Intercepting data of 1-3 bits of data to be analyzed, and generating analysis data corresponding to the first data domain B by using 'line description' as a label of the intercepted data. Intercepting data of 3-4 bits of data to be analyzed, and generating analysis data corresponding to the first data domain C by taking the 'equipment identification number' as a label of the intercepted data.

Optionally, in S204, "sequentially analyze the data to be analyzed according to the order of each first data field and the attribute information of each first data field", includes the following steps:

determining whether the current first data field points to the second structure; wherein the second structure is any of the candidate structures other than the first structure; under the condition that the data does not point to the second structure body, carrying out data interception on the residual data of the data to be analyzed according to the data length of the current first data field to obtain a target data field; and marking the target data field according to the paraphrase information to be used as analysis data corresponding to the current first data field.

In an embodiment, the first data field may point to the second structure, and when the current first data field does not point to the second structure, the remaining data of the data to be analyzed is intercepted according to the data length of the current first data field, and the intercepted data is labeled according to the paraphrase information, so as to generate the analysis data corresponding to the current first data field.

Optionally, after determining whether the current first data field points to the second structure, the method further includes:

when the second structure body is pointed, taking each candidate data field in the second structure body as a second data field; and analyzing the residual data of the data to be analyzed in sequence according to the sequence of each second data domain and the attribute information of each second data domain.

In an embodiment, if the current first data field points to the second structure, the remaining data of the data to be analyzed is sequentially intercepted according to the data length of each second data field and the sequence of each second data field, and the intercepted data is labeled according to the paraphrase information of each second data field, so as to obtain the analysis data corresponding to each second data field.

Optionally, the analyzing the remaining data of the data to be analyzed sequentially according to the order of each second data field and the attribute information of each second data field includes the following steps:

1) determining whether the target second data domain points to a third structure under the condition that the current second data domain is the target second data domain; wherein the target second data field is the second data field in the last order, and the third structure is any one of the candidate structures other than the first structure and the second structure.

2) Under the condition that the third structure body is not pointed, analyzing the residual data of the data to be analyzed according to the attribute information of the target second data domain, and taking a first data domain with the sequence behind the current first data domain as an updated current first data domain.

3) And under the condition of pointing to the third structure, taking each candidate data field in the third structure as a third data field, and analyzing the residual data of the data to be analyzed in sequence according to the sequence of each third data field and the attribute information of each third data field.

In an embodiment, under the condition that the target second data field does not point to the third structure, data interception is performed on the remaining data of the data to be analyzed according to the data length of the target second data field, and the intercepted data is labeled according to the paraphrase information, so as to generate the analysis data corresponding to the target second data field.

And if the target second data field points to the third structure, intercepting the rest data of the data to be analyzed according to the data length of each third data field and the sequence of each third data field in sequence, and marking the intercepted data according to the paraphrase information of each third data field so as to obtain the analysis data corresponding to each third data field.

For example, the present embodiment exemplifies a specific process of data parsing:

fig. 2B is a schematic diagram of structure orientation according to a second embodiment of the present invention, as shown in fig. 2B, the first structure 200 includes a first data field 201, a first data field 202, a first data field 203, and a first data field 204, the first data field 202 points to a second structure 205, the first data field 203 points to a second structure 206, the second structure 205 includes a second data field 207 and a second data field 208, the second structure 206 includes a second data field 209 and a second data field 210, the second data field 208 points to a third structure 211, and the third structure 211 includes a third data field 212 and a third data field 213.

According to the structure body pointing form in fig. 2B, the corresponding data parsing process is as follows:

fig. 2C is a schematic diagram of data parsing according to a second embodiment of the present invention, as shown in fig. 2C, 214 represents data to be parsed, 215 represents a corresponding data field, and 216 represents a structure to which the data field belongs.

If the first data field 201 in the first structure 200 does not point to the second structure, intercepting the data 214 to be analyzed according to the data length of the first data field 201, and labeling the intercepted data according to the paraphrase information of the first data field 201 to obtain the analysis data 217 corresponding to the first data field 201.

If the first data field 202 points to the second structure 205, the remaining data of the data to be analyzed 214 is intercepted according to the data length of the second data field 207, and the intercepted data is labeled according to the paraphrase information of the second data field 207, so as to obtain the analysis data 218 corresponding to the second data field 207. The second data field 208 is the second data field in the last order, i.e. the target second data field, and the second data field 208 points to the third structure 211, then the remaining data of the data to be analyzed 214 is intercepted according to the data length of the third data field 212, and the intercepted data is labeled according to the paraphrase information of the third data field 212, so as to obtain the analysis data 219 corresponding to the third data field 212. And intercepting the residual data of the data to be analyzed 214 according to the data length of the third data field 213 according to the sequence, and marking the intercepted data according to the paraphrase information of the third data field 213 to obtain the analysis data 220 corresponding to the third data field 213.

And then returning to the first structure body 200, wherein the first data field 203 points to the second structure body 206, intercepting the remaining data of the data to be analyzed 214 according to the data length of the second data field 209, and labeling the intercepted data according to the paraphrasing information of the second data field 209 to obtain the analysis data 221 corresponding to the second data field 209. If the second data field 210 does not point to the third structure, the remaining data of the data to be analyzed 214 is intercepted according to the data length of the second data field 210, and the intercepted data is labeled according to the paraphrase information of the second data field 210, so as to obtain the analysis data 222 corresponding to the second data field 210.

And then returning to the first structure 200, if the first data field 204 does not point to the second structure, intercepting the remaining data of the data to be analyzed 214 according to the data length of the first data field 204, and labeling the intercepted data according to the paraphrasing information of the first data field 204 to obtain the analysis data 223 corresponding to the first data field 204.

The whole data analysis process is completed. It should be noted that each third data field in the third structure may also continuously point to other structures, and the process is the same as above, and the description of this embodiment is not repeated.

S205, determining analysis data according to an analysis result, and storing the analysis data and the original data according to a preset storage path; wherein the preset storage path comprises at least one of a local end and a database.

In one embodiment, the set of analysis data corresponding to each data field is used as the analysis data corresponding to the data to be analyzed. And after the analysis data is obtained, storing the analysis data and the original data in a local end or a database.

Specifically, if the preset storage path is a local end, the parsed data and the original data are recorded in TXT log files, and each log file further includes additional contents which are convenient to screen, such as log file generation time, original data acquisition time, and an IP address for acquiring the original data. The log file can be configured into a time mode or a capacity mode, a new log file can be generated after the set time is reached in the time mode, and the new log file is generated and recorded after the storage size of the log text reaches the set storage capacity in the capacity mode.

If the preset storage path is a database, converting the analysis data and the original data into json format and storing the json format into the corresponding database, wherein the database takes RBC equipment as a unit, namely the analysis data and the original data of each RBC equipment are stored in one corresponding database. The database can be selected as mongodb database, and the information storage mode is opened in advance.

S206, visually displaying the analysis data and the original data according to a query instruction of a user.

In one embodiment, the user generates a query instruction according to actual business requirements to view the stored data.

Specifically, the query includes, but is not limited to, the following three forms:

A. real-time query

The original data acquisition and data analysis device has a front-end interface for displaying data records, a tester can refresh and check the latest original data and analysis data through the display of an interface list, and meanwhile, the list interface has certain caching capacity, and the tester can review the record messages within a certain number through rolling.

B. Log query

And searching a corresponding log file at the local end to perform inquiry and editing processing.

C. Database query

The mode is realized through the springboot, the back-end service acquires data to be displayed through inquiring the database, the latest original data and the analyzed data can be displayed in real time according to the updating of the database, and meanwhile, the data can be further screened or graphically displayed through the man-machine interaction of the front end of the webpage. The mode is more ergodic for the query and the filtering of the data than the local query, and can query all the stored data in the database according to time, characteristic values and the like. In addition, the mode can realize that a plurality of people can check and use simultaneously, and is convenient for problem analysis in a cooperation mode.

According to the technical scheme provided by the embodiment of the invention, the network layer data and the link layer data in the original data are removed to obtain the safety data, the safety layer data in the safety data are removed under the condition that the safety data accord with the preset safety protocol, and the residual data are used as the data to be analyzed, so that the effects of data filtering and safety verification on the original data are realized; the first structure body is determined from the candidate structure bodies, each candidate data domain in the first structure body is used as a first data domain, and data to be analyzed are sequentially analyzed according to the sequence of each first data domain and the attribute information of each first data domain, so that the effects of improving the data analysis efficiency and increasing the flexibility of data analysis are achieved; the analysis data is determined according to the analysis result, and the analysis data and the original data are stored according to the preset storage path, so that the data storage effect is realized, and a foundation is laid for subsequent data query; the analytical data and the original data are visually displayed according to the query instruction of the user, so that the data query requirement of the user is met, and the user experience is improved.

EXAMPLE III

Fig. 3 is a schematic structural diagram of a data analysis apparatus according to a third embodiment of the present invention, which is capable of executing a data analysis method according to any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method. As shown in fig. 3, the apparatus may include:

a target analysis file determining module 31, configured to determine a target analysis file from the candidate analysis files according to a selection operation of a user;

the data analysis module 32 is configured to analyze the data to be analyzed according to the candidate structure in the target analysis file and the attribute information of the candidate data field in the candidate structure; wherein the attribute information includes data length and paraphrase information.

On the basis of the foregoing embodiment, the data analysis module 32 is specifically configured to:

determining a first structure from the candidate structures, and taking each candidate data field in the first structure as a first data field;

and analyzing the data to be analyzed in sequence according to the sequence of each first data field and the attribute information of each first data field.

On the basis of the foregoing embodiment, the data analysis module 32 is further specifically configured to:

determining whether the current first data field points to the second structure; wherein the second structure is any of the candidate structures other than the first structure;

under the condition that the data does not point to the second structure body, carrying out data interception on the residual data of the data to be analyzed according to the data length of the current first data field to obtain a target data field;

and marking the target data field according to the paraphrase information to be used as analysis data corresponding to the current first data field.

On the basis of the foregoing embodiment, the apparatus further includes a second data domain determining module, specifically configured to:

when the second structure body is pointed, taking each candidate data field in the second structure body as a second data field;

and analyzing the residual data of the data to be analyzed in sequence according to the sequence of each second data domain and the attribute information of each second data domain.

On the basis of the foregoing embodiment, the second data domain determining module is specifically further configured to:

determining whether the target second data domain points to a third structure under the condition that the current second data domain is the target second data domain; wherein the target second data field is the second data field in last order and the third structure is any of the candidate structures other than the first structure and the second structure;

under the condition that the third structure body is not pointed, analyzing the residual data of the data to be analyzed according to the attribute information of the target second data domain, and taking a first data domain with the sequence behind the current first data domain as an updated current first data domain;

and under the condition of pointing to the third structure, taking each candidate data field in the third structure as a third data field, and analyzing the residual data of the data to be analyzed in sequence according to the sequence of each third data field and the attribute information of each third data field.

On the basis of the above embodiment, the apparatus further includes a to-be-analyzed data acquisition module, which is specifically configured to:

removing network layer data and link layer data in the original data to obtain safety data;

and under the condition that the safety data accord with a preset safety protocol, removing safety layer data in the safety data, and taking the residual data as the data to be analyzed.

On the basis of the above embodiment, the apparatus further includes a data storage module, specifically configured to:

determining analysis data according to an analysis result, and storing the analysis data and the original data according to a preset storage path; wherein the preset storage path comprises at least one of a local end and a database.

On the basis of the above embodiment, the apparatus further includes a data query module, specifically configured to:

and visually displaying the analysis data and the original data according to a query instruction of a user.

The data analysis device provided by the embodiment of the invention can execute the data analysis method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method. For technical details that are not described in detail in this embodiment, reference may be made to a method for checking instruction validity provided in any embodiment of the present invention.

Example four

Fig. 4 is a schematic structural diagram of an apparatus according to a fourth embodiment of the present invention. Fig. 4 illustrates a block diagram of an exemplary device 400 suitable for use in implementing embodiments of the present invention. The apparatus 400 shown in fig. 4 is only an example and should not bring any limitations to the functionality or scope of use of the embodiments of the present invention.

As shown in FIG. 4, device 400 is in the form of a general purpose computing device. The components of device 400 may include, but are not limited to: one or more processors or processing units 401, a system memory 402, and a bus 403 that couples the various system components (including the system memory 402 and the processing unit 401).

Bus 403 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example, such architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, micro-channel architecture (MAC) bus, enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

Device 400 typically includes a variety of computer system readable media. Such media can be any available media that is accessible by device 400 and includes both volatile and nonvolatile media, removable and non-removable media.

The system memory 402 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM)404 and/or cache memory 405. The device 400 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 406 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 4, and commonly referred to as a "hard drive"). Although not shown in FIG. 4, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 403 by one or more data media interfaces. Memory 402 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.

A program/utility 408 having a set (at least one) of program modules 407 may be stored, for example, in memory 402, such program modules 407 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 407 generally perform the functions and/or methods of the described embodiments of the invention.

Device 400 may also communicate with one or more external devices 409 (e.g., keyboard, pointing device, display 410, etc.), with one or more devices that enable a user to interact with device 400, and/or with any devices (e.g., network card, modem, etc.) that enable device 400 to communicate with one or more other computing devices. Such communication may be through input/output (I/O) interface 411. Also, device 400 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet) through network adapter 412. As shown, the network adapter 412 communicates with the other modules of the device 400 over the bus 403. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with device 400, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.

The processing unit 401 executes various functional applications and data processing by running the program stored in the system memory 402, for example, to implement the data parsing method provided by the embodiment of the present invention, including:

EXAMPLE five

An embodiment of the present invention further provides a computer-readable storage medium, where the computer-executable instructions, when executed by a computer processor, are configured to perform a data parsing method, where the method includes:

Of course, the storage medium provided by the embodiment of the present invention contains computer-executable instructions, and the computer-executable instructions are not limited to the method operations described above, and may also perform related operations in a data parsing method provided by any embodiment of the present invention. The computer-readable storage media of embodiments of the invention may take any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A data parsing method, comprising:

2. The method according to claim 1, wherein the parsing the data to be parsed according to the structure in the target parsing file and the attribute information of the data field in the structure includes:

3. The method according to claim 2, wherein the sequentially parsing the data to be parsed according to the order of each of the first data fields and the attribute information of each of the first data fields comprises:

4. The method of claim 3, after determining whether the current first data field points to the second structure, further comprising:

5. The method according to claim 4, wherein the sequentially parsing the remaining data of the data to be parsed according to the order of each of the second data fields and the attribute information of each of the second data fields comprises:

6. The method of claim 1, prior to parsing the data to be parsed, further comprising:

7. The method of claim 6, after parsing the data to be parsed, further comprising:

8. The method of claim 7, after storing the parsed data and the raw data according to a preset storage path, further comprising:

9. A data analysis device, comprising:

10. An electronic device, characterized in that the electronic device further comprises:

one or more processors;

a storage device for storing one or more programs,

when executed by the one or more processors, cause the one or more processors to implement a data parsing method as recited in any of claims 1-8.

11. A computer-readable medium, on which a computer program is stored which, when being executed by a processor, carries out a data parsing method according to any one of claims 1-8.