CN115118793B - BLF file analysis fault tolerance method and device and computer equipment - Google Patents
BLF file analysis fault tolerance method and device and computer equipment Download PDFInfo
- Publication number
- CN115118793B CN115118793B CN202210665305.9A CN202210665305A CN115118793B CN 115118793 B CN115118793 B CN 115118793B CN 202210665305 A CN202210665305 A CN 202210665305A CN 115118793 B CN115118793 B CN 115118793B
- Authority
- CN
- China
- Prior art keywords
- message
- header
- data segment
- length
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/04—Protocols for data compression, e.g. ROHC
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/22—Parsing or analysis of headers
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention provides a BLF file analysis fault-tolerant method, a BLF file analysis fault-tolerant device and computer equipment, wherein the BLF file analysis fault-tolerant method comprises the following steps: when the file header of the BLF file is resolved abnormally, continuously searching a start mark of the data segment header so as to continuously resolve the BLF file from the data segment header; when the decompression of the compressed data segment is abnormal, discarding the compressed data recorded in the data segment header of the current data segment by the analysis program, and searching for a start mark of the data segment header of the next data segment; calculating the length of the current data segment after data compression according to the starting position of the data segment head of the next data segment, intercepting compressed data and decompressing; and in a message analysis stage after data decompression, reading the message length in the current message header, judging whether the message length is an effective length, if not, actively searching the starting mark of the next message header backwards, calculating the effective length of the current message according to the starting position of the next message header, and intercepting the current message according to the effective length for analysis.
Description
Technical Field
The invention relates to the technical field of computer application, in particular to a BLF file analysis fault-tolerant method, a BLF file analysis fault-tolerant device and computer equipment.
Background
With the gradual transition of control of various systems of the automobile to automation and intellectualization, the automobile electrical system is gradually complicated, and the automobile bus technology is generated. The automobile bus communication data is recorded in the log file so as to restore the scene and further analyze and position the problem, wherein the restored actual scene plays a vital role in problem positioning and analysis.
The BLF (Binary Logging Format) log file is widely applied to the recording of automobile bus communication data due to small occupied space. The method is characterized in that a professional tool is required to be used for file analysis for reading message information in the BLF file, the current analysis mechanism is used for carrying out data analysis according to a standard BLF file format, but problems such as incomplete file, data dislocation and data error may exist in the BLF file generated in practical application, if the problematic log file is analyzed according to the standard BLF file format, the analysis program is stopped in advance, partial messages cannot be analyzed, and practical scene restoration and problem analysis positioning cannot be carried out.
Disclosure of Invention
The present disclosure provides a BLF file parsing fault tolerance method, device and computer equipment, which are used for overcoming at least one technical problem existing in the prior art.
According to a first aspect, according to an embodiment of the present disclosure, there is provided a BLF file parsing fault tolerance method, where the BLF file includes a file header, a data segment header, and a compressed data segment, the data segment header is provided with a start flag indicating a start position of the data segment header, decompressed data includes a header and a message, a message type and a message length are recorded in the header, and the header is provided with a start flag indicating the start position of the header, and the method includes:
when the file header of the BLF file is in analysis abnormality, continuously searching for a start mark of the data segment header so as to continuously analyze the BLF file from the data segment header;
when the decompression of the compressed data segment is abnormal, discarding the compressed data recorded in the data segment header of the current data segment by the analysis program, and searching for a start mark of the data segment header of the next data segment; the compressed data comprises a length before data compression and a length after data compression;
calculating the length of the current data segment after data compression according to the starting position of the data segment head of the next data segment; intercepting compressed data and decompressing according to the compressed length of the current data segment so as to restore the data before compression;
and in a message analysis stage after data decompression, reading the message length in the current message header, judging whether the message length in the current message header is an effective length, if not, actively searching a starting mark of a next message header backwards, calculating the effective length of the current message according to the starting position of the next message header, and then intercepting the current message according to the effective length of the current message for analysis.
Optionally, the judging whether the header parsing of the BLF file is abnormal includes:
judging whether the BLF file has a file header or not;
if the BLF file does not have the file header, judging that the file header of the BLF file is abnormal in analysis;
if the BLF file has a file header, judging whether the message number of the file header is more than 0;
if the number of the messages is greater than 0, entering a decompression stage of a compressed data segment;
and if the number of the messages is not more than 0, judging that the file header of the BLF file is abnormal in analysis.
Optionally, the compressed data segment decompression stage further includes:
continuing to read the BLF file, and judging whether the file is the tail part of the BLF file;
if the judgment result is yes, the analysis is finished;
if the judgment result is negative, the compressed data in the data segment head of the current data segment is read;
analyzing the length before data compression of the current data segment according to the compressed data in the data segment header of the current data segment;
decompressing the current data segment by using a decompression algorithm, and restoring the data before the current data segment is compressed according to the length before the data of the current data segment is compressed;
when the decompression of the current data segment is successful, continuing to read the BLF file, and judging whether the current data segment is the tail part of the data segment;
if yes, repeating the process of the decompression stage of the compressed data segment;
if the judgment result is negative, entering the message analysis stage.
Further optionally, intercepting compressed data and decompressing according to the compressed length of the current data segment to restore data before compression specifically includes:
intercepting compressed data according to the length of the current data segment after data compression;
decompressing the intercepted compressed data, detecting the overflow risk of the array in real time, and dynamically expanding the array according to the decompression requirement of the intercepted compressed data.
Optionally, the message parsing stage further includes:
if the message length in the current message header is the effective length, intercepting a byte array with a corresponding length according to the message length of the current message, and analyzing the message by using a message analysis method;
and after the message analysis is successful, outputting a message.
Optionally, calculating the effective length of the current message according to the starting position of the next message header, and then intercepting the current message according to the effective length of the current message to analyze specifically includes:
calculating the effective length of the current message according to the starting position of the next message head;
intercepting a byte array with a corresponding length according to the effective length of the current message, and analyzing the intercepted byte array by using a message analysis method;
and after the message analysis is successful, outputting a message.
Further optionally, the determining whether the message length in the current message header is an effective length includes:
acquiring a preset message length, and matching the message length in the current message header with the preset message length; the preset message length comprises a message length allowed by any known message type;
if the matching is successful, judging that the message length in the current message header is an effective length;
if the matching fails, judging that the message length in the current message header is not the effective length.
Still further optionally, the obtaining a preset message length, and matching the message length in the current message header with the preset message length specifically includes:
reading the message type in the current message header;
acquiring the message length allowed by the message type according to the message type;
calculating the difference value between the message length in the current message header and the current message header length to obtain the data segment length;
and comparing the length of the data segment with the length of the message allowed by the message type.
According to a second aspect, according to an embodiment of the present disclosure, there is further provided a BLF file parsing fault-tolerant device, where the BLF file includes a file header, a data segment header, and a compressed data segment, the data segment header is provided with a start flag indicating a start position of the data segment header, decompressed data includes a header and a message, a message type and a message length are recorded in the header, and the header is provided with a start flag indicating the start position of the header, and the device includes:
the file header fault-tolerant module is used for continuously searching for a start mark of the data segment header when the file header of the BLF file is in analysis abnormality so as to continuously analyze the BLF file from the data segment header;
the data segment header fault-tolerant module is used for discarding the compressed data recorded in the data segment header of the current data segment by the analysis program when the decompression of the compressed data segment is abnormal, searching a start mark of the data segment header of the next data segment, calculating the compressed length of the current data segment according to the start position of the data segment header of the next data segment, and then intercepting the compressed data and decompressing according to the compressed length of the current data segment to restore the data before the compression; the compressed data comprises a length before data compression and a length after data compression;
and the message header fault-tolerant module is used for reading the message length in the current message header in a message analysis stage after data decompression, judging whether the message length in the current message header is an effective length, if not, actively searching the starting mark of the next message header backwards, calculating the effective length of the current message according to the starting position of the next message header, and then intercepting the current message according to the effective length of the current message for analysis.
In a third aspect, according to embodiments of the present specification, there is also provided a computer device comprising a memory storing a computer program and a processor implementing the steps of the method of the first aspect when the processor executes the computer program.
In a fourth aspect, according to an embodiment of the present specification, there is also provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method of the first aspect.
The beneficial effects of the embodiment of the specification are as follows:
aiming at three most common errors in the BLF file analysis process, three corresponding fault-tolerant mechanisms are provided, namely a file header fault-tolerant mechanism, a data segment header fault-tolerant mechanism and a message header fault-tolerant mechanism, and by utilizing the three fault-tolerant mechanisms, a BLF file analysis tool can analyze messages in BLF log files with some format errors to the maximum extent, so that a communication scene is restored for problem analysis and positioning.
Drawings
In order to more clearly illustrate the embodiments of the present description or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of the overall structure of a BLF file;
FIG. 2 is a schematic diagram of a BLF header;
FIG. 3 is a schematic diagram of a header structure of a data segment;
FIG. 4 is a schematic diagram of the structure of the compressed data after decompression;
FIG. 5 is a schematic diagram of a message header;
fig. 6 is a schematic diagram of an analysis flow chart of a BLF file analysis fault tolerance method according to an embodiment of the present disclosure;
FIG. 7 is a schematic diagram of a segment header structure of a data segment of scene 3;
fig. 8 is a schematic diagram of the structure of the header of scenario 4.
Detailed Description
The technical solutions of the embodiments of the present specification will be clearly and completely described below with reference to the drawings of the embodiments of the present specification, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without any inventive effort, are intended to be within the scope of the invention.
It should be noted that the terms "comprising" and "having" and any variations thereof in the embodiments and figures herein are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus.
To better illustrate the BLF file parsing fault tolerance method provided in the embodiments of the present disclosure, the following description describes the standard format of the BLF file:
as shown in fig. 1, the BLF file includes a header, a data segment header, a compressed data segment, and a stuff byte, and is a binary file composed of the header, the data segment header, the compressed data segment, and the stuff byte, and other parts are repeated except for the header, wherein the data segment header is provided with a start flag indicating a start position of the data segment header. As shown in fig. 2, the header of the file is composed of 144 bytes, and the character string corresponding to the first 4 bytes is "log", indicating the start of the file. As shown in fig. 3, the header of the data segment is composed of 32 bytes, the character string corresponding to the first 4 bytes is "LOBG", which indicates the beginning of the data segment header, 9-12 bytes store an integer indicating the total byte length of the data segment including the header, and the last 8 bytes store an integer indicating the decompressed byte length of the data segment following the data segment header. The stuff bytes have no real data and can be ignored.
As shown in fig. 4, the decompressed data of the compressed data segment includes a header, a message and a stuff byte, and the decompressed data of the compressed data segment is composed of the header, the message and the stuff byte, wherein the header, the message and the stuff byte are repeatedly arranged. As shown in fig. 5, the message header records information such as the message type, the message length, the time stamp, and the like, and is provided with a start mark indicating the start position of the message header. Message information is stored in the message bytes. Also, there is no real data in the stuff bytes, which is negligible.
The embodiment of the specification discloses a BLF file analysis fault-tolerant method, which comprises a file header fault-tolerant mechanism, a data segment header fault-tolerant mechanism and a message header fault-tolerant mechanism, wherein the three fault-tolerant mechanisms can analyze messages in BLF log files with some format errors to the maximum extent aiming at various common error scenes, and restore the message communication scenes to analyze and position the problems. The following will describe in detail.
File header fault tolerance mechanism: when the file header of the BLF file is in analysis abnormality, performing first fault-tolerant processing on the file header, namely continuously searching for a start mark of the data segment header so as to continuously analyze the BLF file from the data segment header.
Specifically, the first 144 bytes of the standard BLF file are a header, and the header records information such as the total number of messages and the total number of bytes of the file. In the prior art, the data in the header of the BLF file generated in actual work may be wrong or even have no header, for example, some BLF log generating tools write the header first, set the information such as the total number of messages and the total number of bytes to 0, then append the message, and when writing is finished, change the information such as the total number of messages and the total number of bytes in the header according to the statistical data, but when the fields are forgotten to be updated due to program defects, read the header according to the standard parsing flow, parse the header, find that the total number of messages and the total number of bytes are both 0, and consider that the file has no message and exits from file parsing.
In view of the above problems, the BLF file parsing fault-tolerant method in the embodiment of the present disclosure is designed with a header fault-tolerant mechanism, and even if the file does not include a header or the total number of messages, bytes, etc. is 0, the file can be continuously read, and a start flag (i.e., a flag "LOBG") of the header of the data segment is found, so that the file is continuously parsed from the header of the data segment.
Data segment header fault tolerance mechanism: when the decompression of the compressed data segment is abnormal, performing a second fault-tolerant process on the data segment header of the data segment, namely discarding the compressed data recorded in the data segment header of the current data segment by the analysis program, and searching for a start mark of the data segment header of the next data segment; calculating the length of the current data segment after data compression according to the starting position of the data segment head of the next data segment; and intercepting the compressed data according to the length of the compressed data of the current data segment and decompressing the compressed data to restore the data before compression. The compressed data comprises a length before data compression and a length after data compression.
In an embodiment, compressed data is intercepted according to the length of the compressed data of the current data segment, then the intercepted compressed data is decompressed, the risk of array overflow is detected in real time, and the dynamic expansion of the array is required according to the decompression of the intercepted compressed data.
Specifically, compressed data, that is, a length before data compression and a length after data compression, is recorded in the header of the data segment. If the data in the header of the data segment is correct, the data in the compressed data segment can be parsed according to the standard parsing flow, but if the data in the header of the data segment is wrong, the parsing procedure may be abnormal and stop in advance. For example, an error of array index overflow occurs in the decompression process of compressed data due to incorrect length of data before compression in the header of the data segment caused by abnormal writing procedure of the BLF file.
In view of the above problems, the BLF file parsing fault tolerance method in the embodiments of the present disclosure is designed with a data segment header fault tolerance mechanism. When an abnormality occurs in the decompression process of a compressed data segment, the analysis program discards two data of the length before data compression and the length after data compression recorded in the data segment header, then actively searches the starting position of the next data segment header backwards, thereby calculating the real length after data compression of the segment data, then intercepting the real compressed data and decompressing the compressed data, simultaneously detecting the array overflow risk in real time during decompression, dynamically expanding the array according to the decompression requirement, and avoiding the array overflow so as to restore the data before compression.
Message header fault tolerance mechanism: and in a message analysis stage after data decompression, reading the message length in the current message header, judging whether the message length in the current message header is an effective length, if not, performing third fault-tolerant processing on the current message header, namely actively searching a starting mark of the next message header backwards, calculating the effective length of the current message according to the starting position of the next message header, and then intercepting the current message according to the effective length of the current message for analysis.
Specifically, the message header records information such as message type, timestamp and the like, and also records the byte length of the message, namely the message length, and the character string corresponding to the first 4 bytes of the message header is LOBG, which represents the start mark of the initial position of the message header. The normal parsing program can judge whether the initial position of the message header is the initial position according to the LOBG start mark, and intercept the message byte according to the message length information recorded in the message header to parse, so that when the message length data is wrong, the message parsing error can be caused, and the correct parsing of the next message can be influenced.
In order to solve the above problem, the BLF file parsing fault tolerance method in the embodiment of the present disclosure is designed with a header fault tolerance mechanism. When the message length field in the message header is read, it is first determined whether the length field is a valid length, and the rule is to check whether the length matches the length allowed by any known message type. If the message is not the effective length data, actively searching the starting position of the next message head backwards, thus calculating the effective length of the message, and then intercepting out the real message data for analysis.
The following is a detailed analysis of the analysis flow of the BLF file analysis fault tolerance method provided in the embodiment of the present disclosure with reference to fig. 6.
When the BLF file analysis is started, firstly, the file header analysis is carried out, whether the file header analysis of the BLF file is abnormal or not is judged, specifically, whether the BLF file has a file header or not is firstly judged, and if the BLF file does not have the file header, the file header analysis of the BLF file is judged to be abnormal; if the BLF file has a file header, continuing to judge whether the message number is larger than 0, if the message number is larger than 0, entering a decompression stage of the compressed data segment, and if the message number is not larger than 0, judging that the file header of the BLF file is abnormal in analysis. When the file header of the BLF file is analyzed abnormally, performing first fault-tolerant processing on the file header, and in detail, continuously reading the BLF file, namely continuously searching a start mark of the data segment header, continuously analyzing the BLF file from the data segment header, and entering a decompression stage of the compressed data segment.
After entering a decompression stage of the compressed data segment, continuing to read the BLF file, judging whether the BLF file is the file tail of the BLF file, and if so, ending the analysis; if the judgment result is negative, the compressed data in the data segment head of the current data segment is read, the pre-compression length of the data of the current data segment is analyzed according to the compressed data in the data segment head of the current data segment, then the current data segment is decompressed by a decompression algorithm, and the pre-compression data of the current data segment is restored according to the pre-compression length of the data of the current data segment. If the current data segment is successfully decompressed, the BLF file is continuously read, whether the BLF file is the tail of the data segment is judged, if the BLF file is the tail of the data segment, the process of the decompression stage of the compressed data segment is repeated, and if the BLF file is not the tail of the data segment, the BLF file is entered into the message analysis stage; if decompression of the current data segment fails, namely decompression of the compressed data segment of the BLF file is abnormal, second fault-tolerant processing is carried out on the data segment header of the data segment, the analysis program discards compressed data recorded in the data segment header of the current data segment (namely, two data of the length before data compression and the length after data compression), a starting mark of the data segment header of the next data segment is searched, so that the starting position of the data segment header of the next data segment is determined, the length after data compression of the current data segment is calculated according to the starting position of the data segment header of the next data segment, compressed data is intercepted and decompressed according to the length after data compression of the current data segment, so that data before compression is restored, meanwhile, the overflow risk of an array is detected in real time, and the array is dynamically expanded according to the decompression requirement of the compressed data, so that data before data is restored.
When the compressed data is decompressed, the parsing process enters the message parsing stage.
And in a message analysis stage after data decompression, reading a current message header, analyzing the message length of the current message, and judging whether the message length of the current message is an effective length. If the judgment result is yes, intercepting a byte array with corresponding length according to the message length of the current message, analyzing the message by using a message analysis method, and outputting a message after the message analysis is successful. If the judgment result is negative, the third fault-tolerant processing is carried out on the current message header, the starting mark of the next message header is actively searched backwards in detail to determine the starting position of the next message header, the effective length of the current message is calculated according to the starting position of the next message header, then the byte array with the corresponding length is intercepted according to the effective length of the current message, the message analysis is carried out by utilizing the message analysis method, and a message is output after the message analysis is successful.
The BLF file parsing fault tolerance method is described below by way of example in connection with some specific scenarios.
Scene 1: the BLF file is not written with 144 bytes of the file header, but is written with message data.
Scene 2: the BLF file is written with a header, but the value of the "number of messages" field in the header is 0, and the message data is written.
Under a normal parsing mechanism, both of the above scenarios may lead to early termination of the parsing process. For scenario 1, the parser exits parsing because the string corresponding to the first 4 bytes of the BLF file is not "log," but the input file is not considered to conform to the BLF file format. For scenario 2, the parser will consider that the BLF file does not include a message and exit parsing.
After the BLF file parsing fault-tolerant method in the embodiment of the present disclosure adds the header fault-tolerant mechanism, in both the above two scenarios, the method may enter the message parsing stage to parse the message in the file, and will not end the parsing procedure in advance.
Scene 3: as shown in fig. 7, if there are 20 pieces of message data for a total of 1024 bytes, the compressed length is 128 bytes. Because of the program logic error when writing the BLF file, the length of the compressed data after decompression in the header of the compressed data segment is wrongly written as the length of the compressed data.
Under a normal parsing mechanism, the parsing program will parse the header content of the data segment first, obtain the value of the length after decompression of the compressed data, then initialize an array with a length of 128 bytes according to the value to store the restored data, and then decompress the data segment according to the length of the compressed data to intercept the content with a corresponding length as the compressed data (128 bytes). However, when the byte length of the decompressed data exceeds 128 bytes, the array overflows, and the parsing program considers that the decompression is wrong, so that the parsing of the segment of data can be skipped.
After the BLF file parsing fault-tolerant method in the embodiment of the present disclosure adds a data segment header fault-tolerant mechanism, when a program captures an error of a decompressed program, the program starts the fault-tolerant mechanism, and actively searches and calculates a value of "length after decompression of compressed data", and then initializes an array to store restored data according to the value, an array overflow detection mechanism is added to the decompressed program of the fault-tolerant mechanism, when an impending array overflow error is detected, array expansion is performed, and after the decompressed program is executed, a decompressed result is returned to the main program to continue to be executed downwards.
Scene 4: as shown in fig. 8, assuming that 2 CAN messages are total in the data segment obtained after the compressed data is decompressed, since the length of each CAN message is 24 bytes, the messages in the decompressed data segment are arranged as follows: header 1 (32 bytes), header 1 (24 bytes), header 2 (32 bytes), header 2 (24 bytes). Assuming message 1 has a timestamp millisecond of 1334, program logic makes an error when writing the message to the BLF file, and the timestamp millisecond 1334 is written to the file as the message length. All data of message 2 are normal.
Under a normal analysis mechanism, an analysis program firstly reads a message length field in the message header 1, subtracts the message header length 32 from the message length, namely 1334-32=1302, to obtain the length 1302 of a data segment, and then tries to intercept data of 1302 bytes behind the message header 1 as the data segment of the CAN message to analyze the message, in which case an analysis error is generated because the data length behind the message header 1 is smaller than 1302, so that the analysis program exits the analysis of the data segment.
After adding a message header fault tolerance mechanism to the BLF file parsing fault tolerance method in the embodiment of the present disclosure, the parsing program first reads a message length field in the message header 1, subtracts the message header length 32 from the message length, i.e., 1334-32=1302, to obtain a length 1302 of a data segment, and then reads a message type field in the message header 1, where a value of 1 is read to indicate that the data is a CAN type message data. At this time, the fault tolerance mechanism will check whether the data segment length value 1302 is a valid value according to the message type, and since the length of the CAN message CAN only be 24 bytes, the check result is False, which indicates that 1302 is not a valid value. The parser then discards 1302 the invalid value, and searches back for the next start byte "LOBG" from the first byte after header 1, and if the first byte after header 1 is marked as the 1 st byte, then "LOBG" is found after 24 bytes, that is, the 25 th to 28 th bytes. Then the analysis program calculates the number of bytes between two message heads as 24, calculates the effective length of the message 1 as 24 according to the rule of the BLF supplementary bytes, then sends 24 bytes behind the message head 1 into the message analysis program to analyze the message 1, and then continues to analyze the next message. Thus, the parsing error caused by the data error of the message header is avoided.
The embodiment of the specification also discloses a BLF file analysis fault-tolerant device which comprises a file header fault-tolerant module, a data segment header fault-tolerant module and a message header fault-tolerant module.
The file header fault-tolerant module is used for continuously searching for a start mark of the data segment header when the file header of the BLF file is in analysis abnormality so as to continuously analyze the BLF file from the data segment header.
The data segment header fault-tolerant module is used for discarding the compressed data recorded in the data segment header of the current data segment by the analysis program when the decompression of the compressed data segment is abnormal, searching the start mark of the data segment header of the next data segment, calculating the length of the current data segment after the data compression according to the start position of the data segment header of the next data segment, and then intercepting the compressed data and decompressing according to the length of the current data segment after the data compression so as to restore the data before the compression.
The message header fault-tolerant module is used for reading the message length in the current message header in a message analysis stage after data decompression, judging whether the message length in the current message header is an effective length, if not, actively searching the starting mark of the next message header backwards, calculating the effective length of the current message according to the starting position of the next message header, and then intercepting the current message according to the effective length of the current message for analysis.
For specific limitations on the BLF file parsing fault tolerance apparatus, reference may be made to the above limitation on the BLF file parsing fault tolerance mechanism, and will not be repeated herein. The various modules in the BLF file parsing fault tolerance apparatus described above may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
The embodiment of the specification also discloses a computer device, which comprises a memory and a processor, wherein the memory stores a computer program, and the processor realizes the steps of the BLF file analysis fault-tolerant mechanism when executing the computer program.
The embodiments of the present specification also disclose a computer readable storage medium having stored thereon a computer program which when executed by a processor implements the steps of the BLF file parsing fault tolerance mechanism described above.
In summary, the present disclosure discloses a BLF file parsing fault-tolerant method, apparatus, computer device and storage medium, and the present disclosure proposes three corresponding fault-tolerant mechanisms, namely a header fault-tolerant mechanism, a data segment header fault-tolerant mechanism and a message header fault-tolerant mechanism, for three most common errors in a BLF file parsing process, by which a BLF file parsing tool can parse out messages in a BLF log file with some format errors to the maximum extent, thereby restoring a communication scene to perform problem analysis and positioning.
Those of ordinary skill in the art will appreciate that: the drawing is a schematic diagram of one embodiment and the modules or flows in the drawing are not necessarily required to practice the invention.
Those of ordinary skill in the art will appreciate that: the modules in the apparatus of the embodiments may be distributed in the apparatus of the embodiments according to the description of the embodiments, or may be located in one or more apparatuses different from the present embodiments with corresponding changes. The modules of the above embodiments may be combined into one module, or may be further split into a plurality of sub-modules.
Finally, it should be noted that: the above examples are only specific embodiments of the present invention, and are not intended to limit the scope of the present invention, but it should be understood by those skilled in the art that the present invention is not limited thereto, and that the present invention is described in detail with reference to the foregoing examples: any person skilled in the art may modify or easily conceive of the technical solution described in the foregoing embodiments, or perform equivalent substitution of some of the technical features, while remaining within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. The BLF file analysis fault-tolerant method is characterized in that the BLF file comprises a file header, a data segment header and a compressed data segment, wherein the data segment header is provided with a start mark for marking the starting position of the data segment header, the decompressed data of the compressed data segment comprises a message header and a message, the message header records a message type and a message length, and the message header is provided with a start mark for marking the starting position of the message header, and the method comprises the following steps:
when the file header of the BLF file is in analysis abnormality, continuously searching for a start mark of the data segment header so as to continuously analyze the BLF file from the data segment header;
when the decompression of the compressed data segment is abnormal, discarding the compressed data recorded in the data segment header of the current data segment by the analysis program, and searching for a start mark of the data segment header of the next data segment; the compressed data comprises a length before data compression and a length after data compression;
calculating the length of the current data segment after data compression according to the starting position of the data segment head of the next data segment; intercepting compressed data and decompressing according to the compressed length of the current data segment so as to restore the data before compression;
and in a message analysis stage after data decompression, reading the message length in the current message header, judging whether the message length in the current message header is an effective length, if not, actively searching a starting mark of a next message header backwards, calculating the effective length of the current message according to the starting position of the next message header and the starting position of the current message header, and then intercepting the current message according to the effective length of the current message for analysis.
2. The BLF file resolution fault tolerance method of claim 1, wherein the determining whether the header resolution of the BLF file is abnormal comprises:
judging whether the BLF file has a file header or not;
if the BLF file does not have the file header, judging that the file header of the BLF file is abnormal in analysis;
if the BLF file has a file header, judging whether the message number of the file header is more than 0;
if the number of the messages is greater than 0, entering a decompression stage of a compressed data segment;
and if the number of the messages is not more than 0, judging that the file header of the BLF file is abnormal in analysis.
3. The BLF file resolution fault tolerance method of claim 1, wherein the compressed data segment decompression stage further comprises:
continuing to read the BLF file, and judging whether the file is the tail part of the BLF file;
if the judgment result is yes, the analysis is finished;
if the judgment result is negative, the compressed data in the data segment head of the current data segment is read;
analyzing the length before data compression of the current data segment according to the compressed data in the data segment header of the current data segment;
decompressing the current data segment by using a decompression algorithm, and restoring the data before the current data segment is compressed according to the length before the data of the current data segment is compressed;
when the decompression of the current data segment is successful, continuing to read the BLF file, and judging whether the current data segment is the tail part of the data segment;
if yes, repeating the process of the decompression stage of the compressed data segment;
if the judgment result is negative, entering the message analysis stage.
4. The BLF file resolution fault tolerance method of claim 3, wherein intercepting compressed data and decompressing according to the compressed length of the current data segment to restore data before compression specifically comprises:
intercepting compressed data according to the length of the current data segment after data compression;
decompressing the intercepted compressed data, detecting the overflow risk of the array in real time, and dynamically expanding the array according to the decompression requirement of the intercepted compressed data.
5. The BLF file resolution fault tolerance method of claim 1, wherein the message resolution stage further comprises:
if the message length in the current message header is the effective length, intercepting a byte array with a corresponding length according to the message length of the current message, and analyzing the message by using a message analysis method;
and after the message analysis is successful, outputting a message.
6. The BLF file parsing fault tolerance method of claim 1, wherein calculating the effective length of the current message according to the starting position of the next header, and then intercepting the current message according to the effective length of the current message to parse the current message specifically comprises:
calculating the effective length of the current message according to the starting position of the next message head;
intercepting a byte array with a corresponding length according to the effective length of the current message, and analyzing the intercepted byte array by using a message analysis method;
and after the message analysis is successful, outputting a message.
7. The BLF file resolution fault tolerance method of claim 6, wherein the determining whether the message length in the current header is an effective length comprises:
acquiring a preset message length, and matching the message length in the current message header with the preset message length; the preset message length comprises a message length allowed by any known message type;
if the matching is successful, judging that the message length in the current message header is an effective length;
if the matching fails, judging that the message length in the current message header is not the effective length.
8. The BLF file resolution fault tolerance method of claim 7, wherein the obtaining a preset message length, and matching the message length in the current header with the preset message length specifically comprises:
reading the message type in the current message header;
acquiring the message length allowed by the message type according to the message type;
calculating the difference value between the message length in the current message header and the current message header length to obtain the data segment length;
and comparing the length of the data segment with the length of the message allowed by the message type.
9. The utility model provides a BLF file analysis fault-tolerant device which characterized in that, BLF file includes file head, data section head, compressed data section, data section head is provided with the start mark that marks the starting position of data section head, the data after the compressed data section decompression includes message head, message, recorded message type, message length in the message head, the message head is provided with the start mark that marks the starting position of message head, the device includes:
the file header fault-tolerant module is used for continuously searching for a start mark of the data segment header when the file header of the BLF file is in analysis abnormality so as to continuously analyze the BLF file from the data segment header;
the data segment header fault-tolerant module is used for discarding the compressed data recorded in the data segment header of the current data segment by the analysis program when the decompression of the compressed data segment is abnormal, searching a start mark of the data segment header of the next data segment, calculating the compressed length of the current data segment according to the start position of the data segment header of the next data segment, and then intercepting the compressed data and decompressing according to the compressed length of the current data segment to restore the data before the compression; the compressed data comprises a length before data compression and a length after data compression;
and the message header fault-tolerant module is used for reading the message length in the current message header in a message analysis stage after data decompression, judging whether the message length in the current message header is an effective length, if not, actively searching the starting mark of the next message header backwards, calculating the effective length of the current message according to the starting position of the next message header and the starting position of the current message header, and then intercepting the current message according to the effective length of the current message for analysis.
10. A computer device comprising a memory storing a computer program and a processor implementing the steps of the method of any of claims 1-8 when the computer program is executed by the processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210665305.9A CN115118793B (en) | 2022-06-14 | 2022-06-14 | BLF file analysis fault tolerance method and device and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210665305.9A CN115118793B (en) | 2022-06-14 | 2022-06-14 | BLF file analysis fault tolerance method and device and computer equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115118793A CN115118793A (en) | 2022-09-27 |
| CN115118793B true CN115118793B (en) | 2023-07-07 |
Family
ID=83329462
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210665305.9A Active CN115118793B (en) | 2022-06-14 | 2022-06-14 | BLF file analysis fault tolerance method and device and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115118793B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102708155A (en) * | 2012-04-20 | 2012-10-03 | 西安电子科技大学 | JSAX (joint simple API (application program interface) for XML (extensible markup language)) parser and parsing method based on syntactic analysis of backtracking automaton |
| CN103188104A (en) * | 2011-12-31 | 2013-07-03 | 中国移动通信集团浙江有限公司 | Method and device for analyzing user behaviors |
| CN107832415A (en) * | 2017-11-07 | 2018-03-23 | 广东电网有限责任公司电力调度控制中心 | A kind of fast resolving method of COMTRADE binary data files |
| CN112214462A (en) * | 2020-10-22 | 2021-01-12 | 新华三信息安全技术有限公司 | Multi-layer decompression method of compressed file, electronic equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10360198B2 (en) * | 2016-01-13 | 2019-07-23 | American Express Travel Related Services Company, Inc. | Systems and methods for processing binary mainframe data files in a big data environment |
-
2022
- 2022-06-14 CN CN202210665305.9A patent/CN115118793B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103188104A (en) * | 2011-12-31 | 2013-07-03 | 中国移动通信集团浙江有限公司 | Method and device for analyzing user behaviors |
| CN102708155A (en) * | 2012-04-20 | 2012-10-03 | 西安电子科技大学 | JSAX (joint simple API (application program interface) for XML (extensible markup language)) parser and parsing method based on syntactic analysis of backtracking automaton |
| CN107832415A (en) * | 2017-11-07 | 2018-03-23 | 广东电网有限责任公司电力调度控制中心 | A kind of fast resolving method of COMTRADE binary data files |
| CN112214462A (en) * | 2020-10-22 | 2021-01-12 | 新华三信息安全技术有限公司 | Multi-layer decompression method of compressed file, electronic equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| Hao Qiu ; Ziming Qi.A new motors fault tolerance control strategy to 4WID electric vehicle.《2015 6th International Conference on Automation, Robotics and Applications (ICARA)》.2015,全文. * |
| 基于CAN总线的车辆数据采集与远程监控系统研发;张海逢;《中国优秀硕士学位论文全文数据库》;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115118793A (en) | 2022-09-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110795287A (en) | Data recovery method, system, electronic device and computer storage medium | |
| CN109284331B (en) | Certificate making information acquisition method based on service data resources, terminal equipment and medium | |
| CN109918221B (en) | Hard disk error reporting analysis method, system, terminal and storage medium | |
| CN111966339B (en) | Buried point parameter input method and device, computer equipment and storage medium | |
| KR20070079033A (en) | Methods and systems for derivation of missing data objects from test data | |
| CN115118793B (en) | BLF file analysis fault tolerance method and device and computer equipment | |
| CN109582504A (en) | A kind of data reconstruction method and device for apple equipment | |
| CN110515792B (en) | Monitoring method and device based on web version task management platform and computer equipment | |
| CN111026604B (en) | Log file analysis method and device | |
| CN112181992A (en) | Method for realizing acquisition of oracle database incremental data | |
| CN113792017A (en) | Method and system for checking template content imported into excel | |
| CN105988917B (en) | Abnormal information acquisition method and device | |
| CN113448786A (en) | PCIe equipment testing method, device, equipment and readable storage medium | |
| CN112527557A (en) | Data recovery method, device and equipment and computer readable storage medium | |
| CN109560964B (en) | Equipment compliance checking method and device | |
| CN113037521A (en) | Method for identifying state of communication equipment, communication system and storage medium | |
| CN108415814B (en) | Method for automatically recording field change, application server and computer readable storage medium | |
| US8775528B2 (en) | Computer readable recording medium storing linking keyword automatically extracting program, linking keyword automatically extracting method and apparatus | |
| CN111552959B (en) | Program feature sequence generation method and device | |
| CN116933143B (en) | Flight parameter data classification method | |
| CN115952172B (en) | Data matching method and device based on database temporary table | |
| CN115827404A (en) | Log generation method and device, electronic equipment and storage medium | |
| JP2005267036A (en) | Trace output control method and trace system | |
| CN113127317B (en) | Log acquisition processing method, system, device and storage medium | |
| CN118227438A (en) | Information prompting method, device, equipment and medium for abnormal stack |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |