CN115118793A - BLF file parsing fault-tolerant method and device and computer equipment - Google Patents
BLF file parsing fault-tolerant method and device and computer equipment Download PDFInfo
- Publication number
- CN115118793A CN115118793A CN202210665305.9A CN202210665305A CN115118793A CN 115118793 A CN115118793 A CN 115118793A CN 202210665305 A CN202210665305 A CN 202210665305A CN 115118793 A CN115118793 A CN 115118793A
- Authority
- CN
- China
- Prior art keywords
- message
- header
- data segment
- length
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/04—Protocols for data compression, e.g. ROHC
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/22—Parsing or analysis of headers
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention provides a BLF file analysis fault-tolerant method, a device and computer equipment, wherein the method comprises the following steps: when the analysis of the file header of the BLF file is abnormal, continuously searching the start mark of the data segment header so as to continuously analyze the BLF file from the data segment header; when the decompression of the compressed data segment is abnormal, the analysis program abandons the compressed data recorded in the data segment head of the current data segment and searches for the start mark of the data segment head of the next data segment; calculating the length of the current data segment after data compression according to the initial position of the data segment head of the next data segment, intercepting and decompressing the compressed data; reading the message length in the current message header at the message analysis stage after data decompression, judging whether the message length is the effective length, if not, actively searching the starting mark of the next message header backwards, calculating the effective length of the current message according to the initial position of the next message header, and intercepting the current message according to the effective length for analysis.
Description
Technical Field
The invention relates to the technical field of computer application, in particular to a BLF file analysis fault-tolerant method, a BLF file analysis fault-tolerant device and computer equipment.
Background
With the gradual change of the control of each system of the automobile to automation and intellectualization, the electrical system of the automobile is gradually complicated, and the bus technology of the automobile comes along. The automobile bus communication data are recorded in the log file so as to restore scenes and further perform problem analysis and positioning, wherein the restored actual scenes play a vital role in problem positioning and analysis.
The log file in the blf (binary Logging format) format is widely applied to recording of automobile bus communication data because of small occupied space. Reading message information in a BLF file requires file analysis by using a professional tool, most of the current analysis mechanisms strictly analyze data according to a standard BLF file format, but the generated BLF file in practical application has the problems of incomplete file, data dislocation, data error and the like, and if the log file with the problems is analyzed strictly according to the standard BLF file format, the analysis program is stopped in advance, part of messages cannot be analyzed, and further, the actual scene reduction and problem analysis positioning cannot be performed.
Disclosure of Invention
The present specification provides a fault-tolerant method, apparatus and computer device for BLF file parsing, which are used to overcome at least one technical problem in the prior art.
In a first aspect, according to an embodiment of the present specification, a BLF file parsing fault-tolerant method is provided, where the BLF file includes a file header, a data segment header, and a compressed data segment, where the data segment header is provided with a start mark indicating an initial position of the data segment header, data after decompression of the compressed data segment includes a packet header and a packet, where the packet header records a packet type and a packet length, and the packet header is provided with a start mark indicating an initial position of the packet header, and the method includes:
when the analysis of the file header of the BLF file is abnormal, continuously searching the starting mark of the data segment header so as to continuously analyze the BLF file from the data segment header;
when the decompression of the compressed data segment is abnormal, the analysis program abandons the compressed data recorded in the data segment head of the current data segment and searches for the start mark of the data segment head of the next data segment; the compressed data comprises a length before data compression and a length after data compression;
calculating the length of the current data segment after data compression according to the initial position of the data segment head of the next data segment; intercepting and decompressing the compressed data according to the data compression length of the current data segment to restore the data before compression;
reading the message length in the current message header at the message analysis stage after data decompression, judging whether the message length in the current message header is the effective length, if not, actively searching the starting mark of the next message header backwards, calculating the effective length of the current message according to the initial position of the next message header, and then intercepting the current message according to the effective length of the current message for analysis.
Optionally, the determining whether the file header parsing of the BLF file is abnormal includes:
judging whether the BLF file has a file header;
if the BLF file does not have a file header, judging that the file header of the BLF file is analyzed abnormally;
if the BLF file has a file header, judging whether the number of messages of the file header is greater than 0;
if the number of the messages is more than 0, entering a compressed data segment decompression stage;
and if the number of the messages is not more than 0, judging that the analysis of the file header of the BLF file is abnormal.
Optionally, the compressed data segment decompression stage further includes:
continuously reading the BLF file, and judging whether the BLF file is the file tail of the BLF file;
if the judgment result is yes, the analysis is finished;
if the judgment result is negative, reading the compressed data in the data segment head of the current data segment;
analyzing the length of the current data segment before data compression according to the compressed data in the data segment header of the current data segment;
decompressing the current data segment by using a decompression algorithm, and restoring the data of the current data segment before compression according to the length of the current data segment before data compression;
when the current data segment is successfully decompressed, the BLF file is continuously read, and whether the current data segment is the tail of the data segment is judged;
if yes, repeating the process of the compressed data segment decompression stage;
if the judgment result is negative, entering the message analysis stage.
Further optionally, the intercepting and decompressing the compressed data according to the length of the current data segment after data compression to restore the data before compression specifically includes:
intercepting compressed data according to the length of the compressed data of the current data segment;
decompressing the intercepted compressed data, detecting the risk of array overflow in real time, and dynamically expanding the array according to the decompression requirement of the intercepted compressed data.
Optionally, the message parsing stage further includes:
if the message length in the current message header is the effective length, intercepting a byte array with the corresponding length according to the message length of the current message, and analyzing the message by using a message analyzing method;
and outputting a message after the message is successfully analyzed.
Optionally, the calculating the effective length of the current packet according to the starting position of the next packet header, and then intercepting the current packet according to the effective length of the current packet to perform parsing specifically includes:
calculating the effective length of the current message according to the initial position of the next message header;
intercepting the byte array with the corresponding length according to the effective length of the current message, and analyzing the intercepted byte array by using a message analysis method;
and outputting a message after the message is successfully analyzed.
Further optionally, the determining whether the packet length in the current packet header is the effective length includes:
acquiring a preset message length, and matching the message length in the current message header with the preset message length; the preset message length comprises the message length allowed by any known message type;
if the matching is successful, judging the message length in the current message header as the effective length;
if the matching fails, judging that the message length in the current message header is not the effective length.
Further optionally, the obtaining of the preset message length and the matching of the message length in the current message header and the preset message length specifically include:
reading the message type in the current message header;
acquiring the message length allowed by the message type according to the message type;
calculating the difference value between the message length in the current message header and the current message header length to obtain the length of the data segment;
and comparing the length of the data segment with the message length allowed by the message type.
In a second aspect, according to an embodiment of the present specification, there is further provided a BLF file parsing fault-tolerant apparatus, where the BLF file includes a file header, a data segment header, and a compressed data segment, the data segment header is provided with a start mark indicating a start position of the data segment header, data after decompression of the compressed data segment includes a packet header and a packet, a packet type and a packet length are recorded in the packet header, and the packet header is provided with a start mark indicating the start position of the packet header, the apparatus includes:
a file header fault-tolerant module, configured to, when a file header of the BLF file is analyzed abnormally, continue to search for a start marker of the data segment header, so as to continue to analyze the BLF file from the data segment header;
a data segment header fault-tolerant module, configured to discard, by an analysis program, compressed data recorded in a data segment header of a current data segment when decompression of the compressed data segment is abnormal, search for a start marker of a data segment header of a next data segment, calculate a length of the current data segment after data compression according to an initial position of the data segment header of the next data segment, and then intercept and decompress the compressed data according to the length of the current data segment after data compression to restore data before compression; the compressed data comprises a length before data compression and a length after data compression;
and the message header fault-tolerant module is used for reading the message length in the current message header in a message analysis stage after data decompression, judging whether the message length in the current message header is the effective length, actively searching the starting mark of the next message header backwards if the message length in the current message header is the effective length, calculating the effective length of the current message according to the initial position of the next message header, and intercepting and analyzing the current message according to the effective length of the current message.
In a third aspect, according to an embodiment of the present specification, there is also provided a computer device, including a memory and a processor, where the memory stores a computer program, and the processor implements the steps of the method according to the first aspect when executing the computer program.
In a fourth aspect, according to an embodiment of the present specification, there is further provided a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps of the method of the first aspect.
The beneficial effects of the embodiment of the specification are as follows:
aiming at three most common errors in the BLF file analysis process, three corresponding fault-tolerant mechanisms are provided, namely a file header fault-tolerant mechanism, a data segment header fault-tolerant mechanism and a message header fault-tolerant mechanism, and by utilizing the three fault-tolerant mechanisms, a BLF file analysis tool can analyze messages in BLF log files with certain format errors to the maximum extent, so that a communication scene is restored for problem analysis and positioning.
Drawings
In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present disclosure, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic diagram of the general structure of a BLF file;
fig. 2 is a schematic diagram of a BLF file header;
FIG. 3 is a block diagram of a data block;
FIG. 4 is a schematic diagram of a decompressed compressed data structure;
FIG. 5 is a schematic diagram of a structure of a header;
fig. 6 is a schematic parsing flow diagram of a fault-tolerant method for parsing a BLF file according to an embodiment of the present disclosure;
FIG. 7 is a schematic diagram of a segment header structure of a data segment of scene 3;
fig. 8 is a schematic structural diagram of a header in scenario 4.
Detailed Description
The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without inventive effort based on the embodiments of the present invention, are within the scope of the present invention.
It should be noted that the terms "including" and "having" and any variations thereof in the embodiments of the present specification and the drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.
To better illustrate the BLF file parsing fault-tolerant method provided in the embodiments of the present specification, the following describes a standard format of a BLF file:
as shown in fig. 1, the BLF file includes a file header, a data segment header, a compressed data segment, and stuff bytes, and the BLF file is a binary file composed of a file header, a data segment header, a compressed data segment, and stuff bytes, and the other parts except the file header are repeated, wherein the data segment header is provided with a start mark indicating a start position of the data segment header. As shown in FIG. 2, the header of the file consists of 144 bytes, and the first 4 bytes correspond to a character string "LOGG" indicating the beginning of the file. As shown in fig. 3, the segment header of the data segment is composed of 32 bytes, the character string corresponding to the first 4 bytes is "LOBG" indicating the beginning of the data segment header, 9-12 bytes store an integer indicating the total byte length of the data segment including the segment header, and the last 8 bytes store an integer indicating the byte length of the data segment following the data segment header after decompression. The stuffing bytes have no real data and can be ignored.
As shown in fig. 4, the decompressed data of the compressed data segment includes a message header, a message and stuff bytes, and the decompressed data of the compressed data segment includes the message header, the message and the stuff bytes, wherein the message header, the message and the stuff bytes are arranged repeatedly. As shown in fig. 5, information such as a packet type, a packet length, and a timestamp is recorded in a packet header, and the packet header is provided with a start tag that marks an initial position of the packet header. The message bytes store message information. Likewise, there is no real data in the stuff bytes and it can be ignored.
The embodiment of the specification discloses a BLF file analysis fault-tolerant method which comprises a file header fault-tolerant mechanism, a data segment header fault-tolerant mechanism and a message header fault-tolerant mechanism, wherein the three fault-tolerant mechanisms can be used for analyzing messages in BLF log files with certain format errors to the maximum extent aiming at various common error scenes, and reducing message communication scenes so as to analyze and locate problems. The following are detailed descriptions.
File header fault tolerance mechanism: when the analysis of the file header of the BLF file is abnormal, the first fault-tolerant processing is carried out on the file header, namely, the starting mark of the data segment header is continuously searched, so that the BLF file is continuously analyzed from the data segment header.
Specifically, the first 144 bytes of a standard BLF file are a file header, and the file header records information such as the total number of messages and the total number of bytes of the file. In the prior art, data in a file header of a BLF file generated in actual work may be erroneous, or even have no file header, for example, some BLF log generation tools write a file header first, set information such as a total number of messages and a total number of bytes to 0, then add a message, and change information such as a total number of messages and a total number of bytes in the message header according to statistical data after the completion of writing, but when forgetting to update these fields due to a program defect, read the total number of the parsing messages and the total number of bytes of the file header according to a standard parsing procedure, find that both are 0, and then consider that there is no message in the file and quit file parsing.
In view of the above problems, the BLF file parsing fault tolerance method in this embodiment of the present disclosure is designed with a file header fault tolerance mechanism, so that even if a file does not include a file header or information such as a total number of messages and a total number of bytes is 0, the file can be continuously read, and a start marker (i.e., a flag "LOBG") of a data segment header is searched, so as to continuously parse the file from the data segment header.
Data segment header fault tolerance mechanism: when the decompression of the compressed data segment is abnormal, performing second fault-tolerant processing on the data segment head of the data segment, namely discarding the compressed data recorded in the data segment head of the current data segment by an analysis program, and searching for a start mark of the data segment head of the next data segment; calculating the length of the current data segment after data compression according to the initial position of the data segment head of the next data segment; and intercepting and decompressing the compressed data according to the length of the compressed data of the current data segment so as to restore the data before compression. The compressed data comprises a length before data compression and a length after data compression.
In an embodiment, the compressed data is intercepted according to the length of the compressed data of the current data segment, then the intercepted compressed data is decompressed, the risk of array overflow is detected in real time, and the array is dynamically expanded according to the decompression requirement of the intercepted compressed data.
Specifically, compressed data is recorded in the data segment header, that is, a length before data compression and a length after data compression are recorded. If the data in the data segment header is correct, the data in the compressed data segment can be analyzed according to a standard analysis flow, but if the data in the data segment header has errors, the analysis program may be abnormal and stopped in advance. For example, if the length of data in the header of the data segment before compression is incorrect due to the exception of the BLF file writer, an error of array index overflow during decompression of the compressed data may be caused.
In view of the above problems, the BLF file parsing fault tolerance method in the embodiment of the present specification is designed with a data segment header fault tolerance mechanism. When an abnormality occurs in the decompression process of a compressed data segment, an analysis program abandons two data of the length before data compression and the length after data compression recorded in a data segment head, then actively searches the initial position of the next data segment head backwards so as to calculate the real compressed length of the segment of data, then intercepts the real compressed data and decompresses the compressed data, simultaneously detects the overflow risk of an array in real time during decompression, dynamically expands the array according to the decompression requirement, avoids the overflow of the array, and restores the data before compression.
Message header fault tolerance mechanism: reading the message length in the current message header at the message analysis stage after data decompression, judging whether the message length in the current message header is the effective length, if not, carrying out third fault-tolerant processing on the current message header, namely actively searching a starting mark of a next message header backwards, calculating the effective length of the current message according to the initial position of the next message header, and then intercepting the current message according to the effective length of the current message for analysis.
Specifically, the message header records information such as the message type and the timestamp, and also records the byte length of the message, that is, the message length, and a character string corresponding to the first 4 bytes of the message header is an "LOBG" which indicates a start mark of the start position of the message header. The normal parsing program can judge whether the message is the initial position of the message header according to the LOBG starting mark, and can intercept message bytes to parse the message according to message length information recorded in the message header, so that when the message length data is wrong, not only can the parsing error of the message be caused, but also the correct parsing of the next message can be influenced.
In view of the above problems, a message header fault tolerance mechanism is designed in the BLF file parsing fault tolerance method in the embodiment of the present specification. When reading the message length field in the message header, firstly, judging whether the length field is a valid length, wherein the judgment rule is to check whether the length is matched with the length allowed by any known message type. If the data is not an effective length data, the initial position of the next message header is actively searched backwards, so that the effective length of the message is calculated, and then the real message data is intercepted and analyzed.
The following describes a detailed analysis of an analysis flow of the BLF file analysis fault-tolerant method provided in this specification with reference to fig. 6.
When the BLF file analysis is started, first performing file header analysis, and determining whether the file header analysis of the BLF file is abnormal, specifically, determining whether the BLF file has a file header, and if the BLF file does not have a file header, determining that the file header analysis of the BLF file is abnormal; if the BLF file has a file header, continuously judging whether the number of the messages is greater than 0, if so, entering a compressed data segment decompression stage, and if not, judging that the file header of the BLF file is analyzed abnormally. When the analysis of the file header of the BLF file is abnormal, the first fault-tolerant processing is performed on the file header, and in detail, the BLF file is continuously read, i.e., the start flag of the data segment header is continuously searched, the BLF file is continuously analyzed from the data segment header, and the compressed data segment decompression stage is entered.
After entering a compressed data segment decompression stage, continuously reading the BLF file, judging whether the BLF file is the tail part of the BLF file, and if so, ending the analysis; if the judgment result is negative, reading the compressed data in the data segment header of the current data segment, analyzing the length of the current data segment before data compression according to the compressed data in the data segment header of the current data segment, decompressing the current data segment by using a decompression algorithm, and restoring the data of the current data segment before data compression according to the length of the current data segment before data compression. If the current data segment is successfully decompressed, the BLF file is continuously read, whether the current data segment is the tail of the data segment is judged, if the current data segment is the tail of the data segment, the process of the decompression stage of the compressed data segment is repeated, and if the current data segment is not the tail of the data segment, the message analysis stage is started; if the decompression of the current data segment fails, that is, the decompression of the compressed data segment of the BLF file is abnormal, the second fault-tolerant processing is performed on the data segment header of the data segment, in detail, the parser discards the compressed data (i.e., two data of the length before the data compression and the length after the data compression) recorded in the data segment header of the current data segment, searches for the start marker of the data segment header of the next data segment, thereby determining the start position of the data segment header of the next data segment, calculating the length of the current data segment after data compression according to the initial position of the data segment head of the next data segment, intercepting and decompressing the compressed data according to the length of the compressed data of the current data segment to restore the data before compression, simultaneously detecting the overflow risk of the array in real time, and dynamically expanding the array according to the decompression requirement of the compressed data to avoid the overflow of the array, thereby restoring the data before compression.
When the compressed data is decompressed, the parser enters the message parsing stage.
And reading the current message header in the message analysis stage after data decompression, analyzing the message length of the current message, and judging whether the message length of the current message is the effective length. If the judgment result is yes, intercepting the byte array with the corresponding length according to the message length of the current message, analyzing the message by using a message analyzing method, and outputting a message after the message is successfully analyzed. If the judgment result is negative, third fault-tolerant processing is carried out on the current message header, in detail, the starting mark of the next message header is actively searched backwards to determine the initial position of the next message header, the effective length of the current message is calculated according to the initial position of the next message header, then the byte array with the corresponding length is intercepted according to the effective length of the current message, message analysis is carried out by using a message analysis method, and a message is output after the message analysis is successful.
The BLF file parsing fault-tolerant method is described below by way of example with reference to some specific scenarios.
Scene 1: the BLF file has no 144 bytes written in the header, but has message data written.
Scene 2: the BLF file has a header written therein, but the value of the "number of messages" field in the header is 0, and message data is written therein.
Under normal parsing mechanism, both scenarios will cause the parsing procedure to end early. For scenario 1, the parser may exit parsing because the character string corresponding to the first 4 bytes of the BLF file is not "LOGG", and the input file is considered not to conform to the BLF file format. For scenario 2, the parser will consider that the BLF file does not contain any message and exit parsing.
After the BLF file parsing fault-tolerant method in the embodiments of the present description is added to the file header fault-tolerant mechanism, in both of the above two scenarios, the method may enter the message parsing stage to parse the message in the file, and the parsing procedure will not be ended in advance.
Scene 3: as shown in fig. 7, if there are 20 pieces of message data having 1024 bytes in total, the length after compression is 128 bytes. Due to program logic error when writing BLF file, the length of decompressed compressed data in the header of compressed data segment is wrongly written as the length of compressed data.
Under a normal analysis mechanism, an analysis program firstly analyzes the header content of a data segment, obtains a value of 'length of decompressed compressed data', then initializes an array with length of 128 bytes according to the value to store the restored data, and then intercepts the content with corresponding length as the compressed data (128 bytes) according to the 'length of the compressed data' decompressed data segment. However, if the byte length of the decompressed data exceeds 128 bytes, the array overflows, and the parsing program determines that the decompression is wrong, and the parsing of the data segment is skipped.
After the BLF file parsing fault-tolerant method in the embodiments of this specification is added to a data segment header fault-tolerant mechanism, when a program captures a decompression program error, the fault-tolerant mechanism is started, the program actively searches and calculates a value of "length after decompression of compressed data", then initializes an array according to the value to store the restored data, adds an array overflow detection mechanism to a decompression program of the fault-tolerant mechanism, expands the array when detecting that an array overflow error is about to occur, and returns a decompression result to a main program to continue to execute downwards after the decompression program is executed.
Scene 4: as shown in fig. 8, assuming that there are 2 CAN messages in the data segment obtained after the compressed data is decompressed, since the length of each CAN message is 24 bytes, the messages in the decompressed data segment are arranged as follows: header 1(32 bytes), packet 1(24 bytes), header 2(32 bytes), and packet 2(24 bytes). Assuming that the timestamp millisecond number of the message 1 is 1334, when the message is written into the BLF file, the program logic is in error, and the timestamp millisecond number 1334 is wrongly written into the file as the message length. All data of message 2 is normal.
Under a normal parsing mechanism, a parsing program first reads a message length field in a message header 1, subtracts a message header length 32 from the message length, that is, 1334-32 is 1302, to obtain a length 1302 of a data segment, and then tries to intercept data of 1302 bytes behind the message header 1 as the data segment of a CAN message to parse the message, in which case, since the data length behind the message header 1 is smaller than 1302, a parsing error is generated, and the parsing program exits parsing of the data segment.
After the fault-tolerant method for analyzing the BLF file in this embodiment is added to the message header fault-tolerant mechanism, the analysis program first reads the message length field in the message header 1, subtracts the message header length 32 from the message length, that is, 1334-32 is 1302, to obtain the length 1302 of the data segment, and then reads the message type field in the message header 1, where the read value is 1, which indicates that the message is the CAN-type message data. At this time, the fault tolerance mechanism may check whether the data segment length value 1302 is an effective value according to the message type, and since the length of the CAN message is only 24 bytes, the check result is False, which indicates that 1302 is not an effective value. The parser then discards 1302 this invalid value, starting with the first byte after header 1 and looking back for the next header start byte "LOBG", and if the first byte after header 1 is marked as the 1 st byte, it finds "LOBG" after 24 bytes, i.e. 25 th to 28 th bytes. Then the analysis program calculates the byte number between two message headers as 24, calculates the effective length of the message 1 as 24 according to the rule of BLF complementary bytes, then sends the 24 bytes behind the message header 1 to the message analysis program to analyze the message 1, and then continues to analyze the next message. Thus, the analysis error caused by the data error of the message header is avoided.
The embodiment of the specification also discloses a BLF file parsing fault-tolerant device, which comprises a file header fault-tolerant module, a data segment header fault-tolerant module and a message header fault-tolerant module.
The file header fault-tolerant module is used for continuously searching a starting mark of a data segment header when the file header of the BLF file is analyzed abnormally so as to continuously analyze the BLF file from the data segment header.
The data segment header fault-tolerant module is used for discarding the compressed data recorded in the data segment header of the current data segment by the analysis program when the decompression of the compressed data segment is abnormal, searching the start mark of the data segment header of the next data segment, calculating the length of the current data segment after the data compression according to the initial position of the data segment header of the next data segment, and then intercepting and decompressing the compressed data according to the length of the current data segment after the data compression so as to restore the data before the compression.
The message header fault-tolerant module is used for reading the message length in the current message header in the message analysis stage after data decompression, judging whether the message length in the current message header is the effective length, if not, actively searching the starting mark of the next message header backwards, calculating the effective length of the current message according to the initial position of the next message header, and then intercepting the current message according to the effective length of the current message for analysis.
For specific limitations of the BLF file parsing fault-tolerant device, reference may be made to the above-mentioned limitations on the BLF file parsing fault-tolerant mechanism, which is not described herein again. The modules in the BLF file parsing error-tolerant apparatus may be wholly or partially implemented by software, hardware, or a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
The embodiment of the present specification further discloses a computer device, which includes a memory and a processor, where the memory stores a computer program, and the processor implements the step of the BLF file parsing fault-tolerant mechanism when executing the computer program.
The embodiment of the specification also discloses a computer readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the steps of the BLF file parsing fault tolerance mechanism.
To sum up, the present specification discloses a BLF file parsing fault-tolerant method, apparatus, computer device and storage medium, and the invention provides three corresponding fault-tolerant mechanisms, namely, a file header fault-tolerant mechanism, a data segment header fault-tolerant mechanism and a message header fault-tolerant mechanism, for three most common errors in the BLF file parsing process.
Those of ordinary skill in the art will understand that: the figures are merely schematic representations of one embodiment, and the blocks or flow diagrams in the figures are not necessarily required to practice the present invention.
Those of ordinary skill in the art will understand that: modules in the devices in the embodiments may be distributed in the devices in the embodiments according to the description of the embodiments, or may be located in one or more devices different from the embodiments with corresponding changes. The modules of the above embodiments may be combined into one module, or further split into multiple sub-modules.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A BLF file analyzing fault-tolerant method is characterized in that the BLF file comprises a file header, a data segment header and a compressed data segment, the data segment header is provided with a start mark for marking the initial position of the data segment header, the data after the compressed data segment is decompressed comprises a message header and a message, the message type and the message length are recorded in the message header, and the message header is provided with a start mark for marking the initial position of the message header, and the method comprises the following steps:
when the analysis of the file header of the BLF file is abnormal, continuously searching the starting mark of the data segment header so as to continuously analyze the BLF file from the data segment header;
when the decompression of the compressed data segment is abnormal, the analysis program abandons the compressed data recorded in the data segment head of the current data segment and searches for the start mark of the data segment head of the next data segment; the compressed data comprises a length before data compression and a length after data compression;
calculating the length of the current data segment after data compression according to the initial position of the data segment head of the next data segment; intercepting and decompressing the compressed data according to the data compression length of the current data segment to restore the data before compression;
reading the message length in the current message header at the message analysis stage after data decompression, judging whether the message length in the current message header is the effective length, if not, actively searching the starting mark of the next message header backwards, calculating the effective length of the current message according to the initial position of the next message header, and then intercepting the current message according to the effective length of the current message for analysis.
2. The BLF file parsing fault tolerant method according to claim 1, wherein the determining whether the header parsing of the BLF file is abnormal comprises:
judging whether the BLF file has a file header;
if the BLF file does not have a file header, judging that the file header of the BLF file is analyzed abnormally;
if the BLF file has a file header, judging whether the number of messages of the file header is greater than 0;
if the number of the messages is more than 0, entering a compressed data segment decompression stage;
and if the number of the messages is not more than 0, judging that the analysis of the file header of the BLF file is abnormal.
3. The BLF file parsing fault tolerant method of claim 1, wherein said compressed data segment decompression phase further comprises:
continuously reading the BLF file, and judging whether the BLF file is the file tail of the BLF file;
if the judgment result is yes, the analysis is finished;
if the judgment result is negative, reading the compressed data in the data segment header of the current data segment;
analyzing the length of the current data segment before data compression according to the compressed data in the data segment header of the current data segment;
decompressing the current data segment by using a decompression algorithm, and restoring the data of the current data segment before compression according to the length of the current data segment before data compression;
when the current data segment is successfully decompressed, the BLF file is continuously read, and whether the current data segment is the tail of the data segment is judged;
if yes, repeating the process of the compressed data segment decompression stage;
if the judgment result is negative, entering the message analysis stage.
4. The BLF file parsing fault tolerant method according to claim 3, wherein intercepting and decompressing the compressed data according to the length of the current data segment after data compression to restore the data before compression specifically comprises:
intercepting compressed data according to the length of the compressed data of the current data segment;
decompressing the intercepted compressed data, detecting the risk of array overflow in real time, and dynamically expanding the array according to the decompression requirement of the intercepted compressed data.
5. The BLF file parsing fault tolerant method according to claim 1, wherein said message parsing stage further comprises:
if the message length in the current message header is the effective length, intercepting a byte array with the corresponding length according to the message length of the current message, and analyzing the message by using a message analyzing method;
and outputting a message after the message is successfully analyzed.
6. The BLF file parsing fault-tolerant method according to claim 1, wherein the calculating the effective length of the current packet according to the starting position of the next packet header, and then intercepting the current packet according to the effective length of the current packet for parsing specifically comprises:
calculating the effective length of the current message according to the initial position of the next message header;
intercepting the byte array with corresponding length according to the effective length of the current message, and performing message analysis on the intercepted byte array by using a message analysis method;
and outputting a message after the message is successfully analyzed.
7. The BLF file parsing fault tolerant method according to claim 6, wherein the determining whether the packet length in the current packet header is the valid length comprises:
acquiring a preset message length, and matching the message length in the current message header with the preset message length; the preset message length comprises the message length allowed by any known message type;
if the matching is successful, judging the message length in the current message header as the effective length;
if the matching fails, the message length in the current message header is judged not to be the effective length.
8. The BLF file parsing fault-tolerant method according to claim 7, wherein the obtaining a preset packet length and matching the packet length in the current packet header with the preset packet length specifically comprises:
reading the message type in the current message header;
acquiring the message length allowed by the message type according to the message type;
calculating the difference value between the message length in the current message header and the current message header length to obtain the length of the data segment;
and comparing the length of the data segment with the message length allowed by the message type.
9. The utility model provides a BLF file analysis fault-tolerant device, its characterized in that, the BLF file includes file header, data segment head, compressed data section, data segment head is provided with the initial mark that marks the initial position of data segment head, the data after compressed data section decompresses includes message head, message type, message length have been recorded in the message head, the message head is provided with the initial mark that marks the initial position of message head, the device includes:
a file header fault-tolerant module, configured to, when a file header of the BLF file is analyzed abnormally, continue to search for a start marker of the data segment header, so as to continue to analyze the BLF file from the data segment header;
a data segment header fault-tolerant module, configured to discard, by an analysis program, compressed data recorded in a data segment header of a current data segment when decompression of the compressed data segment is abnormal, search for a start marker of a data segment header of a next data segment, calculate a length of the current data segment after data compression according to an initial position of the data segment header of the next data segment, and then intercept and decompress the compressed data according to the length of the current data segment after data compression to restore data before compression; the compressed data comprises a length before data compression and a length after data compression;
and the message header fault-tolerant module is used for reading the message length in the current message header in a message analysis stage after data decompression, judging whether the message length in the current message header is the effective length, actively searching the starting mark of the next message header backwards if the message length in the current message header is the effective length, calculating the effective length of the current message according to the initial position of the next message header, and intercepting and analyzing the current message according to the effective length of the current message.
10. A computer arrangement, characterized by a memory storing a computer program and a processor implementing the steps of the method of any of claims 1-8 when executing the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210665305.9A CN115118793B (en) | 2022-06-14 | 2022-06-14 | BLF file analysis fault tolerance method and device and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210665305.9A CN115118793B (en) | 2022-06-14 | 2022-06-14 | BLF file analysis fault tolerance method and device and computer equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115118793A true CN115118793A (en) | 2022-09-27 |
| CN115118793B CN115118793B (en) | 2023-07-07 |
Family
ID=83329462
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210665305.9A Active CN115118793B (en) | 2022-06-14 | 2022-06-14 | BLF file analysis fault tolerance method and device and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115118793B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102708155A (en) * | 2012-04-20 | 2012-10-03 | 西安电子科技大学 | JSAX (joint simple API (application program interface) for XML (extensible markup language)) parser and parsing method based on syntactic analysis of backtracking automaton |
| CN103188104A (en) * | 2011-12-31 | 2013-07-03 | 中国移动通信集团浙江有限公司 | Method and device for analyzing user behaviors |
| US20170199896A1 (en) * | 2016-01-13 | 2017-07-13 | American Express Travel Related Services Company, | Systems and methods for processing binary mainframe data files in a big data environment |
| CN107832415A (en) * | 2017-11-07 | 2018-03-23 | 广东电网有限责任公司电力调度控制中心 | A kind of fast resolving method of COMTRADE binary data files |
| CN112214462A (en) * | 2020-10-22 | 2021-01-12 | 新华三信息安全技术有限公司 | Multi-layer decompression method of compressed file, electronic equipment and storage medium |
-
2022
- 2022-06-14 CN CN202210665305.9A patent/CN115118793B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103188104A (en) * | 2011-12-31 | 2013-07-03 | 中国移动通信集团浙江有限公司 | Method and device for analyzing user behaviors |
| CN102708155A (en) * | 2012-04-20 | 2012-10-03 | 西安电子科技大学 | JSAX (joint simple API (application program interface) for XML (extensible markup language)) parser and parsing method based on syntactic analysis of backtracking automaton |
| US20170199896A1 (en) * | 2016-01-13 | 2017-07-13 | American Express Travel Related Services Company, | Systems and methods for processing binary mainframe data files in a big data environment |
| CN107832415A (en) * | 2017-11-07 | 2018-03-23 | 广东电网有限责任公司电力调度控制中心 | A kind of fast resolving method of COMTRADE binary data files |
| CN112214462A (en) * | 2020-10-22 | 2021-01-12 | 新华三信息安全技术有限公司 | Multi-layer decompression method of compressed file, electronic equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| HAO QIU; ZIMING QI: "A new motors fault tolerance control strategy to 4WID electric vehicle", 《2015 6TH INTERNATIONAL CONFERENCE ON AUTOMATION, ROBOTICS AND APPLICATIONS (ICARA)》 * |
| 张海逢: "基于CAN总线的车辆数据采集与远程监控系统研发", 《中国优秀硕士学位论文全文数据库》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115118793B (en) | 2023-07-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US4701851A (en) | Compound word spelling verification | |
| CN114710224A (en) | Frame synchronization method and device, computer readable medium and electronic device | |
| CN109284331B (en) | Certificate making information acquisition method based on service data resources, terminal equipment and medium | |
| CN110888791A (en) | Log processing method, device, equipment and storage medium | |
| CN111966339B (en) | Buried point parameter input method and device, computer equipment and storage medium | |
| CN115118793B (en) | BLF file analysis fault tolerance method and device and computer equipment | |
| CN110442439B (en) | Task process processing method and device and computer equipment | |
| CN111026604B (en) | Log file analysis method and device | |
| CN110928793A (en) | Regular expression detection method and device and computer readable storage medium | |
| US20160253374A1 (en) | Data file writing method and system, and data file reading method and system | |
| US10055341B2 (en) | To-be-stubbed target determining apparatus, to-be-stubbed target determining method and non-transitory recording medium storing to-be-stubbed target determining program | |
| CN109560964B (en) | Equipment compliance checking method and device | |
| CN113792017A (en) | Method and system for checking template content imported into excel | |
| CN112527557A (en) | Data recovery method, device and equipment and computer readable storage medium | |
| CN110298018B (en) | Text data processing method, device, computer equipment and storage medium | |
| CN111444033A (en) | Springbatch-based task fault-tolerant processing method and related equipment | |
| CN112949231A (en) | Module verification system, method and equipment based on UVM verification platform | |
| CN113037521A (en) | Method for identifying state of communication equipment, communication system and storage medium | |
| US20210397498A1 (en) | Information processing apparatus, control method, and program | |
| CN111698330A (en) | Data recovery method and device of storage cluster and server | |
| CN111953544B (en) | Fault detection method, device, equipment and storage medium of server | |
| CN116701810B (en) | Website operation playback method and device | |
| CN107301183B (en) | File storage method and device | |
| CN112579327B (en) | Fault detection method, device and equipment | |
| CN117271202B (en) | Optimal extraction method for multi-pass retransmission data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |