CN107241231B - Rapid and accurate positioning method for original network data packet - Google Patents
Rapid and accurate positioning method for original network data packet Download PDFInfo
- Publication number
- CN107241231B CN107241231B CN201710617660.8A CN201710617660A CN107241231B CN 107241231 B CN107241231 B CN 107241231B CN 201710617660 A CN201710617660 A CN 201710617660A CN 107241231 B CN107241231 B CN 107241231B
- Authority
- CN
- China
- Prior art keywords
- network data
- original network
- data packet
- data packets
- original
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0677—Localisation of faults
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/06—Generation of reports
- H04L43/067—Generation of reports using time frame reporting
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a method for quickly and accurately positioning an original network data packet. The method creates a storage mode of the original network data packet, adds the timestamp and the number information, writes the timestamp and the number information into the log, and facilitates the client to position the original network data packet through the timestamp and the number information. The client can screen out a small range of original network data packets through the timestamp, and then further screens the original network data packets through number matching, so that quick and accurate positioning is realized. The invention has low realization cost, high efficiency and good use value.
Description
Technical Field
The invention belongs to the technical field of network data analysis, and particularly relates to a method for quickly and accurately positioning an original network data packet.
Background
Currently, when a network data packet is abnormal, a related transaction log or an alarm log is generated to remind a user, but many users cannot know the condition of generating the original network data packet corresponding to the alarm log at all by using the log, and cannot perform positioning, and even positioning takes a lot of time.
Therefore, there is a need to provide a technique that enables a user to locate an original network packet accurately as soon as possible by logging.
Disclosure of Invention
In order to solve the above problems, the present invention provides a method for quickly and accurately positioning an original network data packet, comprising the following steps:
the method comprises the following steps: the server collects network flow data packets, records the time stamp of each original network data packet, and numbers the original network data packets with the same time stamp. And then sequentially storing the original network data packets to the server by taking the time stamps as indexes.
Step two: when a transaction log or an alarm log is generated in the process of analyzing the data packets, the server stores the log and records a time stamp and a number of each original network data packet generating the transaction log or the alarm log in the log.
Step three: the client queries the generated transaction logs or alarm logs.
Step four: and the client locates the original network data packet corresponding to each transaction log or alarm log according to the timestamp and the serial number.
Further, in the first step, the storage mode of the original network data packet is as follows: the fixed length header information is written first to store the related information of the original network data packet. And then writing the original content of the original network data packet.
Further, in the first step, the information included in the header information includes: the time stamp of the original network data packet, the number of the original network data packet and the original content length of the original network data packet.
Further, the fourth step is specifically:
step 4.1: and reading the time stamp and the number recorded in the transaction log or the alarm log to be positioned.
Step 4.2: and reading all original network data packets which are stored in the server and are the same as the time stamp by taking the time stamp as an index.
Step 4.3: and (4) filtering the original network data packets read in the step (4.1) through the serial numbers, and finding out the original network data packets with the same serial numbers.
Step 4.4: and repeating the step 4.1 to the step 4.3 until all the transaction logs or the original network data packets corresponding to the alarm logs are positioned.
Further, step 4.3 specifically includes: starting from the initial storage position of the original network data packet acquired in step 4.1, the header information written in the first original network data packet is checked, and whether the stored number is matched with the number of the original network data packet to be positioned is judged. If the data packets are matched, the original network data packet is the original network data packet needing positioning, the original content in the original network data packet is the data packet content needing positioning, if the data packets are not matched, the next original network data packet is matched, and the data packets are matched one by one until the matched original network data packet is found or the number of the read original network data packet is larger than the number of the data packet needing positioning.
Furthermore, each original network data packet with the same timestamp has a different offset, the offset is equal to the sum of the lengths of the original network data packets before the original network data packet, and in step 4.3, the original network data packets are matched one by one through the change of the offset.
Further, the offset of the original network packet at the start position is 0.
Further, the length of the original network data packet is the sum of the length of the header and the length of the original content.
Further, in step two, when the transaction log or the alarm log is generated by a plurality of consecutive data packets, the numbers of the data packets are recorded in a range of the interval.
The invention has the beneficial effects that:
the invention innovates the storage mode of the original network data packet, adds the timestamp and the number information, and writes the timestamp and the number information into the log, thereby facilitating the client to position the original network data packet through the timestamp and the number information. The client can screen out a small range of original network data packets through the timestamp, and then further screens the original network data packets through number matching, so that quick and accurate positioning is realized.
Detailed Description
The method comprises the following steps:
the method comprises the following steps: the method comprises the following steps that a server collects network flow data packets, records a timestamp of each original network data packet, and numbers the original network data packets; and then sequentially storing the original network data packets to a disk of the server by taking the time stamps as indexes.
In order to realize positioning, the embodiment innovates a storage mode of the original network data packet, and marks the data packet by using a time stamp and a serial number. The storage mode of the original network data packet is as follows: writing fixed-length header information to store some related information of the original network data packet; and then writing the original content of the original network data packet. Wherein the header information at least comprises the following field information: the timestamp of the original network data packet, the number of the original network data packet, and the original content length of the original network data packet (i.e. the length of the content to be written after the header is written). The length of the entire original network packet is equal to the sum of the length of the header information and the length of the original content.
Step two: when a transaction log or an alarm log is generated in the process of analyzing the data packets, the server stores the log and records a time stamp and a number of each original network data packet generating the transaction log or the alarm log in the log. Preferably, when the transaction log or the alarm log is generated by a plurality of consecutive data packets, the number of the data packets is recorded in a range format. The original network data packet can be positioned as soon as possible by using the timestamp and the number.
Step three: the client queries the generated transaction logs or alarm logs.
The timestamp and number information of these packets are also passed to the client during the query.
Step four: and the client locates the original network data packet corresponding to each transaction log or alarm log according to the timestamp and the serial number.
The specific method comprises the following steps:
step 4.1: and reading the time stamp and the number of each original network data packet in the transaction log or the alarm log to be positioned.
Step 4.2: and reading all original network data packets which are stored in a server disk and are the same as the time stamp by taking the time stamp as an index.
Step 4.3: and (4) filtering the original network data packets read in the step (4.1) through the serial numbers, and finding out the original network data packets with the same serial numbers.
The method specifically comprises the following steps: starting from the initial storage position (namely offset is 0) of the original network data packet acquired in the step 4.1, checking the header information written in the first original network data packet, and judging whether the stored number is matched with the number of a certain original network data packet to be positioned or not; if the data packets are matched, the original network data packets are the original network data packets needing positioning, the client reads the original content in the original network data packets as the data packet content needing positioning, if the data packets are not matched, the next original network data packets are matched, and the data packets are matched one by one until the matched original network data packets are found or the number of the read original network data packets is larger than the number of the data packets needing positioning.
Further, the process of matching the original network data packets with the same time stamp one by one is realized by calculating the offset, and the offset of each original network data packet is the sum of the total length of the previous original network data packets. Such as: the offset of the first original network packet is 0, if the header fixed length is a, the following original content length is b (the original memory length b is stored in the header information), and the offset of the second original network packet is a + b. And checking the written header information of the second original network data packet with the offset of a + b, and judging whether the stored number is matched with the numbers of other data packets to be determined. The original network packet offset for the start location is generally considered to be 0.
Step 4.4: and (4.1) repeatedly executing the step 4.1 to the step 4.3 until the accurate positioning of the original network data packet corresponding to all the transaction logs or the alarm logs is completed.
Claims (7)
1. A method for quickly and accurately positioning an original network data packet is characterized by comprising the following steps:
the method comprises the following steps: the method comprises the following steps that a server collects network flow data packets, records the time stamp of each original network data packet, and numbers the original network data packets with the same time stamp; then, sequentially storing the original network data packets to a server by taking the time stamps as indexes;
step two: when a transaction log or an alarm log is generated in the process of analyzing the data packets, the server stores the log and records the time stamp and the number of each original network data packet generating the transaction log or the alarm log in the log;
step three: the client side inquires each generated transaction log or alarm log;
step four: the client locates the original network data packet corresponding to each transaction log or alarm log according to the timestamp and the serial number;
in the first step, the storage mode of the original network data packet is as follows: writing fixed-length header information to store the related information of the original network data packet; writing the original content of the original network data packet;
the information contained in the header information includes: the time stamp of the original network data packet, the number of the original network data packet and the original content length of the original network data packet.
2. The method for fast and accurately positioning an original network data packet according to claim 1, wherein the fourth step is specifically:
step 4.1: reading a timestamp and a number recorded in a transaction log or an alarm log to be positioned;
step 4.2: taking the time stamp as an index, and reading all original network data packets which are stored in the server and are the same as the time stamp;
step 4.3: filtering the original network data packets read in the step 4.2 through the numbers, and finding out the original network data packets with the same numbers;
step 4.4: and repeating the step 4.1 to the step 4.3 until the positioning of the original network data packet corresponding to all the transaction logs or the alarm logs is completed.
3. The method for fast and accurately positioning an original network data packet according to claim 2, wherein the step 4.3 is specifically as follows: starting from the initial storage position of the original network data packet acquired in the step 4.2, checking the header information written in the first original network data packet, and judging whether the stored number is matched with the number of the original network data packet to be positioned; if the data packets are matched, the original network data packet is the original network data packet needing positioning, the original content in the original network data packet is the data packet content needing positioning, if the data packets are not matched, the next original network data packet is matched, and the data packets are matched one by one until the matched original network data packet is found or the number of the read original network data packet is larger than the number of the data packet needing positioning.
4. A method for fast and accurate positioning of original network data packets according to claim 3, characterized in that each original network data packet with the same timestamp has a different offset, the offset is equal to the sum of the lengths of the original network data packets before it, and in step 4.3, the original network data packets are matched one by one through the change of the offset.
5. The method as claimed in claim 4, wherein the offset of the original network packet at the start position is 0.
6. The method of claim 3, wherein the original network packet has a length that is the sum of the header length and the original content length.
7. The method as claimed in claim 1, wherein in step two, when the transaction log or the alarm log is generated by a plurality of consecutive data packets, the number of the data packets is recorded in a range manner.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710617660.8A CN107241231B (en) | 2017-07-26 | 2017-07-26 | Rapid and accurate positioning method for original network data packet |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710617660.8A CN107241231B (en) | 2017-07-26 | 2017-07-26 | Rapid and accurate positioning method for original network data packet |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107241231A CN107241231A (en) | 2017-10-10 |
| CN107241231B true CN107241231B (en) | 2020-04-03 |
Family
ID=59989295
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710617660.8A Active CN107241231B (en) | 2017-07-26 | 2017-07-26 | Rapid and accurate positioning method for original network data packet |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107241231B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111061684B (en) * | 2019-12-31 | 2023-04-07 | 科来网络技术股份有限公司 | Data packet file time sequence recovery system |
| CN113419885B (en) * | 2021-06-18 | 2023-05-26 | 杭州海康威视数字技术股份有限公司 | Data integrity processing method and device and electronic equipment |
| CN114422551B (en) * | 2022-01-24 | 2023-06-20 | 成都秦川物联网科技股份有限公司 | Centralized operation type energy metering device with built-in gateway and Internet of things system |
| CN117851696B (en) * | 2024-03-06 | 2024-05-03 | 北京新宇航星科技有限公司 | Method, device and storage medium for retrieving data packet |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101163265A (en) * | 2007-11-20 | 2008-04-16 | 中兴通讯股份有限公司 | Distributed database based on multimedia message log inquiring method and system |
| CN101620617A (en) * | 2009-07-28 | 2010-01-06 | 中兴通讯股份有限公司 | Method and device for searching and processing log file |
| US7991744B2 (en) * | 2008-07-10 | 2011-08-02 | International Business Machines Corporation | Method and system for dynamically collecting data for checkpoint tuning and reduce recovery time |
| CN106776622A (en) * | 2015-11-20 | 2017-05-31 | 北京国双科技有限公司 | The querying method and device of access log |
| WO2017092444A1 (en) * | 2015-12-02 | 2017-06-08 | 中兴通讯股份有限公司 | Log data mining method and system based on hadoop |
-
2017
- 2017-07-26 CN CN201710617660.8A patent/CN107241231B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101163265A (en) * | 2007-11-20 | 2008-04-16 | 中兴通讯股份有限公司 | Distributed database based on multimedia message log inquiring method and system |
| US7991744B2 (en) * | 2008-07-10 | 2011-08-02 | International Business Machines Corporation | Method and system for dynamically collecting data for checkpoint tuning and reduce recovery time |
| CN101620617A (en) * | 2009-07-28 | 2010-01-06 | 中兴通讯股份有限公司 | Method and device for searching and processing log file |
| CN106776622A (en) * | 2015-11-20 | 2017-05-31 | 北京国双科技有限公司 | The querying method and device of access log |
| WO2017092444A1 (en) * | 2015-12-02 | 2017-06-08 | 中兴通讯股份有限公司 | Log data mining method and system based on hadoop |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107241231A (en) | 2017-10-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107241231B (en) | Rapid and accurate positioning method for original network data packet | |
| CN106656536B (en) | Method and equipment for processing service calling information | |
| CN102867071B (en) | Management method for massive network management historical data | |
| US8028194B2 (en) | Sequencing technique to account for a clock error in a backup system | |
| CN106970935B (en) | Real-time data storage structure, data writing method and data reading method | |
| US8396840B1 (en) | System and method for targeted consistency improvement in a distributed storage system | |
| CN105373541B (en) | The processing method and system of the data operation request of database | |
| US8468134B1 (en) | System and method for measuring consistency within a distributed storage system | |
| CN104881481A (en) | Method and device for accessing mass time sequence data | |
| CN105068917B (en) | A kind of input and output IO characteristic analysis methods and system | |
| CN105099833A (en) | Business test method , device and system | |
| CN111177272B (en) | Big data credible audit method based on block chain | |
| CN109240917A (en) | A kind of blog management method and relevant apparatus | |
| CN107729234A (en) | The scheduling of test case performs method, apparatus, equipment and computer-readable storage medium | |
| CN106897342A (en) | A kind of data verification method and equipment | |
| CN106600303A (en) | Method and device for assessment of advertisement putting rationality | |
| CN109377383A (en) | Product data synchronous method, device, computer equipment and storage medium | |
| CN107644033B (en) | Method and equipment for querying data in non-relational database | |
| CN105303430B (en) | Transaction index acquisition method and device | |
| CN108959497A (en) | distributed file system log processing method, device, equipment and storage medium | |
| CN110516124A (en) | A kind of document analysis method, apparatus and computer readable storage medium | |
| US7587513B1 (en) | Efficient storage of network and application data | |
| CN109271097A (en) | Data processing method, data processing equipment and server | |
| CN113297245A (en) | Method and device for acquiring execution information | |
| CN110825940B (en) | Network data packet storage and query method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20200803 Address after: 41401-41406, unit 1, building 4, No. 966, north section of Tianfu Avenue, Chengdu hi tech Zone, Sichuan 610041 Patentee after: Chengdu Kelai Network Technology Co., Ltd Address before: 610041, 966, 4, 1, 13 and 14 building, north section of Tianfu Road, Chengdu hi tech Zone, Sichuan Patentee before: COLASOFT Co.,Ltd. |
|
| CP03 | Change of name, title or address | ||
| CP03 | Change of name, title or address |
Address after: 610041 12th, 13th and 14th floors, unit 1, building 4, No. 966, north section of Tianfu Avenue, Chengdu hi tech Zone, China (Sichuan) pilot Free Trade Zone, Chengdu, Sichuan Patentee after: Kelai Network Technology Co.,Ltd. Address before: 41401-41406, 14th floor, unit 1, building 4, No. 966, north section of Tianfu Avenue, Chengdu hi tech Zone, China (Sichuan) pilot Free Trade Zone, Chengdu hi tech Zone, Sichuan 610041 Patentee before: Chengdu Kelai Network Technology Co.,Ltd. |