CN114153807A - Message processing method and device, electronic equipment and computer readable storage medium - Google Patents
Message processing method and device, electronic equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN114153807A CN114153807A CN202111483630.5A CN202111483630A CN114153807A CN 114153807 A CN114153807 A CN 114153807A CN 202111483630 A CN202111483630 A CN 202111483630A CN 114153807 A CN114153807 A CN 114153807A
- Authority
- CN
- China
- Prior art keywords
- message
- information
- session
- target
- original
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
Abstract
The application provides a message processing method, a message processing device, electronic equipment and a computer readable storage medium, wherein the method comprises the following steps: collecting an original message from a physical network interface; analyzing the original message to determine the message information of the original message; storing the message information into a buffer queue; updating the session information of the target session to which the original message belongs according to the message information; judging whether the message quantity of the target session reaches a preset quantity or not; if the message quantity of the target session reaches a preset quantity, storing the current session information of the target session into a specified storage medium; and storing the message information in the cache queue as a target file, wherein the target file is a file with a viewable format.
Description
Technical Field
The present application relates to the field of packet processing technologies, and in particular, to a packet processing method, an apparatus, an electronic device, and a computer-readable storage medium.
Background
In order to realize the traffic tracing, all original messages in the network can be stored, and the inquiry and evidence collection after the fact can be conveniently carried out. The general traffic tracing is to store the original message into a specific format, and when the original message needs to be checked, a special tool can be used to convert the original message into a standard viewable file, and the file conversion depends on the special tool, so that the complexity and inconvenience of the traffic tracing are greatly increased.
Disclosure of Invention
The application aims to provide a message processing method, a message processing device, electronic equipment and a computer readable storage medium, which can solve the problems of complexity and inconvenience in traffic tracing.
In a first aspect, the present invention provides a method for processing a packet, including:
collecting an original message from a physical network interface;
analyzing the original message to determine the message information of the original message;
storing the message information into a buffer queue;
updating the session information of the target session to which the original message belongs according to the message information;
judging whether the message quantity of the target session reaches a preset quantity or not;
if the message quantity of the target session reaches a preset quantity, storing the current session information of the target session into a specified storage medium;
and storing the message information in the cache queue as a target file, wherein the target file is a file with a viewable format.
In an optional embodiment, the storing the message information in the cache queue as a target file includes:
reading message information from the cache queue;
storing the message information into an intermediate memory block;
and if the size of the intermediate memory block reaches a preset value, writing the intermediate memory block into a file with a target format to form a target file.
In an optional embodiment, the storing the message information into the intermediate memory block includes:
determining the storage sequence of the messages in the message information according to the storage positioning information in the message information;
and storing the messages in the message information into an intermediate memory block according to the storage sequence.
In an optional embodiment, the analyzing the original packet to determine the packet information of the original packet includes:
determining a target session to which the original message belongs according to the original message;
and determining storage positioning information in the message information of the original message according to the session information of the target session and the original message.
In an alternative embodiment, the deposit location information includes a deposit file name and a deposit offset; the determining, according to the session information of the target session and the original packet, storage location information in the packet information of the original packet includes:
determining the storage file name of the original message according to the time of receiving the original message and the original message collection source;
and determining the storage offset of the original message according to the message length of the original message and the historical offset in the session information.
In an optional embodiment, the determining, according to the session information of the target session, whether the number of packets of the target session reaches a preset amount includes:
judging whether the target session is ended or not;
judging whether the message quantity of the target session reaches a preset quantity or not;
and if the target session is ended or the number of the messages of the target session reaches a preset number, representing that the number of the messages of the target session reaches the preset number.
In an alternative embodiment, the method further comprises:
searching sessions to be exported from prestored session information sets according to the retrieval request;
and extracting each message corresponding to the session to be exported from the target file according to the positioning information of the session to be exported.
In a second aspect, the present invention provides a packet processing apparatus, including:
the acquisition module is used for acquiring an original message from a physical network interface;
the analysis module is used for analyzing the original message to determine the message information of the original message;
the first storing module is used for storing the message information into a cache queue;
the judging module is used for judging whether the message quantity of the target session reaches a preset quantity or not according to the session information of the target session;
the updating module is used for updating the session information of the target session to which the original message belongs according to the message information;
the second storage module is used for storing the current session information of the target session into a specified storage medium if the message quantity of the target session reaches a preset quantity;
and the third storage module is used for storing the message information in the cache queue as a target file, wherein the target file is a file with a format capable of being checked.
In a third aspect, the present invention provides an electronic device comprising: a processor, a memory storing machine readable instructions executable by the processor, the machine readable instructions when executed by the processor perform the steps of the method of any of the preceding embodiments when the electronic device is run.
In a fourth aspect, the present invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method according to any of the preceding embodiments.
The beneficial effects of the embodiment of the application are that: by analyzing the received messages in stages, the required events when the historical messages need to be inquired can be reduced. Furthermore, the messages can be stored in the files in the target format in order through message-based analysis to form target files convenient to check, so that the files do not need to be converted again in the process of tracing, and the convenience of tracing can be improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.
Fig. 1 is a block diagram of an electronic device according to an embodiment of the present disclosure;
fig. 2 is a flowchart of a message processing method according to an embodiment of the present application;
fig. 3 is a schematic diagram of a functional module of a message processing apparatus according to an embodiment of the present application.
Detailed Description
The technical solution in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. Meanwhile, in the description of the present application, the terms "first", "second", and the like are used only for distinguishing the description, and are not to be construed as indicating or implying relative importance.
Traditional security devices, such as firewalls, intrusion detection and other security devices, analyze original messages to achieve attack detection and security audit. But the security device typically retains a security log. Therefore, when attack detection and security audit are implemented, the security logs need to be manually determined to know the original messages corresponding to the security logs. The flow tracing technology can meet the requirements of attack detection, safety audit and the like because all original messages are reserved. Therefore, traffic tracing is increasingly important in the field of network security, especially in network security protection of financial and national defense units.
However, the inventor has appreciated that the general traffic tracing is to store the original message into a specific format, and when the original message needs to be viewed, the original message needs to be converted into a standard pcap file by using a special tool. Since conversion of pcap files relies on specialized tools, this adds complexity and inconvenience to traffic tracing.
Based on the above research, the embodiment of the present application provides a message processing method, which can store the acquired message in a standard pcap format and analyze the message in real time. The following describes a message processing method according to the present application with some embodiments.
To facilitate understanding of the present embodiment, first, an electronic device that executes the message processing method disclosed in the embodiment of the present application is described in detail.
As shown in fig. 1, is a block schematic diagram of an electronic device. The electronic device 100 may include a memory 111, a processor 113. It will be understood by those of ordinary skill in the art that the structure shown in fig. 1 is merely exemplary and is not intended to limit the structure of the electronic device 100. For example, electronic device 100 may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.
The aforementioned components of the memory 111 and the processor 113 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, the components may be electrically connected to each other via one or more communication buses or signal lines. The processor 113 is used to execute the executable modules stored in the memory.
The Memory 111 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like. The memory 111 is configured to store a program, and the processor 113 executes the program after receiving an execution instruction, and the method executed by the electronic device 100 defined by the process disclosed in any embodiment of the present application may be applied to the processor 113, or implemented by the processor 113.
The processor 113 may be an integrated circuit chip having signal processing capability. The Processor 113 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the Integrated Circuit may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The electronic device 100 in this embodiment may be a security device, which may be a firewall.
The electronic device 100 in this embodiment may be configured to perform each step in each method provided in this embodiment. The following describes the implementation process of the message processing method in detail through several embodiments.
Please refer to fig. 2, which is a flowchart illustrating a message processing method according to an embodiment of the present application. The specific process shown in fig. 2 will be described in detail below.
In this embodiment, the data processing unit may be used to obtain the original packet of the physical network interface.
Alternatively, a corresponding number of data processing units may be started depending on the number of physical network interfaces. For example, the number of the data processing units may be the same as the number of the physical network interfaces, so that the data processing units are used for collecting messages of one physical network interface. Wherein each data processing unit may be a thread.
Illustratively, each data processing unit may correspond to a unique identity.
Optionally, a file in a target format may be constructed for each data processing unit, so as to write the packet of each physical network interface into the corresponding file in the target format.
Optionally, the data processing unit may further record the number of messages received for each session. For example, each session corresponds to a counting variable, and the counting variable corresponding to the session may be updated each time a message of one session is received.
The message information may include an original message and storage location information.
The storage positioning information can determine the corresponding storage positioning information according to the target session to which the original message belongs.
In this embodiment, the target session to which the original packet belongs may be determined by information carried by the original packet.
The session information corresponding to each group of sessions may be stored in the session storage area. When the session to which the original message belongs needs to be determined, a target session to which the original message belongs can be determined based on the session information in the session storage area and the data corresponding to the original message.
If the original message is not the message of the new session, the session storage area stores the session information of the target session corresponding to the original message; if the original message is a message of a new session, the session storage area does not store the session information of the target session corresponding to the original message, and the session information corresponding to the original message can be newly created and stored in the session storage area.
The session information may include five tuples, protocol name, and element information. The initial creation of the session information can only comprise five-tuple, and the protocol name and element information can be completed when more messages are received.
The quintuple may include: source IP, destination IP, source port, destination port, transport protocol. The protocol name is the application protocol name carried by the session, such as HTTP, SMTP, POP3, FTP, etc. The element information may include: protocol element information and positioning element information.
The protocol element information may be determined by the kind of application protocol, and the protocol elements may be different for different application protocols. For example, HTTP may include domain names, request methods, request parameters, URLs, etc.; SMTP may include mail sender, mail receiver, mail header, etc.; the FTP may include account numbers, commands, return results, transfer file names, etc. The location element information includes a file name, an offset, and a length of each packet.
Because the quintuple can uniquely identify one group of sessions, the target session to which the original message belongs can be determined by the quintuple corresponding to the original message.
In an alternative embodiment, step 220 may include steps 221 through 223.
Step 221, determining a target session to which the original message belongs according to the original message.
In this embodiment, if the protocol name in the session information of the target session corresponding to the original packet is null, the application protocol type may be identified with the packet load feature of the original packet, and after the application protocol type is identified, the protocol name is filled in the protocol name in the session information of the target session. And calling different protocol analysis plug-ins according to different application protocols. For example, if the application protocol is HTTP, an HTTP parsing plug-in is called, and if the application protocol is SMTP, an SMTP parsing plug-in is called. And the protocol analysis plug-in extracts the protocol element information corresponding to the protocol.
Step 222, determining storage positioning information in the message information of the original message according to the session information of the target session and the original message.
The storage location information may include a storage file name and a storage offset.
Step 222, may include: determining the storage file name of the original message according to the time of receiving the original message and the original message collection source; and determining the storage offset of the original message according to the message length of the original message and the historical offset in the session information.
The storage file name of the original message can be determined according to the time of receiving the original message and the data processing unit used for collecting the original message. The deposit file name may be, for example, the time at which the message was collected and the identification of the data processing unit used to collect the message.
Alternatively, the message may be written last to the file in the target format. The message in the target format can be viewed. For example, the target format may be pcap file format, and the file naming rule may be: year _ month _ day _ did. For example, the time for collecting the message is 1 month and 1 day in 2021, and the unique identity of the data processing unit for collecting the message is 001, the name of the stored file may be: 2021_01_01_001. pcap.
The pcap file is composed of a 24-byte file header and a plurality of original message information, and each original message information comprises a 16-byte message header and an original message. The deposit OFFSET represents the position of the current message in the pcap file, and since the file header of the pcap file occupies 24 bytes, the initial value of the OFFSET is 24, that is, when the first message to be stored in the PFILE file arrives, the position of the OFFSET in the PFILE file is 24. The Packet Length (PLEN) written into the PFILE file is the packet header length (16 bytes) plus the original packet length.
After the message is processed, the location information of the message can be represented by a location list as: { OFFSET1, PLEN1}, { OFFSET2, PLEN2} … { OFFSETn, PLENn } ]. { OFFSET1, PLEN1} indicates the position of the first packet of the session in the pcap file, { OFFSET2, PLEN2} indicates the position … of the second packet of the session in the pcap file, and so on, { OFFSETn, PLENn } indicates the position of the nth packet of the session in the pcap file.
Based on the above analysis, the deposit offset of each packet may be calculated based on the current offset of the packet that the PFILE file already needs to deposit:
after a message is processed, storing OFFSET, wherein the formula is as follows:
OFFSETn＝OFFSET(n-1)+PLENn；
wherein, OFFSETn represents the storage offset of the current message; OFFSET (n-1) represents the storage OFFSET of the previous message; PLENn indicates the length of the current packet.
And 240, updating the session information of the target session to which the original message belongs according to the message information.
Optionally, the session information may also record the number of messages of each currently received session.
In one example, a complete HTTP session information is illustrated as follows:
sip, dip, sport, dport, pro, quintuple information for uniquely identifying a session; l7pro is the protocol name; url, host and method are HTTP protocol element information; the position is positioning element information, including a pcap file name pcap _ file and a position of each packet in the pcap file, as shown in the example: the offset of the first message in the file is 24, and the message length is 68 bytes; the offset of the second packet in the file is 200 and the length of the packet is 128 bytes. It will be appreciated that in different examples, the protocol element information may have more or less content than in the above examples.
If the number of messages of the target session reaches the preset amount, step 260 is executed.
Alternatively, the session information of the target session may be stored in the specified storage medium after one session is ended.
Considering that the duration of some sessions is long, if the session information of the target session is not stored in the specified storage medium for a long time, the message may not be traced for a long time. Therefore, when the message quantity of the target session reaches a certain quantity, the session information of the target session can be stored in the specified storage medium.
Step 250 may include: step 251 and step 252.
Step 251, determining whether the target session is ended.
Illustratively, whether the target session is ended may be determined by session information of the target session. And if the physical network interface does not receive the message for the specified duration, the target session can be judged to be ended. And if the physical network interface and the message sending end finish the communication connection, the target session can be judged to be finished. Of course, whether the target session is ended may also be determined in other ways.
Step 252, determining whether the number of the messages of the target session reaches a preset number.
The preset number may be determined according to a hardware resource of the electronic device executing the message processing method of the embodiment, for example, the larger the memory owned by the electronic device is, the larger the preset number may be.
And if the target session is ended or the number of the messages of the target session reaches a preset number, representing that the number of the messages of the target session reaches the preset number.
For example, it may be determined whether the number of messages of the target session reaches a preset number according to a record of the data processing unit on the number of received messages of the target session. For example, the number of messages of the target session may be determined according to the value of the counting variable corresponding to the target session.
And step 260, storing the current session information of the target session into a specified storage medium.
The specified storage medium may be, for example, a fixed storage medium. When the original message needs to be queried, the location of the message needing to be queried in the target file can be determined from the execution storage medium based on the received request.
Optionally, the message information may be acquired from the cache queue in real time and written into the target file.
Optionally, the message information may also be obtained from the buffer queue according to a set time rule, and written into the target file. The set time rule may be that every preset time, the message information is obtained from the buffer queue and written into the target file. The preset duration may be set according to specific situations, and the preset duration is not set as a limit in this embodiment.
Optionally, after storing one item of message information into the cache queue, the message information may be obtained from the cache queue and written into the target file.
The target file is a file in a viewable format. For example, the target format may be a pcap formatted file.
Alternatively, step 270 may be performed by a data storage unit. The data storage unit may be another thread than the data processing unit described above.
The data storage unit reads message information from the buffer queue in sequence. The original message and the storage positioning information corresponding to the original message can be determined from the message information. According to the storage positioning information, the file name of the file in the target format and the specific position in the file PFILE, where the original message can be stored, can be determined.
In this embodiment, the file in the target format may be created when the data processing unit is started, or may be created when a message needs to be written.
Taking the case that the target format file is a pcap file, if the pcap file does not exist when the message needs to be written into the target format file, the pcap file is created first, and a 24-byte pcap file header is written according to the pcap file specification. And then writing the pcap message header and the read original message into a pcap file.
In order to reduce the number of times of writing in the file, the message information in the buffer queue may be stored in the file block first, and when the file block reaches a certain amount, the message information is written in the target format file again. Illustratively, step 270 may include steps 271 through 273.
Step 271, reading the message information from the buffer queue.
Step 272, store the message information into the middle memory block.
If the size of the middle memory block reaches the preset value, execute step 273.
Optionally, the storage sequence of the messages in the message information may be determined according to the storage positioning information in the message information; and storing the messages in the message information into an intermediate memory block according to the storage sequence.
Step 273, writing the intermediate memory block into the file in the target format to form a target file.
In the present embodiment, the order of the steps shown in fig. 2 is merely exemplary, and the specific execution may not be performed in the order shown in fig. 2. For example, step 270 may be performed after step 230, and step 270 may also be performed after step 240.
Based on the foregoing steps 210 to 270, the received message can be stored according to a format that can be checked, and the message processing method according to the embodiment of the present application can more conveniently trace the source of the message on the basis of storing the message in the target format. Based on this, the message processing method may further include.
And step 280, searching the session to be exported from the pre-stored session information set according to the retrieval request.
Illustratively, the pre-stored set of session information may be stored in the session storage area.
Step 290, extracting each message corresponding to the session to be exported from the target file according to the positioning information of the session to be exported.
Optionally, each read message corresponding to the session to be exported may be written into a file in a new target format, so that the file in the new target format only contains the message of the session to be exported.
For example, each message corresponding to the session to be exported may be extracted from the target file by using an offset of each message recorded in the positioning information of the session information.
In one example, the HTTP session information example sequentially reads 68, 128, 64, and 512 bytes from the 2021_10_16_001 pcap file at offsets 24, 200, 512, and 1024, and writes a new pcap file, where the new pcap file only contains the message of the session to be exported, and is also a standard pcap file.
In this embodiment, the steps 280 and 290 of implementing the message retrieval may be implemented by one retrieval unit.
In the message processing method provided by the embodiment of the application, by performing staged analysis on the received message, the fact that the user needs to do when needing to inquire the historical message can be reduced. Furthermore, the messages can be stored in the files in the target format in order through message-based analysis to form target files convenient to check, so that the files do not need to be converted again in the process of tracing, and the convenience of tracing can be improved.
Further, messages can be stored in a standard pcap format, and message viewing can be independent of special tools. Furthermore, the received message can be analyzed through a real-time protocol, and the scene requirement of real-time statistic analysis can be met.
Based on the same application concept, a message processing apparatus corresponding to the message processing method is also provided in the embodiments of the present application, and since the principle of the apparatus in the embodiments of the present application for solving the problem is similar to that in the embodiments of the message processing method described above, the apparatus in the embodiments of the present application may be implemented by referring to the description in the embodiments of the method described above, and repeated details are not described again.
Please refer to fig. 3, which is a schematic diagram of a functional module of a message processing apparatus according to an embodiment of the present application. Each module in the message processing apparatus in this embodiment is configured to execute each step in the foregoing method embodiment. The message processing device comprises: the system comprises an acquisition module 310, an analysis module 320, a first storage module 330, an update module 340, a judgment module 350 and a second storage module 360; wherein the content of the first and second substances,
an acquisition module 310, configured to acquire an original packet from a physical network interface;
the parsing module 320 is configured to parse the original packet to determine packet information of the original packet;
a first storing module 330, configured to store the message information into a cache queue;
an updating module 340, configured to update session information of a target session to which the original packet belongs according to the packet information;
a determining module 350, configured to determine whether the number of messages of the target session reaches a preset amount according to the session information of the target session;
a second storing module 360, configured to store the current session information of the target session into a specified storage medium if the number of messages of the target session reaches a preset amount;
a third storing module 370, configured to store the message information in the cache queue as a target file, where the target file is a file in a viewable format.
In one possible implementation, the third logging module 370 includes: reading sub-modules, storing the sub-modules and writing the sub-modules;
the reading submodule is used for reading the message information from the cache queue;
the storage submodule is used for storing the message information into the middle memory block;
and the writing sub-module is used for writing the intermediate memory block into a file in a target format to form a target file if the size of the intermediate memory block reaches a preset value.
In one possible embodiment, the submodule is stored for:
determining the storage sequence of the messages in the message information according to the storage positioning information in the message information;
and storing the messages in the message information into an intermediate memory block according to the storage sequence.
In a possible implementation, the parsing module 320 includes: a session determining submodule and a positioning determining submodule;
the session determining submodule is used for determining a target session to which the original message belongs according to the original message;
and the positioning determining submodule is used for determining the storage positioning information in the message information of the original message according to the session information of the target session and the original message.
In one possible embodiment, the deposit location information includes a deposit file name and a deposit offset; a location determination submodule for:
determining the storage file name of the original message according to the time of receiving the original message and the original message collection source;
and determining the storage offset of the original message according to the message length of the original message and the historical offset in the session information.
In a possible implementation, the determining module 350 is configured to:
judging whether the target session is ended or not;
judging whether the message quantity of the target session reaches a preset quantity or not;
and if the target session is ended or the number of the messages of the target session reaches a preset number, representing that the number of the messages of the target session reaches the preset number.
In a possible implementation manner, the message processing apparatus in this embodiment further includes:
the searching module is used for searching the session to be exported from the pre-stored session information set according to the retrieval request;
and the extraction module is used for extracting each message corresponding to the session to be exported from the target file according to the positioning information of the session to be exported.
In addition, an embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program performs the steps of the message processing method in the foregoing method embodiment.
The computer program product of the message processing method provided in the embodiment of the present application includes a computer-readable storage medium storing a program code, where instructions included in the program code may be used to execute the steps of the message processing method in the foregoing method embodiment, which may be referred to specifically in the foregoing method embodiment, and details are not described here again.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes. It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application. It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A message processing method is characterized by comprising the following steps:
collecting an original message from a physical network interface;
analyzing the original message to determine the message information of the original message;
storing the message information into a buffer queue;
updating the session information of the target session to which the original message belongs according to the message information;
judging whether the message quantity of the target session reaches a preset quantity or not;
if the message quantity of the target session reaches a preset quantity, storing the current session information of the target session into a specified storage medium;
and storing the message information in the cache queue as a target file, wherein the target file is a file with a viewable format.
2. The method of claim 1, wherein storing the packet information in the buffer queue as a target file comprises:
reading message information from the cache queue;
storing the message information into an intermediate memory block;
and if the size of the intermediate memory block reaches a preset value, writing the intermediate memory block into a file with a target format to form a target file.
3. The method according to claim 2, wherein the storing the message information into an intermediate memory block comprises:
determining the storage sequence of the messages in the message information according to the storage positioning information in the message information;
and storing the messages in the message information into an intermediate memory block according to the storage sequence.
4. The method of claim 1, wherein the parsing the original packet to determine the packet information of the original packet comprises:
determining a target session to which the original message belongs according to the original message;
and determining storage positioning information in the message information of the original message according to the session information of the target session and the original message.
5. The method of claim 4, wherein the deposit location information includes a deposit file name and a deposit offset; the determining, according to the session information of the target session and the original packet, storage location information in the packet information of the original packet includes:
determining the storage file name of the original message according to the time of receiving the original message and the original message collection source;
and determining the storage offset of the original message according to the message length of the original message and the historical offset in the session information.
6. The method according to claim 1, wherein the determining whether the number of packets of the target session reaches a preset number according to the session information of the target session comprises:
judging whether the target session is ended or not;
judging whether the message quantity of the target session reaches a preset quantity or not according to the session information of the target session;
and if the target session is ended or the number of the messages of the target session reaches a preset number, representing that the number of the messages of the target session reaches the preset number.
7. The method according to any one of claims 1-6, further comprising:
searching sessions to be exported from prestored session information sets according to the retrieval request;
and extracting each message corresponding to the session to be exported from the target file according to the positioning information of the session to be exported.
8. A message processing apparatus, comprising:
the acquisition module is used for acquiring an original message from a physical network interface;
the analysis module is used for analyzing the original message to determine the message information of the original message;
the first storing module is used for storing the message information into a cache queue;
the updating module is used for updating the session information of the target session to which the original message belongs according to the message information;
the judging module is used for judging whether the message quantity of the target session reaches a preset quantity or not;
the second storage module is used for storing the current session information of the target session into a specified storage medium if the message quantity of the target session reaches a preset quantity;
and the third storage module is used for storing the message information in the cache queue as a target file, wherein the target file is a file with a format capable of being checked.
9. An electronic device, comprising: a processor, a memory storing machine-readable instructions executable by the processor, the machine-readable instructions when executed by the processor performing the steps of the method of any of claims 1 to 7 when the electronic device is run.
10. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, is adapted to carry out the steps of the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111483630.5A CN114153807A (en) | 2021-12-07 | 2021-12-07 | Message processing method and device, electronic equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111483630.5A CN114153807A (en) | 2021-12-07 | 2021-12-07 | Message processing method and device, electronic equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114153807A true CN114153807A (en) | 2022-03-08 |
Family
ID=80453024
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111483630.5A Pending CN114153807A (en) | 2021-12-07 | 2021-12-07 | Message processing method and device, electronic equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114153807A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114615355A (en) * | 2022-05-13 | 2022-06-10 | 恒生电子股份有限公司 | Message processing method and message analysis module |
-
2021
- 2021-12-07 CN CN202111483630.5A patent/CN114153807A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114615355A (en) * | 2022-05-13 | 2022-06-10 | 恒生电子股份有限公司 | Message processing method and message analysis module |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8813220B2 (en) | Methods and systems for internet protocol (IP) packet header collection and storage | |
| CN109600317B (en) | Method and device for automatically identifying traffic and extracting application rules | |
| CN112039904A (en) | Network traffic analysis and file extraction system and method | |
| US20130191890A1 (en) | Method and system for user identity recognition based on specific information | |
| US8762515B2 (en) | Methods and systems for collection, tracking, and display of near real time multicast data | |
| CN106330584B (en) | A kind of recognition methods of Business Stream and identification device | |
| CN102724317A (en) | Network data flow classification method and device | |
| CN103139315A (en) | Application layer protocol analysis method suitable for home gateway | |
| CN108900374A (en) | A kind of data processing method and device applied to DPI equipment | |
| CN102938764A (en) | Application identification processing method and device | |
| US20240146753A1 (en) | Automated identification of false positives in dns tunneling detectors | |
| CN112532614A (en) | Safety monitoring method and system for power grid terminal | |
| CN114153807A (en) | Message processing method and device, electronic equipment and computer readable storage medium | |
| CN108282414A (en) | A kind of bootstrap technique of data flow, server and system | |
| CN105100246A (en) | Network flow management and control method based on downloaded resource name | |
| CN111198806B (en) | Service call data statistical analysis method and system based on service open platform | |
| CN112887289A (en) | Network data processing method and device, computer equipment and storage medium | |
| CN101036370A (en) | Method for processing a data flow according to the content thereof | |
| CN109492655B (en) | Feature extraction method and device and terminal | |
| CN110868360B (en) | Flow statistics method, electronic equipment, system and medium | |
| CN110661799B (en) | ARP (Address resolution protocol) deception behavior detection method and system | |
| CN111163184B (en) | Method and device for extracting message features | |
| CN114328925A (en) | Flow data processing method and device, probe equipment and storage medium | |
| CN110661796B (en) | User action flow identification method and device | |
| CN109842511B (en) | Method and system for determining TCP performance parameters |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |