CN110719233B

CN110719233B - Method and device for transmitting information

Info

Publication number: CN110719233B
Application number: CN201910962981.0A
Authority: CN
Inventors: 刘宇; 李莅
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2019-10-11
Filing date: 2019-10-11
Publication date: 2023-10-31
Anticipated expiration: 2039-10-11
Also published as: CN110719233A

Abstract

The embodiment of the disclosure discloses a method and a device for sending information, and relates to the field of cloud computing. One embodiment of the method comprises the following steps: in response to the existence of a packet to be sent in a buffer queue, packaging the packet to be sent in the buffer queue into an information packet, and sending the information packet to an information server; and in response to receiving the information storage success message corresponding to the information packet sent by the information server, sending an offset mark updating instruction to the information acquisition protocol. The embodiment realizes the parallel acquisition of the log information and improves the acquisition efficiency of the log information.

Description

Method and device for transmitting information

Technical Field

The embodiment of the disclosure relates to the technical field of computers, in particular to a method and a device for sending information.

Background

The log file is a record file or a file set for recording system operation events on the terminal, and can be divided into event logs and message logs. Has important functions of processing historical data, tracking diagnosis problems, understanding activities of the system and the like.

The server can acquire the log data of the log file on the terminal, and process the running state of the terminal, data analysis and other data through the log data.

Disclosure of Invention

The embodiment of the disclosure provides a method and a device for transmitting information.

In a first aspect, embodiments of the present disclosure provide a method for transmitting information, the method comprising: responding to the existence of a packet to be sent in a buffer queue, packaging the packet to be sent in the buffer queue into an information packet, and sending the information packet to an information server, wherein the packet to be sent is sent to the buffer queue by an information acquisition protocol, and the information acquisition protocol is used for acquiring information from a log file to be processed and packaging the acquired information into the packet to be sent; and responding to the information storage success message which is sent by the information server and corresponds to the information package, and sending an offset mark updating instruction to the information acquisition cooperative program, wherein the offset mark updating instruction is used for indicating the information acquisition cooperative program to acquire the starting point of the information in the log file to be processed.

In some embodiments, the above method further comprises: and setting an information acquisition co-procedure and an information acquisition period of the information acquisition co-procedure for each log file to be processed in the at least one log file to be processed in response to detecting the at least one log file to be processed.

In some embodiments, the names of the packets to be sent include names of the log files to be processed and names of offset identifiers, the offset identifiers are used for marking starting positions of information to be read in the log files to be processed, and the packaging the packets to be sent in the buffer queue into the information packets includes: dividing the packets to be sent in the buffer queue according to the names of the packets to be sent to obtain at least one packet sequence to be sent, wherein the packets to be sent in the packet sequence to be sent belong to the same log file to be processed, and the packets to be sent in the packet sequence to be sent are ordered according to the names of the offset identifiers; and for the to-be-transmitted packet sequence in the at least one to-be-transmitted packet sequence, packaging the to-be-transmitted packets in the to-be-transmitted packet sequence into information packets, and setting the information packet names for the information packets.

In some embodiments, the encapsulating the packets to be sent in the sequence of packets to be sent into the information packets includes: and extracting the packets to be transmitted from the sequence of the packets to be transmitted in sequence, and packaging the packets to be transmitted into information packets.

In some embodiments, the setting a packet name for the packet includes: and setting the name of the information packet for the information packet according to the name of the offset identifier of the packet to be sent in the information packet.

In some embodiments, the sending the information packet to an information server includes: dividing the packet into at least two sub-packets not exceeding a data amount threshold in response to the data amount of the packet being greater than the data amount threshold; and transmitting the sub-information package in the at least two sub-information packages to an information server.

In a second aspect, embodiments of the present disclosure provide a method for transmitting information, the method comprising: acquiring an offset identifier of a log file to be processed, wherein the offset identifier is used for marking the starting point position of information to be read in the log file to be processed; taking the information to be read indicated by the offset mark as an information starting point, and acquiring the information to be sent from the log file to be processed; and packaging the information to be transmitted into a packet to be transmitted and transmitting the packet to a buffer queue.

In some embodiments, the obtaining the offset identifier of the log file to be processed includes: inquiring a log file to be processed in an information acquisition period set at intervals; and responding to the update information of the log file to be processed, and acquiring the offset identification of the log file to be processed.

In some embodiments, the obtaining the offset identifier of the log file to be processed includes: and acquiring the offset identification of the log file to be processed in response to the information quantity of the update information being greater than a set information quantity threshold.

In some embodiments, the information in the log file to be processed is provided with corresponding time information, and the obtaining the information to be sent from the log file to be processed using the information to be read indicated by the offset flag as an information starting point includes: and taking the information to be read indicated by the offset mark as an information starting point, taking the information of the last time information in the log file to be processed as an information ending point, and setting the information between the information starting point and the information ending point as information to be sent.

In some embodiments, the encapsulating the information to be sent into a packet to be sent and sending the packet to a buffer queue includes: and packaging the offset identifier and the information to be transmitted into a packet to be transmitted, and setting a name for the packet to be transmitted, wherein the name comprises the name of the log file to be processed and the name of the offset identifier.

In some embodiments, the above method further comprises: and setting an updating offset mark for the last piece of information corresponding to the information to be sent in the log file to be processed in response to receiving an offset mark updating instruction, wherein the updating offset mark is used for marking the starting point position of the information to be read next time in the log file to be processed.

In a third aspect, embodiments of the present disclosure provide an apparatus for transmitting information, the apparatus comprising: the first information sending unit is used for responding to the existence of a packet to be sent in a buffer queue, packaging the packet to be sent in the buffer queue into an information packet and sending the information packet to an information server, wherein the packet to be sent is sent to the buffer queue by an information acquisition protocol, and the information acquisition protocol is used for acquiring information from a log file to be processed and packaging the acquired information into the packet to be sent; the second information sending unit is configured to send an offset mark updating instruction to the information collection cooperative program in response to receiving the information storage success message corresponding to the information packet sent by the information server, wherein the offset mark updating instruction is used for indicating the start point of the information in the log file to be processed collected by the information collection cooperative program.

In some embodiments, the apparatus further comprises: an information collection co-procedure setting unit, in response to detecting at least one log file to be processed, is configured to set an information collection co-procedure and an information collection period of the information collection co-procedure for each of the at least one log file to be processed.

In some embodiments, the name of the packet to be sent includes a name of the log file to be processed and a name of an offset identifier, where the offset identifier is used to mark a start position of information to be read in the log file to be processed, and the first information sending unit includes: the sequence dividing subunit is configured to divide the packets to be sent in the buffer queue according to the names of the packets to be sent to obtain at least one packet sequence to be sent, wherein the packets to be sent in the packet sequence to be sent belong to the same log file to be processed, and the packets to be sent in the packet sequence to be sent are ordered according to the names of the offset identifiers; and the information packet encapsulation subunit is configured to encapsulate the to-be-sent packets in the to-be-sent packet sequence into information packets for the to-be-sent packet sequence in the at least one to-be-sent packet sequence, and set information packet names for the information packets.

In some embodiments, the packet encapsulation subunit includes: and the information packet encapsulation module is configured to extract the packets to be sent from the sequence of the packets to be sent in sequence and encapsulate the packets to be sent into the information packets.

In some embodiments, the packet encapsulation subunit includes: and the information packet name setting module is configured to set the information packet name for the information packet according to the name of the offset identifier of the packet to be sent in the information packet.

In some embodiments, the first information sending unit includes: a packet dividing subunit configured to divide the packet into at least two sub-packets not exceeding the data amount threshold in response to the data amount of the packet being greater than the data amount threshold; and a first information transmitting subunit configured to transmit the sub-packets of the at least two sub-packets to the information server.

In a fourth aspect, embodiments of the present disclosure provide an apparatus for transmitting information, the apparatus comprising: an offset identifier obtaining unit configured to obtain an offset identifier of a log file to be processed, where the offset identifier is used to mark a start position of information to be read in the log file to be processed; the information to be transmitted acquisition unit is configured to take the information to be read indicated by the offset mark as an information starting point and acquire the information to be transmitted from the log file to be processed; and the third information sending unit is configured to package the information to be sent into a packet to be sent and send the packet to the buffer queue.

In some embodiments, the offset identifier obtaining unit includes: the log inquiry subunit is configured to inquire the log files to be processed at intervals of the set information acquisition period; and the offset identifier acquisition subunit is used for responding to the update information of the log file to be processed and is configured to acquire the offset identifier of the log file to be processed.

In some embodiments, the offset identifier obtaining subunit includes: and the offset identifier acquisition module is used for responding to the fact that the information quantity of the update information is larger than a set information quantity threshold value and is configured to acquire the offset identifier of the log file to be processed.

In some embodiments, the information in the log file to be processed is provided with corresponding time information, and the information obtaining unit to be sent includes: and the information to be transmitted acquisition subunit is configured to take the information to be read indicated by the offset mark as an information starting point, take the last information of the time information in the log file to be processed as an information ending point, and set the information between the information starting point and the information ending point as the information to be transmitted.

In some embodiments, the third information sending unit includes: and the third information sending subunit is configured to package the offset identifier and the information to be sent into a packet to be sent, and set a name for the packet to be sent, wherein the name comprises the name of the log file to be processed and the name of the offset identifier.

In some embodiments, the apparatus further comprises: and the offset mark updating unit is used for setting an updating offset mark for the last piece of information corresponding to the information to be sent in the log file to be processed in response to receiving an offset mark updating instruction, and the updating offset mark is used for marking the starting point position of the information to be read next time in the log file to be processed.

In a fifth aspect, embodiments of the present disclosure provide an electronic device, comprising: one or more processors; and a memory having one or more programs stored thereon, which when executed by the one or more processors, cause the one or more processors to perform the method for transmitting information of the first aspect or the method for transmitting information of the second aspect.

In a sixth aspect, embodiments of the present disclosure provide a computer readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method for transmitting information of the first aspect or the method for transmitting information of the second aspect described above.

When a packet to be sent exists in a buffer queue, the packet to be sent in the buffer queue is packaged into an information packet, and the information packet is sent to an information server, wherein the packet to be sent is sent to the buffer queue by an information acquisition protocol, and the information acquisition protocol is used for acquiring information from a log file to be processed and packaging the acquired information into the packet to be sent; and then when receiving the information storage success message corresponding to the information packet sent by the information server, sending an offset mark updating instruction to the information acquisition cooperative program, wherein the offset mark updating instruction is used for indicating the information acquisition cooperative program to acquire the starting point of the information in the log file to be processed. The method and the device can acquire the information of a plurality of log files to be processed simultaneously, realize parallel acquisition of the log information and improve the acquisition efficiency of the log information.

Drawings

Other features, objects and advantages of the present disclosure will become more apparent upon reading of the detailed description of non-limiting embodiments, made with reference to the following drawings:

FIG. 1 is an exemplary system architecture diagram in which an embodiment of the present disclosure may be applied;

FIG. 2 is a flow chart of one embodiment of a method for transmitting information according to the present disclosure;

FIG. 3 is a schematic illustration of one application scenario of a method for transmitting information according to the present disclosure;

FIG. 4 is a flow chart of another embodiment of a method for transmitting information according to the present disclosure;

FIG. 5 is a schematic structural diagram of one embodiment of an apparatus for transmitting information according to the present disclosure;

FIG. 6 is a schematic structural diagram of another embodiment of an apparatus for transmitting information according to the present disclosure;

fig. 7 is a schematic diagram of an electronic device suitable for use in implementing embodiments of the present disclosure.

Detailed Description

The present disclosure is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings.

It should be noted that, without conflict, the embodiments of the present disclosure and features of the embodiments may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.

Fig. 1 illustrates an exemplary system architecture 100 to which a method for transmitting information or an apparatus for transmitting information of an embodiment of the present disclosure may be applied.

As shown in fig. 1, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, a log collection device 105, and an information server 106. The network 104 is a medium used to provide communication links between the terminal devices 101, 102, 103, the log collection device 105, and the information server 106. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The terminal devices 101, 102, 103 interact with the log collection device 105 through the network 104 to receive or send messages or the like. Various communication client applications, such as a web browser application, a shopping class application, a search class application, an instant messaging tool, a mailbox client, social platform software, etc., may be installed on the terminal devices 101, 102, 103.

The terminal devices 101, 102, 103 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, they may be various electronic devices having a display screen and supporting data processing, including but not limited to smartphones, tablet computers, electronic book readers, MP3 players (Moving Picture Experts Group Audio Layer III, dynamic video expert compression standard audio plane 3), MP4 (Moving Picture Experts Group Audio Layer IV, dynamic video expert compression standard audio plane 4) players, laptop and desktop computers, and the like. When the terminal devices 101, 102, 103 are software, they can be installed in the above-listed electronic devices. Which may be implemented as multiple software or software modules (e.g., to provide distributed services), or as a single software or software module, without limitation.

The log collection device 105 may be configured to set an information collection co-procedure for the log files on the terminal devices 101, 102, 103, so as to perform data collection on the log files through the information collection co-procedure. The log collection device 105 may send the received log data to the information server 106.

Note that the log collection device 105 may be hardware or software. When the log collection device 105 is hardware, it may be implemented as a distributed server cluster composed of a plurality of servers, or as a single server. When the log collection device 105 is software, it may be implemented as a plurality of software or software modules (e.g., to provide distributed services), or as a single software or software module, which is not specifically limited herein.

The information server 106 may be a server that provides various services, such as a server that processes log data sent from the log collection device 105. The server may analyze the received log data and the like, and store the processing result to the server.

It should be noted that, the method for sending information provided by the embodiments of the present disclosure is generally performed by the log collection device 105, and accordingly, the apparatus for sending information is generally disposed in the log collection device 105.

The server may be hardware or software. When the server is hardware, the server may be implemented as a distributed server cluster formed by a plurality of servers, or may be implemented as a single server. When the server is software, it may be implemented as a plurality of software or software modules (for example, to provide a distributed service), or may be implemented as a single software or software module, which is not specifically limited herein.

It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

With continued reference to fig. 2, a flow 200 of one embodiment of a method for transmitting information according to the present disclosure is shown. The method for transmitting information includes the steps of:

in step 201, in response to the existence of the packet to be sent in the buffer queue, the packet to be sent in the buffer queue is encapsulated into an information packet, and the information packet is sent to an information server.

In this embodiment, an execution body (for example, the log collection device 105 shown in fig. 1) of the method for sending information may receive, by a wired connection manner or a wireless connection manner, a packet to be sent by an information collection protocol on the terminal devices 101, 102, 103 to a buffer sequence, where the buffer sequence is located on the execution body and is used to store the packet to be sent. The to-be-sent packet contains information in a log file to be processed, which is acquired by the information acquisition protocol. It should be noted that the wireless connection may include, but is not limited to, 3G/4G connections, wiFi connections, bluetooth connections, wiMAX connections, zigbee connections, UWB (Ultra Wideband) connections, and other now known or later developed wireless connection means.

In the existing log information collection method, although periodic collection of log information can be realized, the log information can only be read and processed according to rows, and the log information cannot be efficiently pushed to the information server 106. In addition, in the existing log information acquisition method, a large number of small files can be generated due to time intervals and other reasons in the process of acquiring log information, and a plurality of log files on the same equipment cannot be acquired in parallel.

When there is a packet to be transmitted in a buffer queue on an execution body, the present application encapsulates the packet to be transmitted in the buffer queue into an information packet, and transmits the information packet to the information server 106. The execution body does not send the packets to be sent to the information server when each packet to be sent is received, but needs to encapsulate the packets to be sent into the information packets before sending the packets to the information server. The packaging process can effectively reduce the times and the quantity of sending the packets to be sent, and effectively reduce the occurrence of small files. And the packets to be sent are sent to a buffer queue by the information acquisition protocol. The information collection protocol can be used for collecting information from the log file to be processed and packaging the collected information into a package to be sent. The information collection coroutine can be a thread or a process running on the equipment where the log file to be processed is located, and is specifically set according to actual needs.

In some optional implementations of this embodiment, the method may further include: and setting an information acquisition co-procedure and an information acquisition period of the information acquisition co-procedure for each log file to be processed in the at least one log file to be processed in response to detecting the at least one log file to be processed.

The execution body may establish information connection with a plurality of terminal apparatuses 101, 102, 103 at the same time and monitor information change on the terminal apparatuses 101, 102, 103. When the execution body monitors that at least one new log file to be processed exists on the terminal device 101, 102, 103, an information collection coroutine may be set for each log file to be processed in the at least one log file to be processed, respectively. For example, the execution body may send an instruction to the terminal device 101, 102, 103 instructing the terminal device 101, 102, 103 to allocate a thread or process for collecting log information of the corresponding log file to be processed. After receiving the instruction, the terminal device 101, 102, 103 allocates a thread or process dedicated for collecting information to the log file to be processed. Thus, parallel information acquisition of a plurality of log files is realized. After that, the execution body may further set an information collection period of the information collection protocol, so that the information server 106 can obtain log information relatively stably, avoid network pressure caused by transmission of a large amount of log information in a short time, and reduce generation of small files.

In some optional implementations of this embodiment, the name of the packet to be sent may include a name of the log file to be processed and a name of an offset identifier, where the offset identifier is used to mark a starting point position of information to be read in the log file to be processed, and the encapsulating the packet to be sent in the buffer queue into an information packet may include the following steps:

the first step, dividing the packets to be sent in the buffer queue according to the names of the packets to be sent to obtain at least one sequence of the packets to be sent.

The information collection protocol may set a name for the packet to be sent in order to mark the packet to be sent. The name of the packet to be sent may include the name of the log file to be processed and the name of the offset identifier. The offset mark is used for marking the starting point position of the information to be read in the information acquisition co-procedure acquisition log file to be processed. The name of the offset identifier may be a line number of the log file to be processed, etc. and is used for characterizing the sequence of the information in the log file to be processed. As can be seen from the above description, the execution body may obtain the packets to be sent of the plurality of log files to be processed in parallel. In order to avoid information confusion in the information transmission process, the execution main body may further divide the packets to be sent in the buffer queue according to the names of the packets to be sent, so as to obtain at least one sequence of packets to be sent. Wherein, the packets to be sent in the sequence of packets to be sent belong to the same log file to be processed. The packets to be transmitted within the sequence of packets to be transmitted may be ordered by the name of the offset identifier.

The names of the offset identifiers can characterize the sequence of the information in the log file to be processed. Thus, the execution body may sort the packets to be transmitted within the sequence of packets to be transmitted by the name of the offset identification. And the information collection protocol collects information in the log file to be processed according to the offset mark and sends the packet to be sent. I.e. there is a consecutive order between each packet to be sent. Thus, the packets to be transmitted within the sequence of packets to be transmitted obtained by the execution body are also continuous.

And a second step of packaging the packets to be transmitted in the sequence of the packets to be transmitted into information packets and setting the information packet names for the information packets.

The execution body may encapsulate the packets to be sent in each sequence of packets to be sent to avoid encapsulating the packets to be sent of different log files to be processed together. In this way, it is also convenient to set the packet name for the packet.

In some optional implementations of this embodiment, the encapsulating the packets to be sent in the sequence of packets to be sent into the information packet may include: and extracting the packets to be transmitted from the sequence of the packets to be transmitted in sequence, and packaging the packets to be transmitted into information packets.

In order to avoid information confusion, the executing body may further extract the packets to be sent from the sequence of packets to be sent in order and package the packets to be sent into information packets.

In some optional implementations of this embodiment, the setting a packet name for the packet may include: and setting the name of the information packet for the information packet according to the name of the offset identifier of the packet to be sent in the information packet.

The execution body may also set a packet name for the packet according to the name of the offset identifier of the packet to be transmitted in the packet. For example, the information packet includes three packets to be transmitted. The names of the three packets to be sent may be: file 1-50 th line; file 1-150 th line; file 1-300 lines. Wherein, "File 1" is the name of the log file to be processed; "line 50", "line 150", "line 300" may be the name of the offset identifier. The packet name may be: file 1-lines 50-300.

In some optional implementations of this embodiment, the sending the information packet to the information server may include the following steps:

in a first step, in response to the data amount of a packet being greater than a data amount threshold, the packet is divided into at least two sub-packets that do not exceed the data amount threshold.

To ensure the success rate of each transmission of the packet to the information server, the execution body may limit the data amount of the packet according to the data amount threshold. When the data amount of the packet is greater than the data amount threshold, the execution body may divide the packet into at least two sub-packets that do not exceed the data amount threshold. For example, the packet name may be: the names of the two sub-packets obtained by dividing the file 1 to the 50 th-300 th line can be as follows: "File 1-50-300 lines-package 1" and "File 1-50-300 lines-package 2".

And a second step of transmitting the sub-packets of the at least two sub-packets to an information server.

The execution body may then send the sub-packets to the information server in the divided order.

Step 202, in response to receiving the information storage success message corresponding to the information packet sent by the information server, sending an offset flag update instruction to the information acquisition protocol.

When the information server receives the information packet and stores it, it can send information storage success message to the execution body. I.e. the information storage success message is used to characterize that the information package has been successfully stored by the information server. After receiving the information storage success message, the execution body can send an offset mark updating instruction to the information acquisition protocol. After the information acquisition protocol receives the offset mark updating instruction, the position of the information in the log file to be processed can be modified so as to avoid repeated information acquisition. That is, the offset flag update instruction may be used to instruct the information collection co-procedure to collect the start point of the information in the log file to be processed.

With continued reference to fig. 3, fig. 3 is a schematic diagram of an application scenario of the method for transmitting information according to the present embodiment. In the application scenario of fig. 3, there are a packet to be sent of the terminal device 102 acquired by the information acquisition co-procedure 1 and a packet to be sent of the terminal device 103 acquired by the information acquisition co-procedure 2 in a buffer queue of the log acquisition device 105. The log collection device 105 may encapsulate the packets to be sent into packets, and then send the packets to the information server 106. When the log collection device 105 receives the information storage success message sent from the information server 106, it is indicated that the information packet sent from the log collection device 105 has been successfully stored by the information server 106. The log collection device 105 may send an offset flag update instruction to the information collection co-procedure 1 and the information collection co-procedure 2, to instruct the information collection co-procedure 1 and the information collection co-procedure 2 to adjust the next collection start points of the log files to be processed on the terminal device 102 and the terminal device 103.

When a packet to be sent exists in a buffer queue, the method provided by the embodiment of the disclosure encapsulates the packet to be sent in the buffer queue into an information packet, and sends the information packet to an information server; and then, when receiving the information storage success message corresponding to the information packet sent by the information server, sending an offset mark updating instruction to the information acquisition protocol. The method and the device can acquire the information of a plurality of log files to be processed simultaneously, realize parallel acquisition of the log information and improve the acquisition efficiency of the log information.

With further reference to fig. 4, a flow 400 of yet another embodiment of a method for transmitting information is shown. The flow 400 of the method for transmitting information comprises the steps of:

step 401, obtaining an offset identifier of a log file to be processed.

In this embodiment, before acquiring information of a log file to be processed, an execution subject (for example, terminal devices 101, 102, 103 shown in fig. 1) of a method for transmitting information may first acquire an offset identifier of the log file to be processed through an information acquisition protocol. The offset mark is used for marking the starting point position of the information to be read in the log file to be processed.

In some optional implementations of this embodiment, the obtaining the offset identifier of the log file to be processed may include the following steps:

firstly, inquiring a log file to be processed at intervals of a set information acquisition period.

The change condition of the information in the log file to be processed is variable. In order to collect information in the log file to be processed in time, the situation that the information is collected after a large amount of new information appears in the log file to be processed is avoided, so that the information server 106 cannot acquire the log information in time. The execution body can inquire the log file to be processed at intervals of the set information acquisition period.

And secondly, responding to the update information of the log file to be processed, and acquiring the offset identification of the log file to be processed.

When the update information (i.e., the information which is not collected by the information collection protocol) of the log file to be processed exists, the execution subject may acquire the offset identifier of the log file to be processed.

In some optional implementations of this embodiment, the obtaining the offset identifier of the log file to be processed may include: and acquiring the offset identification of the log file to be processed in response to the information quantity of the update information being greater than a set information quantity threshold.

If the amount of information of the update information is small, the execution body may not update it to avoid sending the packets to be sent with a small amount of information excessively. When the information amount of the updated information is greater than the set information amount threshold, the execution body may acquire the offset identifier of the log file to be processed, so as to acquire information in the log file to be processed.

And step 402, taking the information to be read indicated by the offset mark as an information starting point, and acquiring the information to be transmitted from the log file to be processed.

The offset mark is used for marking the starting point position of the information to be read in the log file to be processed, and the execution body can take the information to be read indicated by the offset mark as the information starting point to acquire the information to be sent from the log file to be processed.

In some optional implementations of this embodiment, the information in the log file to be processed is provided with corresponding time information, and the obtaining the information to be sent from the log file to be processed using the information to be read indicated by the offset flag as an information starting point may include: and taking the information to be read indicated by the offset mark as an information starting point, taking the information of the last time information in the log file to be processed as an information ending point, and setting the information between the information starting point and the information ending point as information to be sent.

The information in the log file to be processed is typically provided with corresponding time information to mark the time of occurrence of the information. For this purpose, the execution body may use the information to be read indicated by the offset flag as an information start point, use the information of the last time information in the log file to be processed as an information end point, and set the information between the information start point and the information end point as information to be transmitted.

And step 403, packaging the information to be sent into a buffer queue for sending the packets to be sent to the terminal.

After reading the information to be read, the execution body may encapsulate the information to be sent into a packet to be sent and send the packet to the buffer queue of the log acquisition device 105.

In some optional implementations of this embodiment, the encapsulating the information to be sent into the packet to be sent and sending the packet to the buffer queue may include: and packaging the offset identifier and the information to be transmitted into a packet to be transmitted, and setting a name for the packet to be transmitted.

In order to facilitate the operations of the information server 106 on archiving the log information, the execution body may encapsulate the offset identifier and the information to be sent into a packet to be sent, and set a name for the packet to be sent. The names may include names of the log files to be processed and names of the offset identifiers.

In some optional implementations of this embodiment, the method may further include: and in response to receiving the offset mark updating instruction, setting an updating offset mark for the last piece of information corresponding to the information to be sent in the log file to be processed.

When the information server 106 receives the corresponding information, it sends an information storage success message to the log acquisition device 105. The log collection device 105 then sends an offset flag update instruction to the executing body. After the execution body receives the offset mark updating instruction, an updating offset mark can be set for the last piece of information corresponding to the information to be sent in the log file to be processed. The update offset flag may be used to mark a starting position of information to be read next in the log file to be processed.

With further reference to fig. 5, as an implementation of the method shown in the foregoing figures, the present disclosure provides an embodiment of an apparatus for transmitting information, where the embodiment of the apparatus corresponds to the embodiment of the method shown in fig. 2, and the apparatus may be specifically applied to various electronic devices.

As shown in fig. 5, the apparatus 500 for transmitting information of the present embodiment may include: a first information transmission unit 501 and a second information transmission unit 502. The first information sending unit 501 encapsulates, in response to the existence of a packet to be sent in the buffer queue, the packet to be sent in the buffer queue into an information packet, and is configured to send the information packet to the information server, where the packet to be sent is sent to the buffer queue by an information collection protocol, and the information collection protocol is used to collect information from a log file to be processed and encapsulate the collected information into the packet to be sent; the second information sending unit 502 is configured to send an offset flag update instruction to the information collection co-procedure in response to receiving the information storage success message corresponding to the information packet sent by the information server, where the offset flag update instruction is used to instruct the information collection co-procedure to collect the start point of the information in the log file to be processed.

In some optional implementations of this embodiment, the apparatus 500 for sending information may further include: an information collection co-procedure setting unit (not shown in the figure) is configured to set an information collection co-procedure and an information collection period of the information collection co-procedure for each of the at least one log file to be processed in response to detection of the at least one log file to be processed.

In some optional implementations of this embodiment, the name of the packet to be sent includes a name of the log file to be processed and a name of an offset identifier, where the offset identifier is used to mark a start position of information to be read in the log file to be processed, and

the first information transmitting unit 501 may include: a sequence dividing subunit (not shown) and a packet encapsulation subunit (not shown). The sequence dividing subunit is configured to divide the packets to be sent in the buffer queue according to the names of the packets to be sent to obtain at least one packet sequence to be sent, wherein the packets to be sent in the packet sequence to be sent belong to the same log file to be processed, and the packets to be sent in the packet sequence to be sent are ordered according to the names of the offset identifiers; and the information packet encapsulation subunit is configured to encapsulate the to-be-sent packets in the to-be-sent packet sequence into information packets for the to-be-sent packet sequence in the at least one to-be-sent packet sequence, and set information packet names for the information packets.

In some optional implementations of this embodiment, the packet encapsulation subunit may include: and the information packet encapsulation module (not shown in the figure) is configured to extract the packets to be sent from the sequence of packets to be sent in sequence and encapsulate the packets to be sent into the information packets.

In some optional implementations of this embodiment, the packet encapsulation subunit may include: a packet name setting module (not shown in the figure) configured to set a packet name for the packet according to the name of the offset identifier of the packet to be transmitted within the packet.

In some optional implementations of this embodiment, the first information sending unit 501 may include: a packet dividing subunit (not shown) and a first information transmitting subunit (not shown). Wherein, the information packet divides the subunit, respond to the data volume of the information packet is greater than the data volume threshold value, is configured to divide the above-mentioned information packet into at least two sub-information packets not exceeding the above-mentioned data volume threshold value; the first information transmitting subunit is configured to transmit a sub-packet of the at least two sub-packets to the information server.

With further reference to fig. 6, as an implementation of the method shown in the foregoing figures, the present disclosure provides an embodiment of an apparatus for transmitting information, which corresponds to the method embodiment shown in fig. 4, and which is particularly applicable to various electronic devices.

As shown in fig. 6, the apparatus 600 for transmitting information of the present embodiment may include: an offset identification acquisition unit 601, a to-be-transmitted information acquisition unit 602, and a third information transmission unit 603. The offset identifier obtaining unit 601 is configured to obtain an offset identifier of the log file to be processed, where the offset identifier is used to mark a starting point position of information to be read in the log file to be processed; a to-be-transmitted information obtaining unit 602 configured to obtain to-be-transmitted information from the to-be-processed log file, with the to-be-read information indicated by the offset flag as an information start point; the third information sending unit 603 is configured to encapsulate the information to be sent into a packet to be sent and send the packet to the buffer queue.

In some optional implementations of the present embodiment, the offset identifier obtaining unit 601 may include: a log query subunit (not shown) and an offset identification acquisition subunit (not shown). The log inquiring subunit is configured to inquire the log files to be processed in an information acquisition period set at intervals; and the offset identifier acquisition subunit is used for responding to the update information of the log file to be processed and is configured to acquire the offset identifier of the log file to be processed.

In some optional implementations of this embodiment, the offset identifier obtaining subunit may include: an offset identifier obtaining module (not shown in the figure) is configured to obtain an offset identifier of the log file to be processed in response to the information amount of the update information being greater than a set information amount threshold.

In some optional implementations of this embodiment, the information in the log file to be processed is provided with corresponding time information, and the information to be sent obtaining unit 602 may include: and a to-be-transmitted information obtaining subunit (not shown in the figure) configured to take the to-be-read information indicated by the offset mark as an information starting point, take the last information of the time information in the to-be-processed log file as an information ending point, and set the information between the information starting point and the information ending point as to-be-transmitted information.

In some optional implementations of this embodiment, the third information sending unit 603 may include: and a third information sending subunit (not shown in the figure) configured to encapsulate the offset identifier and the information to be sent into a packet to be sent, and set a name for the packet to be sent, where the name includes the name of the log file to be processed and the name of the offset identifier.

In some optional implementations of this embodiment, the apparatus 600 for sending information may further include: an offset flag updating unit (not shown in the figure) is configured to set an update offset flag for the last piece of information corresponding to the information to be transmitted in the log file to be processed, in response to receiving an offset flag update instruction, the update offset flag being used to mark a start position of information to be read next in the log file to be processed.

The embodiment also provides an electronic device, including: one or more processors; and a memory having one or more programs stored thereon, which when executed by the one or more processors, cause the one or more processors to perform the method for transmitting information described above.

The present embodiment also provides a computer-readable medium having stored thereon a computer program which, when executed by a processor, implements the above-described method for transmitting information.

Referring now to FIG. 7, a schematic diagram of a computer system 700 suitable for use with an electronic device (e.g., log collection device 105 of FIG. 1) implementing embodiments of the present disclosure is shown. The electronic device shown in fig. 7 is only one example and should not impose any limitations on the functionality and scope of use of embodiments of the present disclosure.

As shown in fig. 7, the electronic device 700 may include a processing means (e.g., a central processor, a graphics processor, etc.) 701, which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 702 or a program loaded from a storage means 708 into a Random Access Memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 are also stored. The processing device 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.

In general, the following devices may be connected to the I/O interface 705: input devices 706 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, and the like; an output device 707 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 708 including, for example, magnetic tape, hard disk, etc.; and a communication device 709. The communication means 709 may allow the electronic device 700 to communicate wirelessly or by wire with other devices to exchange data. While fig. 7 shows an electronic device 700 having various means, it is to be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead. Each block shown in fig. 7 may represent one device or a plurality of devices as needed.

In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via communication device 709, or installed from storage 708, or installed from ROM 702. The above-described functions defined in the methods of the embodiments of the present disclosure are performed when the computer program is executed by the processing device 701.

It should be noted that, the above-mentioned computer readable medium according to the embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the above-mentioned two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In an embodiment of the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. Whereas in embodiments of the present disclosure, the computer-readable signal medium may comprise a data signal propagated in baseband or as part of a carrier wave, with computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.

The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: responding to the existence of a packet to be sent in a buffer queue, packaging the packet to be sent in the buffer queue into an information packet, and sending the information packet to an information server, wherein the packet to be sent is sent to the buffer queue by an information acquisition protocol, and the information acquisition protocol is used for acquiring information from a log file to be processed and packaging the acquired information into the packet to be sent; and responding to the information storage success message which is sent by the information server and corresponds to the information package, and sending an offset mark updating instruction to the information acquisition cooperative program, wherein the offset mark updating instruction is used for indicating the information acquisition cooperative program to acquire the starting point of the information in the log file to be processed.

Computer program code for carrying out operations of embodiments of the present disclosure may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units involved in the embodiments described in the present disclosure may be implemented by means of software, or may be implemented by means of hardware. The described units may also be provided in a processor, for example, described as: a processor includes a first information transmitting unit and a second information transmitting unit. The names of these units do not constitute a limitation of the unit itself in some cases, and for example, the first information transmission unit may also be described as "a unit for transmitting log information to an information server".

The foregoing description is only of the preferred embodiments of the present disclosure and description of the principles of the technology being employed. It will be appreciated by those skilled in the art that the scope of the invention referred to in this disclosure is not limited to the specific combination of features described above, but encompasses other embodiments in which features described above or their equivalents may be combined in any way without departing from the spirit of the invention. Such as those described above, are mutually substituted with the technical features having similar functions disclosed in the present disclosure (but not limited thereto).

Claims

1. A method for transmitting information, comprising:

responding to a to-be-sent packet in a buffer queue, packaging the to-be-sent packet in the buffer queue into an information packet, and sending the information packet to an information server, wherein the to-be-sent packet is sent to the buffer queue by an information acquisition protocol, and the information acquisition protocol is used for acquiring information from a log file to be processed and packaging the acquired information into the to-be-sent packet;

responding to the information storage success message which is sent by the information server and corresponds to the information package, and sending an offset mark updating instruction to the information acquisition cooperative program, wherein the offset mark updating instruction is used for indicating the information acquisition cooperative program to acquire the starting point of information in a log file to be processed;

Wherein the name of the packet to be sent comprises the name of the log file to be processed and the name of an offset identifier, the offset identifier is used for marking the starting point position of the information to be read in the log file to be processed, and

the packaging the packets to be sent in the buffer queue into information packets includes:

dividing the packets to be sent in the buffer queue according to the names of the packets to be sent to obtain at least one packet sequence to be sent, wherein the packets to be sent in the packet sequence to be sent belong to the same log file to be processed, and the packets to be sent in the packet sequence to be sent are ordered according to the names of the offset identifiers;

and for a to-be-transmitted packet sequence in the at least one to-be-transmitted packet sequence, packaging the to-be-transmitted packets in the to-be-transmitted packet sequence into information packets, and setting information packet names for the information packets.

2. The method of claim 1, wherein the method further comprises:

in response to detecting at least one log file to be processed, an information acquisition co-procedure and an information acquisition period of the information acquisition co-procedure are set for each log file to be processed in the at least one log file to be processed.

3. The method of claim 1, wherein encapsulating the packets to be transmitted within the sequence of packets to be transmitted into packets of information comprises:

And extracting the packets to be transmitted from the sequence of the packets to be transmitted according to the sequence, and packaging the packets to be transmitted into information packets.

4. The method of claim 1, wherein said setting a packet name for a packet comprises:

and setting the name of the information packet for the information packet according to the name of the offset identifier of the packet to be sent in the information packet.

5. The method of any of claims 1 to 4, wherein the sending the information packet to an information server comprises:

responsive to the data amount of a packet being greater than a data amount threshold, dividing the packet into at least two sub-packets that do not exceed the data amount threshold;

and transmitting the sub-information package in the at least two sub-information packages to an information server.

6. A method for transmitting information, comprising:

acquiring an offset identifier of a log file to be processed, wherein the offset identifier is used for marking the starting point position of information to be read in the log file to be processed;

taking the information to be read indicated by the offset mark as an information starting point, and acquiring the information to be sent from the log file to be processed;

packaging the information to be sent into a packet to be sent and sending the packet to be sent to a buffer queue, wherein the method comprises the following steps: and packaging the offset identifier and the information to be sent into a packet to be sent, and setting a name for the packet to be sent, wherein the name comprises the name of the log file to be processed and the name of the offset identifier, so that the log acquisition equipment divides the packet to be sent in the buffer queue according to the name of the packet to be sent to obtain at least one packet sequence to be sent, wherein the packets to be sent in the packet sequence to be sent belong to the same log file to be processed, and the packets to be sent in the packet sequence to be sent are ordered according to the name of the offset identifier.

7. The method of claim 6, wherein the obtaining the offset identification of the log file to be processed comprises:

inquiring a log file to be processed in an information acquisition period set at intervals;

and responding to the update information of the log file to be processed, and acquiring the offset identification of the log file to be processed.

8. The method of claim 7, wherein the obtaining the offset identification of the log file to be processed comprises:

and acquiring the offset identification of the log file to be processed in response to the information quantity of the update information being greater than a set information quantity threshold.

9. The method of claim 6, wherein the information within the log file to be processed is provided with corresponding time information, and

the step of obtaining the information to be sent from the log file to be processed by taking the information to be read indicated by the offset mark as an information starting point includes:

and taking the information to be read indicated by the offset mark as an information starting point, taking the information of the last time information in the log file to be processed as an information ending point, and setting the information between the information starting point and the information ending point as information to be sent.

10. The method according to any one of claims 6 to 9, wherein the method further comprises:

And setting an updating offset mark for the last piece of information corresponding to the information to be sent in the log file to be processed in response to receiving an offset mark updating instruction, wherein the updating offset mark is used for marking the starting point position of the information to be read next time in the log file to be processed.

11. An apparatus for transmitting information, comprising:

the first information sending unit is used for responding to the existence of a packet to be sent in a buffer queue, packaging the packet to be sent in the buffer queue into an information packet and sending the information packet to an information server, wherein the packet to be sent is sent to the buffer queue by an information acquisition protocol, and the information acquisition protocol is used for acquiring information from a log file to be processed and packaging the acquired information into the packet to be sent;

the second information sending unit is used for responding to the information storage success message which is sent by the information server and corresponds to the information package, and is configured to send an offset mark updating instruction to the information acquisition cooperative program, wherein the offset mark updating instruction is used for indicating the information acquisition cooperative program to acquire the starting point of information in a log file to be processed;

The first information transmitting unit includes:

the sequence dividing subunit is configured to divide the packets to be sent in the buffer queue according to the names of the packets to be sent to obtain at least one packet sequence to be sent, wherein the packets to be sent in the packet sequence to be sent belong to the same log file to be processed, and the packets to be sent in the packet sequence to be sent are ordered according to the names of the offset identifiers;

and the information packet encapsulation subunit is configured to encapsulate the to-be-sent packets in the to-be-sent packet sequence into information packets for the to-be-sent packet sequence in the at least one to-be-sent packet sequence, and set information packet names for the information packets.

12. The apparatus of claim 11, wherein the apparatus further comprises:

an information collection co-procedure setting unit configured to set an information collection co-procedure and an information collection period of the information collection co-procedure for each of at least one log file to be processed in response to detection of the at least one log file to be processed.

13. The apparatus of claim 11, wherein the packet encapsulation subunit comprises:

and the information packet encapsulation module is configured to extract the packets to be sent from the sequence of the packets to be sent in sequence and encapsulate the packets to be sent into the information packets.

14. The apparatus of claim 11, wherein the packet encapsulation subunit comprises:

and the information packet name setting module is configured to set the information packet name for the information packet according to the name of the offset identifier of the packet to be sent in the information packet.

15. The apparatus according to any one of claims 11 to 14, wherein the first information transmitting unit includes:

a packet dividing subunit configured to divide the packet into at least two sub-packets not exceeding a data amount threshold in response to the data amount of the packet being greater than the data amount threshold;

and a first information transmitting subunit configured to transmit the sub-packets of the at least two sub-packets to the information server.

16. An apparatus for transmitting information, comprising:

the offset identification acquisition unit is configured to acquire an offset identification of the log file to be processed, wherein the offset identification is used for marking the starting point position of information to be read in the log file to be processed;

the information to be transmitted acquisition unit is configured to take the information to be read indicated by the offset mark as an information starting point and acquire the information to be transmitted from the log file to be processed;

the third information sending unit is configured to package the information to be sent into a packet to be sent and send the packet to a buffer queue;

Wherein the third information transmitting unit includes:

and the third information sending subunit is configured to encapsulate the offset identifier and the information to be sent into a packet to be sent, and set a name for the packet to be sent, wherein the name comprises the name of the log file to be processed and the name of the offset identifier, so that the log acquisition equipment divides the packet to be sent in the buffer queue according to the name of the packet to be sent to obtain at least one packet sequence to be sent, the packets to be sent in the packet sequence to be sent belong to the same log file to be processed, and the packets to be sent in the packet sequence to be sent are ordered according to the name of the offset identifier.

17. The apparatus of claim 16, wherein the offset identification acquisition unit comprises:

the log inquiry subunit is configured to inquire the log files to be processed at intervals of the set information acquisition period;

and the offset identification acquisition subunit is used for responding to the update information of the log file to be processed and is configured to acquire the offset identification of the log file to be processed.

18. The apparatus of claim 17, wherein the offset identification acquisition subunit comprises:

and the offset identification acquisition module is used for responding to the fact that the information quantity of the update information is larger than a set information quantity threshold value and is configured to acquire the offset identification of the log file to be processed.

19. The apparatus of claim 16, wherein the information within the log file to be processed is provided with corresponding time information, and

the information to be transmitted acquisition unit includes:

and the information to be transmitted acquisition subunit is configured to take the information to be read indicated by the offset mark as an information starting point, take the last information of the time information in the log file to be processed as an information ending point, and set the information between the information starting point and the information ending point as the information to be transmitted.

20. The apparatus according to any one of claims 16 to 19, wherein the apparatus further comprises:

and the offset mark updating unit is used for setting an updating offset mark for the last piece of information corresponding to the information to be sent in the log file to be processed in response to receiving an offset mark updating instruction, and the updating offset mark is used for marking the starting point position of the information to be read next time in the log file to be processed.

21. An electronic device, comprising:

one or more processors;

a memory having one or more programs stored thereon,

the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any of claims 1-5 or the method of any of claims 6-10.

22. A computer readable medium on which a computer program is stored, characterized in that the program, when executed by a processor, implements the method of any one of claims 1 to 5 or the method of any one of claims 6 to 10.