CN113934683A

CN113934683A - Log transmission method and device, CDN system and electronic equipment

Info

Publication number: CN113934683A
Application number: CN202010668653.2A
Authority: CN
Inventors: 张景顺
Original assignee: Beijing Kingsoft Cloud Network Technology Co Ltd
Current assignee: Beijing Kingsoft Cloud Network Technology Co Ltd
Priority date: 2020-07-13
Filing date: 2020-07-13
Publication date: 2022-01-14

Abstract

The embodiment of the invention provides a log transmission method and device, a CDN system and electronic equipment. Wherein the method comprises the following steps: acquiring a log to be transmitted; format conversion is carried out according to the information carried by the log to be transmitted and the agreed log format agreed by a preset log template, so that a user-defined log conforming to the agreed log format is generated, wherein the preset log template is pre-configured for a target user to which the log to be transmitted belongs; and uploading the custom log to a preset log storage device. The log formats of the logs uploaded to the log storage device can be configured through the log templates, different log templates can be set for different users, the logs of different users have different log formats, namely, the customization of the log formats is realized, and the bottom layer logic does not need to be changed according to the user requirements, so that the cost of customizing the log formats can be reduced.

Description

Log transmission method and device, CDN system and electronic equipment

Technical Field

The present invention relates to the technical field of Content Delivery Networks (CDNs), and in particular, to a log transmission method and apparatus, a CDN system, and an electronic device.

Background

A Content Delivery Network (CDN) generates a large number of logs during operation, where the logs carry detailed user access information, so that a user of the CDN system can obtain information from the logs and analyze a service running on the CDN based on the obtained information.

According to different log formats, the information carrying modes of the logs are different, for example, a first field in one log format is used for representing the log type, a third field in another log format is used for representing the log type, for example, the log type in one log format is represented in a letter form, and the log type in another log format is represented in a number form. The way of acquiring information from the log should theoretically correspond to the way of carrying information by the log, and different users may have different requirements on the log format because the different users have different actual requirements.

In the related technology, a user may negotiate with a provider of the CDN system, and the provider of the CDN system modifies underlying logic of logs recorded by each CDN node in the CDN system, so that a format in which the logs are recorded by the CDN nodes may meet a requirement of the user, but the cost of modifying the underlying logic is high.

Disclosure of Invention

The embodiment of the invention aims to provide a log transmission method, a log transmission device, a CDN system and electronic equipment so as to reduce the cost of a user-defined log format. The specific technical scheme is as follows:

in a first aspect of the embodiments of the present invention, a log transmission method is provided, where the log transmission method is applied to any electronic device with a log transmission function in a content delivery network CDN system, and the method includes:

acquiring a log to be transmitted;

format conversion is carried out according to the information carried by the log to be transmitted and the agreed log format agreed by a preset log template, so that a user-defined log conforming to the agreed log format is generated, wherein the preset log template is pre-configured for a target user to which the log to be transmitted belongs;

and uploading the custom log to a preset log storage device.

In a possible embodiment, the preset log template includes at least one field format information, and each field format information is used for indicating a generation mode of a log field in the log conforming to the agreed log format;

the method for generating the custom log according with the agreed log format comprises the following steps of carrying out format conversion according to the information carried by the log to be transmitted and the agreed log format agreed by a preset log template, wherein the format conversion comprises the following steps:

generating a custom log field according to the information carried by the log to be transmitted and the generation mode represented by each field format information in a preset log template;

and combining all the custom log fields to obtain a custom log.

In a possible embodiment, the field format information includes assignment information and identification information, the assignment information is used to indicate an obtaining manner of a value, and the identification information is used to indicate an identification manner of the value;

the generation generates the custom log field according to the generation mode represented by each field format information in the preset log template, and the generation method comprises the following steps:

acquiring a value according to an acquisition mode represented by assignment information in the field format information;

and identifying the obtained value according to the identification mode represented by the identification information in the field format information to obtain the custom log field.

In a possible implementation, the assignment information includes an original field identifier, where the original field identifier is an identifier of a field in the log to be transmitted;

the obtaining a value according to the obtaining mode indicated by the assignment information in the field format information includes:

and acquiring the value of the field marked as the original field mark from the log to be transmitted.

In a possible embodiment, the assignment information comprises assignment parameters;

and carrying out preset mapping on the assignment parameters in the assignment information of the field format information, and taking the mapping result as the obtained value.

In a possible embodiment, the preset log template includes service information, where the service information is used to indicate a service for generating the log to be transmitted;

before format conversion is performed according to an agreed log format agreed by a preset log template according to the information carried by the log to be transmitted and a custom log conforming to the agreed log format is generated, the method further comprises the following steps:

and determining the information carried by the log to be transmitted according to the log format agreed by the service represented by the service information.

In a possible embodiment, the uploading the custom log to a preset log storage device includes:

and uploading the custom log to a log storage device preset for the target user.

In a second aspect of the embodiments of the present invention, a log transmission module is provided, which is applied to any electronic device with a log transmission function in a CDN, and the apparatus includes:

the log acquisition module is used for acquiring a log to be transmitted;

the format conversion module is used for carrying out format conversion according to the information carried by the log to be transmitted and the agreed log format agreed by a preset log template to generate a custom log which accords with the agreed log format, wherein the preset log template is pre-configured for a target user to which the log to be transmitted belongs;

and the log uploading module is used for uploading the custom log to a preset log storage device.

the format conversion module performs format conversion according to the agreed log format agreed by a preset log template according to the information carried by the log to be transmitted, and generates a custom log conforming to the agreed log format, and the format conversion module comprises:

and combining all the custom log fields to obtain a custom log.

the format conversion module generates a custom log field according to a generation mode represented by each field format information in a preset log template, and the generation mode comprises the following steps:

In a possible embodiment, the assignment information includes an original field identifier, where the original field identifier is an identifier of a field in the log to be transmitted;

the format conversion module acquires the value according to the acquisition mode represented by the assignment information in the field format information, and the acquisition mode comprises the following steps:

the format conversion module is further configured to determine the information carried by the log to be transmitted according to the log format agreed by the service represented by the service information before performing format conversion according to the agreed log format agreed by a preset log template according to the information carried by the log to be transmitted and generating a custom log conforming to the agreed log format.

In a possible embodiment, the uploading the custom log to a preset log storage device by the log uploading module includes:

In a third aspect of the embodiments of the present invention, a CDN system is provided, where the CDN system includes a plurality of CDN nodes;

each CDN node uploads a log in a local cache of the CDN node to a preset log storage device by using the log transmission method according to any one of the first aspects.

In a fourth aspect of embodiments of the present invention, there is provided an electronic device including a computer-readable storage medium storing a computer program;

a processor for implementing the method steps of any of the above first aspects when executing a program stored on a computer readable storage medium.

In a fifth aspect of embodiments of the present invention, there is provided a computer-readable storage medium having stored therein a computer program which, when executed by a processor, performs the method steps of any one of the above-mentioned first aspects.

The embodiment of the invention has the following beneficial effects:

according to the log uploading method and device, the CDN system and the electronic device provided by the embodiment of the invention, the log format of the log uploaded to the log storage device can be configured through the log template, different log templates can be set for different users, the logs of different users have different log formats, namely, the self-definition of the log format is realized, and the bottom layer logic does not need to be changed according to the user requirements, so that the cost for self-defining the log format can be reduced. .

Of course, not all of the advantages described above need to be achieved at the same time in the practice of any one product or method of the invention.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by using the drawings without creative efforts.

Fig. 1 is a schematic flowchart of a log transmission method according to an embodiment of the present invention;

fig. 2 is a schematic structural diagram of a task management module according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a log template configuration interface according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of a configuration information configuration interface according to an embodiment of the present invention;

FIG. 5 is a schematic view of a monitoring graph according to an embodiment of the present invention;

fig. 6 is a schematic structural diagram of a log transmission apparatus according to an embodiment of the present invention;

fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

In order to more clearly describe the log transmission method provided by the embodiment of the present invention, a possible application scenario of the log transmission method provided by the embodiment of the present invention will be exemplarily described below, where the following example is only one possible application scenario of the log transmission method provided by the embodiment of the present invention, and the log transmission method provided by the embodiment of the present invention may also be applied to other possible application scenarios in other possible embodiments.

The CDN system may include a log center and a plurality of CDN nodes, where the CDN nodes are configured to implement services deployed in the CDN system, such as an on-demand service and a live broadcast service, and each CDN node generates a large amount of logs in a service operation process. The CDN node may upload the generated cache to a log center at intervals of a preset upload period, such as 2 hours and 2 days, in a local cache. The CDN system provider can provide a download address of the log stored in the log center for a CDN user, and the user downloads the log from the log center according to the download address.

The log format of the log is determined by the CDN node underlying logic, and it is difficult for the CDN system provider to know the actual requirements of the user on the log format in advance when developing the CDN node underlying logic, so the log format of the log center may not meet the actual requirements of the user. For example, a user may wish that the first field of the log represents the log type, while the log type in the log of the log center may be carried in the third field. Making it difficult for the user to obtain information from the log.

Although the user can reflect the actual demand to the CDN provider, and the CDN provider modifies the underlying logic of the CDN node according to the actual demand of the user, the development cost of redeveloping the underlying logic is high, and it is difficult to modify the underlying logic in time according to the demand of the user.

Based on this, the embodiment of the present invention provides a log transmission method, and the log transmission method provided by the embodiment of the present invention may be applied to any electronic device with a log transmission function in a CDN system, such as a CDN node, a log center, and the like in the CDN system. Referring to fig. 1, fig. 1 is a schematic flow chart of a log transmission method according to an embodiment of the present invention, where the log transmission method includes:

s101, obtaining a log to be transmitted.

S102, according to the information carried by the log to be transmitted, format conversion is carried out according to the agreed log format agreed by the preset log template, and a custom log conforming to the agreed log format is generated.

And S103, uploading the custom log to a preset log storage device.

By adopting the embodiment, the log format of the log uploaded to the log storage device can be configured through the log template, different log formats of different users can be realized by setting different log templates for different users, namely, the self-definition of the log format is realized, and the bottom layer logic does not need to be changed according to the user requirements, so that the cost of self-defining the log format can be reduced.

In S101, the log to be transmitted may be acquired in different manners according to different execution subjects. For example, if the execution subject is a CDN node, the log in the local cache of the CDN node may be obtained, and if the execution subject is a log center, the log uploaded by each CDN node may be obtained.

The execution timing of S101 may also be different according to different application scenarios, and may be executed in real time, or may be executed at preset intervals, for example, once every 2 hours or 2 days. And in a possible embodiment, the two execution modes can be switched between real-time execution and execution every preset period according to requirements. Compared with executing the S101 at preset intervals, executing the S101 in real time enables the user to obtain logs in real time, so that the user can grasp the situation of the CDN system in real time. Compared with the real-time execution of the S101, the execution of the S101 at intervals of the preset period can reduce the number of uploading times, and is convenient for a user to perform centralized processing on the log.

In S102, the preset log template is pre-configured for the target user to which the log to be transmitted belongs, and a configuration manner of the preset log template will be described in detail in the subsequent embodiments, which is not described herein again. The way of presetting the format of the log template for appointing the log can be different according to different application scenes.

For example, in one possible embodiment, the preset log template may include at least one field format information, where each field format information is used to indicate a generation manner of a log field in the log conforming to the agreed log format.

For example, assuming that the log conforming to the agreed log format includes 3 log fields, the preset log template may include three fields of format information. When the custom log is generated, the custom log field can be generated according to the information carried by the log to be transmitted and the generation mode represented by the format information of each field in the preset log template.

Suppose a log field can be represented as "information name: the form of the value, such as "log time: no. 1/18 hours 55 minutes in 2020 "," node: node 3 ", where the information name is used to identify a value, the value may be a string, character, numeric value, etc. used to represent a particular meaning. The field format information may indicate how the value in the log field is obtained and how the obtained value is identified.

For example, in one possible embodiment, the field format information may include assignment information and identification information, where the assignment information is used to indicate an obtaining manner of the value, and the identification information is used to indicate an identification manner of the value. When the custom log field is generated according to the generation mode indicated by the field format information, the value may be acquired according to the acquisition mode indicated by the assignment information in the field format information, and the acquired value may be identified according to the identification mode indicated by the identification information in the field format information, so as to obtain the custom log field.

In a possible embodiment, the obtaining manner indicated by the assignment information may include two manners:

the acquisition method is as follows: obtaining a value carried by a designated field from a log to be transmitted;

and the second acquisition mode is as follows: the parameter mapping method is obtained by mapping preset parameters.

Two modes will be explained separately below. For the first mode, the assignment information may include an original field identifier, where the original field identifier is an identifier of a field in the log to be transmitted, and the original field identifier in the assignment information may be configured according to actual requirements of a user, for example, the original field identifier is an identifier indicating a field carrying a log type in the log to be transmitted. The value is obtained according to the obtaining mode indicated by the assignment information, and the value can be obtained from the log to be transmitted, wherein the field value is identified as the original field identification.

For the second mode, the assignment information may include assignment parameters or may include a mapping mode, where the mapping mode may not be provided according to different application scenarios, and for example, a user-Defined function UDF (user Defined functions) may be used to map preset parameters, that is, the assignment parameters are used as input of the UDF to obtain output of the UDF, where the output is an obtained value. The assignment parameter may be set in advance by a relevant person according to an actual requirement, or may be determined according to a preset rule, which is not limited in this embodiment. In other possible embodiments, the manner represented by the assignment information may also be other than the first manner and the second manner, for example, the manner represented by the assignment information may be: and obtaining a value carried by the designated field from the log to be transmitted, and mapping the obtained value by using preset parameters. For example, assuming that the information carried by the third field in the original format is time, and the carried time is represented by a hexadecimal numerical value, and the time in the log is a decimal numerical value according to actual needs of the user, the assignment information may be set, and the obtaining manner represented by the assignment information is as follows: and acquiring the value carried by the third field from the log to be transmitted, and inputting the acquired value into a function with the parameter being a preset parameter and used for converting the hexadecimal numerical value into the decimal numerical value.

In a possible embodiment, the adopted mapping manner may also be a mapping in which the input and the output are equal, that is, the obtained mapping value is the preset parameter itself, and in this embodiment, the preset parameter may be directly used as the obtained value. And obtaining the value according to the obtaining mode indicated by the assignment information, wherein the obtaining mode can be that the assignment parameters in the assignment information are mapped in advance, and the mapping result is used as the obtained value.

The information carried by the log to be transmitted can be determined from the log to be transmitted, and it can be understood that the log formats of the log to be transmitted generated according to different services are different, so that the meanings of the information carried by the same field of different logs may be different. For example, a first field in a log generated by a download service is used to carry a log generation time, and first fields in a log generated by a live broadcast service may be used to represent nodes.

Therefore, in a possible embodiment, in order to accurately determine the information carried in the log to be transmitted, the preset log template may include service information, where the service information is used to indicate a service for generating the log to be transmitted. In this embodiment, the information carried by the log to be transmitted may be determined according to a log format agreed by the service indicated by the service information. For example, assuming that the service information indicates that the service generating the log to be transmitted is a download service, the time for generating the log may be determined according to a value carried by a first field in the log to be transmitted, and assuming that the service information indicates that the service generating the log to be transmitted is a live broadcast service, the node may be determined according to a value carried by a first field in the log to be transmitted.

The manner of identifying the obtained value may be different according to different application scenarios, for example, the identification information may include an identifier, and the identification manner represented by the identification information may be: the identifier and the preset symbol included in the identifier information, such as colon, pause, comma, etc., are added before the acquired value. For example, assuming that the obtained value is 192.168.1.1 and the identifier included in the identification information is "IP address", the "IP address" may be added before "192.168.1.1": the obtained value is identified by the manner of "i.e. the generated field is" IP address: 192.168.1.1".

In S103, the preset log storage device may be different according to different application scenarios, for example, in a possible embodiment, when the execution subject is a CDN node, the preset log storage device may be a log center in the CDN system.

It can be understood that a CDN system often has many users, and if a large number of users download logs from a log center, bandwidth pressure of the log center may be large, and therefore, in a possible embodiment, the preset storage device may be a log storage device configured in advance for a target user, for example, may be a user server of each user.

In order to more clearly describe the log transmission method provided by the embodiment of the present invention, a CDN node is taken as an execution subject, and a principle of the log transmission method provided by the embodiment of the present invention is described below with reference to a configuration manner of a preset log template.

In one possible embodiment, a collection (input) module, a computation (control) module, a real-time backhaul (output) module, and a control (controller) module may be included on each CDN node.

The system comprises a collecting module, a processing module and a processing module, wherein the collecting module is used for acquiring logs in real time from a cache of a CDN node;

and the calculation module is used for screening out invalid logs from the logs acquired by the collection module, wherein the invalid logs can be logs which do not conform to a preset original format, logs with abnormal carried information, logs which cannot be decoded and the like, and the preset original format is an agreed format of logs generated by the CDN node bottom layer logic. For example, assuming that the third field of the specified log in the preset original format is used for representing the log type, and the log type is represented in the form of letters, and the third field of the log a carries a number, the information carried by the log a can be considered to be abnormal, and the log a can be screened out.

The real-time return module is used for uploading the logs screened by the calculation module to a preset log storage device through any one of the log transmission methods;

the control module is used for acquiring a preset log template configured in advance for a target user from the control center, wherein the control center is a device independent of the CDN node, and the user or related staff can configure the preset log template in the control center locally or in a remote access mode to the control center.

For example, in a possible embodiment, as shown in fig. 2, a log transmission module Edgeon is set in a CDN node, the Edgeon module includes a Task management module Task Manager, and the Task management module may have a structure as shown in fig. 2, and includes a collection module 210, a computation module 220, a real-time backhaul module 230, and a control module 240.

The collection module 210 is configured to obtain logs from a cache Cahce of the CDN node in real time, and the calculation module 220 is configured to decode, filter, and clean the logs obtained by the collection module 210, so as to screen out invalid logs from the logs to be transmitted.

The real-time backhaul module 230 may include a Filter (Filter) sub-module 231, a convert (Handle) sub-module 232, and a pack (Flash) sub-module 233. The filtering submodule 231 is configured to filter the logs filtered by the computing module 220 again to determine the logs to be uploaded, where a rule for determining whether the logs need to be uploaded is preset by a relevant person according to an actual requirement, for example, it may be determined whether a user to which the logs belong is a target user, and if the user to which the logs belong is the target user, the logs are uploaded, where the user to which the logs belong refers to a user to which a service for generating the logs belongs.

The converting submodule 232 is configured to perform format conversion according to the agreed log format agreed by the preset log template according to the information carried by the log to be transmitted, and generate a custom log conforming to the agreed log format, where for format conversion, reference may be made to the related description in S102, which is not described herein again.

The packing submodule 233 is configured to pack and upload the custom log to a user server of a target user, where a packing manner may be different according to different application scenarios, and may be, for example, a form of packing the custom log into an array, or a form of packing the custom log into a non-array.

The control module 240 is configured to obtain configuration information pre-configured for the target user from the control center, where the configuration information includes a preset log template. The following respectively describes the configuration information and the preset log template, and refer to fig. 3, where fig. 3 is a schematic diagram of a log template configuration interface provided in an embodiment of the present invention.

The user or related personnel of the CDN provider can pre-configure the log template through the log template configuration interface. In the example shown in fig. 3, the service includes a download service (i.e., a download platform) and a live broadcast platform (i.e., a live broadcast platform), and in other possible embodiments, the service may also include other services, such as an intelligent operation service. The present embodiment does not limit this.

The field interval symbol is set to indicate the division symbol in different log fields, and may be set to be, for example, a pause sign, a comma, a semicolon, or the like. The line break setting is used to indicate a division symbol between different lines in the log, and may be, for example, a line break \ n, where a null value is used by default to indicate that a value of the null value is replaced when the null value is obtained from the log to be transmitted, and for example, it is assumed that an obtaining manner of a value of a custom log field in a preset log template is: obtaining a value carried by a second field in the log to be transmitted, wherein the value of the second field in the log to be transmitted is a null value, if the value of the custom log field agreed in the preset log template uses a node identifier, and the null value is default to be minus, the custom log field in the generated custom log can be a node: - ".

The custom fields are the aforementioned field format information, and the sequence numbers are used to identify the different custom fields, and in one possible embodiment, the earlier the sequence numbers the earlier the custom fields are in the custom log. The assignment type, assignment parameter, original field identifier, and original field name are used to indicate the manner of obtaining the value in the custom field, i.e., the assignment information. The assignment type can include three types, namely a fixed value, an original value and a user-defined function, when the assignment type is the fixed value or the user-defined function, assignment parameters need to be configured, when the assignment type is the fixed value, the value of the user-defined field is the assignment parameters, and when the assignment type is the user-defined function, the value of the user-defined field is a mapping value obtained by mapping the assignment parameters by using the user-defined function. When the assignment type is an original value, an original field identifier and/or an original field name need to be configured, the original field identifier is an English identifier of a field in the log to be transmitted, the original field name is a Chinese identifier of the field in the log to be transmitted, the configured original field identifier is the same as a field represented by the configured original field name, in a possible embodiment, only the original field identifier can be configured, and the corresponding original field name can be automatically configured according to the configured original field identifier.

The derived field identification, the derived field name, and the derived field type are used to indicate the identification manner of the value, i.e., the aforementioned identification information. The derived field identifier may be an english identifier of the obtained value, the derived field name may be a chinese identifier of the obtained value, and the derived field type may be a representation of the identifier of the value, such as a string (string).

Examples to show examples of custom fields. The upper arrow and the lower arrow of the operation are used for adjusting the sequence of the custom format information, and the garbage can icon is used for deleting the custom format information.

The log template configured according to the configuration interface may be represented by a json file as follows:

12 in order, serial number

"category":3, # assigned type, 3 denotes udf

Retention of type

Value, # null, # Retention

"function": merge ", # assign udf name

"function _ args" [ $9-server _ addr _ server _ port, $ cache _ layer ], "|", # assigned parameter

"source_field_name":"9-server_addr_server_port",

Source _ field _ name _ CN, local IP address local port and Chinese identification

"result_field_name":"",

"result_field_name_CN":"",

"result_field_name_type":"",

"result_field_name_sample":null

}

The configuration interface of the configuration information may be as shown in fig. 4, where the task name, the user ID, the service platform, and the domain name range are used to identify an object of the real-time task, that is, to represent a log to be transmitted.

As an example, assuming that a task name, a user ID, a service platform, and a domain name range in the configuration information indicate that a log to be transmitted is a log 1-100, and a template name in the configuration information is used to identify a log template a, the CDN node may perform format conversion on the log 1-100 according to an agreed log format agreed by the log template a to generate a custom log, and upload the generated custom log to a user server of a user corresponding to the configuration information.

The task type and the return method are used for configuring a mode that the CDN node uploads the custom log to the preset storage device, when the task type is a real-time task, the CDN node uploads the custom log in real time, and when the task type is an off-line task, the CDN node uploads the custom log once every preset period. And when the back-transmission mode is GET, the CDN node uploads the custom log in a GET mode, and when the back-transmission mode is POST, the CDN node uploads the custom log in a POST mode.

Whether the user-defined logs are uploaded in the form of packaging the user-defined logs into an array or in the form of packaging the user-defined logs into a character string when the user-defined logs are uploaded in the form of array return. The return address is used to indicate an address of the predetermined log storage device, for example, assuming that the predetermined log storage device is a user server of the user, the return address may be configured as an address of the user server of the user. The UA is used for designating an agent used in the process of uploading the custom log to the preset log storage device.

The back-transmission mode is used for configuring a path for uploading the custom log to the preset log storage device, when the back-transmission mode is edge back-transmission, the custom log can be directly uploaded to the preset log storage device by each CDN node, when the back-transmission mode is center back-transmission, the custom log can be uploaded to a log center in the CDN system by each CDN node, and then the custom log uploaded by each CDN node is uploaded to the preset log storage device by the log center in a unified mode.

The trigger conditions are used for configuring trigger conditions for uploading the custom logs by the CDN nodes, and the values configured in the plurality of sending online lines represent: when the number of the custom logs in the CDN node reaches the value, uploading the custom logs in the CDN node, wherein the value configured when the single batch is sent with the maximum delay represents: the CDN node will upload the custom log at least once every interval of the value.

In some possible embodiments, in order to facilitate the user or the related personnel of the CDN system provider to monitor the log uploading situation, a fighter monitoring chart may be set in the preset display device, where the monitoring chart may be as shown in fig. 5, where the abscissa represents the date, the ordinate represents the log number, the curve on the left side represents the change of the number of logs successfully uploaded with time, and the curve on the lower right side represents the change of the number of logs failed to upload with time.

Referring to fig. 6, fig. 6 is a schematic structural diagram of a log transmission apparatus according to an embodiment of the present invention, where the log transmission apparatus may include:

a log obtaining module 601, configured to obtain a log to be transmitted;

a format conversion module 602, configured to perform format conversion according to an agreed log format agreed by a preset log template according to information carried by the log to be transmitted, and generate a custom log conforming to the agreed log format, where the preset log template is pre-configured for a target user to which the log to be transmitted belongs;

a log uploading module 603, configured to upload the user-defined log to a preset log storage device.

the format conversion module 602 performs format conversion according to an agreed log format agreed by a preset log template according to the information carried by the log to be transmitted, and generates a custom log conforming to the agreed log format, including:

and combining all the custom log fields to obtain a custom log.

the format conversion module 602 generates a custom log field according to a generation manner indicated by each field format information in a preset log template, including:

the format conversion module 602 obtains the value according to the obtaining mode indicated by the assignment information in the field format information, including:

In a possible embodiment, the log uploading module 603 uploads the custom log to a preset log storage device, including:

An embodiment of the present invention further provides an electronic device, as shown in fig. 7, including a processor 701, a communication interface 702, a memory 703 and a communication bus 704, where the processor 701, the communication interface 702, and the memory 703 complete mutual communication through the communication bus 704,

a memory 703 for storing a computer program;

the processor 701 is configured to implement the following steps when executing the program stored in the memory 703:

acquiring a log to be transmitted;

and uploading the custom log to a preset log storage device.

and combining all the custom log fields to obtain a custom log.

the obtaining the value according to the obtaining mode indicated by the assignment information in the field format information includes:

The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.

The communication interface is used for communication between the electronic equipment and other equipment.

The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.

The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.

In yet another embodiment provided by the present invention, a computer-readable storage medium is further provided, in which a computer program is stored, and the computer program, when executed by a processor, implements the steps of any of the log transmission methods described above.

In yet another embodiment, a computer program product containing instructions is provided, which when run on a computer causes the computer to perform any of the log transmission methods of the above embodiments.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the embodiments of the apparatus, the electronic device, the CDN system, the computer readable storage medium, and the computer program product, since they are substantially similar to the method embodiments, the description is relatively simple, and it suffices to refer to the partial description of the method embodiments for relevant points.

The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims

1. A log transmission method is applied to any electronic equipment with a log transmission function in a Content Delivery Network (CDN) system, and comprises the following steps:

acquiring a log to be transmitted;

and uploading the custom log to a preset log storage device.

2. The method according to claim 1, wherein the preset log template comprises at least one field format information, and each field format information is used for representing a generation mode of a log field in the log conforming to the agreed log format;

and combining all the custom log fields to obtain a custom log.

3. The method according to claim 2, wherein the field format information includes assignment information and identification information, the assignment information is used for indicating an obtaining mode of the value, and the identification information is used for indicating an identification mode of the value;

the generating of the custom log field according to the generating mode represented by each field format information in the preset log template includes:

4. The method according to claim 3, wherein the assignment information includes an original field identifier, and the original field identifier is an identifier of a field in the log to be transmitted;

5. The method of claim 3, wherein the assignment information includes assignment parameters;

6. The method according to claim 1, wherein the preset log template comprises service information, and the service information is used for representing a service for generating the log to be transmitted;

7. The method of claim 1, wherein uploading the custom log to a preset log storage device comprises:

8. A log transmission device is applied to any electronic equipment with a log transmission function in a CDN, and the device comprises:

the log acquisition module is used for acquiring a log to be transmitted;

9. A CDN system, wherein the CDN system comprises a plurality of CDN nodes;

each CDN node uploads a log cached locally by the CDN node to a preset log storage device by the log transmission method of any one of claims 1 to 7.

10. The system according to claim 9, wherein the CDN node comprises a collection module, a computation module, a real-time backhaul module, and a control module;

the collection module is used for acquiring logs from the cache of the CDN node in real time;

the calculation module is used for screening out invalid logs from the logs acquired by the collection module;

the real-time return module is used for uploading the logs screened by the computing module to a preset log storage device through the log transmission method according to any one of claims 1 to 7;

and the control module is used for acquiring a preset log template which is configured in advance for a target user from the control center.

11. The system according to claim 10, wherein the real-time backhaul module comprises a filtering sub-module, a converting sub-module, and a packaging sub-module;

the filtering submodule is used for determining logs to be transmitted from the logs screened by the computing module;

the conversion submodule is used for carrying out format conversion according to an agreed log format agreed by a preset log template according to information carried by the log to be transmitted, and generating a custom log which accords with the agreed log format, wherein the preset log template is pre-configured for a target user to which the log to be transmitted belongs;

and the packaging submodule is used for packaging and uploading the custom log to a preset log storage device.

12. An electronic device, comprising:

a computer-readable storage medium for storing a computer program;

a processor for implementing the method steps of any of claims 1-7 when executing a program stored on a computer readable storage medium.

13. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of claims 1 to 7.