CN112380001A - Log output method, load balancing device and computer readable storage medium - Google Patents
Log output method, load balancing device and computer readable storage medium Download PDFInfo
- Publication number
- CN112380001A CN112380001A CN202011187866.XA CN202011187866A CN112380001A CN 112380001 A CN112380001 A CN 112380001A CN 202011187866 A CN202011187866 A CN 202011187866A CN 112380001 A CN112380001 A CN 112380001A
- Authority
- CN
- China
- Prior art keywords
- log
- core
- session information
- cores
- type
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 90
- 238000012545 processing Methods 0.000 claims abstract description 58
- 230000008569 process Effects 0.000 claims abstract description 42
- 238000004590 computer program Methods 0.000 claims description 5
- 230000004931 aggregating effect Effects 0.000 claims description 3
- 238000013439 planning Methods 0.000 claims description 2
- 239000000126 substance Substances 0.000 claims description 2
- 230000006399 behavior Effects 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 230000003993 interaction Effects 0.000 description 4
- 230000001174 ascending effect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000008092 positive effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 235000019580 granularity Nutrition 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000012384 transportation and delivery Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 238000013024 troubleshooting Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1734—Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a log output method, load balancing equipment and a computer readable storage medium. In the invention, when receiving the service flow, the designated work core in the multi-core CPU carries out data plane forwarding processing on the service flow and dispatches the processed service flow to the back-end server, meanwhile, the generated session information corresponding to various log types is recorded to the ring queue corresponding to the working core in the processing process, and the designated log core in the multi-core CPU is set in the process that the working core executes the processing, periodically acquiring session information from a ring queue corresponding to the working core according to the preset frequency corresponding to various log types and outputting the session information as a log file, the forwarding of the service flow and the log output are not interfered with each other, and simultaneously the log files of various log types can be ensured to be output in a balanced way, therefore, under the condition that the number of the CPU cores is limited, various attack behaviors can be found in time, and corresponding early warning is given, so that the protection safety is improved.
Description
Technical Field
The embodiment of the invention relates to the technical field of computer networks, in particular to a log output method, load balancing equipment and a computer readable storage medium.
Background
Logs are data generated during the running of programs. The complete log has positive effects on the aspects of program maintenance, fault backtracking, user behavior analysis and the like, and particularly has positive effects on the development, operation and maintenance of a large-scale real-time system, such as various load balancing programs and schemes suitable for the application scene of a Content Delivery Network (CDN).
At present, in order to facilitate troubleshooting, in a program and a scheme implemented based on a load balancing technology, a log is generally required to be output to a service traffic forwarded by a load balancing device, so as to facilitate subsequent positioning analysis.
However, the output of the log is often performed directly during the transfer of the traffic flow. Although this method can realize the output of the log, since the I/O read-write speed is much slower than the speed of a Central Processing Unit (CPU) processing the service traffic, even if the I/O read-write operation is processed in batch by using a cache method, the forwarding speed of the service traffic is still seriously affected by the log output of the CPU performing the service traffic forwarding during the service traffic forwarding process.
To address this problem, a Data Plane Development Kit (DPDK) framework based log processing system application is created. The log processing system realizes independent operation of service flow forwarding and log output by introducing the multi-core CPU and dividing the multi-core CPU into the log core and the working core. However, the working principle of the DPDK framework is that the log core processes the logs based on a polling method, which results in that only a certain type of logs are output in a period of time, and when all the types of logs are output, other types of logs can be processed, thereby seriously affecting the monitoring of other logs.
Disclosure of Invention
An embodiment of the present invention provides a log output method, a load balancing device, and a computer-readable storage medium, which are used to solve the above technical problems.
In order to solve the above technical problem, an embodiment of the present invention provides a log output method, which is applied to a load balancing device of a CPU having a multi-core processor, where the log output method includes:
receiving service flow, performing data plane forwarding processing on the service flow by using a working core in the multi-core CPU, scheduling the processed service flow to a rear-end server, and recording session information corresponding to various log types generated in the processing process into an annular queue corresponding to the working core;
and for each type of log type, in the process of carrying out data plane forwarding processing on the service flow by the working core, acquiring corresponding session information from the annular queue by using the log core in the multi-core CPU at a preset frequency corresponding to the log type, and outputting the session information as a log file corresponding to the log type.
An embodiment of the present invention further provides a load balancing apparatus, including:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the at least one processor is a multi-core processor comprising a working core and a log core, the memory storing instructions executable by the at least one processor to enable the at least one processor to perform the log output method as described above.
Embodiments of the present invention also provide a computer-readable storage medium storing a computer program which, when executed by a processor, implements the log output method as described above.
In the log output method, the load balancing device and the computer readable storage medium provided by the embodiment of the invention, by adopting the load balancing device of the multi-core CPU, when receiving the service traffic, the specified work core in the multi-core CPU performs data plane forwarding processing on the service traffic, and schedules the processed service traffic to the back-end server, and simultaneously records the generated session information corresponding to various log types to the ring queue corresponding to the work core in the processing process, and sets the specified log core in the multi-core CPU to periodically acquire the session information from the ring queue corresponding to the work core according to the preset frequency corresponding to various log types in the process of executing the processing by the work core, and outputs the acquired session information as a log file, thereby realizing that the forwarding of the service traffic and the output of the log are not interfered with each other, and ensuring the forwarding speed of the service traffic, the logs corresponding to various log types can be output in real time, so that the user experience is guaranteed, and the subsequent analysis and positioning according to the log files can be facilitated.
In addition, according to the log output method, the load balancing device and the computer readable storage medium provided by the embodiment of the invention, when the log output is performed by setting the designated log core in the multi-core CPU, the log file is output at the preset frequency corresponding to different log types, so that the log file of each log type is prevented from occupying the log core for a long time, and the log files of each log type can be output in a balanced manner, and therefore, under the condition that the number of the CPU cores is limited, the output frequency of each log can be dynamically planned according to the log types corresponding to different service flows, the integrity and the real-time performance of the output log files of each log type are further ensured, and thus, various attack behaviors can be found in time according to the real-time and complete logs of each output log type, and further, corresponding early warning is made, so that the protection safety is improved.
In addition, before the obtaining, by using the log core in the multi-core CPU, the corresponding session information from the ring queue at a preset frequency corresponding to the log type and outputting the session information as a log file corresponding to the log type, the method further includes: determining the number and the length of the session information corresponding to the log type to obtain a first number and a first length; and planning the preset frequency corresponding to the log type according to the first quantity and the first length. According to the embodiment of the invention, the preset frequency corresponding to the log type of the log type is planned according to the number and the length of the session information corresponding to the log type, so that the dynamic adjustment of the preset frequency corresponding to each type of log type is realized, and the log core can more reasonably and uniformly output various types of log files.
In addition, before the recording the session information corresponding to various types of log types generated in the processing process into the ring queue corresponding to the working core, the method further includes: for each type of log type, distributing a ring queue corresponding to the log type for the working core; recording session information corresponding to various log types generated in the processing process into a ring queue corresponding to the working core, wherein the recording of the session information corresponding to various log types generated in the processing process into the ring queue corresponding to the working core comprises the following steps: and recording the session information corresponding to each type of log type generated in the processing process into a ring queue distributed for the log type in the working core. According to the embodiment of the invention, the ring queues for recording the session information of different log types are distributed to the working cores, so that the log types can be classified from the source, and the log cores can output log files of different types by acquiring the session information from the ring queues corresponding to the different log types.
In addition, the recording the session information corresponding to each type of the log types generated in the processing process into a ring queue allocated to the log types in the work core includes: counting the number of the session information corresponding to each type of the log types generated in the processing process to obtain a second number; judging whether the second quantity meets a first preset condition or not; and if so, recording the session information into a ring queue distributed for the log type in the working core by taking the second quantity as a write-once batch. According to the embodiment of the invention, the session information is recorded into the corresponding annular queue in a batch input mode, so that the processing efficiency of the working core is improved, and the consumption of the working core resources is effectively reduced.
In addition, the acquiring the corresponding session information from the circular queue at the preset frequency corresponding to the log type, and outputting the session information as a log file corresponding to the log type includes: counting the number of the session information recorded in the annular queue to obtain a third number; judging whether the third quantity meets a second preset condition or not; if so, acquiring the batch by taking the third quantity as one time, acquiring the corresponding session information from the annular queue by taking the preset frequency corresponding to the log type, and outputting the session information as a log file corresponding to the log type. According to the embodiment of the invention, the session information is acquired from the annular queue in a batch reading mode by setting the log cores, and the session information is output as the log file, so that the throughput of the log file is greatly improved, and the consumption of log core resources is effectively reduced.
In addition, the working cores are multiple; for each log type, in the process of performing data plane forwarding processing on the service traffic by the work core, acquiring, by using a log core in the multi-core CPU, the corresponding session information from the ring queue at a preset frequency corresponding to the log type, and outputting the session information as a log file corresponding to the log type, including: for each type of log type, in the process of performing data plane forwarding processing on the service flow by the working core, acquiring corresponding session information from the ring queue corresponding to each working core by using the log core in the multi-core CPU at a preset frequency corresponding to the log type; and aggregating the session information acquired from the annular queue corresponding to each working core, and outputting the session information as a log file corresponding to the log type.
In addition, before the receiving the traffic, the method further includes: determining the number of cores of the multi-core CPU; determining the number of the working cores according to a preset working core proportion and the number of the working cores, and selecting the cores with the number of the working cores from the multi-core CPU as the working cores; and determining the number of log cores according to a preset log core proportion and the number of the cores, and selecting the cores with the number of the log cores from the rest cores in the multi-core CPU as the log cores.
In addition, the selecting, as the log core, a core of the number of log cores from the cores remaining in the multi-core CPU includes: acquiring the load state of each residual core in the multi-core CPU; and selecting the cores with the number of the log cores from the rest cores in the multi-core CPU as the log cores according to the load states of the rest cores.
Drawings
One or more embodiments are illustrated by way of example in the accompanying drawings, which correspond to the figures in which like reference numerals refer to similar elements and which are not to scale unless otherwise specified.
Fig. 1 is a schematic diagram of interaction between a work core and a log core in a log output method according to a first embodiment of the present invention;
fig. 2 is a schematic diagram of interaction between a load balancing device and a client and a backend server, where the load balancing device is applied to the log output method according to the first embodiment of the present invention;
fig. 3 is a schematic diagram of interaction between a work core and a log core in the log output method provided by the first embodiment of the present invention;
fig. 4 is a schematic diagram of interaction between a work core and a log core in the log output method provided by the second embodiment of the present invention;
fig. 5 is a schematic structural diagram of a load balancing apparatus according to a third embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, it will be appreciated by those of ordinary skill in the art that in various embodiments of the invention, numerous technical details are set forth in order to provide a better understanding of the present application. However, the technical solution claimed in the present application can be implemented without these technical details and various changes and modifications based on the following embodiments. The following embodiments are divided for convenience of description, and should not constitute any limitation to the specific implementation manner of the present invention, and the embodiments may be mutually incorporated and referred to without contradiction.
The first embodiment of the invention relates to a log output method, which is applied to load balancing equipment, and a processor in the load balancing equipment is a multi-core CPU.
In practical applications, the load balancing device of the multi-core CPU is, for example, a backend server, and for convenience of description, the load balancing device is taken as an example for description in this embodiment.
To facilitate understanding of the technical solution provided in this embodiment, it should be first described that the multi-core CPU in the load balancing device in this embodiment is at least divided into a working core for forwarding service traffic and a log core for outputting a log.
Regarding the division of the working core and the log core, the division may be specifically determined according to the number of cores of the multi-core CPU and a preset ratio, such as a working core ratio and a log core ratio.
Specifically, in practical application, before the load balancing device implements the log output method provided in this embodiment, the number of cores of the multi-core CPU adopted by the load balancing device may be determined; then, determining the number of the working cores according to a preset working core proportion and the number of the cores, and determining the number of the log cores according to a preset log core proportion and the number of the cores; then, the cores of the number of the working cores are selected from the multi-core CPU to be the working cores, and the cores of the number of the log cores are selected from the cores remaining in the multi-core CPU to be the log cores.
For example, for any multi-core CPU, for convenience of explanation, the number of cores of the CPU is represented by N, and the ratios of the appointed operating cores are all
The ratio of the log cores is
For an 8-core CPU, 4 cores may be set as a working core, and 1 core may be set as a log core; for a 16-core CPU, 8 cores are required to be set as working cores, and 1 core is required to be set as a log core; for a 32-core CPU, 16 cores thereof need to be set as work cores, and 1 core needs to be set as a log core.
For example, for any multi-core CPU, the duty ratio of the working core can be predetermined to be
With the proportion of the log core being the working core
I.e. integral
For an 8-core CPU, 4 cores are required to be set as working cores, and 1 core is required to be set as a log core; for a 16-core CPU, 8 cores are required to be set as working cores, and 2 cores are required to be set as log cores; for a 32-core CPU, 16 cores thereof need to be set as work cores, and 4 cores need to be set as log cores.
It should be understood that the above examples are only examples for better understanding of the technical solution of the present embodiment, and are not to be taken as the only limitation to the present embodiment.
In addition, it is worth mentioning that, in practical applications, when the cores with the number of the log cores are selected from the cores remaining in the multi-core CPU, and the cores are used as the log cores, it may be specifically determined which of the remaining cores are selected as the log cores according to the load states of the cores remaining in the multi-core CPU.
Specifically, during implementation, the load state of each remaining core in the multi-core CPU may be obtained first; then, according to the load states of the remaining cores, cores of the number of the log cores are selected from the remaining cores in the multi-core CPU as the log cores.
For example, the load states of the remaining cores are sorted in an ascending order, and then the cores corresponding to the sorted load states are sequentially selected as the log cores until the number of the selected log cores is equal to the determined number of the log cores.
Taking a 32-core CPU as an example, if the determined number of the work cores is 16 and the number of the log cores is 4, after 16 cores are selected from the 32 cores of the 32-core CPU as the work cores, it is necessary to sort the work cores in an ascending order according to the load states of the remaining 16 cores, that is, to arrange the core with the smallest current load state in front, and select the first 4 cores from the sorted cores as the log cores in the order.
As can be seen from the above description, the selection of the log core is selected according to the dynamically changing load state, that is, the log core determined at the current time is satisfied with the current performance of the load balancing device.
In addition, in practical application, for an application scenario in which the load state change of each core is small, the working core and the log core are determined based on the above manner, and a plurality of cores can be directly fixed as the log core as required.
It should be understood that the above examples are only examples for better understanding of the technical solution of the present embodiment, and are not to be taken as the only limitation to the present embodiment.
For convenience of understanding the log output method of the present embodiment, details of the implementation of the log output method of the present embodiment are described below with reference to fig. 1 to 3.
It should be understood that the following are implementation details provided only for ease of understanding and are not necessary to practice the present solution.
As shown in fig. 1, for a working core of the multi-core CPU, when receiving a service traffic, a data plane forwarding process is performed on the service traffic, and session information corresponding to various types of logs generated in the processing process is recorded in a ring queue corresponding to the working core.
Meanwhile, after the data plane forwarding processing of the service traffic is completed, the processed service traffic is directly dispatched to a back-end server.
It should be understood that, in practical applications, different service flows are often processed by different backend servers, and thus after completing the data plane forwarding processing on the service flows, the worker core essentially schedules the processed service flows to a specific backend server to which the service flows are directed.
In addition, in this embodiment, the service traffic is generated according to a service operation triggered by the client in practical application, that is, the service traffic comes from the client.
For ease of understanding, the following description is made in conjunction with FIG. 2:
as shown in fig. 2, it is assumed that, in a certain time period, i clients initiate service processing requests to a load balancing device including a j-core CPU, and the load balancing device acquires, according to the service processing requests initiated by each client, service traffic corresponding to the service processing requests, in order to distinguish the service traffic generated between the clients and determine which one of k backend servers needs to process the service traffic corresponding to the clients. The acquired service flow corresponding to each client at least needs to carry a source IP address and a source port of the client, and a destination IP address and a destination port of a back-end server which needs to be accessed.
Assuming that a destination IP address and a destination port carried in a service flow corresponding to the client 1 indicate that a backend server which provides a service for the client is the backend server 1, the service flow corresponding to the client is subjected to data plane forwarding processing by a work core (specifically, which one of the CPU1 and the CPU2.. CPUj is predetermined according to a preset configuration requirement) in the load balancing device, and then is directly scheduled to the backend server 1 by the work core in the load balancing device.
It should be understood that the values of i and k are required to be integers greater than 1, and the value of j is required to be at least an integer greater than 2, usually an integer multiple of 8.
In addition, as can be seen from fig. 1, in the process that the work core performs data plane forwarding processing on the service traffic and records session information corresponding to various types of logs generated in the processing process into the ring queue corresponding to the work core, the log core performs the following operations without affecting the processing of the work core on the service traffic:
for each type of log type, the log core acquires the session information corresponding to the current log type from the corresponding ring queue, and outputs the acquired session information as a log file corresponding to the current log type, that is, only the session information corresponding to the current log type is recorded in the generated log file.
As can be seen from the above description, the forwarding of the service traffic by the load balancing device is performed by the single work core, and the output of the log is performed by the single log core, so that the forwarding of the service traffic and the output of the log may not interfere with each other.
Further, in practical applications, in order to facilitate the clustering of the log files output by the log core according to the log types, the log files are output, that is, the session information corresponding to different log types is output as different log files, for example, for the session information of the CC Attack (Challenge Collapsar attach) type mainly used for attacking the page, the session information is output as a CC Attack log file, and the session information of the session speed-limiting type is output as a session speed-limiting log file.
For each type of log type, a ring queue corresponding to each log type may be allocated to the work core in advance.
Correspondingly, when the working core records the session information corresponding to various log types generated in the processing process into the ring queue corresponding to the working core, the session information corresponding to each log type generated in the processing process is specifically recorded into the ring queue allocated for the log type by the working core, for example, the session information of the CC attack type is separately recorded into the CC attack ring queue, and the session information of the session type speed limit is separately recorded into the session speed limit ring queue.
Therefore, by distributing the ring queues for recording the session information of different log types to the working cores, the log types can be classified from the source, and the log cores can output log files of different types by acquiring the session information from the ring queues corresponding to the different log types.
It should be noted that the session is generally called "session control" in a computer, especially in a network application, and is used for storing attributes and configuration information required for a specific user session.
Further, in practical applications, in order to improve the performance of the work core and the log core as much as possible and reduce the consumption of resources of the work core and the log core, for the work core, session information generated in the processing process may be recorded in batch to a corresponding ring queue, and for the log core, session information may be acquired in batch from the corresponding ring queue.
Specifically, to implement batch recording of session information, the following implementation logic may be set for the work core:
firstly, counting the number of the session information corresponding to each type of the log types generated in the processing process to obtain a second number.
Then, it is determined whether the second number satisfies a first predetermined condition, such as reaching a threshold.
For example, in practical applications, the threshold may be set according to the maximum number of pieces of data that can be processed by the working core at a time or the data amount, such as 32 pieces of data or 100KB of data for the first preset condition.
Therefore, the second number may be the number of pieces of session information or the size of data amount.
Correspondingly, if the number of the session information corresponding to a certain log type generated at the current moment is determined to meet the first preset condition through judgment, the second number is taken as a write-once batch, and the session information is recorded into the ring queue distributed for the log type in the working core.
Correspondingly, in order to achieve batch acquisition of session information, a similar implementation flow may be set for the log core:
firstly, the number of the session information recorded in the ring queue is counted to obtain a third number.
And then, judging whether the third quantity meets a second preset condition.
Correspondingly, if the log type is satisfied, the third quantity is used as one time to obtain the batch, the corresponding session information is obtained from the annular queue according to the preset frequency corresponding to the log type, and the session information is output as the log file corresponding to the log type.
It should be noted that, in practical applications, the multi-core CPU in the load balancing device may reserve several cores for processing other services besides dividing the cores into several cores as the working cores and several cores as the log cores. For example, a main CPU core for uniformly receiving service flows sent by different clients particularly obtains a load state of each working core, and then allocates the currently received service flows to corresponding working cores according to the load state to perform plane data forwarding processing and scheduling of the service flows.
Correspondingly, the quantity statistics and judgment operations required for batch recording of session information may be implemented by the corresponding work core, or by the main CPU core.
Similarly, the number statistics and judgment operations that need to be performed when session information is obtained in batches may also be implemented by the log core or the main CPU core.
In addition, it is worth mentioning that in practical applications, in general, a plurality of work cores are determined in the load balancing device, and for this case, when the log core acquires session information from the ring queues corresponding to the work cores and outputs the session information as a log file, specifically:
for each type of log type, in the process of performing data plane forwarding processing on the service flow by the working core, acquiring corresponding session information from the ring queue corresponding to each working core by using the log core in the multi-core CPU at a preset frequency corresponding to the log type;
and aggregating the session information acquired from the annular queue corresponding to each working core, and outputting the session information as a log file corresponding to the log type.
For ease of understanding, the following description is made in conjunction with FIG. 3:
assume that there are 3 cores in the multi-core CPU of the current load balancing device as working cores, which are respectively the working core 1, the working core 2, and the working core 3 in fig. 3.
And according to the session information generated in the process of carrying out data plane forwarding processing on the service flow by the 3 working cores, the session information generated by the working core 1 is classified into an A log type and an A 'log type, the session information generated by the working core 2 is classified into the A log type, and the session information generated by the working core 3 is classified into the A log type and the A' log type.
Based on this, the work core 1 is allocated with a ring queue for recording the a-log type and a ring queue for recording the a 'log type, the work core 2 is allocated with a ring queue for recording the a-log type, and the work core 3 is allocated with a ring queue for recording the a-log type and a ring queue for recording the a' log type.
Correspondingly, when session information a (session information a1, session information a 2.) of an a log type generated in the process of performing service traffic processing by the worker core 1 meets a certain preset condition, batch recording the session information a into a ring queue of the a log type corresponding to the worker core 1; when session information a ' (session information a ' 1, session information a ' 2.) of an a ' log type generated in the process of processing the service traffic by the working core 1 meets a certain preset condition, the session information a ' 1 and the session information a ' 2 are recorded in batch into an annular queue of the a ' log type corresponding to the working core 1.
The recording process types of the working core 2 and the working core 3 are not described in detail here.
Accordingly, it is assumed that the preset frequency set for the a log type is to acquire session information of one preset batch per minute, and the preset frequency set for the a' log type is to acquire session information of one preset batch per 30 seconds.
Then, for the log core, in a certain work cycle, the session information in the corresponding a log type ring queue and the session information in the a ' log type ring queue in the 3 work cores are alternately acquired at the above two time granularities, and further, the session information acquired from the corresponding a log type ring queue of each of the 3 work cores is output as an a log type log file, and the session information acquired from the corresponding a ' log type ring queue of each of the 3 work cores is output as an a ' log type log file. By the mode of outputting the log files at the preset frequencies of different log types, the log file of a single type is effectively prevented from occupying a log core for a long time, and the integrity and the real-time performance of the log files of various log types are ensured.
It should be understood that the above examples are only examples for better understanding of the technical solution of the present embodiment, and are not to be taken as the only limitation to the present embodiment.
It is not difficult to find out through the above description that, in the log output method provided in this embodiment, by using the load balancing device of the multi-core CPU, when receiving the service traffic, the service traffic is subjected to data plane forwarding processing by the designated work core in the multi-core CPU, and the processed service traffic is scheduled to the back-end server, and at the same time, session information corresponding to various types of generated logs is recorded in the ring queue corresponding to the work core during the processing, and the designated log core in the multi-core CPU is set to periodically obtain the session information from the ring queue corresponding to the work core according to the preset frequency corresponding to various types of logs during the processing performed by the work core, and output the obtained session information as a log file, thereby implementing non-interference between the forwarding of the service traffic and the output of the logs, and while ensuring the forwarding speed of the service traffic, the log can be output in real time, so that the user experience is guaranteed, and the subsequent analysis and positioning can be conveniently carried out according to the log file.
In addition, according to the log output method provided by this embodiment, when the log core specified in the multi-core CPU is configured to output the log file at the preset frequency of different log types, the log file of a single type is prevented from occupying the log core for a long time, so that the log files of various log types can be output in a balanced manner, and therefore, the integrity and the real-time performance of the log files of various log types are ensured.
A second embodiment of the present invention relates to a log output method. The second embodiment is further improved on the basis of the first embodiment, and the main improvements are as follows: the preset frequency of the log core for acquiring the session information from the ring-shaped queues corresponding to different log types is dynamically determined according to the number of the session information generated in real time when the work core processes the service flow and the length of each piece of session information.
As shown in fig. 2, before the log core obtains the session information from the ring queue corresponding to each log type corresponding to the work core, the log core needs to first obtain a first quantity and a first length of the session information recorded in the corresponding ring queue from the work core, and then plan a preset frequency of the corresponding log type according to the obtained first quantity and the first length, that is, according to the quantity of the session information, the length or the size of each piece of session information, how often the session information of the current log type is obtained is determined.
Specifically, the first number and the first length acquired by the log core may be determined by the work core, or may be determined by the main CPU core described in the first embodiment.
Accordingly, for the manner determined by the worker core, the log core is obtained directly from the worker core for the first number and the first length; for the mode determined by the main CPU core, the log core needs to acquire the first number and the first length from the main CPU core, or initiates a request for acquiring the first number and the first length to the work core, and the work core acquires the first number and the first length from the main CPU core and feeds back the first number and the first length to the log core.
It should be understood that the above examples are only examples for better understanding of the technical solution of the present embodiment, and are not to be taken as the only limitation to the present embodiment.
Therefore, according to the log output method provided by the embodiment, the preset frequency corresponding to the log type of the type is planned according to the number and the length of the session information corresponding to the log type, so that the dynamic adjustment of the preset frequency corresponding to each type of log type is realized, and the log core can more reasonably and uniformly output various types of log files.
It should be understood that the above steps of the various methods are divided for clarity, and the implementation may be combined into one step or split into a plurality of steps, and all that includes the same logical relationship is within the protection scope of the present patent; it is within the scope of the patent to add insignificant modifications to the algorithms or processes or to introduce insignificant design changes to the core design without changing the algorithms or processes.
A third embodiment of the present invention relates to a load balancing apparatus, as shown in fig. 5, including at least one processor 501; and a memory 502 communicatively coupled to the at least one processor 501; wherein the at least one processor 501 is a multi-core processor, the multi-core processor includes a working core and a log core, and the memory 502 stores instructions executable by the at least one processor 501, and the instructions are executed by the at least one processor 501, so that the at least one processor 501 can execute the log output method described in the above first or second embodiment.
The memory 502 and the processor 501 are coupled by a bus, which may include any number of interconnected buses and bridges that couple one or more of the various circuits of the processor 501 and the memory 502 together. The bus may also connect various other circuits such as peripherals, voltage regulators, power management circuits, and the like, which are well known in the art, and therefore, will not be described any further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver may be one element or a plurality of elements, such as a plurality of receivers and transmitters, providing a means for communicating with various other apparatus over a transmission medium. The data processed by the processor 501 is transmitted over a wireless medium via an antenna, which further receives the data and transmits the data to the processor 501.
The processor 501 is responsible for managing the bus and general processing and may also provide various functions including timing, peripheral interfaces, voltage regulation, power management, and other control functions. And memory 502 may be used to store data used by processor 501 in performing operations.
A fourth embodiment of the present invention relates to a computer-readable storage medium storing a computer program. The computer program, when executed by a processor, implements the log output method embodiments described above.
That is, as can be understood by those skilled in the art, all or part of the steps in the method for implementing the embodiments described above may be implemented by a program instructing related hardware, where the program is stored in a storage medium and includes several instructions to enable a device (which may be a single chip, a chip, or the like) or a processor (processor) to execute all or part of the steps of the method described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
It will be understood by those of ordinary skill in the art that the foregoing embodiments are specific embodiments for practicing the invention, and that various changes in form and details may be made therein without departing from the spirit and scope of the invention in practice.
Claims (10)
1. A log output method is applied to load balancing equipment of a CPU (central processing unit) of a multi-core processor, and comprises the following steps:
receiving service flow, performing data plane forwarding processing on the service flow by using a working core in the multi-core CPU, scheduling the processed service flow to a rear-end server, and recording session information corresponding to various log types generated in the processing process into an annular queue corresponding to the working core;
and for each type of log type, in the process of carrying out data plane forwarding processing on the service flow by the working core, acquiring corresponding session information from the annular queue by using the log core in the multi-core CPU at a preset frequency corresponding to the log type, and outputting the session information as a log file corresponding to the log type.
2. The log outputting method according to claim 1, wherein before the obtaining, by using a log core in the multi-core CPU, the corresponding session information from the ring queue at a preset frequency corresponding to the log type and outputting the session information as a log file corresponding to the log type, the method further comprises:
determining the number and the length of the session information corresponding to the log type to obtain a first number and a first length;
and planning the preset frequency corresponding to the log type according to the first quantity and the first length.
3. The log outputting method according to claim 1, wherein before the recording session information corresponding to each type of log generated during the processing process into the ring queue corresponding to the work core, the method further comprises:
for each type of log type, distributing a ring queue corresponding to the log type for the working core;
recording session information corresponding to various log types generated in the processing process into a ring queue corresponding to the working core, wherein the recording of the session information corresponding to various log types generated in the processing process into the ring queue corresponding to the working core comprises the following steps:
and recording the session information corresponding to each type of log type generated in the processing process into a ring queue distributed for the log type in the working core.
4. The log outputting method according to claim 3, wherein the recording the session information corresponding to each type of the log types generated in the processing procedure into a ring queue allocated to the log type in the work core comprises:
counting the number of the session information corresponding to each type of the log types generated in the processing process to obtain a second number;
judging whether the second quantity meets a first preset condition or not;
and if so, recording the session information into a ring queue distributed for the log type in the working core by taking the second quantity as a write-once batch.
5. The log outputting method according to claim 4, wherein the obtaining the corresponding session information from the circular queue at a preset frequency corresponding to the log type, and outputting the session information as a log file corresponding to the log type, comprises:
counting the number of the session information recorded in the annular queue to obtain a third number;
judging whether the third quantity meets a second preset condition or not;
if so, acquiring the batch by taking the third quantity as one time, acquiring the corresponding session information from the annular queue by taking the preset frequency corresponding to the log type, and outputting the session information as a log file corresponding to the log type.
6. A log output method as claimed in any one of claims 1 to 5, wherein there are a plurality of said working cores;
for each log type, in the process of performing data plane forwarding processing on the service traffic by the work core, acquiring, by using a log core in the multi-core CPU, the corresponding session information from the ring queue at a preset frequency corresponding to the log type, and outputting the session information as a log file corresponding to the log type, including:
for each type of log type, in the process of performing data plane forwarding processing on the service flow by the working core, acquiring corresponding session information from the ring queue corresponding to each working core by using the log core in the multi-core CPU at a preset frequency corresponding to the log type;
and aggregating the session information acquired from the annular queue corresponding to each working core, and outputting the session information as a log file corresponding to the log type.
7. The log output method of any of claims 1 to 5, wherein prior to said receiving traffic, the method further comprises:
determining the number of cores of the multi-core CPU;
determining the number of the working cores according to a preset working core proportion and the number of the working cores, and selecting the cores with the number of the working cores from the multi-core CPU as the working cores;
and determining the number of log cores according to a preset log core proportion and the number of the cores, and selecting the cores with the number of the log cores from the rest cores in the multi-core CPU as the log cores.
8. The log output method according to claim 7, wherein the selecting, as the log core, the core of the number of log cores from the cores remaining in the multi-core CPU includes:
acquiring the load state of each residual core in the multi-core CPU;
and selecting the cores with the number of the log cores from the rest cores in the multi-core CPU as the log cores according to the load states of the rest cores.
9. A load balancing device, comprising:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the at least one processor is a multi-core processor comprising a working core and a log core, the memory storing instructions executable by the at least one processor to enable the at least one processor to perform the log output method of any of claims 1 to 8.
10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the log output method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011187866.XA CN112380001A (en) | 2020-10-30 | 2020-10-30 | Log output method, load balancing device and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011187866.XA CN112380001A (en) | 2020-10-30 | 2020-10-30 | Log output method, load balancing device and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112380001A true CN112380001A (en) | 2021-02-19 |
Family
ID=74577426
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011187866.XA Pending CN112380001A (en) | 2020-10-30 | 2020-10-30 | Log output method, load balancing device and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112380001A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113485661A (en) * | 2021-07-01 | 2021-10-08 | 深圳市同泰怡信息技术有限公司 | Four-way server and method for outputting log information by four-way server |
| CN114257549A (en) * | 2021-12-21 | 2022-03-29 | 北京锐安科技有限公司 | Flow forwarding method, device, equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20120075622A (en) * | 2010-12-20 | 2012-07-09 | 엔에이치엔비즈니스플랫폼 주식회사 | Method, system and compueter readable medium to store log data effective in the game service |
| US20140196050A1 (en) * | 2013-01-07 | 2014-07-10 | Ki Soo Yu | Processing system including a plurality of cores and method of operating the same |
| CN109547580A (en) * | 2019-01-22 | 2019-03-29 | 网宿科技股份有限公司 | A kind of method and apparatus handling data message |
| CN110222503A (en) * | 2019-04-26 | 2019-09-10 | 西安交大捷普网络科技有限公司 | Database audit method, system and equipment under a kind of load of high amount of traffic |
| CN111045782A (en) * | 2019-11-20 | 2020-04-21 | 北京奇艺世纪科技有限公司 | Log processing method and device, electronic equipment and computer readable storage medium |
-
2020
- 2020-10-30 CN CN202011187866.XA patent/CN112380001A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20120075622A (en) * | 2010-12-20 | 2012-07-09 | 엔에이치엔비즈니스플랫폼 주식회사 | Method, system and compueter readable medium to store log data effective in the game service |
| US20140196050A1 (en) * | 2013-01-07 | 2014-07-10 | Ki Soo Yu | Processing system including a plurality of cores and method of operating the same |
| CN109547580A (en) * | 2019-01-22 | 2019-03-29 | 网宿科技股份有限公司 | A kind of method and apparatus handling data message |
| CN110222503A (en) * | 2019-04-26 | 2019-09-10 | 西安交大捷普网络科技有限公司 | Database audit method, system and equipment under a kind of load of high amount of traffic |
| CN111045782A (en) * | 2019-11-20 | 2020-04-21 | 北京奇艺世纪科技有限公司 | Log processing method and device, electronic equipment and computer readable storage medium |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113485661A (en) * | 2021-07-01 | 2021-10-08 | 深圳市同泰怡信息技术有限公司 | Four-way server and method for outputting log information by four-way server |
| CN113485661B (en) * | 2021-07-01 | 2023-08-22 | 深圳市同泰怡信息技术有限公司 | Four-way server and method for outputting log information thereof |
| CN114257549A (en) * | 2021-12-21 | 2022-03-29 | 北京锐安科技有限公司 | Flow forwarding method, device, equipment and storage medium |
| CN114257549B (en) * | 2021-12-21 | 2023-01-10 | 北京锐安科技有限公司 | Flow forwarding method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10831562B2 (en) | Method and system for operating a data center by reducing an amount of data to be processed | |
| CN108776934B (en) | Distributed data calculation method and device, computer equipment and readable storage medium | |
| US10445850B2 (en) | Technologies for offloading network packet processing to a GPU | |
| CN105812276B (en) | Message forwarding adaptive system and method adopting multi-core processor | |
| US10120820B2 (en) | Direct memory access transmission control method and apparatus | |
| CN108268372B (en) | Mock test processing method and device, storage medium and computer equipment | |
| CN112162865A (en) | Server scheduling method and device and server | |
| CN109726004B (en) | Data processing method and device | |
| CN112380001A (en) | Log output method, load balancing device and computer readable storage medium | |
| DE112017003294T5 (en) | Technologies for scalable sending and receiving of packets | |
| CN109828790B (en) | Data processing method and system based on Shenwei heterogeneous many-core processor | |
| US11838384B2 (en) | Intelligent scheduling apparatus and method | |
| WO2017107456A1 (en) | Method and apparatus for determining resources consumed by task | |
| US11438271B2 (en) | Method, electronic device and computer program product of load balancing | |
| CN111163018A (en) | Network equipment and method for reducing transmission delay thereof | |
| US20240152395A1 (en) | Resource scheduling method and apparatus, and computing node | |
| CN112422448A (en) | FPGA accelerator card network data transmission method and related components | |
| CN112422251B (en) | Data transmission method and device, terminal and storage medium | |
| CN116248699A (en) | Data reading method, device, equipment and storage medium in multi-copy scene | |
| CN115712572A (en) | Task testing method and device, storage medium and electronic device | |
| CN110209475B (en) | Data acquisition method and device | |
| CN112417015A (en) | Data distribution method and device, storage medium and electronic device | |
| CN111240867A (en) | Information communication system and method | |
| US20220245474A1 (en) | Implementation of Rules in a Computing System | |
| CN111221858B (en) | Data processing method, device and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |