CN108471387B - Log flow decentralized control method and system - Google Patents

Log flow decentralized control method and system Download PDF

Info

Publication number
CN108471387B
CN108471387B CN201810257323.7A CN201810257323A CN108471387B CN 108471387 B CN108471387 B CN 108471387B CN 201810257323 A CN201810257323 A CN 201810257323A CN 108471387 B CN108471387 B CN 108471387B
Authority
CN
China
Prior art keywords
log
service node
cluster service
target
collector cluster
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810257323.7A
Other languages
Chinese (zh)
Other versions
CN108471387A (en
Inventor
陈建锋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Agricultural Bank of China
Original Assignee
Agricultural Bank of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Agricultural Bank of China filed Critical Agricultural Bank of China
Priority to CN201810257323.7A priority Critical patent/CN108471387B/en
Publication of CN108471387A publication Critical patent/CN108471387A/en
Application granted granted Critical
Publication of CN108471387B publication Critical patent/CN108471387B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00Traffic control in data switching networks
    • H04L47/10Flow control; Congestion control
    • H04L47/12Avoiding congestion; Recovering from congestion
    • H04L47/125Avoiding congestion; Recovering from congestion by balancing the load, e.g. traffic engineering
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1001Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • H04L67/1004Server selection for load balancing

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Debugging And Monitoring (AREA)

Abstract

In the method, a target log collector cluster service node compares the total amount of the received logs with the preset target log flow; if the total amount of the received logs is not greater than the preset target log flow, writing the logs of the target front-end application into a log engine; if the total amount of the received logs is larger than the preset target log flow, the logs applied by the target front end are discarded, and the log collection flow of the cluster service node of the target log collector can be controlled, so that in the cluster of the log collector, the cluster service node of the log collector can realize the control of the log collection flow.

Description

Log flow decentralized control method and system
Technical Field
The present application relates to the field of log traffic control technologies, and in particular, to a method and a system for decentralized control of log traffic.
Background
Nowadays, the internet has become an integral part of people's daily life, and shopping, chatting, reading and the like are performed on the internet. Because people leave their own personalized data on the internet while using the internet application system, the internet application system generates and records valuable log data every day, and the internet application system generally needs to collect the log data for performing related analysis.
Currently, log data of a plurality of distributed application systems are generally collected by a distributed log collector in a centralized manner, but how to control the log data collection traffic of the distributed log collector becomes a problem.
Disclosure of Invention
In order to solve the above technical problems, embodiments of the present application provide a method and a system for distributed control of log traffic, so as to achieve the purpose of distributed control of log collection traffic by multiple service nodes of a log collector cluster, and the technical scheme is as follows:
a log traffic decentralized control method comprises the following steps:
a target log collector cluster service node receives a log request sent by a target front-end application;
the cluster service node of the target log collector acquires the log of the target front-end application from the log request, counts the log of the target front-end application, and takes the counted result as the total amount of the log received by the target front-end application on the cluster service node of the target log collector;
the target log collector cluster service node acquires preset target log traffic of the target front-end application;
the target log collector cluster service node compares the received total log amount with the preset target log flow;
if the total amount of the received logs is not larger than the preset target log flow, writing the logs applied by the target front end into a log engine;
and if the total amount of the received logs is larger than the preset target log flow, discarding the logs applied by the target front end.
Preferably, the determining of the target log collector cluster service node includes:
the node to which the target front-end application belongs obtains a log collector cluster service node list and the weight of each log collector cluster service node from a distributed configuration center;
and determining a node used for receiving the log of the target front-end application in each log collector cluster service node as the target log collector cluster service node by using a load balancing algorithm, the log collector cluster service node list and the weight of each log collector cluster service node.
Preferably, the preset target log flow is configured by the distributed configuration center;
the configuration process of the log collector cluster service node list comprises the following steps:
the distributed configuration center registers each log collector cluster, each log collector cluster service node and the weight of each log collector cluster service node;
and the distributed configuration center writes the information of each log collector cluster service node into a list to obtain the log collector cluster service node list.
Preferably, the obtaining, by the target front-end application, the log collector cluster service node list and the weight of each log collector cluster service node from the distributed configuration center includes:
and the target front-end application acquires the log collector cluster service node list and the weight of each log collector cluster service node from the distributed configuration center in an asynchronous notification mode.
Preferably, the acquiring, by the cluster service node of the target log collector, the preset target log traffic of the target front-end application includes:
the target log collector cluster service nodes acquire preset target log total flow of each front-end application, the log collector cluster service node list and the weight of each log collector cluster service node from the distributed configuration center in an asynchronous notification mode;
the target log collector cluster service node calculates preset target log flow of each front end applied to the target log collector cluster service node according to the preset target log total flow and the proportion of the weight of each log collector cluster service node;
the target log collector cluster service node stores preset target log flow of each front-end application in a local cache;
and the target log collector cluster service node acquires preset target log flow of the target front-end application from the local cache.
A log traffic decentralized control system, comprising: a target log collector cluster service node, the target log collector cluster service node comprising:
the receiving module is used for receiving a log request sent by a target front-end application;
the counting module is used for acquiring the logs of the target front-end application from the log request, counting the logs of the target front-end application, and taking the counting result as the total amount of the logs received by the target front-end application on the cluster service node of the target log collector;
the first acquisition module is used for acquiring preset target log flow of the target front-end application;
the comparison module is used for comparing the received total log amount with the preset target log flow, if the received total log amount is not larger than the preset target log flow, the writing module is executed, and if the received total log amount is larger than the preset target log flow, the discarding module is executed;
the writing module is used for writing the log of the target front-end application into a log engine;
the discarding module is configured to discard the log of the target front-end application.
Preferably, the system further comprises: the node to which the target front-end application belongs and the distributed configuration center, wherein the node to which the target front-end application belongs comprises:
the second acquisition module is used for acquiring a log collector cluster service node list and the weight of each log collector cluster service node from the distributed configuration center;
and the determining module is used for determining a node which is used for receiving the log applied by the target front end in each log collector cluster service node as the target log collector cluster service node by utilizing a load balancing algorithm, the log collector cluster service node list and the weight of each log collector cluster service node.
Preferably, the distributed configuration center is configured to configure the preset target log traffic;
the distributed configuration center includes:
the log collector cluster service node comprises a registration module, a storage module and a processing module, wherein the registration module is used for registering each log collector cluster, each log collector cluster service node and the weight of each log collector cluster service node;
and the list generation module is used for writing the information of each log collector cluster service node into a list to obtain the log collector cluster service node list.
Preferably, the second obtaining module is specifically configured to obtain, in an asynchronous notification manner, the log collector cluster service node list and the weight of each log collector cluster service node from the distributed configuration center.
Preferably, the first obtaining module includes:
the first obtaining unit is used for obtaining the total flow of the preset target logs of each front-end application, the log collector cluster service node list and the weight of each log collector cluster service node from the distributed configuration center in an asynchronous notification mode;
the calculation unit is used for calculating the preset target log flow of each front end applied to the cluster service node of the target log collector according to the preset target log total flow and the weight proportion of each cluster service node of the log collector;
the storage unit is used for storing the preset target log flow of each front-end application in a local cache;
and the second acquisition unit is used for acquiring the preset target log flow of the target front-end application from the local cache.
Compared with the prior art, the beneficial effect of this application is:
in the application, the target log collector cluster independently compares the received total log amount with the preset target log flow through each service node; if the total amount of the received logs is not larger than the preset target log flow, writing the logs applied by the target front end into a log engine; if the total amount of the received logs is larger than the preset target log flow, the logs applied by the target front end are discarded, and the log flow applied by the target front end can be controlled, so that each node in the log collector cluster respectively controls partial log flow in such a way, the purpose of dispersedly controlling the log collection flow by the multi-service nodes of the log collector cluster is achieved, and the overall control of the log flow is realized.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive labor.
Fig. 1 is a flowchart of a log traffic decentralized control method provided in the present application;
fig. 2 is a flowchart of a log traffic distribution control method provided in the present application;
FIG. 3 is a sub-flowchart of a log traffic dispersion control method provided by the present application;
fig. 4 is another sub-flowchart of the log traffic dispersion control method provided in the present application;
fig. 5 is a schematic logic structure diagram of a log traffic decentralized control system provided in the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.
The embodiment of the application discloses a log flow decentralized control method, which comprises the steps of receiving a log request sent by a target front-end application through a cluster service node of a target log collector; the cluster service node of the target log collector acquires the logs of the target front-end application from the log request, counts the logs of the target front-end application, and takes the counted result as the total amount of the received logs; the target log collector cluster service node acquires preset target log traffic of the target front-end application; the target log collector cluster service node compares the received total log amount with the preset target log flow; if the total amount of the received logs is not greater than the preset target log flow, writing the logs of the target front-end application into a log engine; if the total amount of the received logs is larger than the preset target log flow, the logs applied by the target front end are discarded, and the control of the log collection flow of each log collector cluster service node can be realized.
Next, a method for controlling distributed log traffic disclosed in an embodiment of the present application is described, with reference to fig. 1, where the method may include:
step S11, the cluster service node of the target log collector receives a log request sent by the target front-end application.
In this embodiment, the target front-end application and the target log collector cluster service node may communicate in a point-to-point direct communication manner, thereby avoiding the transfer of the load balancing device, saving the cost of the load balancing device, and being capable of breaking away from the performance bottleneck display of the load balancing device.
Step S12, the cluster service node of the target log collector obtains the log of the target front-end application from the log request, counts the log of the target front-end application, and takes the counted result as the total amount of the log received by the cluster service node of the target log collector by the target front-end application.
Step S13, the target log collector cluster service node obtains the preset target log flow of the target front-end application.
The preset target log traffic can be understood as: an upper limit value for the number of logs that a target log collector cluster server node can collect.
It should be noted that each log collector cluster service node in the log collector cluster corresponds to a preset target log traffic, and the preset target log traffic corresponding to each log collector cluster service node is different.
Step S14, the target log collector cluster service node compares the received total log amount with the preset target log flow.
If the total received log amount is not greater than the preset target log flow, executing step S15; and if the total amount of the received logs is greater than the preset target log flow, executing step S16.
And S15, writing the log of the target front-end application into a log engine.
And S16, discarding the log of the target front-end application.
In the application, the target log collector cluster independently compares the received total log amount with the preset target log flow through each service node; if the total amount of the received logs is not larger than the preset target log flow, writing the logs applied by the target front end into a log engine; if the total amount of the received logs is larger than the preset target log flow, the logs applied by the target front end are discarded, and the log flow applied by the target front end can be controlled, so that each node in the log collector cluster respectively controls partial log flow in such a way, the purpose of dispersedly controlling the log collection flow by the multi-service nodes of the log collector cluster is achieved, and the overall control of the log flow is realized.
Further, based on the fact that each log collector cluster service node in the log collector cluster corresponds to a preset target log flow and the preset target log flows corresponding to the log collector cluster service nodes are different from each other, by adopting the log flow decentralized control method, the target log collector cluster service node can independently control the log collection flow, and the problem of performance bottleneck caused by synchronization of used flow counting among the log collector cluster service nodes related to centralized control of the log flow is avoided.
In another embodiment of the present application, the process of determining the target log collector cluster service node is described, referring to fig. 2, which may include:
and S21, the node to which the target front-end application belongs acquires a log collector cluster service node list and the weight of each log collector cluster service node from a distributed configuration center.
It will be appreciated that the higher the resource configuration capability of the log collector cluster service node, the greater the weight.
Step S22, determining, by using a load balancing algorithm, the log collector cluster service node list, and the weight of each log collector cluster service node, a node in each log collector cluster service node, which is used for receiving the log of the target front-end application, as the target log collector cluster service node.
It can be understood that, the target front-end application nodes asynchronously obtain the target log collector cluster service node list and the weights of the log collector cluster service nodes, determine the target log collector cluster service nodes by using a load balancing algorithm, and send the logs to the target log collector cluster service nodes point to point, thereby realizing the soft load balancing of the target front-end application log traffic.
The target log collector cluster service node may be understood as: the log collection traffic is sufficient for the nodes of the log collection of the target front-end application.
In another embodiment of the present application, introducing the preset target log traffic may specifically include:
the preset target log traffic may be configured by the distributed configuration center.
It should be noted that the distributed configuration center configures a preset target log traffic for each log collector cluster service node, and the preset target log traffic corresponding to each log collector cluster service node is different.
In this embodiment, a configuration process of the log collector cluster service node list is introduced at the same time, please refer to fig. 3, which may specifically include:
step S31, the distributed configuration center registers each log collector cluster, each log collector cluster service node, and the weight of each log collector cluster service node.
And step S32, the distributed configuration center writes the information of each log collector cluster service node into a list to obtain the log collector cluster service node list.
In another embodiment of the present application, introducing the log collector cluster service node list and the weight of each log collector cluster service node obtained by the target front-end application from the distributed configuration center may include:
and the target front-end application acquires the log collector cluster service node list and the weight of each log collector cluster service node from the distributed configuration center in an asynchronous notification mode.
And acquiring the log collector cluster service node list and the weight of each log collector cluster service node from the distributed configuration center in an asynchronous notification mode, so that network resources can be saved.
Based on the content of the foregoing embodiments, in another embodiment of the present application, the obtaining, by the target log collector cluster service node, the preset target log traffic of the target front-end application is described, referring to fig. 4, where the method may include:
step S41, the target log collector cluster service node obtains the total preset target log traffic of each front-end application from the distributed configuration center in an asynchronous notification manner.
The target log collector cluster service node acquires the total preset target log flow of each front-end application from the distributed configuration center in an asynchronous notification mode, so that network resources can be saved.
Step S42, the target log collector cluster service node calculates, according to the preset target total log traffic and the ratio of the weights of the log collector cluster service nodes, a preset target log traffic of each front end application on the target log collector cluster service node.
Step S43, the target log collector cluster service node stores the preset target log traffic of each front-end application in a local cache.
Step S44, the cluster service node of the target log collector obtains the preset target log traffic of the target front-end application from the local cache.
Referring to fig. 5, the distributed log traffic control system provided in the present application is described next, and includes: a target log collector cluster service node 1, the target log collector cluster service node 1 comprising: a receiving module 11, a counting module 12, a first obtaining module 13, a comparing module 14, a writing module 15 and a discarding module 16.
The receiving module 11 is configured to receive a log request sent by a target front-end application.
A counting module 12, configured to obtain the log of the target front-end application from the log request, count the log of the target front-end application, and use a count result as a total amount of logs received by the target front-end application on the cluster service node of the target log collector.
A first obtaining module 13, configured to obtain a preset target log flow of the target front-end application.
A comparing module 14, configured to compare the total amount of the received logs with the preset target log traffic, if the total amount of the received logs is not greater than the preset target log traffic, execute a writing module 15, and if the total amount of the received logs is greater than the preset target log traffic, execute a discarding module 16.
The writing module 15 is configured to write the log of the target front-end application into a log engine.
The discarding module 16 is configured to discard the log of the target front-end application.
In this embodiment, the log traffic distributed control system may further include: the node of the target front-end application and the distributed configuration center.
Accordingly, the node to which the target front-end application belongs may include:
the second acquisition module is used for acquiring a log collector cluster service node list and the weight of each log collector cluster service node from the distributed configuration center;
and the determining module is configured to determine, by using a load balancing algorithm, the log collector cluster service node list, and the weight of each log collector cluster service node, a node, which is used for receiving the log of the target front-end application, in each log collector cluster service node as the target log collector cluster service node 1.
In this embodiment, the distributed configuration center may be configured to configure the preset target log traffic;
the distributed configuration center may include:
the log collector cluster service node comprises a registration module, a storage module and a processing module, wherein the registration module is used for registering each log collector cluster, each log collector cluster service node and the weight of each log collector cluster service node;
and the list generating module is used for writing the information of the cluster service nodes of the log collector into a list to obtain a cluster service node list of the log collector.
In this embodiment, the second obtaining module may be specifically configured to obtain, in an asynchronous notification manner, the log collector cluster service node list and the weight of each log collector cluster service node from the distributed configuration center.
In this embodiment, the first obtaining module may include:
a first obtaining unit, configured to obtain, in an asynchronous notification manner, a preset target total log traffic of each front-end application, the log collector cluster service node list, and a weight of each log collector cluster service node from the distributed configuration center;
the calculation unit is used for calculating the preset target log flow of each front end applied to the cluster service node of the target log collector according to the preset target log total flow and the weight proportion of each cluster service node of the log collector;
the storage unit is used for storing the preset target log flow of each front-end application in a local cache;
and the second obtaining unit is used for obtaining the preset target log flow of the target front-end application from the local cache.
It should be noted that, in this specification, each embodiment is described in a progressive manner, and each embodiment focuses on differences from other embodiments, and portions that are the same as and similar to each other in each embodiment may be referred to. For the device-like embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
Finally, it should also be noted that, in this document, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising one of 8230; \8230;" 8230; "does not exclude the presence of additional like elements in a process, method, article, or apparatus that comprises the element.
For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functionality of the units may be implemented in one or more software and/or hardware when implementing the present application.
From the above description of the embodiments, it is clear to those skilled in the art that the present application can be implemented by software plus necessary general hardware platform. Based on such understanding, the technical solutions of the present application or portions thereof that contribute to the prior art may be embodied in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, or the like, and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute the method according to the embodiments or some portions of the embodiments of the present application.
The method and the system for decentralized control of log traffic provided by the present application are described in detail above, and a specific example is applied in the present application to explain the principle and the implementation of the present application, and the description of the above embodiment is only used to help understand the method and the core idea of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims (8)

1. A log traffic decentralized control method is characterized by comprising the following steps:
a target log collector cluster service node receives a log request sent by a target front-end application;
the cluster service node of the target log collector acquires the log of the target front-end application from the log request, counts the log of the target front-end application, and takes the counted result as the total amount of the log received by the target front-end application on the cluster service node of the target log collector;
the target log collector cluster service node acquires preset target log traffic of the target front-end application; each log collector cluster service node in the log collector cluster corresponds to a preset target log flow, and the preset target log flows corresponding to the log collector cluster service nodes are different;
the target log collector cluster service node compares the received total log amount with the preset target log flow;
if the total amount of the received logs is not greater than the preset target log flow, writing the logs of the target front-end application into a log engine;
if the total amount of the received logs is larger than the preset target log flow, discarding the logs applied by the target front end;
the determination process of the target log collector cluster service node comprises the following steps:
the node to which the target front-end application belongs obtains a log collector cluster service node list and the weight of each log collector cluster service node from a distributed configuration center;
and determining a node used for receiving the log of the target front-end application in each log collector cluster service node as the target log collector cluster service node by using a load balancing algorithm, the log collector cluster service node list and the weight of each log collector cluster service node.
2. The method of claim 1, wherein the preset target log traffic is configured by the distributed configuration center;
the configuration process of the log collector cluster service node list comprises the following steps:
the distributed configuration center registers each log collector cluster, each log collector cluster service node and the weight of each log collector cluster service node;
and the distributed configuration center writes the information of each log collector cluster service node into a list to obtain the log collector cluster service node list.
3. The method of claim 1, wherein the target front-end application obtaining a list of log collector cluster service nodes and weights for each log collector cluster service node from a distributed configuration center, comprises:
and the target front-end application acquires a log collector cluster service node list and the weight of each log collector cluster service node from the distributed configuration center in an asynchronous notification mode.
4. The method of claim 1, wherein the obtaining, by the target log collector cluster service node, a preset target log traffic of the target front-end application comprises:
the target log collector cluster service node acquires preset target log total flow of each front-end application, the log collector cluster service node list and the weight of each log collector cluster service node from the distributed configuration center in an asynchronous notification mode;
the target log collector cluster service node calculates preset target log flow of each front end applied to the target log collector cluster service node according to the preset target log total flow and the weight proportion of each log collector cluster service node;
the target log collector cluster service node stores preset target log flow of each front-end application in a local cache;
and the target log collector cluster service node acquires preset target log flow of the target front-end application from the local cache.
5. A log traffic decentralized control system, comprising: a target log collector cluster service node, the target log collector cluster service node comprising:
the receiving module is used for receiving a log request sent by a target front-end application;
the counting module is used for acquiring the logs of the target front-end application from the log request, counting the logs of the target front-end application, and taking the counting result as the total amount of the logs received by the target front-end application on the cluster service node of the target log collector;
the first acquisition module is used for acquiring preset target log flow of the target front-end application; each log collector cluster service node in the log collector cluster corresponds to a preset target log flow, and the preset target log flows corresponding to the log collector cluster service nodes are different;
the comparison module is used for comparing the received total log amount with the preset target log flow, if the received total log amount is not larger than the preset target log flow, the write-in module is executed, and if the received total log amount is larger than the preset target log flow, the discard module is executed;
the writing module is used for writing the log of the target front-end application into a log engine;
the discarding module is used for discarding the log of the target front-end application;
the node to which the target front-end application belongs and the distributed configuration center, wherein the node to which the target front-end application belongs comprises:
the second acquisition module is used for acquiring a log collector cluster service node list and the weight of each log collector cluster service node from the distributed configuration center;
and the determining module is used for determining a node which is used for receiving the log of the target front-end application in each log collector cluster service node and is used as the target log collector cluster service node by utilizing a load balancing algorithm, the log collector cluster service node list and the weight of each log collector cluster service node.
6. The system of claim 5, wherein the distributed configuration center is configured to configure the pre-set target log traffic;
the distributed configuration center includes:
the log collector cluster service node comprises a registration module, a judging module and a judging module, wherein the registration module is used for registering each log collector cluster, each log collector cluster service node and the weight of each log collector cluster service node;
and the list generation module is used for writing the information of each log collector cluster service node into a list to obtain the log collector cluster service node list.
7. The system of claim 5, wherein the second obtaining module is specifically configured to obtain the list of log collector cluster service nodes and the weight of each log collector cluster service node from the distributed configuration center in an asynchronous notification manner.
8. The system of claim 5, wherein the first obtaining module comprises:
the first obtaining unit is used for obtaining the total flow of the preset target logs of each front-end application, the log collector cluster service node list and the weight of each log collector cluster service node from the distributed configuration center in an asynchronous notification mode;
the calculation unit is used for calculating the preset target log flow of each front end applied to the cluster service node of the target log collector according to the preset target log total flow and the weight proportion of each cluster service node of the log collector;
the storage unit is used for storing the preset target log flow of each front-end application in a local cache;
and the second obtaining unit is used for obtaining the preset target log flow of the target front-end application from the local cache.
CN201810257323.7A 2018-03-27 2018-03-27 Log flow decentralized control method and system Active CN108471387B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810257323.7A CN108471387B (en) 2018-03-27 2018-03-27 Log flow decentralized control method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810257323.7A CN108471387B (en) 2018-03-27 2018-03-27 Log flow decentralized control method and system

Publications (2)

Publication Number Publication Date
CN108471387A CN108471387A (en) 2018-08-31
CN108471387B true CN108471387B (en) 2022-10-21

Family

ID=63265896

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810257323.7A Active CN108471387B (en) 2018-03-27 2018-03-27 Log flow decentralized control method and system

Country Status (1)

Country Link
CN (1) CN108471387B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109327353B (en) * 2018-09-29 2022-01-11 创新先进技术有限公司 Service flow determination method and device and electronic equipment
CN112231191B (en) * 2020-10-23 2023-03-31 新华三信息安全技术有限公司 Log collection method and device

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101849384A (en) * 2007-11-06 2010-09-29 朗讯科技公司 Method for controlling load balance of network system, client, server and network system
CN103220354A (en) * 2013-04-18 2013-07-24 广东宜通世纪科技股份有限公司 Method for achieving load balancing of server cluster
CN103458013A (en) * 2013-08-21 2013-12-18 成都云鹰科技有限公司 Streaming media server cluster load balancing system and balancing method
CN104158911A (en) * 2014-09-01 2014-11-19 青岛海信电器股份有限公司 Server cluster-oriented control method and system

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104092650B (en) * 2013-12-04 2018-04-03 腾讯数码(天津)有限公司 A kind of method and apparatus for distributing service request
CN103838659B (en) * 2014-02-17 2017-09-01 大唐移动通信设备有限公司 The control method and device of a kind of system journal
CN104023083B (en) * 2014-06-23 2017-12-12 广东睿江云计算股份有限公司 The method and device of log collection cluster load balance
CN105915646A (en) * 2016-06-20 2016-08-31 飞天诚信科技股份有限公司 Method and device for dispersing load of servers
CN106210120B (en) * 2016-07-29 2019-07-09 广州南方仕通网络科技有限公司 A kind of recommended method and its device of server
CN107426055B (en) * 2017-07-31 2021-09-07 努比亚技术有限公司 Abnormal flow monitoring method and device and computer readable storage medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101849384A (en) * 2007-11-06 2010-09-29 朗讯科技公司 Method for controlling load balance of network system, client, server and network system
CN103220354A (en) * 2013-04-18 2013-07-24 广东宜通世纪科技股份有限公司 Method for achieving load balancing of server cluster
CN103458013A (en) * 2013-08-21 2013-12-18 成都云鹰科技有限公司 Streaming media server cluster load balancing system and balancing method
CN104158911A (en) * 2014-09-01 2014-11-19 青岛海信电器股份有限公司 Server cluster-oriented control method and system

Also Published As

Publication number Publication date
CN108471387A (en) 2018-08-31

Similar Documents

Publication Publication Date Title
WO2021004063A1 (en) Cache server bandwidth scheduling method and device
JP6126099B2 (en) Marketplace for timely event data distribution
CN103067297B (en) A kind of dynamic load balancing method based on resource consumption prediction and device
CN108282514B (en) Distributed service establishing method and device
KR20170106648A (en) High-capacity network data processing techniques
JP2017500641A (en) Method, system, and computer-readable medium for network function virtualization information concentrator
CN109618002A (en) A kind of micro services gateway optimization method, device and storage medium
CN108319598B (en) Data caching method, device and system
CN106713396B (en) Server scheduling method and system
US11570078B2 (en) Collecting route-based traffic metrics in a service-oriented system
CN111966289B (en) Partition optimization method and system based on Kafka cluster
CN109976989B (en) Cross-node application performance monitoring method and device and high-performance computing system
CN109032800A (en) A kind of load equilibration scheduling method, load balancer, server and system
US20220200902A1 (en) Method, apparatus and storage medium for application identification
CN108574645B (en) Queue scheduling method and device
CN111562889B (en) Data processing method, device, system and storage medium
CN108471387B (en) Log flow decentralized control method and system
US20210334791A1 (en) Method and device for blockchain-based data traffic calculation
CN110740293A (en) data transmission method, system, device and computer storage medium
CN102932271A (en) Method and device for realizing load balancing
CN113722276A (en) Log data processing method, system, storage medium and electronic equipment
CN104731660B (en) data distributing method, device and system
CN103607731A (en) Method and device for processing measurement reports
US20240022507A1 (en) Information flow recognition method, network chip, and network device
CN105376155B (en) A kind of Intelligent routing system and method based on distributed type assemblies framework

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant