WO2019096046A1 - Système et procédé de traitement de données, et procédé de gestion de jeton - Google Patents

Système et procédé de traitement de données, et procédé de gestion de jeton Download PDF

Info

Publication number
WO2019096046A1
WO2019096046A1 PCT/CN2018/114348 CN2018114348W WO2019096046A1 WO 2019096046 A1 WO2019096046 A1 WO 2019096046A1 CN 2018114348 W CN2018114348 W CN 2018114348W WO 2019096046 A1 WO2019096046 A1 WO 2019096046A1
Authority
WO
WIPO (PCT)
Prior art keywords
token
unit
service
tokens
management unit
Prior art date
Application number
PCT/CN2018/114348
Other languages
English (en)
Chinese (zh)
Inventor
戴红斌
Original Assignee
阿里巴巴集团控股有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 阿里巴巴集团控股有限公司 filed Critical 阿里巴巴集团控股有限公司
Publication of WO2019096046A1 publication Critical patent/WO2019096046A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/10—Network architectures or network communication protocols for network security for controlling access to devices or network resources
    • H04L63/108—Network architectures or network communication protocols for network security for controlling access to devices or network resources when the policy decisions are valid for a limited amount of time
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0207—Discounts or incentives, e.g. coupons or rebates
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/06—Network architectures or network communication protocols for network security for supporting key management in a packet data network
    • H04L63/068—Network architectures or network communication protocols for network security for supporting key management in a packet data network using time-dependent keys, e.g. periodically changing keys
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/12—Messaging; Mailboxes; Announcements

Definitions

  • the present specification relates to the field of computer network technologies, and in particular, to a data processing system, method, and token management method.
  • each service unit may be a physical machine or a virtual machine mounted on a physical machine. These service units can run in parallel. Multiple service units or multiple processes of the same service unit may initiate concurrent access to the same resource at the same time. If they are not controlled, the system will affect the ability of the system to process data.
  • the embodiment of the present specification provides a data processing system, including a service unit, a token management unit, and a processing unit, wherein the service unit is configured to send a token acquisition request to the token management unit to obtain the token. And then providing the service data to the processing unit for processing; the token management unit is configured to manage at least one token set, and maintain the number of tokens in the token set according to a specified rule; the token management unit is Receiving the token acquisition request, transmitting the token in the token set to a service unit that issues the token acquisition request; the processing unit is configured to have the token according to a specified processing rule The business data provided by the business unit is processed.
  • the embodiment of the present specification provides a data processing method, which is applied to a service unit.
  • the method includes: the service unit sends a token acquisition request to the token management unit; and receives a token fed back by the token management unit; wherein the token
  • the management unit manages at least one token set and maintains the number of tokens in the token set according to a specified rule; providing the service data and the token to the processing unit; wherein the processing unit has the specified processing rule pair
  • the business data of the token is processed.
  • the embodiment of the present specification provides a data processing method, which is applied to a token management unit; the method includes:
  • Receiving a token acquisition request issued by the service unit transmitting a token in the token set to the service unit, where the service unit provides the token and service data to a processing unit; wherein the processing unit Processing the service data provided by the service unit having the token; wherein the number of the token sets is at least one, and the number of tokens in the token set is maintained according to a specified rule.
  • the embodiment of the present specification provides a data processing method, which is applied to a processing unit, where the method includes: receiving a service processing request of a service unit; and if the service processing request has a token, the service is performed according to a specified processing rule. Processing the service data accompanying the request for processing; wherein the service unit obtains the token from the token management unit.
  • the embodiment of the present specification provides a token management method, which is applied to a token management unit, where the token management unit maintains at least two token sets, including: selecting one of the token sets; and setting the token set Comparing the number of tokens with the adjacent token set, adding a specified number of tokens in a smaller number of token sets; or comparing the token set with the adjacent token set for the number of tokens, A larger number of tokens reduces the specified number of tokens.
  • the token implementation unit sends a token to the service unit through the token management unit, and the processing unit processes the service data of the service unit only when the service unit obtains the token.
  • the method limits the speed of data traffic in the network, adjusts the data traffic output rate, reduces the possibility of system overload, and improves the efficiency and performance of service data processing.
  • FIG. 1 is a schematic diagram of a workflow of a data processing system according to an embodiment of the present disclosure
  • FIG. 2 is a structural block diagram of a data processing system provided by an embodiment of the present specification
  • FIG. 3 is a schematic diagram of a scenario of a data processing system according to an embodiment of the present disclosure
  • FIG. 4 is a schematic diagram showing an example of a scenario of supplementing and consuming tokens when there is only one token set provided by an embodiment of the present specification
  • FIG. 5 is a schematic diagram of an example of a scenario of supplementing and consuming tokens when there are multiple token sets provided by an embodiment of the present disclosure
  • FIG. 6 is a schematic diagram of a workflow of a data processing method applied to a service unit according to an embodiment of the present disclosure
  • FIG. 7 is a schematic flowchart of a data processing method applied to a token management unit according to an embodiment of the present disclosure
  • FIG. 8 is a schematic flowchart of a data processing method applied to a processing unit according to an embodiment of the present disclosure
  • FIG. 9 is a schematic diagram of a workflow of a token management method according to an embodiment of the present disclosure.
  • an embodiment of the present specification provides a data processing system including a service unit 20, a token management unit 22, and a processing unit 24.
  • the data processing system may be a system in which hardware or software components are distributed on computers of different networks or hardware devices that can be networked, and communication and coordination can be performed between each other through message transmission. More ordinary computers can be used to provide external services. The more computers, the more CPU, memory, storage resources, etc., the greater the amount of concurrent access that can be processed.
  • the service unit 20 transmits a token acquisition request to the token management unit 22.
  • the service unit 20 is configured to send a token acquisition request to the token management unit 22 to obtain the token and then provide the service data to the processing unit 24 for processing.
  • the service unit 20 may include a network node device having data in the network that needs to be processed and possessing a network address, and software necessary to drive the hardware device to work.
  • the business unit 20 may also be only a hardware device with data processing capabilities, or only software running in a hardware device. Network data communication with the network node device is possible through the network address.
  • the service unit 20 includes: a workstation, a server, a computer, and other network devices of the user terminal, and the service unit 20 can provide a network interface for data interaction.
  • the service unit 20 can perform data information interaction in accordance with a network communication protocol.
  • the service unit 20 can implement a certain service function.
  • the service data can be provided to the processing unit 24 after the token is obtained to implement the service function. .
  • the token may be data that follows a certain format, and is used to represent a certain authority. After the business unit 20 obtains the token, it has the authority indicated by the token, so that resource access can be performed based on the authority. When data information is sent from one network node to another, only the network node that obtained the token has the right to send data.
  • the token acquisition request may be a string or code that satisfies the communication protocol and follows a certain format, and is used to indicate that the service unit 20 that issued the token acquisition request requests to obtain the corresponding token.
  • the token management unit 22 is configured to manage at least one token set and maintain the number of tokens in the token set according to a specified rule; the token management unit 22 receives the order In the case of a card acquisition request, the token in the token set is sent to the business unit 20 that issued the token acquisition request.
  • the token management unit 22 may be a server or other electronic device.
  • the token management unit 22 may include a hardware device having a data information processing function and software necessary to drive the hardware device to operate.
  • the token management unit 22 may also be only a hardware device with data processing capability, or only software running in a hardware device.
  • the token management unit 22 can be provided with an interface through which data interaction can be performed.
  • the token management unit 22 can perform network data information interaction based on a network communication protocol that can be followed.
  • the token management unit 22 is pre-configured with a certain number of tokens.
  • the token set may be a set of the tokens in the token management unit 22. At least one token is included in the token set.
  • the token management unit 22 may be from the at least one token set.
  • the token unit collects at least one token to allocate the service unit 20, and ensures that the service unit 20 corresponding to the plurality of token acquisition requests can acquire the token.
  • the token set may have a maximum number of tokens.
  • the maximum number of tokens of the token set is less than or equal to the processing unit's ability to process the service data. If the processing unit 24 is larger than the processing unit 24, the service data of the acquiring token is greater than the processing of the processing unit 24. The ability to cause some business data can not be processed, nor can it achieve the role of current limiting.
  • the token management unit 22 may maintain the number of tokens in the token set according to a specified rule, and the designation rule may be preset or may be set according to current network conditions. When the number of token sets satisfies at least one specified rule, the token management unit 22 supplements the token with the token set.
  • the processing unit 24 can process the service data according to a preset rate and manner or change the processing rate and mode according to the current network condition to adapt to the current network condition and make the network status. Maintained within a manageable range.
  • the specifying the rule may include: adding a specified number of tokens to the token set every specified time interval; and after the service unit 20 uses the obtained token, the token management unit 22 The token set adds a corresponding number of tokens.
  • sending the token in the token set to the service unit 20 that issues the token acquisition request may include, as shown in step 11 in FIG. 1, the token management unit 22 receives the token. Get the request. After receiving the token acquisition request sent by the service unit 20, the token management unit 22 checks whether the number of tokens in the token set is greater than the token acquisition, as shown in step 12 in FIG. The amount, the token management unit 20 may check the number of tokens in the current at least one token set and compare it with the token acquisition amount sent by the business unit 20. If the number of tokens in the at least one token set of the token management unit 22 is greater than the token acquisition amount sent by the service unit, the token is sent to the service as shown in step S13 in FIG. Unit 20.
  • a corresponding number of tokens may be deducted from the token set, and the token is sent to the service unit 20 according to the network address of the service unit 20 of the request token. If the current total token amount of the token management unit 22 is smaller than the token acquisition amount sent by the service unit 20, as shown in step S15 of FIG. 1, the service unit waits for the token management unit to supplement the token. Specifically, the token management unit 22 may feed back information to the service unit 20, and include the acquisition request sent by the service unit 20 in the request queue, and wait for the token management unit 22 to supplement the token; The token acquisition request is placed in the request queue. When the token has a token, the service unit 20 corresponding to the token acquisition request sends notification information to notify the service unit 20 to resend the token acquisition request. After the token management unit supplements the token, as shown in step S16 of FIG. 1, the token is sent to the service unit 20. Specifically, the token management unit 22 may perform the service according to the request token. The network address of unit 20 sends the token to the service unit 20.
  • the processing unit 24 is configured to process the service data provided by the service unit 20 having the token according to a specified processing rule.
  • the processing unit 24 may include a server capable of processing parallel data or software necessary for other electronic devices and driving hardware devices to operate.
  • the processing may also be only a hardware device with data processing capability, or only software running in a hardware device.
  • the parallel data may be a set of processing units contained within a server or other electronic device, each processing unit storing one or more data elements. When the machine executes the sequential program, the data stored in all or part of the internal processing unit can be operated simultaneously.
  • the processing unit 24 can be provided with an interface through which data interaction can be performed.
  • the processing unit can perform network data information interaction based on a network communication protocol that can be followed.
  • the processing unit 24 may be a gateway. In the message delivery process, at least one service unit 20 simultaneously sends a message to the user terminal. By calling the gateway interface, the gateway repackages and sends the message sent by the at least one service unit 20.
  • the processing unit 24 may also be a search engine. The user inputs the search information through the user terminal, and the thread that executes the search task invokes the search engine interface, and the search engine searches for the corresponding information.
  • the processing unit 24 can also process other business functions, but as long as the functions or effects thereof are the same or similar to the present application, It should be covered by the scope of this application.
  • the specified processing rule may be preset or may be changed according to current network conditions. As shown in steps S14 and S17 of FIG. 1, the service unit 24 processes the service data, and when the service data provided by the service unit 24 satisfies at least one predetermined specified processing rule, the service data is processed.
  • the specified processing rule may include: when the service data provided by the service unit 20 includes a token, the processing unit 24 processes the service data; when a plurality of service data having different priorities and including the token When arriving at the same time, the service data with higher priority is prioritized.
  • the data processing system is structured as a platform server, a token server, and an information sending gateway.
  • the platform organizes a time-limited promotion.
  • the platform server sends the information about the limited-time promotion to the mobile phone of the registered member of the platform in the form of a short message.
  • the number of SMS messages to be sent by the platform server is 200,000.
  • the interface of the information sending gateway needs to be called to send the 200,000 SMS messages to the mobile phone of the registered member.
  • the platform The server also sends a token acquisition request to the token server to obtain the token, so that the 200,000 messages can be sent to the registered member's mobile phone through the information sending gateway, and the platform server stipulates that when there is no short message to be sent, the server no longer The token server sends a token get request.
  • the ability of the messaging gateway to process and send text messages to registered members' mobile phones per second is 20,000.
  • the token server manages a token set that contains 20,000 tokens, and the token server sets a token to the token set every 1 second, each time Fill up the capacity of the token set.
  • the token server receives 200,000 token acquisition requests sent by the platform server, queries 20,000 tokens in the token set, sends the 20,000 tokens to the platform server, and sends the tokens to the platform.
  • the server sends feedback information to notify the platform server that the number of tokens in the current token set is 0, so that the platform server sends 20,000 token acquisition requests every 1 second after receiving the token until the data processing is completed.
  • the token replenishment unit begins to replenish the token after the token and feedback information has been sent.
  • the service server randomly allocates 20,000 tokens to 20,000 threads that perform SMS sending tasks, and each thread executes one SMS sending task. And the 20,000 threads simultaneously call the interface of the information sending gateway, and the information sending gateway sends the 20,000 text messages to the registered member's mobile phone.
  • the platform server sends 20,000 token acquisition requests to the token server one second after receiving the token, and the token server sends the supplemental 20,000 tokens to the platform server, and the platform server again will Ten thousand tokens are randomly assigned to 20,000 threads that perform SMS sending tasks. These threads call the information sending gateway interface and send out 20,000 SMS messages to the registered member's mobile phone. After repeating 8 times again, 200,000 text messages will be sent to the registered member's mobile phone.
  • the service data provided by the service unit is processed by the processing unit only after the service unit acquires the token from the at least one token set of the token management unit, which can reduce the occurrence of the network transmission data.
  • the network congestion in turn sends a packet loss condition, and the network traffic can be further controlled by controlling the token replenishment rate of the token set.
  • the data processing system is structured as a plurality of mobile terminals, a token server, and a search engine.
  • three users send search requests for searching for World Cup information through the mobile terminal, and the search engine responds and processes the search requests to search for World Cup information.
  • the user sends a token acquisition request to the token server through the mobile terminal, and the token server extracts the token from the token set managed by the token server to the corresponding mobile terminal, and the mobile terminal searches after obtaining the token.
  • the engine processes the search request sent by the mobile terminal.
  • the token server manages 1 token set, the maximum number of tokens in the token set is 2 tokens, and sets 2 tokens to be added every 1 second, if the number of tokens reaches the maximum The number of tokens is no longer added, and if the maximum number of tokens is not reached, continue to add.
  • three users search for World Cup information through the mobile terminal, and the three user terminals send a token acquisition request for obtaining one token to the token server, and two token acquisition requests arrive at the token at the same time.
  • the server receives two tokens from the token set by the token server, respectively sends the two tokens to the mobile terminal that sends the two token acquisition requests, and replenishes the token after one second.
  • the search request for searching for World Cup information sent by the two mobile terminals that obtain the token is processed by the search engine, and the two users obtain the World Cup information through the search engine.
  • the third token acquisition request arrives at the token server after the first two token acquisition requests reach the token server.
  • the token server manages the token set without the token, and the token server sends the token.
  • the three tokens acquire the requested mobile terminal feedback information, and notify the third mobile terminal to send the token acquisition request again after 1 second, and after 1 second, the third mobile terminal sends the token acquisition request again, and the token server receives the command.
  • the card acquisition request one token is extracted from the two tokens in the token set and assigned to the mobile terminal.
  • the token server replenishes the token again.
  • the search request sent by the third mobile terminal for searching for World Cup information is processed by the search engine, and the user obtains the World Cup information through the search engine. After that, no user terminal sends the token acquisition request again.
  • the data processing system is structured as a plurality of mobile terminals, a token management unit, and a search engine.
  • one user transmits a search request for searching for World Cup information through the mobile terminal, and the search engine responds and processes the search request to search for World Cup information.
  • the user sends a token acquisition request to the token server through the mobile terminal, and the token server extracts the token from the token set managed by the token server to the corresponding mobile terminal, and the mobile terminal searches after obtaining the token.
  • the engine processes the search request sent by the mobile terminal.
  • the token server manages five token sets.
  • the maximum number of tokens for the five token sets is 10, 7, 9, 4, and 8, each of the current token set.
  • the number of tokens is 9, 6, 8, 3, and 7, respectively, and the number of these 5 token sets is numbered. And set to add 1 token every 1 second interval. If the number of tokens reaches the maximum number of tokens, it will not be added. If the maximum number of tokens is not reached, continue to add.
  • one user searches for World Cup information through the mobile terminal, and the mobile terminal first sends a token acquisition request requesting to obtain one token, and the token acquisition request arrives at the token server at the same time, and the token server randomly selects Token Set 2, the number of tokens in the token set is retrieved, and one of the six tokens in the token set 2 is taken out and sent to the user's mobile terminal. After the mobile terminal obtains the token, the search request sent is searched. The engine processes the user's search results for World Cup information.
  • the token server queries the number of tokens in the token set 1 and the token set 3 adjacent to the token set 2, There are 9 and 8 respectively.
  • the token server chooses to take 1 token from the token set 1 and send it to the mobile terminal. After the mobile terminal obtains the token, the search request sent by the mobile terminal is processed by the search engine, and the user obtains the World Cup information. Search results.
  • the token server adds 1 token per second, the token server randomly selects a token set, selects token set 3, and token token 3 has 8 tokens.
  • Token server query and order The number of tokens in token set 2 and token set 4 adjacent to card set 3 is 6 and 3 respectively, wherein the number of tokens in token set 4 is the least, and in token set 4 The number of tokens does not reach the maximum number of tokens, and one token is added to token set 4.
  • the service unit has the service data that needs to be processed by the processing unit before sending the token acquisition request to the token management unit.
  • the service unit when the service unit has service data to be processed by the processing unit, the service unit may send a token acquisition request to the token management unit, thereby acquiring the token from the token management unit; When there is no data required by the processing unit, the service unit may not send a token acquisition request to the token management unit, and thus the token cannot be obtained.
  • the service unit can The token management unit sends a token acquisition request, which improves the token allocation efficiency and reduces the number of tokens available in the token set when there is a service unit demand token that needs to be processed by the service data, and no service data is needed.
  • the processed business unit has a waste of network resources caused by idle tokens.
  • the service unit includes a plurality of tasks having different task priorities; the task having the specified task priority can send the to the token management unit before having the service data Token get request.
  • the service unit includes a plurality of tasks.
  • the service unit can implement a certain service function, and the implementation of the service function requires the service unit to perform a plurality of different tasks. Having threads executing different tasks in the service unit, corresponding at least one piece of service data may be allocated to different threads for execution, and the threads may acquire tokens and/or call processing from the token management unit.
  • the unit interface allows the processing unit to process the plurality of service data to implement a service function.
  • the plurality of tasks have different task priorities.
  • the service unit may have threads for performing different tasks, and corresponding service data may be allocated to different threads for execution, and the service unit sets different priorities for different task threads, wherein, priority is given.
  • Business data in a high-level thread can be executed with priority.
  • the specified priority may be preset.
  • the thread executing the task may prioritize other threads, and generate service data that needs to be processed by the processing unit.
  • the token acquisition request can be sent before the token management unit.
  • the platform server pushes a message to the user client, and the user can be divided into a VIP user and a normal user.
  • the platform server sets the thread that performs the push of the VIP user message to a high priority, and the thread that performs the normal user message push is executed. Set to low priority.
  • the thread that performs the push of the VIP user message can send the token acquisition request to the token management unit in advance before the message is pushed.
  • the token can be obtained immediately, and the message pushed to the VIP user can be preferentially processed.
  • the task having the specified priority may send the token acquisition request to the token management unit before having the service data. Ensure that important business data can be prioritized, and reduce the possibility of important business data being lost, further improving the efficiency and performance of data processing.
  • the service units have different service priorities; the task of the service unit with the specified service priority can send the token acquisition request to the token management unit before having the service data. .
  • the service units have different service priorities. Different business functions can be implemented for different business units. The implementation of business functions requires execution of certain tasks, and the business unit has at least one thread to perform tasks. Different business functions to be implemented by different business units may have different priorities, so different priorities are set for different business units. Corresponding service data can be allocated to the execution thread of each respective service unit for execution, wherein the service data in the thread with higher priority can be executed preferentially.
  • the specified service priority may be preset, and when the service unit meets a preset priority requirement, the thread that executes the service data of the service unit may be prioritized by other threads, and needs to be processed.
  • a token acquisition request may be sent to the token management unit before the unit processed business data is generated.
  • the user accesses the website through the user terminal, and the user can be divided into a VIP user and a normal user.
  • the VIP user logs in to the user terminal, and the user terminal is in the thread that performs the website access task.
  • the priority information with the priority being high is set; the ordinary user logs in to the user terminal, and the user terminal sets the priority information with the lower priority in the thread that performs the website access.
  • the thread that the VIP user terminal performs website access can send a token acquisition request to the token management unit before the user wants to visit the website, and the ordinary user terminal can only send the token to the token management unit when the user visits the website. Obtaining a request ensures that VIP users can prioritize access to the website.
  • the service unit having the specified service priority may send the token acquisition request to the token management unit before having the service data. It ensures that the business data of important business units can be prioritized, and the possibility that the business data of important business units is lost is also reduced, further improving the efficiency and performance of data processing.
  • the specified rule includes at least one of: adding a specified number of tokens to the token set every specified time interval; or, after the service unit uses the obtained token, the The token management unit adds a corresponding number of tokens to the token set.
  • the specified time may be preset or may be set according to a current network condition, and the token management unit adds to the token set every time after the specified time interval Token.
  • the specified time may be constant or varied.
  • the token management unit presets that a certain number of tokens are added to the token set every 2 seconds; or the token management unit presets to send a token to the business unit for the first time. After that, the token is added to the token set at intervals of 2 seconds. After the token is sent to the service unit for the second time, the token is added to the token set at intervals of 5 seconds. After each token is sent to the service unit, the interval is supplemented at different intervals. Card; or when there is more network data to be processed at present, the token is added to the token set every 1 second. When the network data to be processed is small, the specified time is adjusted, and the interval is 5 seconds. brand.
  • the specified number may be preset or may be set according to a current network condition, and the token management unit concentrates on the token every time the interval is preset. Add the specified number of tokens.
  • the specified number can be constant or varied.
  • the token management unit presets to add two tokens to the token set every 2 seconds, and adds two tokens each time the token issuance interval; or The token management unit presets that after sending the token to the service unit for the first time, the token set is supplemented with two tokens at intervals of 2 seconds, and after the token is sent to the service unit for the second time, the token is sent to the token for 5 seconds.
  • the set adds 5 tokens, each time a token is added to the token set, and a different number of tokens are sent at different intervals; or when there are more network data to be processed, 5 tokens are added each time, when it needs to be processed When the network data is small, adjust the specified quantity and add one token at a time.
  • the token management unit presets or according to the current acquisition network status setting, how many tokens are used by the service unit after the service unit acquires and uses the token acquired from the token management unit. How many tokens are added to the token set by the token management unit.
  • the token management unit sends 5 tokens to the service unit, and the service unit uses all 5 tokens. At this time, the token management unit adds 5 tokens to the token set.
  • the token management unit adds a token to the token set by specifying a certain rule. According to the network status, control the rate and quantity of token replenishment, further improve the shaping of network traffic and avoid network congestion.
  • the addition of tokens to the token set is stopped.
  • the number of token sets reaching the maximum number of tokens may be, the token set has a quantity threshold, and the threshold is the maximum capacity of the token set.
  • the threshold of the number of tokens must be less than or equal to the maximum processing capacity of the processing unit, otherwise it will not function as a current limit.
  • the token management unit may preset that when the number of tokens in the token set reaches the maximum number of tokens, the token is no longer replenished to the token set regardless of any token supplementation rules.
  • the token can be replenished to the token set only if the token in the token set does not reach the token maximum token count and the predetermined token supplementation rule is met.
  • the manner in which the token management unit replenishes the token set to the token set is restricted by setting the maximum number of tokens. Stopping the token to the token set after the token set reaches the maximum number of tokens can save network resources.
  • the token management unit when the token management unit receives the token acquisition request, if the token set has no token, the token management unit processes the token acquisition request.
  • the method includes at least one of: sending a feedback message to a service unit that issues the token acquisition request to notify the service unit to reissue a token acquisition request; or placing the token acquisition request into a request queue,
  • the token is allocated to the service unit corresponding to the token acquisition request in the request queue; or the token acquisition request is placed in the request queue, where the token is
  • the service unit corresponding to the token acquisition request in the request queue issues notification information to notify the service unit to reissue the token acquisition request.
  • the feedback information may be a character string or code written in a fixed format
  • the token management unit queries the number of tokens in the current token set. If the number of tokens in the token set is 0, the service is sent to the service. The unit sends the feedback information.
  • the feedback information may include: the number of tokens in the current token set, and how long the service unit can send the token acquisition request, the number of tokens that can be requested, and the like. Notifying the service unit that the token acquisition fails, and transmitting the time information of the token acquisition request, etc., after a certain time interval or immediately.
  • the service unit sends a token acquisition request to the token management unit, and the token management unit queries the number of tokens in the current token set.
  • the number of tokens in the current token set is 0, and the token management unit sends feedback to the service unit.
  • the information informs the service unit that the current token set number of tokens is 0, and notifies the service unit to send the token acquisition request to the token management unit again after 1 second.
  • the request queue may be a buffer area or a storage unit, and is configured to store the token acquisition request that is not requested to the token.
  • the request queue obtains the request number of the token that enters the request queue.
  • the numbering method may include: numbering according to the time of entry; obtaining the priority number of the service unit or task according to the sending token, the higher the priority, the higher the number
  • the token is obtained by assigning the token to the service unit corresponding to the token acquisition request.
  • the multiple service units may simultaneously send a token acquisition request to the token management unit, and the token management unit queries the number of tokens in the current token set. When the number of tokens in the token set is 0, the token may be acquired. The request is placed in the request queue and numbered according to the request time. After the token set is supplemented with the token, the service unit corresponding to the token acquisition request in the request queue is sequentially assigned the token according to the number.
  • the token supplementing unit may sequentially send the notification information to the service unit corresponding to the token acquisition request in the request queue according to the number, and notify the service unit that the service unit can immediately Or send the token acquisition request again after a period of time.
  • the token in the case where there is no token in the token set, the token is allocated by transmitting feedback information to the service unit and/or placing the token acquisition request in the request queue.
  • the service data processing failure caused by the absence of the token in the token set when the service unit sends the token acquisition request is reduced, and the efficiency and stability of processing the service data are improved.
  • the number of the token sets is plural; the specifying rule includes: selecting one token set in a plurality of token sets, and performing the token set with an adjacent token set number Compare, add a specified number of tokens in a small number of token sets.
  • the token management unit may manage a plurality of token sets, and by managing a plurality of token sets, prevent the service unit from being reliable in a system in which the number of reads and writes per second of the single token set is increased. Reduced sex.
  • the token management unit may randomly select a token set, or may select a token set by directivity or according to a preset rule. Query the current number of tokens of the selected token set, and query the number of tokens of the token set adjacent to or adjacent to or adjacent to the selected token set. Compare the number of tokens for these token sets, select a smaller number of token sets, and replenish the tokens for a smaller number of token sets.
  • the specified number may be preset or may be set according to current network conditions, and may include: increasing the number of fixed tokens; increasing the number of changed tokens; and increasing the fixed interval time.
  • the number of tokens; the fixed interval increases the number of tokens changed; the interval change time increases the number of fixed tokens; the interval change time increases the number of tokens changed, and so on.
  • the number of tokens of the selected token set and the adjacent token set is compared, and the number of tokens is small.
  • the manner in which the token set is supplemented with the token further reduces the possibility that the token management unit selects the empty token set to allocate the token to the service unit after the service unit sends the token acquisition request, and reduces the token acquisition. waiting time.
  • the addition of the token is stopped when the number of tokens in the selected token set and its neighboring token set reaches the maximum number of tokens.
  • the token management unit may preset, after selecting the token set, query the current token number of the selected token set and the token set adjacent to the selected token set.
  • the current number of tokens if the number of tokens of the selected token set and the token set adjacent to the selected token set reaches their respective maximum number of tokens, then the stop order is satisfied regardless of any token supplemental rules. Card supplement.
  • the token management unit is restricted from replenishing the token set to the selected token set and the token set adjacent to the selected token set by setting the maximum number of tokens. Stopping the token to the token set after the token set reaches the maximum number of tokens can save network resources.
  • the number of the token sets is multiple; each of the token sets corresponds to a maximum number of tokens; wherein, the maximum number of tokens of at least part of the token set and the maximum number of other tokens The number of cards is different.
  • the token management unit manages a plurality of token sets, wherein each token set corresponds to a maximum number of tokens, and the maximum number of tokens is less than or equal to the service data processing capability of the processing unit. At least one of the token sets managed by the token management unit may be different from the number of tokens of other token sets, or the number of tokens of all token sets may be different.
  • the token management unit manages five token sets, and the maximum number of tokens in the five token sets is five, four, six, seven, and two, and the maximum token between the token sets The numbers are different from each other.
  • the token management unit can arbitrarily set the maximum number of tokens of the plurality of token sets, and the total number of tokens managed by the token management unit is less than or equal to the processing capability of the processing unit, and no The maximum number of tokens for the token set is set to the same as necessary.
  • the number of the token sets is multiple; after receiving the token acquisition request, the token management unit selects a token set, and the token set and the adjacent token The set performs a token quantity comparison, and assigns a token to a business unit that issues the token acquisition request in a larger number of token sets.
  • the token management unit may compare the selected token set with the number of tokens of the token set adjacent to the selected token set. Select a token set with a large number of tokens, and assign a token to the corresponding business unit from the token set with a larger number of tokens.
  • the token set allocation token with more tokens is selected for the corresponding business unit.
  • the way. The possibility that the service unit can obtain the token without waiting after sending the token acquisition request is improved, and the efficiency of processing the corresponding service data is improved.
  • the number of the token sets is multiple, and after receiving the token obtaining request, the token management unit selects a token set, and the number of tokens in the token set cannot match.
  • the token management unit selects a token set, and the number of tokens in the token set cannot match.
  • the token management unit may select a token set to allocate a token to the service unit.
  • the token management unit may query the token set adjacent to the selected token set. The current number of tokens.
  • the token is assigned from the token set that satisfies the request quantity of the token acquisition request.
  • Commercian unit if there is one request quantity satisfying the token acquisition request from the token set adjacent to the selected token set, the token is assigned from the token set that satisfies the request quantity of the token acquisition request.
  • a token set may be randomly selected from the adjacent token set, and the token is allocated to the service unit;
  • the neighbor tokens select the token set with the largest number of tokens, and assign tokens to the business unit.
  • the service unit if the current number of tokens of the adjacent token set cannot satisfy the request amount of the token acquisition request, the service unit is notified to wait for a period of time to acquire again.
  • the token set selected by the token management unit cannot satisfy the service by assigning a token to the service unit from a token set adjacent to the selected token set.
  • the problem that the number of requests in the token acquisition request sent by the unit is caused by the business unit not acquiring the token or waiting to be able to acquire the token. Thereby, the possibility that the business unit acquires the token is further improved, and the efficiency of the business data processing is improved.
  • the data processing system provided by the embodiment of the present specification sends a token acquisition request to the token management unit through the service unit, and the token management unit allocates the token to the service unit from the at least one token group managed by the token management unit, and the service unit
  • the way service data can be processed by the processing unit in the case of tokens limits the speed of data traffic in the network, adjusts the data traffic output rate, reduces the possibility of system overload, and improves the processing of service data. Efficiency and performance, and the system can be applied to any scenario that requires traffic shaping and rate limiting.
  • an embodiment of the present specification provides a data processing method, which may be applied to a service unit.
  • the method can include the following steps.
  • Step S60 The service unit sends a token acquisition request to the token management unit.
  • Step S62 Receive a token fed back by the token management unit; wherein the token management unit manages at least one token set and maintains the number of tokens in the token set according to a specified rule.
  • Step S64 Providing the service data and the token to the processing unit; wherein the processing unit processes the service data having the token according to a specified processing rule.
  • the subject performing the above steps may be a business unit.
  • the service unit may send a token acquisition request, receive a token fed back by the token management unit, provide service data to the processing unit, and the token.
  • the service unit may be provided with a predetermined port through which token acquisition request information may be sent to the token management unit, a token assigned by the token management unit is received, service data is provided to the processing unit, and Describe the token.
  • the service server may perform network data interaction with the token management unit based on a network protocol such as HTTP, TCP/IP or FTP.
  • an embodiment of the present specification provides a data processing method, which may be applied to a token management unit.
  • the method can include the following steps.
  • Step S70 Receive a token acquisition request sent by the service unit.
  • Step S72 Sending a token in the token set to the service unit, where the service unit provides the token and service data to a processing unit; wherein the processing unit pairs have the token The service data provided by the service unit is processed; wherein the number of the token sets is at least one, and the number of tokens in the token set is maintained according to a specified rule.
  • an embodiment of the present specification provides a data processing method, which may be applied to a processing unit.
  • the method can include the following steps.
  • Step S80 Receive a service processing request of the service unit.
  • Step S82 If the service processing request has a token, process the service data attached to the service processing request according to the specified processing rule; wherein the service unit obtains the token from the token management unit.
  • the service processing request may be a string or code that satisfies a communication protocol and follows a certain format, and may be used to indicate that the service unit that issues the service processing request has service data that needs to be processed by the processing unit.
  • the service processing request may include a search request sent by the user to the search engine by the client; a push request of the platform pushing the message to the user, and the like.
  • the token may be sent to the processing unit in the service processing request, and the processing unit may select whether the token has a service according to the service processing request. Process the business data attached to the request for processing. In the case of having a token, the processing unit may process the corresponding service data according to the service processing request.
  • two users simultaneously use the client to send a search request for searching for entertainment news to the search engine.
  • the two search requests have one token with no token, and the search engine can process the search request with the token to search for related entertainment. News; search requests without tokens can be left unprocessed.
  • the specified processing rule may be preset or may be changed according to current network conditions.
  • the business data can be processed when the business data provided by the business unit satisfies at least one predetermined specified processing rule.
  • the specified processing rule may include: when the service data provided by the service unit includes a token, the processing unit processes the service data; when a plurality of service data having different priorities and containing the token arrive at the same time Priority is given to prioritizing business data with high priority.
  • the processing unit processes the service processing request with the token by receiving the service processing request sent by the service unit, can adjust the data traffic output rate, reduces the possibility of system overload, and improves the The efficiency and performance of business data processing.
  • an embodiment of the present specification provides a token management method, which may be applied to a token management unit, and the token management unit may maintain at least two token sets.
  • the method can include the following steps.
  • S92 Compare the token set with an adjacent token set by a number of tokens, and add a specified number of tokens in a smaller number of token sets; or, combine the token set with an adjacent token The set compares the number of tokens and reduces the specified number of tokens in a larger number of tokens.
  • the manner of selecting one of the token sets may include: randomly selecting one token set from the at least two token sets; and selecting a specified token set from the at least two token sets Specifically, for example, selecting the token set of the specified number for the at least two token set numbers, or selecting a token set according to the location of the token set of the at least two token sets, for example, selecting the most The token set on the left or the rightmost token set.
  • the adjacent token set may include adjacent to the selected one of the token sets, left and right adjacent, up and down adjacent, and the like.
  • the token management unit may query the selected one of the token set and the number of tokens in the token set adjacent to the selected one of the token sets, in a smaller number of orders Add a specified number of tokens in a card set or reduce a specified number of tokens in a larger number of token sets.
  • the specified number may be preset or may be set according to current network conditions, and may include: increasing the number of fixed tokens; increasing the number of changed tokens; and increasing the fixed interval time.
  • the number of tokens; the fixed interval increases the number of tokens changed; the interval change time increases the number of fixed tokens; the interval change time increases the number of tokens changed, and so on.
  • the token management unit compares the number of tokens of the selected token set and the adjacent token set by managing at least two token sets and selecting one token set therefrom. Replenishing a token for a token set with a small number of tokens, reducing the token from a token set with a large number of tokens, and reducing the token selection unit to select an empty command after the service unit sends a token acquisition request.
  • the card set assigns a token to the business unit and reduces the wait time for token acquisition.
  • the present specification provides a data processing system, method, and token management method, the data processing system including: a service unit, a token management unit, and a processing unit, wherein the service unit is configured to send a token to the token management unit Acquiring a request to obtain the token and then providing the service data to the processing unit for processing; the token management unit is configured to manage at least one token set, and maintain the token set token according to a specified rule a quantity; the token management unit, when receiving the token acquisition request, transmitting a token in the token set to a service unit that issues the token acquisition request; the processing unit is configured to follow The specified processing rule processes the business data provided by the business unit having the token.
  • the data processing system provided in this specification limits the speed of data traffic in the network, adjusts the data traffic output rate, reduces the possibility of system overload, and improves the efficiency and performance of service data processing.
  • the various embodiments in the specification are described in a progressive manner, and the same or similar parts between the various embodiments may be referred to each other, and each embodiment focuses on differences from other embodiments.
  • the data processing method applied to the business unit the data processing method applied to the token management unit, the data processing method applied to the processing unit, and the token management method are basically similar to the data processing system
  • the description of the implementation is relatively simple, and the relevant parts can be referred to the description of the implementation of the data processing system.
  • This description can be used in a variety of general purpose or special purpose computer system environments or configurations.

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Computer Security & Cryptography (AREA)
  • Accounting & Taxation (AREA)
  • Signal Processing (AREA)
  • Finance (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Strategic Management (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Development Economics (AREA)
  • Computer Hardware Design (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Business, Economics & Management (AREA)
  • Physics & Mathematics (AREA)
  • Marketing (AREA)
  • Economics (AREA)
  • Game Theory and Decision Science (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Computer And Data Communications (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

La présente invention concerne un système et un procédé de traitement de données et un procédé de gestion de jeton. Le système de traitement de données comprend : une unité de service, une unité de gestion de jeton et une unité de traitement ; l'unité de service est utilisée pour envoyer une demande d'acquisition de jeton à l'unité de gestion de jeton, de façon à fournir des données de service à l'unité de traitement pour un traitement après l'acquisition des jetons ; l'unité de gestion de jeton est utilisée pour gérer au moins un ensemble de jetons, et maintenir la quantité de jetons dans l'ensemble de jetons selon une règle spécifiée ; lorsqu'il a reçu la demande d'acquisition de jeton, l'unité de gestion de jeton envoie les jetons dans l'ensemble de jetons à l'unité de service qui envoie la demande d'acquisition de jeton ; l'unité de traitement est utilisée afin de traiter, selon une règle de traitement spécifiée, les données de service fournies par l'unité de service ayant les jetons. Le système de traitement de données selon la présente invention limite la vitesse des flux de données dans le réseau, réduisant la probabilité de surcharge du système, améliorant les performances de traitement de données.
PCT/CN2018/114348 2017-11-16 2018-11-07 Système et procédé de traitement de données, et procédé de gestion de jeton WO2019096046A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201711138960.4A CN109802895B (zh) 2017-11-16 2017-11-16 数据处理系统、方法及令牌管理方法
CN201711138960.4 2017-11-16

Publications (1)

Publication Number Publication Date
WO2019096046A1 true WO2019096046A1 (fr) 2019-05-23

Family

ID=66539376

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/114348 WO2019096046A1 (fr) 2017-11-16 2018-11-07 Système et procédé de traitement de données, et procédé de gestion de jeton

Country Status (2)

Country Link
CN (1) CN109802895B (fr)
WO (1) WO2019096046A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111585913A (zh) * 2020-04-30 2020-08-25 武汉众邦银行股份有限公司 基于回收令牌的服务流量限制方法及存储介质

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110175078B (zh) * 2019-05-30 2020-12-01 口碑(上海)信息技术有限公司 业务处理方法及装置
CN110708234B (zh) * 2019-09-24 2022-05-17 深圳证券交易所 消息发送的处理方法、消息发送的处理装置及存储介质
CN110912794B (zh) * 2019-11-15 2021-07-16 国网安徽省电力有限公司安庆供电公司 基于令牌集的近似匹配策略
CN111027033B (zh) * 2019-11-27 2022-05-27 中国银行股份有限公司 一种接口的访问方法及装置
CN112866136B (zh) * 2021-03-31 2023-05-02 中国工商银行股份有限公司 业务数据处理方法和装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102640455A (zh) * 2009-11-23 2012-08-15 爱立信(中国)通信有限公司 许可重新分发方法、调解器及其许可控制系统
US20140380324A1 (en) * 2013-06-25 2014-12-25 Amazon Technologies, Inc. Burst-mode admission control using token buckets
CN105409171A (zh) * 2013-06-25 2016-03-16 亚马逊科技公司 突发模式控制
CN106375238A (zh) * 2015-07-21 2017-02-01 深圳市中兴微电子技术有限公司 一种流量监管方法及装置

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN100454897C (zh) * 2005-08-25 2009-01-21 华为技术有限公司 一种网络设备有效防攻击的方法
US7719968B2 (en) * 2005-12-30 2010-05-18 Extreme Networks, Inc. Multi-priority multi-color markers for traffic metering
CN101360049B (zh) * 2008-09-23 2011-02-09 福建星网锐捷网络有限公司 一种报文转发方法及装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102640455A (zh) * 2009-11-23 2012-08-15 爱立信(中国)通信有限公司 许可重新分发方法、调解器及其许可控制系统
US20140380324A1 (en) * 2013-06-25 2014-12-25 Amazon Technologies, Inc. Burst-mode admission control using token buckets
CN105409171A (zh) * 2013-06-25 2016-03-16 亚马逊科技公司 突发模式控制
CN106375238A (zh) * 2015-07-21 2017-02-01 深圳市中兴微电子技术有限公司 一种流量监管方法及装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111585913A (zh) * 2020-04-30 2020-08-25 武汉众邦银行股份有限公司 基于回收令牌的服务流量限制方法及存储介质
CN111585913B (zh) * 2020-04-30 2023-04-07 武汉众邦银行股份有限公司 基于回收令牌的服务流量限制方法及存储介质

Also Published As

Publication number Publication date
CN109802895B (zh) 2023-01-13
CN109802895A (zh) 2019-05-24

Similar Documents

Publication Publication Date Title
WO2019096046A1 (fr) Système et procédé de traitement de données, et procédé de gestion de jeton
US11762703B2 (en) Multi-region request-driven code execution system
US10372499B1 (en) Efficient region selection system for executing request-driven code
Kliazovich et al. CA-DAG: Modeling communication-aware applications for scheduling in cloud computing
US10169060B1 (en) Optimization of packet processing by delaying a processor from entering an idle state
KR101670642B1 (ko) 클라이언트 디바이스 상에서의 패킷 송신을 스케줄링하기 위한 시스템 및 방법
US8832063B1 (en) Dynamic request throttling
WO2018130163A1 (fr) Procédé et dispositif de planification pour plate-forme informatique de nuage mobile
EP2783490B1 (fr) Livraison de données sensibles au temps
US9525644B2 (en) Method and system for managing resources among different clients for an exclusive use
US10228979B1 (en) Dynamic virtual partitioning for delayed queues
WO2020019743A1 (fr) Procédé et dispositif de régulation de trafic
CN111565154B (zh) 图像识别请求处理方法、装置、计算机设备和存储介质
US11316916B2 (en) Packet processing method, related device, and computer storage medium
CN109002364B (zh) 进程间通信的优化方法、电子装置以及可读存储介质
CN112600761A (zh) 一种资源分配的方法、装置及存储介质
CN114155026A (zh) 一种资源分配方法、装置、服务器及存储介质
US10154116B1 (en) Efficient synchronization of locally-available content
CN107294911A (zh) 一种数据包监听方法及装置、远程过程调用系统、设备
WO2022142008A1 (fr) Procédé et appareil de traitement de données, dispositif électronique et support de stockage
CN114911602A (zh) 一种服务器集群的负载均衡方法、装置、设备和存储介质
US10979359B1 (en) Polling resource management system
US20220374259A1 (en) Application-centric design for 5g and edge computing applications
CN106790632B (zh) 一种流数据的并发传输方法和装置
US10728291B1 (en) Persistent duplex connections and communication protocol for content distribution

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18879135

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18879135

Country of ref document: EP

Kind code of ref document: A1