CN111782732B - Data processing method, device and computer readable storage medium - Google Patents
Data processing method, device and computer readable storage medium Download PDFInfo
- Publication number
- CN111782732B CN111782732B CN202010703197.0A CN202010703197A CN111782732B CN 111782732 B CN111782732 B CN 111782732B CN 202010703197 A CN202010703197 A CN 202010703197A CN 111782732 B CN111782732 B CN 111782732B
- Authority
- CN
- China
- Prior art keywords
- key
- keys
- target
- consumption
- target object
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 31
- 238000000034 method Methods 0.000 claims abstract description 32
- 238000012545 processing Methods 0.000 claims abstract description 30
- 230000004044 response Effects 0.000 claims abstract description 20
- 238000004590 computer program Methods 0.000 claims description 15
- 238000007726 management method Methods 0.000 description 33
- 238000010586 diagram Methods 0.000 description 13
- 238000005516 engineering process Methods 0.000 description 10
- 238000004891 communication Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 4
- 239000007787 solid Substances 0.000 description 3
- 238000013500 data storage Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 230000002441 reversible effect Effects 0.000 description 2
- 230000003111 delayed effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000003442 weekly effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Telephonic Communication Services (AREA)
Abstract
The embodiment of the application provides a data processing method, a data processing device and a computer readable storage medium, wherein the method comprises the following steps: receiving an acquisition request of a target object sent by terminal equipment; determining a target key from a plurality of keys corresponding to a target object in response to an acquisition request, wherein each key in the plurality of keys is used for correspondingly recording the consumption and the consumption threshold value of the target object on each key, and the sum of the consumption threshold values recorded by the plurality of keys is a preset total consumption threshold value of the target object; acquiring a first consumption and a first consumption threshold value of a target object on a target key, which are recorded corresponding to the target key, from a storage server; if the first consumption is less than the first consumption threshold, a pass acquire request is determined. By the method, data access can be uniformly distributed to a plurality of keys when the acquisition request of the article is processed, and hot spot data which are frequently read in a storage server are avoided, so that parallel capacity expansion of interface service is facilitated.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a data processing method, apparatus, and computer readable storage medium.
Background
A hotspot is data that is accessed by high frequencies in a distributed environment. When a large number of access requests for certain data occur in the network, the data can become a hot spot. The storage service may use keys (keys) to record some data, such as the total consumption of an item, for which the calculation is performed by the corresponding calculation unit (e.g., CPU core unit).
Currently, in a scenario where there is a limit to the total consumption, when the interface server receives an acquisition request for a certain item, it is necessary to access the total consumption recorded by the key corresponding to the item from the storage service. When the access request is too many, the data of the total consumption of the key record becomes hot spot data and is accessed frequently, and the load of the computing unit corresponding to the key is too large, so that the system has a large processing delay and even fails to process. Therefore, how to avoid hot spot data in a storage service when data access is performed is a problem that needs to be solved at present.
Disclosure of Invention
The embodiment of the invention provides a data processing method, a data processing device and a computer readable storage medium, which can realize that data access is uniformly distributed to a plurality of keys when an acquisition request of an article is processed, avoid hot spot data which are frequently read in a storage server, and further facilitate parallel capacity expansion of interface service.
In a first aspect, an embodiment of the present application provides a data processing method, which is applied to an interface server included in an interface service cluster, where the method includes:
Receiving an acquisition request of a target object sent by terminal equipment;
Determining a target key from a plurality of keys corresponding to a target object in response to an acquisition request, wherein each key in the plurality of keys is used for correspondingly recording the consumption and the consumption threshold value of the target object on each key, and the sum of the consumption threshold values recorded by the plurality of keys is a preset total consumption threshold value of the target object;
Acquiring a first consumption and a first consumption threshold value of a target object on a target key, which are recorded corresponding to the target key, from a storage server;
If the first consumption is less than the first consumption threshold, a pass acquire request is determined.
In a second aspect, an embodiment of the present application provides a data processing method, applied to a control server, where the method includes:
Acquiring performance parameters configured by a management system and configuration parameters of a target object, wherein the performance parameters comprise the maximum concurrency request quantity of an interface service cluster and the maximum concurrency number of each second of a storage server, and the configuration parameters comprise the total consumption threshold of the target object;
determining a plurality of keys corresponding to the target object and consumption thresholds of the target object corresponding to each key in the plurality of keys according to the performance parameters and the configuration parameters, wherein the sum of the consumption thresholds corresponding to the plurality of keys is a preset total consumption threshold of the target object;
and sending a plurality of keys corresponding to the target items to an interface server included in the interface service cluster, and sending the consumption threshold of the target items corresponding to each key to a storage server, so that the storage server stores the consumption threshold of the target items corresponding to each key.
In a third aspect, an embodiment of the present application provides a data processing apparatus, applied to an interface server included in an interface service cluster, where the apparatus includes:
the receiving and transmitting unit is used for receiving the acquisition request of the target object sent by the terminal equipment;
the processing unit is used for determining a target key from a plurality of keys corresponding to the target object in response to the acquisition request, wherein each key in the plurality of keys is used for correspondingly recording the consumption and the consumption threshold value of the target object on each key, and the sum of the consumption threshold values correspondingly recorded by the plurality of keys is a preset total consumption threshold value of the target object;
The acquisition unit is used for acquiring the first consumption and the first consumption threshold value of the target object on the target key, which are recorded corresponding to the target key, from the storage server;
the processing unit is further configured to determine that the request is received if the first consumption is less than a first consumption threshold.
In a fourth aspect, an embodiment of the present application provides a data processing apparatus, applied to a control server, including:
The system comprises an acquisition unit, a storage server and a management system, wherein the acquisition unit is used for acquiring performance parameters configured by the management system and configuration parameters of the target object, the performance parameters comprise the maximum concurrency request quantity of the interface service cluster and the maximum concurrency quantity per second of the storage server, and the configuration parameters comprise the total consumption threshold of the target object;
The processing unit is used for determining a plurality of keys corresponding to the target object and consumption thresholds of the target object corresponding to each key in the plurality of keys according to the performance parameters and the configuration parameters, and the sum of the consumption thresholds corresponding to the plurality of keys is a preset total consumption threshold of the target object;
And the receiving and transmitting unit is used for transmitting the plurality of keys corresponding to the target items to an interface server included in the interface service cluster and transmitting the consumption threshold value of the target items corresponding to each key to the storage server so that the storage server stores the consumption threshold value of the target items corresponding to each key.
In a fifth aspect, an embodiment of the present application provides a data processing apparatus comprising a processor and a memory, the processor and the memory being interconnected, wherein the memory is for storing a computer program, the computer program comprising program instructions, the processor being configured to invoke the program instructions to perform a data processing method as described in the first aspect or a data processing method as described in the second aspect.
In a sixth aspect, embodiments of the present application provide a computer readable storage medium storing a computer program comprising program instructions which, when executed by a processor, cause the processor to perform a data processing method as described in the first aspect, and a data processing method as described in the second aspect.
In a seventh aspect, embodiments of the present application provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions so that the computer device performs the data processing method provided in the various optional manners of the first aspect or the second aspect described above.
In the embodiment of the application, an interface server in an interface service cluster receives an acquisition request of a target object sent by terminal equipment; determining a target key from a plurality of keys corresponding to the target object in response to the acquisition request, wherein each key in the plurality of keys is used for correspondingly recording the consumption and the consumption threshold value of the target object on each key, and the sum of the consumption threshold values correspondingly recorded by the plurality of keys is a preset total consumption threshold value of the target object; acquiring a first consumption and a first consumption threshold value of a target object on a target key, which are recorded corresponding to the target key, from a storage server; if the first consumption is less than the first consumption threshold, a pass acquire request is determined. By the method, data access can be uniformly distributed to a plurality of keys when the acquisition request of the article is processed, and hot spot data which are frequently read in a storage server are avoided, so that parallel capacity expansion of interface service is facilitated.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of an article management and control system according to an embodiment of the present application;
FIG. 2 is a schematic flow chart of a data processing method according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a plurality of keys corresponding to a target object according to an embodiment of the present application;
FIG. 4 is a schematic diagram of another article management and control system according to an embodiment of the present application;
FIG. 5 is a schematic diagram of yet another article management and control system according to an embodiment of the present application;
FIG. 6 is a flowchart of another data processing method according to an embodiment of the present application;
FIG. 7 is a schematic diagram of a process for generating a plurality of keys corresponding to a target object according to an embodiment of the present application;
FIG. 8 is a flowchart of another data processing method according to an embodiment of the present application;
FIG. 9 is a schematic diagram of a unit of a data processing apparatus according to an embodiment of the present application;
fig. 10 is a simplified schematic diagram of a physical structure of a data processing apparatus according to an embodiment of the present application.
Detailed Description
The technical scheme in the embodiment of the application will be described below with reference to the accompanying drawings.
In the prior art, since the control server does not generate multiple keys for the interface server and the storage server, the interface server can determine whether the item acquisition request can pass by acquiring the current consumption and the consumption threshold corresponding to only one Key corresponding to the item stored in the storage server after receiving the item acquisition request sent by the user. And, every time the interface server receives an article acquisition request, the current consumption and the consumption threshold value of the Key are acquired by the storage server to judge. In addition, the storage server only uses one computing unit to process the Key, if the article acquisition requests sent by the users are excessive in a period of time, the computing unit becomes a bottleneck of the whole system due to limited computing capacity, so that the Key becomes a hot spot, and parallel expansion of interface services is difficult. There are two solutions to this phenomenon at present.
One is to increase the frequency of the computing unit in the storage server or to optimize the program performance, but the main frequency of the computing unit has a physical upper limit and the program performance has a limit value, which can only be extended vertically, and the rise space is limited.
Secondly, the total quantity of the articles is cancelled, the total quantity of the articles is calculated in a background quasi-real time mode, and when the total quantity of the articles reaches the limit, an alarm is given, so that related operators increase the total quantity limit or stop the article taking activities of users. However, the alarm after the total quantity of the articles is consumed to be limited may be delayed or omitted, so that the total quantity of the articles dispensed cannot be accurately controlled, and part of the risk of system operation is shared among system users.
Therefore, the two solutions can not solve the problem of the hot Key well.
In order to better understand the embodiments of the present application, the following description refers to the terms and the specialized knowledge involved in the embodiments of the present application:
Cloud technology (Cloud technology) is based on the general terms of network technology, information technology, integration technology, management platform technology, application technology and the like applied by Cloud computing business models, and can form a resource pool, so that the Cloud computing business model is flexible and convenient as required. Cloud computing technology will become an important support. Background services of technical networking systems require a large amount of computing, storage resources, such as video websites, picture-like websites, and more portals. Along with the high development and application of the internet industry, each article possibly has an own identification mark in the future, the identification mark needs to be transmitted to a background system for logic processing, data with different levels can be processed separately, and various industry data needs strong system rear shield support and can be realized only through cloud computing.
Cloud storage (cloud storage) is a new concept that extends and develops in the concept of cloud computing, and a distributed cloud storage system (hereinafter referred to as a storage system for short) refers to a storage system that integrates a large number of storage devices (storage devices are also referred to as storage nodes) of various types in a network to work cooperatively through application software or application interfaces through functions such as cluster application, grid technology, and a distributed storage file system, so as to provide data storage and service access functions for the outside.
At present, the storage method of the storage system is as follows: when creating logical volumes, each logical volume is allocated a physical storage space, which may be a disk composition of a certain storage device or of several storage devices. The client stores data on a certain logical volume, that is, the data is stored on a file system, the file system divides the data into a plurality of parts, each part is an object, the object not only contains the data but also contains additional information such as a data Identification (ID) and the like, the file system writes each object into a physical storage space of the logical volume, and the file system records storage location information of each object, so that when the client requests to access the data, the file system can enable the client to access the data according to the storage location information of each object.
The process of allocating physical storage space for the logical volume by the storage system specifically includes: physical storage space is divided into stripes in advance according to the set of capacity measures for objects stored on a logical volume (which measures tend to have a large margin with respect to the capacity of the object actually to be stored) and redundant array of independent disks (RAID, redundant Array of INDEPENDENT DISK), and a logical volume can be understood as a stripe, whereby physical storage space is allocated for the logical volume.
In order to better understand the embodiments of the present application, a system architecture to which the embodiments of the present application are applicable is described below.
Referring to fig. 1, fig. 1 is a schematic diagram of an article management and control system according to an embodiment of the present application. As shown in fig. 1, the article management and control system includes a management system, a control server, an interface service cluster, and a storage service. The interface service cluster may include a plurality of interface servers, including a first interface server, a second interface server, and a third interface server in fig. 1, each configured with a routing module. In practical application, the interface service cluster may include more than three interface servers, and in the embodiment of the present application, the interface service cluster includes three interface servers as an example. The storage server may assign a plurality of keys to the target item, each key being used to record the consumption of the target item on that key. The storage server may also assign calculation units, namely, "K1/L1" to "Kn/Ln" in the figure, to each Key of the plurality of keys (keys), respectively. Wherein K1-Kn are indexes of each Key respectively, L1-Ln are consumption thresholds configured by each Key in K1-Kn respectively, and sum of consumption thresholds (namely L1, L2, … … and Ln) corresponding to the n keys in K1-Kn is a preset total consumption threshold (Lmax) of the target object.
The management system as shown in fig. 1 may be a system that an operator uses to set parameters such as an item total consumption threshold (Lmax), a single user limit, a maximum concurrent request amount (Qmax) of an interface service cluster, a second maximum concurrent request amount (Pmax) of a storage server, and the like for each item. Wherein the total limit quantity of the articles refers to the total dispensing quantity of a certain article, and the maximum dispensing quantity of the article is limited; the single user limit refers to the number of items that a user can pick up at most; the maximum concurrency request per second refers to the maximum number of item requests that the storage server can receive in one second. The items may refer to items in a virtual game or other product, or other items or data that may be picked up and accessed. The management system may include an interactive interface to configure each performance parameter and configuration parameter by obtaining parameter values entered by the operator for the above parameters. Of course, the management system may also configure each performance parameter and configuration parameters by the operator entering an identifiable computer language.
The control server, as shown in fig. 1, is used to provide control services, and may operate on control classes for one or more interface servers and storage servers in the interface service cluster. The control server can generate keys according to performance parameters and configuration parameters configured by the management system and by combining the working capacity of the specific server. Wherein the control service can control the number of keys generated, as well as the threshold value of each Key; the storage server may also be controlled to assign a corresponding computational unit to the generated Key. For example, if the control server determines that 10 keys are to be generated according to the performance parameters and the configuration parameters obtained from the management system, the control server may send the 10 keys to each interface server in the interface service cluster, and may further control the storage service to allocate 10 computing units to the 10 keys.
Each interface service in the interface service cluster shown in fig. 1 is configured to receive an article acquisition request initiated by a user, and determine whether the article acquisition request can pass according to information acquired from a storage service. Specifically, after receiving an article acquisition request sent by a user, the interface service can determine a target Key from a plurality of keys in the storage server through the routing module, and further judge whether the article acquisition request can pass according to the current consumption and the consumption threshold of the target Key acquired from the storage server, so that in a scene of limiting the total amount of articles, the request received by the interface service can be uniformly distributed to the plurality of keys, and hot spot data which are frequently read in the storage server are avoided, thereby increasing the number of the interface services and avoiding the hot spot data, and facilitating the parallel capacity expansion of the interface service. Each interface server is provided with a routing module, and the routing module can enable each interface server to exchange data with the storage service. For example, the interface server may obtain the current consumption of one or more keys and the consumption threshold for each Key via the routing module.
The storage server as shown in fig. 1 may assign computing units to a plurality of keys generated by the control server. The storage server may also increment the total consumption of the item by one count unit after the interface server has passed an item acquisition request sent by a user. For example, for a single user, only one article can be taken, and the total consumption of the articles is 99 currently, if the storage server receives a notification message that the interface server passes through an article acquisition request, the total consumption of the articles can be increased by one counting unit, namely, the total consumption becomes 100; accordingly, the target Key in the storage server selected by the interface server as the item acquisition request also increases a consumption amount.
In order to eliminate hot spots in the server, the embodiment of the application provides a data processing method, a data processing device and a computer readable storage medium, and the data processing method and the communication device provided by the embodiment of the application are further described in detail below.
Referring to fig. 2, fig. 2 is a flowchart of a data processing method applied to an interface server according to an embodiment of the present application. The execution body of the embodiment is any one interface server in the interface service cluster. The data processing method may include the steps of:
210. And receiving an acquisition request of the target object sent by the terminal equipment.
The user clicks and retrieves the target object through the interaction page of the terminal device, and the terminal device can generate an acquisition request of the target object according to the operation of the user and send the acquisition request of the target object to the interface server.
Before the interface server receives the acquisition request of the target object sent by the terminal device, the interface service also needs to receive a plurality of keys corresponding to the target object sent by the control server, wherein the keys are determined by the control server according to the performance parameters and the configuration parameters configured by the management system. The performance parameters configured by the management system include a maximum concurrency request quantity (Qmax) of the interface service cluster and a maximum concurrency per second (Pmax) of the storage server, and the configuration parameters include a total consumption threshold (Lmax) of the target item.
220. And determining a target key from a plurality of keys corresponding to the target item in response to the acquisition request, wherein each key in the plurality of keys is used for correspondingly recording the consumption and the consumption threshold value of the target item on each key, and the sum of the consumption threshold values recorded by the plurality of keys is a preset total consumption threshold value of the target item.
Specifically, after receiving the acquisition request of the target object, the interface server may determine, in response to the acquisition request, a target key from a plurality of keys corresponding to the target object. The interface server may first process the get request using the old keys, correspondingly consuming one of the old keys. The interface server may then convert the consumed key of the old keys into a plurality of keys sent by the control server.
In one possible implementation, the interface service may determine the target key by a method of polling. The interface server may first obtain the first index of the recorded key in response to the above-mentioned obtaining request. And determining a target key from a plurality of keys corresponding to the target object according to the first index. The first index may be recorded and saved by the interface server. The saved first index may be a number of a key, and instruct the interface server to determine that the key corresponding to the number is the target key. For example, the first index is 8, the interface server may determine that the 7 th key is the target key. The indices of the plurality of keys, that is, K1 to Kn, are counted from 0, that is, K1 has an index of 0 and Kn has an index of n-1.
In one possible implementation manner, before the interface server obtains the first index of the recorded key, it needs to determine whether the first index exists in the interface server, and in the case that the first index exists, the target key may be determined directly according to the first index. If the first index does not exist in the interface server, the interface server needs to generate a random number r, where the random number r may be any positive integer greater than or equal to 0. The fact that the first index does not exist in the interface server may be caused by that the interface server currently receives the acquisition request of the target object sent by the user through the terminal device for the first time, or the first index is lost in the interface server, so that the first index cannot be read. After the interface service generates a random number r, the number n of the plurality of keys is balanced by the random number r, so that a second index can be obtained, and the target key is determined from the plurality of keys corresponding to the target object by the second index. Wherein the number n of the plurality of keys is determined by the control server according to the performance parameters and the configuration parameters configured by the management system. For example, the interface server may determine that the 10 th key of the plurality of keys corresponding to the target item is the target key when the random number generated by the interface service is 99 and the number of the plurality of keys n is 10, and the first index is equal to r% n, that is, 99%10, and the result is 9 and the second index is 9. After determining the target key from the plurality of keys corresponding to the target object according to the second index, the interface server generates the first index according to the second index and the number of the plurality of keys, and stores the first index. The first index may be generated by adding one to the second index, and then adding one to the second index to obtain a sum, and taking the remainder of the number n of the plurality of keys. For example, if the second index is 9, the interface server identifies the target key from the plurality of keys corresponding to the target object based on the second index, and then obtains 10 by 9+1, and obtains 0 by calculating 10%10, and the first index is 0. If the interface server receives the acquisition request of the next target object, the key with the target key index of 0 can be determined to be the target key.
In one possible implementation, the interface server may also determine the target key by means of probabilistic access. The interface server may obtain, in response to the obtaining request, the consumption amount of the target item recorded corresponding to each of the plurality of keys corresponding to the target item. And then determining the access probability of each key according to the consumption of the target object recorded corresponding to each key, and finally determining the target key from the plurality of keys according to the access probability of each key. Referring to fig. 3, fig. 3 is a schematic structural diagram of a plurality of keys corresponding to a target object according to an embodiment of the present application. As shown in fig. 3, the number of the plurality of keys is 5, that is, K1 to K5, the consumption of the target object recorded for each key is a, b, c, d, e, and the threshold value of each key configuration is L1, L2, L3, L4, and L5. Let s=a+b+c+d+e, then K1 has an access probability of e/s, K2 has an access probability of d/s, K3 has an access probability of c/s, K4 has an access probability of b/s, and K5 has an access probability of a/s. It will be appreciated that each key corresponds to a recorded consumption of the target item, arranged in reverse in the molecules of the probability of access. Therefore, the probability of accessing the keys with small consumption is increased, and the probability of accessing the keys with large consumption is reduced, so that each key can be accessed more uniformly, and some keys are prevented from becoming hot spots. For example, if the number of the plurality of keys is 5, that is, K1 to K5, the threshold value of each key is 20. The interface server obtains the consumption of the target object recorded corresponding to each key as 6, 8, 10, 12 and 14 respectively. The interface server can calculate that the access probability of K1 is 14/50, the access probability of K2 is 12/50, the access probability of K3 is 10/50, the access probability of K4 is 8/50, and the access probability of K5 is 6/50. As can be seen from the threshold value of 20 for each key, the remaining consumption amount of K1 is the largest, so the access probability of K1 is the largest; the remaining consumption of K5 is the least, so the access probability of K5 is the least. By the method, access of K1-K5 can be more uniform.
In one possible implementation, the interface server may obtain the locally cached data, and query the locally cached data for the consumption of the target item recorded for each of the plurality of keys corresponding to the target item. If the consumption of the target object of each key exists in the local cache, the consumption of the target object of the local cache is used for calculating the access probability of each key. If the consumption of the target item for each key does not exist in the local cache, the interface server may obtain the consumption of the target item for each key from the storage server.
In one possible implementation, the interface server may also only obtain the size arrangement relation of the consumption amount of the target item recorded corresponding to each of the plurality of keys, that is, without obtaining a specific value. For example, in fig. 3, when the interface server obtains that the consumption of the target items of K1 to K5 is a > b > c > d > e, the interface server may calculate that the access probability of K5 is minimum and the access probability of K1 is maximum. The interface server may determine the target key according to the magnitude arrangement relation of the access probability.
230. The method comprises the steps of obtaining a first consumption amount and a first consumption amount threshold value of a target object on a target key, wherein the first consumption amount and the first consumption amount threshold value are recorded in a corresponding mode of the target key from a storage server.
After the interface server determines the target key, if the acquisition request of the target object sent by the terminal device and received by the interface server can pass, the first consumption and the first consumption threshold of the target object on the target key, which are recorded corresponding to the target key, need to be acquired from the storage server.
240. If the first consumption is less than the first consumption threshold, a pass acquire request is determined.
The interface server may determine a size relationship between the first consumption and the first consumption threshold, and if the first consumption is smaller than the first consumption, the interface server may determine that the consumption of the target item, which represents the target key and the remaining target items, may be used, and may determine an acquisition request of the target item sent through the terminal device. After the interface server determines the acquisition request of the target object sent by the terminal device, a notification message for updating the target key is sent to the storage server, where the notification message is used to instruct the storage server to update the consumption of the target object corresponding to the stored target key. For example, when the current consumption of the target key in the storage server is 19 and the storage server receives the notification message sent by the reverse interface server, the current consumption of the target key may be updated. If the user gets 1 article, the current consumption of the target key in the storage server is updated to 20; if the user has picked up 2 items, the current consumption of the target key in the storage server is updated to 21, and so on.
After the interface server determines the acquisition request of the target object sent by the terminal device, the interface service may update the consumption of the target object corresponding to the target key included in the locally cached data. When the next acquisition request of the target object arrives, the interface server can then correspond to the recorded consumption of the target object according to each of the plurality of keys corresponding to the locally cached target object, without re-acquiring from the storage server. It should be noted that, there may be a certain error between the consumption of the target item recorded by each of the plurality of keys corresponding to the target item locally cached by the interface server and the consumption of the target item recorded by each of the plurality of keys corresponding to the target item in the storage server, because the storage server also receives notification messages of update keys of other interface servers. The error only has a certain influence on the accuracy of the access probability, that is, the error affects at most the average degree of each key in the storage server determined as the target key by the interface server, so that the average degree cannot reach high accuracy. For example, in the case where the average degree is very high, the number of times each key as shown in fig. 3 in the storage server is determined as the target key may be 19 times, 20 times, 19 times, 20 times, respectively. While the error may simply cause the number of times each key is determined as the target key to become less uniform than described above, for example 17 times, 18 times, 20 times, 22 times, 21 times. But the total number of times the target key is determined from the plurality of keys is the same. The error does not affect the storage server's calculation of the total consumption of the target item.
In one possible implementation manner, when the interface server detects the first consumption of the target object recorded by the corresponding target key in the target key, after the target object is acquired through the current acquisition request, the first consumption threshold is just reached, that is, after the target object is acquired through the current acquisition, the interface server just finishes the consumption, and then the interface server automatically skips the target key when determining the target key next time. For example, if the first consumption of the target key determined by the interface server is 99 and the first consumption threshold is 100, the first consumption will become 100 after the target item is acquired this time, that is, the consumption of the target key will reach the threshold. The interface server will not select that key as the target key the next time.
In one possible implementation, for the case that the user retrieves 2 items at a time, if the first consumption of the target key determined by the interface server is 99 and the first consumption threshold is 100, it may happen that the number of target items that the user wants to retrieve is greater than the remaining consumption of the target key. In this case, the interface server may consume one consumption amount of the current target key first, and then determine another target key again from the storage server, so as to complete the request for acquiring 2 target items sent by the user.
In one possible implementation, in the prior art, where the target item is a single user limit, the storage server would assign a key to each user individually to calculate if the target item pickup for each user is out of limit. Therefore, the system can be combined with the architecture of the total amount limitation of the articles in the embodiment of the application, and the system can refer to fig. 4, and fig. 4 is a schematic diagram of another article management and control system provided in the embodiment of the application. The system may be adapted for use in situations where the total amount of items is limited, and where there is a limit to a single user. That is, the item management system may provide two storage services, namely a single user limited storage service and an item total limited storage service, using one storage server. Specifically, after receiving the request for acquiring the target article sent by the terminal device, the interface server may first determine whether the target article acquired by the user exceeds the limit on a plurality of keys limited by a single user, and if not, may determine whether the consumption of the target article reaches the consumption threshold on a plurality of keys limited by the total amount of articles, thereby determining whether the request for acquiring the target article is sent by the terminal device. For example, the consumption threshold of the target articles is 1000, and a single user can get 3 (namely, the upper limit of the single user gets the number), if the interface server knows that the target articles which a certain user gets are 2 on a plurality of keys limited by the single user, the interface server judges that the target articles which the user gets are not out of limit; and further continuing to judge on a plurality of keys for limiting the total amount of the articles, and if the total consumption of the target article is 900, which represents that the target article is not picked up, the interface server can pass the acquisition request of the target article.
In one possible implementation manner, when the interface server fails or is down, the interface server can be directly shut down or taken off line, and each interface server is independent and does not influence each other.
In one possible implementation manner, if the operator increases the total amount of the target object through the management system, the control server may newly generate a plurality of keys according to the performance parameter and the configuration parameter configured by the management system, and after receiving the request, the interface server may determine the target key according to the original plurality of keys and the newly added plurality of keys, so as to determine whether the acquisition request of the target object can pass.
In one possible implementation manner, the embodiment of the application can be applied to effect advertisements, such as limiting the play amount of the advertisements according to the advertisement exposure, click-through amount or expense consumption.
In one possible implementation, please refer to fig. 5, fig. 5 is a schematic diagram of another article management and control system according to an embodiment of the present application. The article management and control system shown in fig. 5 comprises an interface service cluster and a real-time computing server, wherein the real-time computing server can calculate the residual consumption corresponding to each current interface server in real time and push the value of the real-time residual consumption to each interface server. And judging whether the acquisition request of the target object reaches the upper limit or not by each interface server according to the self residual consumption.
In one possible implementation manner, based on the article management and control system shown in fig. 5, each interface server may report its own consumption amount to the real-time computing server, and the real-time computing server may collect the consumption amounts to calculate a total consumption amount, and when the total consumption amount reaches a consumption amount threshold, notify all interface servers that the consumption amount is out of limit, and reject an acquisition request of an article sent by the terminal device.
In a possible implementation manner, the embodiment of the application can also be applied to single IP, single Open ID, and other scenes, that is, the number of items that can be taken by one device corresponding to one IP or one Open ID is limited. In addition, the consumption threshold may be extended to a1 year consumption threshold, a1 month consumption threshold, a1 week consumption threshold, etc., or may be periodic consumption thresholds, such as a weekly consumption threshold, etc.
According to the embodiment of the application, after receiving the acquisition request of the target object sent by the terminal equipment, the interface server determines the target key from the plurality of keys of the corresponding point of the target object, and judges whether the first consumption of the target object on the changed target key is smaller than a first consumption threshold. If yes, the notification message for updating the target key can be further sent to the storage server through the acquisition request of the target item, and the storage server can update the consumption of the stored target item corresponding to the target key. By the method, each key corresponding to the target object can be prevented from becoming an access hot spot, the requests received by the interface server can be uniformly distributed to the plurality of keys, and hot spot data which are frequently read in the storage server are prevented from being generated, so that the hot spot data cannot be generated due to the increase of the number of the interface services, and the parallel capacity expansion of the interface services is facilitated.
Referring to fig. 6, fig. 6 is a flowchart of a data processing method applied to a control server according to an embodiment of the present application. The execution subject of this embodiment is a control server. The data processing method may include the steps of:
610. And acquiring performance parameters configured by the management system and configuration parameters of the target object, wherein the performance parameters comprise the maximum concurrency request quantity of the interface service cluster and the maximum concurrency number of the storage server per second, and the configuration parameters comprise the total consumption threshold of the target object.
The maximum concurrency request quantity of the interface service cluster is Qmax, the maximum concurrency quantity per second of the storage server is Pmax, and the total consumption threshold value of the target object is Lmax.
620. And determining a plurality of keys corresponding to the target object and consumption thresholds of the target object corresponding to each key in the plurality of keys according to the performance parameters and the configuration parameters, wherein the sum of the consumption thresholds corresponding to the plurality of keys is a preset total consumption threshold of the target object.
Specifically, as shown in fig. 7, the control server may generate n keys according to Lmax (total consumption threshold of the target item), qmax (maximum concurrency request amount of the interface service cluster), and Pmax (second maximum concurrency request amount of the storage server). The n keys generated are K1/L1-Kn/Ln in FIG. 7, where K1-Kn are the index of each newly generated key and L1-Ln are the threshold of each newly generated key. The control server first determines the number n of keys corresponding to the target item to be generated based on Qmax and Pmax. The control server then calculates an intermediate parameter t, which t=qmax/Pmax, when t=0, n=1; when t >0 and t is not an integer, n=int (t) +1; when t >0 and t is an integer, n=t.
After determining the number of the plurality of keys corresponding to the target item, the consumption threshold L of the target item corresponding to each of the plurality of keys needs to be determined according to Lmax and the number n of the plurality of keys. Threshold l=int (Lmax/n) +delta, let m=lmax% n, delta=1 for the first m keys; delta=0 for the last n-m bonds.
For example, if qmax=1000, pmax=200, lmax=2000, then t=5, n=t=5, m=2000%5=0; the 5-key corresponding point threshold values L are all the same, i.e., l1=l2=l3=l4=l5=2000/5=400, and l1+l2+l3+l4+l5=400+400+400+400+400=2000 (i.e., the total consumption threshold value Lmax of the target item).
If qmax=1100, pmax=200, lmax=2000, then t=5.5, n=int (5.5) +1=6, m=2000%6=2; the threshold of the first 2 keys l1=l2=int (2000/6) +1=334; the threshold value l3=l4=l5=l6=int (2000/6) =333 for the last 4 keys, and l1+l2+l3+l4+l5+l6=334+334+333+333+333+333=2000 (i.e., the total consumption threshold Lmax of the target item).
630. And sending a plurality of keys corresponding to the target items to an interface server included in the interface service cluster, and sending the consumption threshold of the target items corresponding to each key to a storage server, so that the storage server stores the consumption threshold of the target items corresponding to each key.
According to the embodiment of the application, the control server can acquire the performance parameters configured by the management system and the configuration parameters of the target object, and determine a plurality of keys corresponding to the target object and the consumption threshold value of the target object corresponding to each key in the plurality of keys according to the performance parameters and the configuration parameters of the target object. After the determination, a plurality of keys corresponding to the target items may be sent to each interface server, and the consumption threshold of the target items corresponding to each key may be sent to the storage server, so that the storage server stores the consumption threshold of the target items corresponding to each key. According to the method, the control server can split the keys corresponding to the objects into a plurality of keys, so that the interface server and the storage server are controlled to process data, hot spot data which are frequently read in the storage server can be avoided, the hot spot data can not be caused by increasing the number of the interface services, and the parallel capacity expansion of the interface services is facilitated.
Referring to fig. 8, fig. 8 is a flowchart of another data processing method according to an embodiment of the application. The execution body of the embodiment comprises a control server and an interface server. The data processing method may include the steps of:
810. the control server obtains performance parameters configured by the management system and configuration parameters of the target items, wherein the performance parameters comprise the maximum concurrency request quantity of the interface service cluster and the maximum concurrency quantity per second of the storage server, and the configuration parameters comprise the total consumption threshold of the target items.
820. And the control server determines a plurality of keys corresponding to the target object and a consumption threshold value of the target object corresponding to each key in the plurality of keys according to the performance parameters and the configuration parameters, wherein the sum of the consumption threshold values corresponding to the plurality of keys is a preset total consumption threshold value of the target object.
830. The control server sends a plurality of keys corresponding to the target object to an interface server included in the interface service cluster.
840. The control server sends the consumption threshold of the target item corresponding to each key to the storage server so that the storage server stores the consumption threshold of the target item corresponding to each key.
850. And the interface server receives the acquisition request of the target object sent by the terminal equipment.
860. And the interface server responds to the acquisition request to determine a target key from a plurality of keys corresponding to the target object, wherein each key in the plurality of keys is used for correspondingly recording the consumption and the consumption threshold value of the target object on each key.
870. The interface server obtains the first consumption and the first consumption threshold value of the target object on the target key, which are recorded by the corresponding target key, from the storage server.
880. If the first consumption is less than the first consumption threshold, the interface server determines that the request is passed.
The specific implementation of steps 810 to 880 may be referred to the related descriptions of the foregoing embodiments, and will not be repeated here.
According to the embodiment of the application, the control server determines the plurality of keys corresponding to the target object and the consumption threshold value of the target object corresponding to each key in the plurality of keys through the performance parameters and the configuration parameters of the management system, and the interface server and the storage server perform data processing according to the plurality of keys corresponding to the target object and the consumption threshold value of the target object corresponding to each key generated by the control server, so that the hot spot data read frequently in the storage server are avoided, the hot spot data are not caused even if the number of the interface services is increased, and the parallel capacity expansion of the interface services is facilitated.
Referring to fig. 9, fig. 9 is a schematic diagram of a unit of a data processing apparatus according to an embodiment of the application. The data processing device shown in fig. 9 may be used to perform some or all of the functions of the method embodiments described above with respect to fig. 2, 5 and 8. The device may be a server, a device in a server, or a device that can be used in cooperation with a server. The logic structure of the device may include: a transceiver unit 910, a processing unit 920, and an acquisition unit 930. When the apparatus is applied to an interface server comprised by an interface service cluster:
a transceiver 910, configured to receive an acquisition request of a target object sent by a terminal device;
A processing unit 920, configured to determine a target key from a plurality of keys corresponding to a target item in response to an acquisition request, where each key in the plurality of keys is configured to record a consumption amount and a consumption amount threshold of the target item on each key, and a sum of consumption amount thresholds recorded by the plurality of keys is a preset total consumption amount threshold of the target item;
An obtaining unit 930, configured to obtain, from the storage server, a first consumption amount and a first consumption amount threshold value of the target item on the target key, where the first consumption amount and the first consumption amount threshold value are recorded in correspondence with the target key;
The processing unit 920 is further configured to determine that the request is received if the first consumption is less than the first consumption threshold.
In a possible implementation manner, the obtaining unit 930 is further configured to obtain, in response to the obtaining request, a first index of a key of the record; the processing unit 920 is further configured to determine a target key from a plurality of keys corresponding to the target object according to the first index.
In one possible implementation, if the first index does not exist, the processing unit 920 is further configured to generate a random number; determining a second index based on the random number and the number of the plurality of keys; determining a target key from a plurality of keys corresponding to the target object according to the second index; a first index is generated based on the second index and the number of the plurality of keys, and the first index is stored.
In one possible implementation manner, the acquiring unit 930 is further configured to acquire, in response to the acquiring request, a consumption amount of the target item recorded corresponding to each of a plurality of keys corresponding to the target item; the processing unit 920 is further configured to determine an access probability of each key according to the consumption amount of the target object recorded corresponding to each key; and determining a target key from the plurality of keys according to the access probability of each key.
In a possible implementation manner, the acquiring unit 930 is further configured to acquire locally cached data in response to an acquisition request; the processing unit 920 is further configured to query, from the locally cached data, a consumption amount of the target item recorded corresponding to each of the plurality of keys corresponding to the target item.
In a possible implementation manner, before receiving the request for obtaining the target object sent by the terminal device, the transceiver 910 is further configured to receive a plurality of keys corresponding to the target object sent by the control server, where the plurality of keys are determined by the control server according to a performance parameter configured by the management system, and the performance parameter includes a maximum concurrency request amount of the interface service cluster and a maximum concurrency number of the storage server per second.
When the apparatus is applied to a control server:
An obtaining unit 930, configured to obtain a performance parameter configured by the management system and a configuration parameter of the target object, where the performance parameter includes a maximum concurrency request amount of the interface service cluster and a maximum concurrency number per second of the storage server, and the configuration parameter includes a total consumption threshold of the target object;
A processing unit 920, configured to determine a plurality of keys corresponding to a target item and a consumption threshold of the target item corresponding to each key in the plurality of keys according to the performance parameter and the configuration parameter, where a sum of the consumption thresholds corresponding to the plurality of keys is a preset total consumption threshold of the target item;
And a transceiver unit 910, configured to send, to an interface server included in the interface service cluster, a plurality of keys corresponding to the target items, and send, to the storage server, a consumption threshold of the target item corresponding to each key, so that the storage server stores the consumption threshold of the target item corresponding to each key.
In a possible implementation manner, the processing unit 920 is further configured to determine a plurality of keys corresponding to the target object according to the maximum concurrency request amount of the interface service cluster and the maximum concurrency number per second of the storage server; and determining the consumption threshold of the target object corresponding to each key in the plurality of keys according to the total consumption threshold and the number of the plurality of keys.
Referring to fig. 10, fig. 10 is a simplified schematic diagram of a physical structure of a data processing apparatus according to an embodiment of the present application, where the apparatus includes a processor 1010, a memory 1020, and a communication interface 1030, and the processor 1010, the memory 1020, and the communication interface 1030 are connected by one or more communication buses.
The processor 1010 is configured to support the communication device to perform the functions corresponding to the methods of fig. 2, 5 and 8. The processor 1010 may be a central processor (central processing unit, CPU), a network processor (network processor, NP), a hardware chip, or any combination thereof.
The memory 1020 is used for storing program codes and the like. Memory 1020 may include volatile memory (volatile memory), such as random access memory (random access memory, RAM); the memory 1020 may also include a non-volatile memory (non-volatile memory), such as a read-only memory (ROM), a flash memory (flash memory), a hard disk (HARD DISK DRIVE, HDD) or a solid state disk (solid-state disk (SSD-STATE DRIVE); memory 1020 may also include a combination of the above types of memory.
The communication interface 1030 is used to transmit and receive data, information or messages, etc., and may also be described as a transceiver, transceiver circuitry, etc.
In an embodiment of the present application, when the data processing apparatus is applied to an interface server included in an interface service cluster, the processor 1010 may call a program code stored in the memory 1020 to:
control communication interface 1030 receives a request for acquiring a target item sent by a terminal device;
The processor 1010 calls a program code stored in the memory 1020 to respond to an acquisition request to determine a target key from a plurality of keys corresponding to a target item, wherein each key in the plurality of keys is used for correspondingly recording the consumption and the consumption threshold value of the target item on each key, and the sum of the consumption threshold values corresponding to the plurality of keys is a preset total consumption threshold value of the target item;
the processor 1010 calls the program code stored in the memory 1020 to acquire, from the storage server, the first consumption and the first consumption threshold of the target item on the target key, the first consumption and the first consumption threshold being recorded corresponding to the target key;
The processor 1010 calls program code stored in the memory 1020 to determine a pass acquire request if the first consumption is less than a first consumption threshold.
In one possible implementation, the processor 1010 invokes a first index of keys of the program code stored in the memory 1020 to retrieve a record in response to a retrieval request; the processor 1010 invokes program code stored in the memory 1020 to determine a target key from a plurality of keys corresponding to the target item according to the first index.
In one possible implementation, if the first index is not present, the processor 1010 calls the program code stored in the memory 1020 to generate a random number; determining a second index based on the random number and the number of the plurality of keys; determining a target key from a plurality of keys corresponding to the target object according to the second index; a first index is generated based on the second index and the number of the plurality of keys, and the first index is stored.
In one possible implementation, the processor 1010 invokes program code stored in the memory 1020 to obtain, in response to the obtaining request, a consumption amount of the target item recorded for each of a plurality of keys corresponding to the target item; the processor 1010 calls the program code stored in the memory 1020 to determine the access probability of each key according to the consumption of the target item recorded for each key; and determining a target key from the plurality of keys according to the access probability of each key.
In one possible implementation, the processor 1010 invokes program code stored in the memory 1020 to retrieve locally cached data in response to a retrieval request; the processor 1010 invokes program code stored in the memory 1020 to query the locally cached data for the consumption of the target item recorded for each of a plurality of keys corresponding to the target item.
In one possible implementation manner, before receiving the request for obtaining the target object sent by the terminal device, the control communication interface 1030 receives a plurality of keys corresponding to the target object sent by the control server, where the plurality of keys are determined by the control server according to a performance parameter configured by the management system, and the performance parameter includes a maximum concurrency request amount of the interface service cluster and a maximum concurrency number of the storage server per second.
In an embodiment of the present application, when the apparatus is applied to a control server, the processor 1010 may call the program code stored in the memory 1020 to:
The processor 1010 invokes program code stored in the memory 1020 to obtain performance parameters configured by the management system, wherein the performance parameters include a maximum concurrency request amount of the interface service cluster and a maximum concurrency number per second of the storage server, and configuration parameters of the target item, wherein the configuration parameters include a total consumption threshold of the target item;
The processor 1010 calls the program code stored in the memory 1020 to determine a plurality of keys corresponding to the target item and a consumption threshold of the target item corresponding to each of the plurality of keys according to the performance parameter and the configuration parameter, wherein the sum of the consumption thresholds corresponding to the plurality of keys is a preset total consumption threshold of the target item;
The control communication interface 1030 transmits the plurality of keys corresponding to the target item to the interface server included in the interface service cluster, and transmits the consumption threshold of the target item corresponding to each key to the storage server, so that the storage server stores the consumption threshold of the target item corresponding to each key.
In one possible implementation, the processor 1010 invokes program code stored in the memory 1020 to determine a plurality of keys corresponding to the target item based on the maximum concurrency request of the interface service cluster and the maximum concurrency per second of the storage server; and determining the consumption threshold of the target object corresponding to each key in the plurality of keys according to the total consumption threshold and the number of the plurality of keys.
Embodiments of the present application also provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The computer instructions are read from the computer-readable storage medium by a processor of a computer device, and executed by the processor, cause the computer device to perform the functions corresponding to the methods of fig. 2, 5, and 8.
In the foregoing embodiments, the descriptions of the embodiments are focused on, and for those portions of one embodiment that are not described in detail, reference may be made to the related descriptions of other embodiments.
The steps in the method of the embodiment of the invention can be sequentially adjusted, combined and deleted according to actual needs.
The units in the processing device of the embodiment of the invention can be combined, divided and deleted according to actual needs.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions in accordance with embodiments of the present application are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital subscriber line), or wireless (e.g., infrared, wireless, microwave, etc.). Computer readable storage media can be any available media that can be accessed by a computer or data storage devices, such as servers, data centers, etc., that contain an integration of one or more available media. Usable media may be magnetic media (e.g., floppy disks, storage disks, magnetic tape), optical media (e.g., DVD), or semiconductor media (e.g., solid state storage disk Solid STATE DISK (SSD)), etc.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the application.
Claims (11)
1. A data processing method, applied to an interface server included in an interface service cluster, the method comprising:
Receiving an acquisition request of a target object sent by terminal equipment;
Determining a target key from a plurality of keys corresponding to the target object in response to the acquisition request, wherein each key in the plurality of keys is used for correspondingly recording the consumption and the consumption threshold value of the target object on each key, and the sum of the consumption threshold values recorded by the plurality of keys is a preset total consumption threshold value of the target object; the plurality of keys corresponding to the target object are sent by the control server, the plurality of keys corresponding to the target object are determined by the control server according to the performance parameters configured by the management system and the configuration parameters, the performance parameters configured by the management system comprise the maximum concurrency request quantity of the interface service cluster and the maximum concurrency number of the storage server per second, and the configuration parameters comprise the total consumption threshold value of the target object;
Acquiring a first consumption amount and a first consumption amount threshold value of the target object on the target key, which are recorded corresponding to the target key, from a storage server;
and if the first consumption is smaller than the first consumption threshold, determining that the acquisition request is passed.
2. The method of claim 1, wherein determining the target key from the plurality of keys corresponding to the target item in response to the acquisition request comprises:
acquiring a first index of a recorded key in response to the acquisition request;
And determining a target key from a plurality of keys corresponding to the target object according to the first index.
3. The method according to claim 2, wherein the method further comprises:
If the first index does not exist, generating a random number;
determining a second index based on the random number and the number of the plurality of keys;
Determining a target key from a plurality of keys corresponding to the target object according to the second index;
and generating the first index according to the second index and the number of the keys, and storing the first index.
4. The method of claim 1, wherein determining the target key from the plurality of keys corresponding to the target item in response to the acquisition request comprises:
responding to the acquisition request to acquire the consumption of the target object recorded corresponding to each key in a plurality of keys corresponding to the target object;
Determining the access probability of each key according to the consumption of the target object recorded by the corresponding key;
and determining a target key from the plurality of keys according to the access probability of each key.
5. The method of claim 4, wherein the obtaining, in response to the obtaining request, the consumption of the target item recorded for each of a plurality of keys corresponding to the target item comprises:
responding to the acquisition request to acquire locally cached data;
and inquiring the consumption of the target object recorded corresponding to each key in the plurality of keys corresponding to the target object from the locally cached data.
6. The method according to claim 1, wherein before the receiving the request for acquiring the target article sent by the terminal device, the method further comprises:
And receiving a plurality of keys corresponding to the target object sent by a control server, wherein the keys are determined by the control server according to performance parameters configured by a management system, and the performance parameters comprise the maximum concurrency request quantity of the interface service cluster and the maximum concurrency number per second of the storage server.
7. A data processing method, applied to a control server, the method comprising:
Acquiring performance parameters configured by a management system and configuration parameters of a target object, wherein the performance parameters comprise the maximum concurrency request quantity of an interface service cluster and the maximum concurrency quantity per second of a storage server, and the configuration parameters comprise the total consumption threshold of the target object;
Determining a plurality of keys corresponding to the target object and consumption thresholds of the target object corresponding to each key in the plurality of keys according to the performance parameters and the configuration parameters, wherein the sum of the consumption thresholds corresponding to the plurality of keys is a preset total consumption threshold of the target object;
And sending a plurality of keys corresponding to the target items to an interface server included in the interface service cluster, and sending a consumption threshold of the target items corresponding to each key to the storage server, so that the storage server stores the consumption threshold of the target items corresponding to each key.
8. The method of claim 7, wherein the determining the plurality of keys corresponding to the target item and the consumption threshold of the target item corresponding to each of the plurality of keys based on the performance parameter and the configuration parameter comprises:
determining a plurality of keys corresponding to the target object according to the maximum concurrency request quantity of the interface service cluster and the maximum concurrency number per second of the storage server;
And determining the consumption threshold of the target object corresponding to each key in the plurality of keys according to the total consumption threshold and the number of the plurality of keys.
9. A data processing apparatus, comprising:
a processor and a memory, the processor and the memory being interconnected, wherein the memory is adapted to store a computer program, the computer program comprising program instructions, the processor being configured to invoke the program instructions to perform the data processing method according to any of claims 1 to 6 or the data processing method according to claim 7 or 8.
10. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program comprising program instructions which, when executed by a processor, cause the processor to perform the data processing method according to any one of claims 1 to 6 or the data processing method according to claim 7 or 8.
11. A computer program product comprising computer program/instructions which, when executed by a processor, implement a data processing method as claimed in any one of claims 1 to 6 or a data processing method as claimed in claim 7 or 8.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010703197.0A CN111782732B (en) | 2020-07-21 | 2020-07-21 | Data processing method, device and computer readable storage medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010703197.0A CN111782732B (en) | 2020-07-21 | 2020-07-21 | Data processing method, device and computer readable storage medium |
Publications (2)
Publication Number | Publication Date |
---|---|
CN111782732A CN111782732A (en) | 2020-10-16 |
CN111782732B true CN111782732B (en) | 2024-09-10 |
Family
ID=72763647
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010703197.0A Active CN111782732B (en) | 2020-07-21 | 2020-07-21 | Data processing method, device and computer readable storage medium |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111782732B (en) |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104182508A (en) * | 2014-08-19 | 2014-12-03 | 华为技术有限公司 | Data processing method and data processing device |
CN108334505A (en) * | 2017-01-19 | 2018-07-27 | 阿里巴巴集团控股有限公司 | A kind of data processing method, device, server and inventory system |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2246785B1 (en) * | 2009-04-16 | 2020-01-08 | 2236008 Ontario Inc. | System for managing and arbitrating for cost-constrained resources |
US20170185454A1 (en) * | 2015-12-25 | 2017-06-29 | Le Holdings (Beijing) Co., Ltd. | Method and Electronic Device for Determining Resource Consumption of Task |
CN108960960A (en) * | 2018-06-01 | 2018-12-07 | 中国平安人寿保险股份有限公司 | A kind of method and server handling high concurrent data |
CN110489421B (en) * | 2019-08-22 | 2023-11-14 | 腾讯科技(深圳)有限公司 | Data storage method, apparatus, computer readable storage medium and computer device |
CN110855741B (en) * | 2019-09-29 | 2022-01-11 | 北京奇艺世纪科技有限公司 | Service self-adaptive access method and device, storage medium and electronic device |
-
2020
- 2020-07-21 CN CN202010703197.0A patent/CN111782732B/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104182508A (en) * | 2014-08-19 | 2014-12-03 | 华为技术有限公司 | Data processing method and data processing device |
CN108334505A (en) * | 2017-01-19 | 2018-07-27 | 阿里巴巴集团控股有限公司 | A kind of data processing method, device, server and inventory system |
Also Published As
Publication number | Publication date |
---|---|
CN111782732A (en) | 2020-10-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110032567B (en) | Report query method, device, server and storage medium | |
US11394625B2 (en) | Service level agreement based storage access | |
US10235047B2 (en) | Memory management method, apparatus, and system | |
JP2020173778A (en) | Method, apparatus, electronic facility, computer readable medium, and computer program for allocating resource | |
US20110107053A1 (en) | Allocating Storage Memory Based on Future Use Estimates | |
CN109213604B (en) | Data source management method and device | |
US20170153909A1 (en) | Methods and Devices for Acquiring Data Using Virtual Machine and Host Machine | |
CN110909978A (en) | Resource processing method, device, server and computer readable storage medium | |
CN112965879A (en) | Data processing method and device, electronic equipment and readable storage medium | |
US20240348684A1 (en) | Cloud desktop data migration method, service node, management node, server, electronic device, and computer-readable storage medium | |
CN111562884A (en) | Data storage method and device and electronic equipment | |
CN111192070B (en) | Storage service metering method and system, computer system and readable storage medium | |
CN111782732B (en) | Data processing method, device and computer readable storage medium | |
CN108205559A (en) | A kind of data managing method and its equipment | |
CN113760940B (en) | Quota management method, device, equipment and medium applied to distributed system | |
CN110677463B (en) | Parallel data transmission method, device, medium and electronic equipment | |
CN113672652A (en) | Data access method, device, equipment and storage medium | |
CN114513469A (en) | Traffic shaping method and device for distributed system and storage medium | |
CN112468590A (en) | Storage resource mounting method and device | |
CN111161019B (en) | Data processing system and method | |
CN109672900B (en) | Method and device for generating hot content list | |
US20220043802A1 (en) | Method and system for performing computations in a distributed system | |
JP2024053468A (en) | Control method, control program, and information processing device | |
CN117891779A (en) | Access method and device of network file system, storage medium and electronic equipment | |
CN117573756A (en) | Service counting method and device, storage medium and electronic equipment |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |