CN113032410A - Data processing method and device, electronic equipment and computer storage medium - Google Patents
Data processing method and device, electronic equipment and computer storage medium Download PDFInfo
- Publication number
- CN113032410A CN113032410A CN201911355053.4A CN201911355053A CN113032410A CN 113032410 A CN113032410 A CN 113032410A CN 201911355053 A CN201911355053 A CN 201911355053A CN 113032410 A CN113032410 A CN 113032410A
- Authority
- CN
- China
- Prior art keywords
- data
- distributed consistency
- consistency system
- tenant
- resource consumption
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 25
- 238000012545 processing Methods 0.000 claims abstract description 31
- 238000000034 method Methods 0.000 claims abstract description 28
- 238000004590 computer program Methods 0.000 claims description 10
- 238000010586 diagram Methods 0.000 description 13
- 238000004891 communication Methods 0.000 description 7
- 239000000872 buffer Substances 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000013459 approach Methods 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000011084 recovery Methods 0.000 description 3
- 241000508269 Psidium Species 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 239000011521 glass Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004984 smart glass Substances 0.000 description 2
- 230000026676 system process Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000007429 general method Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Computer And Data Communications (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention provides a data processing method and device, electronic equipment and a computer storage medium, and relates to the technical field of data processing. Wherein the method comprises the following steps: when the distributed consistency system is determined to be in a busy state based on the number of data requests to be processed in the distributed consistency system, resource consumption data of the data requests to be processed in the distributed consistency system are estimated; and when determining that the resource consumption data exceeds the global resource consumption permission data set by the distributed consistency system, performing current limiting operation on a data request sent by a client connected with the distributed consistency system. The embodiment of the invention can simply, conveniently and accurately carry out current limiting on the request of the distributed consistency system, thereby effectively ensuring the service quality of the distributed consistency system.
Description
Technical Field
The embodiment of the application relates to the technical field of data processing, in particular to a data processing method and device, electronic equipment and a computer storage medium.
Background
Currently, QOS (Quality of Service) design is basically implemented by limiting current, i.e., limiting speed, which is simply understood as a funnel with openings at two ends, a large opening at an input side and a small opening at an output side. The output side always outputs at the maximum predictable rate no matter how large the flow rate on the input side is. As shown in fig. 1A, no matter how much the client is stressed, the server can always meet the requirement of stable service quality on the basis of a stably controllable request amount.
Currently, there are two main ways to limit the flow of requests, one is precision rate control. The accurate rate control is generally used for pre-configuring the rate of the traffic to be controlled on the premise of better pre-estimating the system capacity and the service traffic, and the general method is to use a leak Bucket algorithm or a TokenBucket algorithm. Such algorithms are widely used for various types of system speed limits, such as the basic library guava, the web service Nginx, and various language toolkits. However, this approach requires more adaptation work, e.g., different network environments, different server models, possibly different processing rates, and different rate configurations. The other is pressure feedback control. This control is more useful in scenarios where it is desirable to maximize the capacity of the back-end server, and when the server has capacity to handle requests, the requests are accepted, otherwise they are rejected. Due to different deployment environments of the same server, the processing speeds of the servers may have great differences, and a common implementation manner of the method is to allocate a buffer area for the server, where a request first enters the buffer area to be cached, and then a special processing module of the server obtains the request from the buffer area to process the request. When the processing capacity of the processing module reaches an upper limit, the buffer will be filled with the request, thereby starting to reject the request. However, this approach requires additional buffers, which may be complex to implement. Therefore, how to simply and accurately limit the current of the system becomes a technical problem to be solved urgently.
Disclosure of Invention
In view of this, embodiments of the present invention provide a data processing method, an apparatus, an electronic device, and a computer storage medium, so as to solve the technical problem in the prior art how to simply and accurately limit a current to a system request.
According to a first aspect of embodiments of the present invention, a data processing method is provided. The method comprises the following steps: when the distributed consistency system is determined to be in a busy state based on the number of data requests to be processed in the distributed consistency system, resource consumption data of the data requests to be processed in the distributed consistency system are estimated; and when determining that the resource consumption data exceeds the global resource consumption permission data set by the distributed consistency system, performing current limiting operation on a data request sent by a client connected with the distributed consistency system.
According to a second aspect of embodiments of the present invention, there is provided a data processing apparatus. The device comprises: the estimation module is used for estimating resource consumption data of the data requests to be processed in the distributed consistency system when the distributed consistency system is determined to be in a busy state based on the number of the data requests to be processed in the distributed consistency system; and the first current limiting module is used for executing current limiting operation on a data request sent by a client connected with the distributed consistency system when the resource consumption data is determined to exceed the global resource consumption allowable data set by the distributed consistency system.
According to a third aspect of embodiments of the present application, there is provided an electronic apparatus, including: one or more processors; a computer readable medium configured to store one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the data processing method as described in the first aspect of the embodiments above.
According to a fourth aspect of embodiments of the present application, there is provided a computer-readable medium, on which a computer program is stored, which when executed by a processor, implements the data processing method as described in the first aspect of the embodiments above.
According to the technical scheme provided by the embodiment of the application, when the distributed consistency system is determined to be in a busy state based on the number of data requests to be processed in the distributed consistency system, resource consumption data of the data requests to be processed in the distributed consistency system are estimated; when the resource consumption data exceed the global resource consumption allowable data set by the distributed consistency system, performing current limiting operation on data requests sent by a client connected with the distributed consistency system, and compared with the existing other modes, determining whether the distributed consistency system is in a busy state according to the number of data requests to be processed in the distributed consistency system, and estimating the resource consumption data of the data requests to be processed in the distributed consistency system when determining that the distributed consistency system is in the busy state based on the number of the data requests to be processed in the distributed consistency system; when the resource consumption data is determined to exceed the global resource consumption allowable data set by the distributed consistency system, the current limiting operation is performed on the data request sent by the client connected with the distributed consistency system, the current limitation can be simply, conveniently and accurately performed on the request of the distributed consistency system, and therefore the service quality of the distributed consistency system is effectively guaranteed.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the embodiments of the present invention, and it is also possible for a person skilled in the art to obtain other drawings based on the drawings.
FIG. 1A is a schematic diagram of a prior art system for limiting current;
fig. 1B is a schematic diagram of an application scenario of a data processing scheme provided in an embodiment of the present application;
FIG. 1C is a flowchart illustrating steps of a data processing method according to an embodiment of the present disclosure;
FIG. 1D is a diagram illustrating a pending data request queue according to an embodiment of the present disclosure;
FIG. 2 is a flowchart illustrating steps of a data processing method according to a second embodiment of the present application;
FIG. 3 is a schematic structural diagram of a data processing apparatus according to a third embodiment of the present application;
FIG. 4 is a schematic structural diagram of a data processing apparatus according to a fourth embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device in a fifth embodiment of the present application;
fig. 6 is a hardware structure of an electronic device according to a sixth embodiment of the present application.
Detailed Description
In order to make those skilled in the art better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments obtained by a person skilled in the art based on the embodiments of the present invention shall fall within the scope of the protection of the embodiments of the present invention.
The following further describes specific implementation of the embodiments of the present invention with reference to the drawings.
The mythology service is a bottom layer basic module of a flying distributed consistency system, is widely applied to the Alice cloud as a de facto standard of the consistency service in the Alice cloud, supports numerous weight-level cloud products such as ECS, OSS, MaxCommute, TableStore, VPC, Alimail and the like, and also supports core services such as nailing unitization, ant artificial intelligence service platforms and the like in a group. The mythology service provides a typical series of distributed consistency services such as distributed locks, service discovery, metadata storage, etc. As shown in fig. 1B, the connection between the client and the server of the mythology service is session-based, and the client needs to create a session with the server and maintain the lifetime of the session between the client and the server through periodic heartbeat. The specific process is that the client firstly creates a long connection with the server of the mythology service, then creates a session on the long connection, the session creation is completed, and then a normal request can be sent, and at the same time, a heartbeat renewal session is sent periodically (the session is kept valid), but in the scene of session disconnection, the long connection needs to be created again, and the session is tried to be continued. The server may be a distributed consistency system, which is a distributed consistency system implemented based on a consistency protocol (e.g., Paxos, Raft, etc.). When the coherency protocol is Raft, the system may include a leader node and a plurality of follower nodes. In order to effectively ensure the service quality of the distributed consistency system in a normal scene, the flow limitation needs to be performed on the request sent by the client to the distributed consistency system. Currently, there are two main ways to limit the flow of requests, one is precision rate control and the other is pressure feedback control. However, both of these approaches to limiting requests are too complex and do not provide for fine or precise limiting of requests. Therefore, the data processing method provided by the embodiment of the application can simply, conveniently and accurately limit the current of the request of the distributed consistency system, so that the service quality of the distributed consistency system is effectively guaranteed. The method can be applied to ZooKeeper of the Apache community, web services nginx, and toolkits of various languages, such as ratelimit, a Guava current limiting tool. The specific implementation manner of the data processing method provided by the embodiment of the application is as follows:
referring to fig. 1C, a flowchart illustrating steps of a data processing method according to a first embodiment of the present application is shown.
Specifically, the data processing method of the present embodiment includes the following steps:
in step S101, when it is determined that the distributed consistency system is in a busy state based on the number of data requests to be processed in the distributed consistency system, resource consumption data of the data requests to be processed in the distributed consistency system is estimated.
In this embodiment, the data request to be processed may be a service data request sent by a client to a distributed consistency system. When the number of data requests to be processed in the distributed consistency system reaches a certain number, it may be determined that the distributed consistency system is in a busy state. Specifically, if it is determined that the number of the data requests to be processed is greater than or equal to a preset number threshold, it is determined that the distributed consistency system is in a busy state. The preset number threshold may be set by a person skilled in the art according to actual needs, and the embodiment of the present application is not limited in any way. More specifically, as shown in fig. 1D, it can be considered that a data request sent by a tenant a or a tenant B to a distributed coherency system through a client may form a queue of data requests to be processed after entering the system, only the length of the queue needs to be concerned, which means that only one global value needs to be maintained, one is added when the data request enters the queue, and one is subtracted when the distributed coherency system sends a response to the data request to the client, which may reflect the current operating state of the distributed coherency system, if the global value is too large, which means that the current system has an excessive number of data requests to be processed, the data request sent by the client to the distributed coherency system needs to be limited, which is a characteristic that the existing precise rate control method does not have, and the simple precise rate control only forcibly sets the inflow rate of the data request according to the configured rate, the busyness of the current distributed coherency system is not a concern. Furthermore, the resource consumption data may be understood as resource consumption data estimated when the distributed coherency system processes the pending data request. Specifically, the resource consumption data may be time resource data, calculation resource data, or storage resource data consumed by the distributed consistency system when processing the data request to be processed. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.
In some optional embodiments, when resource consumption data of the to-be-processed data request in the distributed consistency system is estimated, determining a request type to which the to-be-processed data request belongs; determining resource consumption weight data corresponding to the data request to be processed based on the request type to which the data request to be processed belongs; and estimating resource consumption data of the data request to be processed in the distributed consistency system based on the resource consumption weight data corresponding to the data request to be processed. Therefore, the resource consumption data of the data request to be processed in the distributed consistency system can be accurately estimated by determining the request type to which the data request to be processed belongs and the resource consumption weight data corresponding to the data request to be processed. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.
In one specific example, the request type to which the pending data request belongs may include at least one of: read request, list request, create request, update request, delete request, heartbeat, session create request, session continue request. Wherein the list request can be understood as a request for requesting a list or a manifest. Before resource consumption data of the data request to be processed in the distributed consistency system is estimated, request types and resource consumption weight data corresponding to each request type can be configured in advance. For example, when the request type is a read request, the resource consumption weight data corresponding to the read request may be 2, when the request type is a list request, the resource consumption weight data corresponding to the list request may be 4, when the request type is a create request, the resource consumption weight data corresponding to the create request may be 3, when the request type is an update request, the resource consumption weight data corresponding to the update request may be 3, when the request type is a delete request, the resource consumption weight data corresponding to the delete request may be 3, when the request type is a heartbeat, the resource consumption weight data corresponding to the heartbeat may be 1, and when the request type is a session creation request, the resource consumption weight data corresponding to the session creation request may be 3, when the request type is a session renewal request, the resource consumption weight data corresponding to the session renewal request may be 1. The resource consumption weight data can be understood as data for measuring resources consumed by the distributed consistency system when the distributed consistency system processes the request of the request type, and the resource consumption weight data can be set according to different characteristics of each system. In addition, the request types may not be limited to the above request types, and may be extended to other possible request types. When determining the request type to which the data request to be processed belongs, the request type to which the data request to be processed belongs may be determined according to the content of a request header in the data request to be processed or a type field in a request carrier. When determining the resource consumption weight data corresponding to the data request to be processed, the corresponding relationship between the pre-configured request type and the resource consumption weight data can be searched according to the request type to which the data request to be processed belongs, so as to determine the resource consumption weight data corresponding to the data request to be processed. When the resource consumption data of the data request to be processed in the distributed consistency system is estimated, the resource consumption weight data corresponding to the data request to be processed can be accumulated to obtain the resource consumption data of the data request to be processed in the distributed consistency system. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.
In a specific example, the number of pending data requests in the distributed coherency system cannot completely reflect the busy level of the current system, because different data requests cause different resource consumptions of the system, for example, the resource consumption caused by reading one node and the system resource consumption caused by reading multiple nodes are different, and the resource consumption caused by reading requests and writing requests are different, so that a resource consumption weight is configured for each request type, a virtual processing counting mode for the data requests is formed, so that a separate processing count does not need to be set for each request, a global resource consumption data can be set for the distributed coherency system, and the system is considered to be stably controllable as long as the resource consumption of the pending data requests in the current system is kept within a reasonable range, once overridden, a throttling operation needs to be performed on data requests sent by the client to the system. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.
In step S102, when it is determined that the resource consumption data exceeds the global resource consumption permission data set by the distributed consistency system, a current limiting operation is performed on a data request sent by a client connected to the distributed consistency system.
In this embodiment, if the resource consumption data of the data request to be processed in the distributed consistency system is less than or equal to the global resource consumption permission data set by the distributed consistency system, it is indicated that the distributed consistency system is stable and controllable, and it is not necessary to perform a current limiting operation on the data request sent by the client to the distributed consistency system. And if the resource consumption data of the data request to be processed in the distributed consistency system is larger than the global resource consumption allowable data set by the distributed consistency system, performing current limiting operation on the data request sent by the client to the distributed consistency system. Specifically, by closing the session between the client and the distributed consistency system, or stopping monitoring the long connection for creating the session, it is possible to implement the current limiting operation on the data request sent by the client to the distributed consistency system. The global resource consumption permission data may be set by a person skilled in the art according to actual needs, and this is not limited in this embodiment of the present application. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.
According to the data processing method provided by the embodiment of the application, when the distributed consistency system is determined to be in a busy state based on the number of data requests to be processed in the distributed consistency system, resource consumption data of the data requests to be processed in the distributed consistency system are estimated; when the resource consumption data exceed the global resource consumption allowable data set by the distributed consistency system, performing current limiting operation on data requests sent by a client connected with the distributed consistency system, and compared with the existing other modes, determining whether the distributed consistency system is in a busy state according to the number of data requests to be processed in the distributed consistency system, and estimating the resource consumption data of the data requests to be processed in the distributed consistency system when determining that the distributed consistency system is in the busy state based on the number of the data requests to be processed in the distributed consistency system; when the resource consumption data is determined to exceed the global resource consumption allowable data set by the distributed consistency system, the current limiting operation is performed on the data request sent by the client connected with the distributed consistency system, the current limitation can be simply, conveniently and accurately performed on the request of the distributed consistency system, and therefore the service quality of the distributed consistency system is effectively guaranteed.
The data processing method of the present embodiment may be performed by any suitable device having data processing capabilities, including but not limited to: cameras, terminals, mobile terminals, PCs, servers, in-vehicle devices, entertainment devices, advertising devices, Personal Digital Assistants (PDAs), tablet computers, notebook computers, handheld game consoles, smart glasses, smart watches, wearable devices, virtual display devices or display enhancement devices (such as Google Glass, Oculus rise, Hololens, Gear VR), and the like.
Referring to fig. 2, a flowchart illustrating steps of a data processing method according to a second embodiment of the present application is shown.
Specifically, the data processing method of the present embodiment includes the following steps:
in step S201, a session creation request with tenant information sent by the client is received.
In this embodiment, a tenant field may be added to a request header or a request bearer of the session creation request, and tenant information is carried by the tenant field. The tenant information may include at least one of: the tenant name, the tenant identification information and the tenant validity information. The tenant information is transmitted to the distributed consistency system through the session creation request, the independent control of the service is mainly realized, and different priorities can be provided for services with different importance according to the degradation principle, so that the important service request can be guaranteed to be processed in time under the abnormal condition. In addition, the distributed consistency system can perform richer current limiting work according to different tenants. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.
In step S202, based on the session creation request, a session corresponding to the tenant information is created, so as to receive, through the session, the pending data request sent by the tenant through the client.
In this embodiment, the session may be understood as a mechanism in which the client creates a lease with the distributed consistency system and needs to periodically send a heartbeat to the distributed consistency system to maintain the lease for a long time. The lease is understood to be a time period occupied by a certain resource in the computer field. If the lease is not extended before the deadline, the resource will be automatically released from its occupation. In addition, a plurality of users often use the system, different users refer to different tenants, and in the system, the data or requests of different tenants are often isolated and do not affect each other. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.
In a specific example, since the system is a session-based distributed consistency system, the tenant and the session can be bound only by adding tenant information to the session creation request, without requiring the tenant information to be transmitted to each data request. Once the session is established, all data requests in the session process are bound on one tenant, and the design simplifies the complexity of the system and reduces the data volume transmitted by the network. Of course, when the tenant information is transferred, each data request may also carry the tenant information, which increases the burden of network transmission, but is applicable to a scenario in which information of multiple tenants is stored in a single session. Such a scenario may occur in a front-end of a distributed consistency system, where the front-end proxies service requests of different tenants and completes service requests of different tenants in one session. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.
In step S203, when it is determined that the distributed consistency system is in a busy state based on the number of the to-be-processed data requests in the distributed consistency system, resource consumption data of the to-be-processed data requests in the distributed consistency system is estimated.
Since step S203 is similar to step S101, it is not described herein again.
In step S204, when it is determined that the resource consumption data exceeds the global resource consumption permission data set by the distributed consistency system, a current limiting operation is performed on the data request sent by the client.
Since step S204 is similar to step S102, it is not repeated herein.
In some optional embodiments, the method further comprises: determining the type of a tenant to which the data request to be processed belongs; and based on the type of the tenant, performing a current limiting operation on a data request sent by the tenant to the distributed consistency system through the client. Therefore, the current limiting operation is performed on the data request sent by the tenant to the distributed consistency system through the client according to the type of the tenant, and the current limiting operation can be performed on the data request sent by the client to the distributed consistency system more finely. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.
In a specific example, when determining the type of the tenant to which the data request to be processed belongs, the type of the tenant to which the data request to be processed belongs may be determined through the session in which the data request to be processed belongs. Since the session and the tenant information have a corresponding relationship, as long as the session where the data request to be processed is located is determined, the type of the tenant to which the data request to be processed belongs can be determined. Wherein the type of the tenant comprises a normal type and/or an unrestricted type. The tenants are divided into two types, because tenants of some core applications need to have the right of preferentially processing service requests, and in some extreme cases, the tenants need to be preferentially recovered, for example, a front-end machine of a distributed consistency system needs to be preferentially recovered, which can help a back-end system to bear pressure in advance, and in extreme cases, service requests of non-limited type tenants can greatly occupy service request space of normal type tenants, so that the highest-priority processing is obtained. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.
In a specific example, when the current limiting operation is performed on the data request sent by the tenant to the distributed consistency system through the client based on the type of the tenant, when the type of the tenant is a normal type, if the resource consumption data of the data request sent by the tenant to the distributed consistency system through the client in the distributed consistency system exceeds the resource consumption permission data set by the distributed consistency system for the tenant of the normal type, the current limiting operation is performed on the data request sent by the tenant to the distributed consistency system through the client, or when the data request sent by the tenant through the client is received by the distributed consistency system, if the resource consumption data of the data request to be processed in the distributed consistency system exceeds the global resource consumption permission data set by the distributed consistency system If so, performing a current limiting operation on a data request sent by the tenant to the distributed consistency system through the client; when the type of the tenant is an unrestricted type, if the resource consumption data of the data request sent by the tenant to the distributed consistency system through the client in the distributed consistency system exceeds the resource consumption permission data set by the distributed consistency system for the tenant of the unrestricted type, performing a current limiting operation on the data request sent by the tenant to the distributed consistency system through the client. The resource consumption permission data set by the distributed consistency system for the normal type of tenant and the resource consumption permission data set by the distributed consistency system for the unrestricted type of tenant may be set by those skilled in the art according to actual needs, which is not limited in this embodiment of the present application. In addition, the type of the tenant is not limited to a normal type and/or an unrestricted type, the type of the tenant can be expanded to more types, and meanwhile, the distributed consistency system sets different resource consumption permission data for different types of tenants. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.
In the session disconnection scene, the session cannot be recovered for a long time, basically because of a large number of invalid connections and invalid reconnection requests. When the client is reconnected with the distributed consistency system, a timeout mechanism exists, if the client is reconnected with the distributed consistency system, other service machines can be connected, but the original machine cannot immediately sense the session disconnection event of the client, and can sense the event only after the service data request in the TCP connection is processed, so that a large number of invalid reconnection requests are caused, the subsequent valid reconnection requests cannot be processed in time and become invalid reconnection requests, and the session cannot be recovered between the client and the distributed consistency system for a long time. However, after the data processing scheme provided by the embodiment of the application is adopted, the data requests sent by the client to the distributed consistency system can be simply and finely limited, so that the distributed consistency system is ensured to process effective data requests to the maximum extent, and then disconnected sessions can be quickly recovered. For example, a session of a front-end machine with a distributed consistency system may be quickly restored. Specifically, by setting the tenant of the front-end to be of an unrestricted type, the session of the front-end and the distributed consistency system is preferentially restored. Then, a specific recovery time can be calculated. Assuming that the speed of the distributed consistency system processing per second is V, the total number of sessions in the distributed consistency system is N, and the theoretical recovery time of a disconnected session is:
meaning that if the total number of sessions in the distributed coherency system is 10 ten thousand sessions and the processing speed per second is 2 ten thousand sessions, a recovery of a disconnected session can be done in 5 seconds theoretically. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.
In practical application, the data processing scheme provided by the embodiment of the application can be applied to all the online mythology clusters. The test is carried out on ten thousand clusters of ODPS lines, and the service quality is effectively ensured. It should be understood that the above description is only exemplary, and the embodiments of the present application are not limited in this respect.
On the basis of the first embodiment, a session creation request carrying tenant information and sent by the client is received, a session corresponding to the tenant information is created based on the session creation request, and the to-be-processed data request sent by the tenant through the client is received through the session. Once the session is established, all requests in the session process are bound with one tenant, and the design simplifies the complexity of the distributed consistency system and simultaneously reduces the data volume transmitted by the network in the session process.
The data processing method of the present embodiment may be performed by any suitable device having data processing capabilities, including but not limited to: cameras, terminals, mobile terminals, PCs, servers, in-vehicle devices, entertainment devices, advertising devices, Personal Digital Assistants (PDAs), tablet computers, notebook computers, handheld game consoles, smart glasses, smart watches, wearable devices, virtual display devices or display enhancement devices (such as Google Glass, Oculus rise, Hololens, Gear VR), and the like.
Referring to fig. 3, a schematic structural diagram of a data processing apparatus according to a third embodiment of the present application is shown.
The data processing apparatus of the present embodiment includes: the estimation module 301 is configured to estimate resource consumption data of the to-be-processed data request in the distributed consistency system when the distributed consistency system is determined to be in a busy state based on the number of the to-be-processed data requests in the distributed consistency system; a first current limiting module 302, configured to perform a current limiting operation on a data request sent by a client connected to the distributed consistency system when it is determined that the resource consumption data exceeds global resource consumption permission data set by the distributed consistency system.
The data processing apparatus of this embodiment is configured to implement the corresponding data processing method in the foregoing multiple method embodiments, and has the beneficial effects of the corresponding method embodiment, which are not described herein again.
Referring to fig. 4, a schematic structural diagram of a data processing apparatus in the fourth embodiment of the present application is shown.
The data processing apparatus of the present embodiment includes: the estimation module 404 is configured to estimate resource consumption data of the to-be-processed data request in the distributed consistency system when it is determined that the distributed consistency system is in a busy state based on the number of the to-be-processed data requests in the distributed consistency system; a first current limiting module 405, configured to perform a current limiting operation on a data request sent by a client connected to the distributed consistency system when it is determined that the resource consumption data exceeds the global resource consumption allowance data set by the distributed consistency system.
Optionally, before the estimating module 404, the apparatus further includes: a first determining module 401, configured to determine that the distributed consistency system is in a busy state if it is determined that the number of the data requests to be processed is greater than or equal to a preset number threshold.
Optionally, before the estimating module 404, the apparatus further includes: a receiving module 402, configured to receive a session creation request with tenant information sent by the client; a creating module 403, configured to create a session corresponding to the tenant information based on the session creating request, so as to receive, through the session, the pending data request sent by the tenant through the client.
Optionally, the estimation module 404 is specifically configured to: determining a request type to which the data request to be processed belongs; determining resource consumption weight data corresponding to the data request to be processed based on the request type to which the data request to be processed belongs; and estimating resource consumption data of the data request to be processed in the distributed consistency system based on the resource consumption weight data corresponding to the data request to be processed.
Optionally, the apparatus further comprises: a second determining module 406, configured to determine a type of a tenant to which the data request to be processed belongs; and a second current limiting module 407, configured to perform a current limiting operation on a data request sent by the tenant to the distributed consistency system through the client based on the type of the tenant.
Optionally, the second current limiting module 407 is specifically configured to: when the type of the tenant is a normal type, if the resource consumption data in the distributed consistency system of the data request sent by the tenant to the distributed consistency system through the client exceeds the resource consumption allowable data set by the distributed consistency system for the tenant of the normal type, performing a throttling operation on a data request sent by the tenant through the client to the distributed consistency system, or when the distributed consistency system receives a data request sent by the tenant through the client, if the resource consumption data of the data request to be processed in the distributed consistency system exceeds the global resource consumption allowed data set by the distributed consistency system, executing a current limiting operation on a data request sent by the tenant to the distributed consistency system through the client; when the type of the tenant is an unrestricted type, if the resource consumption data of the data request sent by the tenant to the distributed consistency system through the client in the distributed consistency system exceeds the resource consumption permission data set by the distributed consistency system for the tenant of the unrestricted type, performing a current limiting operation on the data request sent by the tenant to the distributed consistency system through the client.
The data processing apparatus of this embodiment is configured to implement the corresponding data processing method in the foregoing multiple method embodiments, and has the beneficial effects of the corresponding method embodiment, which are not described herein again.
Fig. 5 is a schematic structural diagram of an electronic device in a fifth embodiment of the present application; the electronic device may include:
one or more processors 501;
a computer-readable medium 502, which may be configured to store one or more programs,
when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the data processing method according to the first embodiment or the second embodiment.
Fig. 6 is a hardware structure of an electronic device according to a sixth embodiment of the present application; as shown in fig. 6, the hardware structure of the electronic device may include: a processor 601, a communication interface 602, a computer-readable medium 603, and a communication bus 604;
wherein the processor 601, the communication interface 602, and the computer readable medium 603 communicate with each other via a communication bus 604;
alternatively, the communication interface 602 may be an interface of a communication module, such as an interface of a GSM module;
the processor 601 may be specifically configured to: when the distributed consistency system is determined to be in a busy state based on the number of data requests to be processed in the distributed consistency system, resource consumption data of the data requests to be processed in the distributed consistency system are estimated; and when determining that the resource consumption data exceeds the global resource consumption permission data set by the distributed consistency system, performing current limiting operation on a data request sent by a client connected with the distributed consistency system.
The Processor 601 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The computer-readable medium 603 may be, but is not limited to, a Random Access Memory (RAM), a Read-Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code configured to perform the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication section, and/or installed from a removable medium. The computer program, when executed by a Central Processing Unit (CPU), performs the above-described functions defined in the method of the present application. It should be noted that the computer readable medium described herein can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access storage media (RAM), a read-only storage media (ROM), an erasable programmable read-only storage media (EPROM or flash memory), an optical fiber, a portable compact disc read-only storage media (CD-ROM), an optical storage media piece, a magnetic storage media piece, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
Computer program code configured to carry out operations for the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may operate over any of a variety of networks: including a Local Area Network (LAN) or a Wide Area Network (WAN) -to the user's computer, or alternatively, to an external computer (e.g., through the internet using an internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions configured to implement the specified logical function(s). In the above embodiments, specific precedence relationships are provided, but these precedence relationships are only exemplary, and in particular implementations, the steps may be fewer, more, or the execution order may be modified. That is, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present application may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor includes an estimation module and a first current limiting module. The names of the modules do not limit the modules themselves under certain conditions, for example, the pre-estimation module can also be described as a module for pre-estimating resource consumption data of the data request to be processed in the distributed consistency system when the distributed consistency system is determined to be in a busy state based on the number of the data requests to be processed in the distributed consistency system.
As another aspect, the present application also provides a computer-readable medium on which a computer program is stored, which when executed by a processor, implements the data processing method as described in the first or second embodiment.
As another aspect, the present application also provides a computer-readable medium, which may be contained in the apparatus described in the above embodiments; or may be present separately and not assembled into the device. The computer readable medium carries one or more programs which, when executed by the apparatus, cause the apparatus to: when the distributed consistency system is determined to be in a busy state based on the number of data requests to be processed in the distributed consistency system, resource consumption data of the data requests to be processed in the distributed consistency system are estimated; and when determining that the resource consumption data exceeds the global resource consumption permission data set by the distributed consistency system, performing current limiting operation on a data request sent by a client connected with the distributed consistency system.
The expressions "first", "second", "said first" or "said second" used in various embodiments of the present disclosure may modify various components regardless of order and/or importance, but these expressions do not limit the respective components. The above description is only configured for the purpose of distinguishing elements from other elements. For example, the first user equipment and the second user equipment represent different user equipment, although both are user equipment. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element, without departing from the scope of the present disclosure.
When an element (e.g., a first element) is referred to as being "operably or communicatively coupled" or "connected" (operably or communicatively) to "another element (e.g., a second element) or" connected "to another element (e.g., a second element), it is understood that the element is directly connected to the other element or the element is indirectly connected to the other element via yet another element (e.g., a third element). In contrast, it is understood that when an element (e.g., a first element) is referred to as being "directly connected" or "directly coupled" to another element (a second element), no element (e.g., a third element) is interposed therebetween.
The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention herein disclosed is not limited to the particular combination of features described above, but also encompasses other arrangements formed by any combination of the above features or their equivalents without departing from the spirit of the invention. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.
Claims (14)
1. A method of data processing, the method comprising:
when the distributed consistency system is determined to be in a busy state based on the number of data requests to be processed in the distributed consistency system, resource consumption data of the data requests to be processed in the distributed consistency system are estimated;
and when determining that the resource consumption data exceeds the global resource consumption permission data set by the distributed consistency system, performing current limiting operation on a data request sent by a client connected with the distributed consistency system.
2. The method of claim 1, wherein the predicting resource consumption data in the distributed coherency system for the pending data request is preceded by:
and if the number of the data requests to be processed is larger than or equal to a preset number threshold, determining that the distributed consistency system is in a busy state.
3. The method of claim 1, wherein the predicting resource consumption data in the distributed coherency system for the pending data request is preceded by:
receiving a session establishing request which is sent by the client and carries tenant information;
and creating a session corresponding to the tenant information based on the session creation request so as to receive the pending data request sent by the tenant through the client through the session.
4. The method of claim 1, wherein predicting resource consumption data of the pending data request in the distributed coherency system comprises:
determining a request type to which the data request to be processed belongs;
determining resource consumption weight data corresponding to the data request to be processed based on the request type to which the data request to be processed belongs;
and estimating resource consumption data of the data request to be processed in the distributed consistency system based on the resource consumption weight data corresponding to the data request to be processed.
5. The method of claim 1, further comprising:
determining the type of a tenant to which the data request to be processed belongs;
and based on the type of the tenant, performing a current limiting operation on a data request sent by the tenant to the distributed consistency system through the client.
6. The method according to claim 5, wherein the performing, based on the type of the tenant, a throttling operation on a data request sent by the tenant through the client to the distributed consistency system comprises:
when the type of the tenant is a normal type, if the resource consumption data in the distributed consistency system of the data request sent by the tenant to the distributed consistency system through the client exceeds the resource consumption allowable data set by the distributed consistency system for the tenant of the normal type, performing a throttling operation on a data request sent by the tenant through the client to the distributed consistency system, or when the distributed consistency system receives a data request sent by the tenant through the client, if the resource consumption data of the data request to be processed in the distributed consistency system exceeds the global resource consumption allowed data set by the distributed consistency system, executing a current limiting operation on a data request sent by the tenant to the distributed consistency system through the client;
when the type of the tenant is an unrestricted type, if the resource consumption data of the data request sent by the tenant to the distributed consistency system through the client in the distributed consistency system exceeds the resource consumption permission data set by the distributed consistency system for the tenant of the unrestricted type, performing a current limiting operation on the data request sent by the tenant to the distributed consistency system through the client.
7. A data processing apparatus, characterized in that the apparatus comprises:
the estimation module is used for estimating resource consumption data of the data requests to be processed in the distributed consistency system when the distributed consistency system is determined to be in a busy state based on the number of the data requests to be processed in the distributed consistency system;
and the first current limiting module is used for executing current limiting operation on a data request sent by a client connected with the distributed consistency system when the resource consumption data is determined to exceed the global resource consumption allowable data set by the distributed consistency system.
8. The apparatus of claim 7, wherein the prediction module is preceded by:
the first determining module is used for determining that the distributed consistency system is in a busy state if the number of the data requests to be processed is determined to be larger than or equal to a preset number threshold.
9. The apparatus of claim 7, wherein the prediction module is preceded by:
the receiving module is used for receiving a session establishing request which is sent by the client and carries tenant information;
a creating module, configured to create a session corresponding to the tenant information based on the session creating request, so as to receive, through the session, the to-be-processed data request sent by the tenant through the client.
10. The apparatus of claim 7, wherein the estimation module is specifically configured to:
determining a request type to which the data request to be processed belongs;
determining resource consumption weight data corresponding to the data request to be processed based on the request type to which the data request to be processed belongs;
and estimating resource consumption data of the data request to be processed in the distributed consistency system based on the resource consumption weight data corresponding to the data request to be processed.
11. The apparatus of claim 7, further comprising:
the second determining module is used for determining the type of the tenant to which the data request to be processed belongs;
and the second current limiting module is used for executing current limiting operation on a data request sent by the tenant to the distributed consistency system through the client based on the type of the tenant.
12. The apparatus of claim 11, wherein the second current limiting module is specifically configured to:
when the type of the tenant is a normal type, if the resource consumption data in the distributed consistency system of the data request sent by the tenant to the distributed consistency system through the client exceeds the resource consumption allowable data set by the distributed consistency system for the tenant of the normal type, performing a throttling operation on a data request sent by the tenant through the client to the distributed consistency system, or when the distributed consistency system receives a data request sent by the tenant through the client, if the resource consumption data of the data request to be processed in the distributed consistency system exceeds the global resource consumption allowed data set by the distributed consistency system, executing a current limiting operation on a data request sent by the tenant to the distributed consistency system through the client;
when the type of the tenant is an unrestricted type, if the resource consumption data of the data request sent by the tenant to the distributed consistency system through the client in the distributed consistency system exceeds the resource consumption permission data set by the distributed consistency system for the tenant of the unrestricted type, performing a current limiting operation on the data request sent by the tenant to the distributed consistency system through the client.
13. An electronic device, comprising:
one or more processors;
a computer readable medium configured to store one or more programs,
when executed by the one or more processors, cause the one or more processors to implement a data processing method as claimed in any one of claims 1-6.
14. A computer-readable medium, on which a computer program is stored which, when being executed by a processor, carries out the data processing method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911355053.4A CN113032410B (en) | 2019-12-25 | 2019-12-25 | Data processing method, device, electronic equipment and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911355053.4A CN113032410B (en) | 2019-12-25 | 2019-12-25 | Data processing method, device, electronic equipment and computer storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113032410A true CN113032410A (en) | 2021-06-25 |
| CN113032410B CN113032410B (en) | 2024-05-03 |
Family
ID=76458094
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911355053.4A Active CN113032410B (en) | 2019-12-25 | 2019-12-25 | Data processing method, device, electronic equipment and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113032410B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113923163A (en) * | 2021-10-20 | 2022-01-11 | 广东亿迅科技有限公司 | Long connection message channel based current limiting method and system |
| CN114254036A (en) * | 2021-11-12 | 2022-03-29 | 阿里巴巴（中国）有限公司 | Data processing method and system |
| CN114629806A (en) * | 2022-04-13 | 2022-06-14 | 腾讯科技(成都)有限公司 | Data processing method, data processing apparatus, electronic device, storage medium, and program product |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040054714A1 (en) * | 2002-09-12 | 2004-03-18 | Randy Wuerfel | System and method for accessing busy IP network resources |
| CN101959252A (en) * | 2009-07-15 | 2011-01-26 | 华为技术有限公司 | Quality of service (QoS) control and strategy configuration method and device |
| US20120054341A1 (en) * | 2009-03-31 | 2012-03-01 | Freestyle Technology Pty Ltd | Communications Process and System Associated with Monitoring and Controlling Resource Consumption |
| US20160241458A1 (en) * | 2013-09-30 | 2016-08-18 | Orange | Management of network connections |
| CN106713171A (en) * | 2015-07-28 | 2017-05-24 | 腾讯科技（深圳）有限公司 | Server and delay-queue-based current-limiting protection system and method |
| CN107204967A (en) * | 2016-03-18 | 2017-09-26 | 阿里巴巴集团控股有限公司 | Resources regulation method and device during safe physical examination |
| CN109120548A (en) * | 2018-07-02 | 2019-01-01 | 联动优势电子商务有限公司 | A kind of flow control methods and device |
| CN109885399A (en) * | 2019-01-17 | 2019-06-14 | 平安普惠企业管理有限公司 | Data processing method, electronic device, computer equipment and storage medium |
| CN110049084A (en) * | 2018-12-10 | 2019-07-23 | 阿里巴巴集团控股有限公司 | Current-limiting method, device and the equipment of distributed system |
| CN110401720A (en) * | 2019-07-31 | 2019-11-01 | 中国工商银行股份有限公司 | Information processing method, device, system, application server and medium |
| CN110417677A (en) * | 2019-07-29 | 2019-11-05 | 北京易捷思达科技发展有限公司 | A kind of QoS control method based on Ceph distributed storage Osd end data Recovery |
| CN110430141A (en) * | 2019-08-08 | 2019-11-08 | 北京字节跳动网络技术有限公司 | Current-limiting method and device |
| CN110460541A (en) * | 2019-07-30 | 2019-11-15 | 秒针信息技术有限公司 | Current-limiting method and device applied to web server |
-
2019
- 2019-12-25 CN CN201911355053.4A patent/CN113032410B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040054714A1 (en) * | 2002-09-12 | 2004-03-18 | Randy Wuerfel | System and method for accessing busy IP network resources |
| US20120054341A1 (en) * | 2009-03-31 | 2012-03-01 | Freestyle Technology Pty Ltd | Communications Process and System Associated with Monitoring and Controlling Resource Consumption |
| CN101959252A (en) * | 2009-07-15 | 2011-01-26 | 华为技术有限公司 | Quality of service (QoS) control and strategy configuration method and device |
| US20160241458A1 (en) * | 2013-09-30 | 2016-08-18 | Orange | Management of network connections |
| CN106713171A (en) * | 2015-07-28 | 2017-05-24 | 腾讯科技（深圳）有限公司 | Server and delay-queue-based current-limiting protection system and method |
| CN107204967A (en) * | 2016-03-18 | 2017-09-26 | 阿里巴巴集团控股有限公司 | Resources regulation method and device during safe physical examination |
| CN109120548A (en) * | 2018-07-02 | 2019-01-01 | 联动优势电子商务有限公司 | A kind of flow control methods and device |
| CN110049084A (en) * | 2018-12-10 | 2019-07-23 | 阿里巴巴集团控股有限公司 | Current-limiting method, device and the equipment of distributed system |
| CN109885399A (en) * | 2019-01-17 | 2019-06-14 | 平安普惠企业管理有限公司 | Data processing method, electronic device, computer equipment and storage medium |
| CN110417677A (en) * | 2019-07-29 | 2019-11-05 | 北京易捷思达科技发展有限公司 | A kind of QoS control method based on Ceph distributed storage Osd end data Recovery |
| CN110460541A (en) * | 2019-07-30 | 2019-11-15 | 秒针信息技术有限公司 | Current-limiting method and device applied to web server |
| CN110401720A (en) * | 2019-07-31 | 2019-11-01 | 中国工商银行股份有限公司 | Information processing method, device, system, application server and medium |
| CN110430141A (en) * | 2019-08-08 | 2019-11-08 | 北京字节跳动网络技术有限公司 | Current-limiting method and device |
Non-Patent Citations (2)
| Title |
|---|
| 杨恩峰;张勇;刘士军;: "基于资源感知的SaaS服务QoS保障机制研究", 小型微型计算机系统, no. 01, 15 January 2013 (2013-01-15) * |
| 蒋勇;: "基于微服务架构的基础设施设计", 软件, no. 05, 15 May 2016 (2016-05-15) * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113923163A (en) * | 2021-10-20 | 2022-01-11 | 广东亿迅科技有限公司 | Long connection message channel based current limiting method and system |
| CN114254036A (en) * | 2021-11-12 | 2022-03-29 | 阿里巴巴（中国）有限公司 | Data processing method and system |
| CN114629806A (en) * | 2022-04-13 | 2022-06-14 | 腾讯科技(成都)有限公司 | Data processing method, data processing apparatus, electronic device, storage medium, and program product |
| CN114629806B (en) * | 2022-04-13 | 2023-12-12 | 腾讯科技(成都)有限公司 | Data processing method, device, electronic equipment, storage medium and program product |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113032410B (en) | 2024-05-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111651253B (en) | Computing resource scheduling method and device | |
| CN113032410B (en) | Data processing method, device, electronic equipment and computer storage medium | |
| US11751004B2 (en) | Methods and systems for communication management | |
| CN108776934A (en) | Distributed data computational methods, device, computer equipment and readable storage medium storing program for executing | |
| CN107645407B (en) | Method and device for adapting QoS | |
| JP5372988B2 (en) | Data synchronization server, system, and data transfer bandwidth control method | |
| US9197687B2 (en) | Prioritized blocking of on-demand requests | |
| US9722930B2 (en) | Exploiting probabilistic latency expressions for placing cloud applications | |
| US10812606B2 (en) | Supporting communications in a stream processing platform | |
| WO2014090075A1 (en) | System and method for estimating an effective bandwidth | |
| CN113141321B (en) | Data transmission method based on edge calculation and electronic equipment | |
| CN109428926B (en) | Method and device for scheduling task nodes | |
| CN114466226B (en) | Bandwidth duration duty cycle determination method, device, equipment and computer readable medium | |
| US10764191B2 (en) | Device and method for managing end-to-end connections | |
| CN110708678B (en) | Communication method and device | |
| CN111181875B (en) | Bandwidth adjusting method and device | |
| US9094321B2 (en) | Energy management for communication network elements | |
| CN113453285B (en) | Resource adjusting method, device and storage medium | |
| CN114884823A (en) | Flow congestion control method and device, computer readable medium and electronic equipment | |
| Emfinger et al. | Analysis, verification, and management toolsuite for cyber-physical applications on time-varying networks | |
| JP2006246119A (en) | Network system for accomplishing quality guaranteed service using priority control, call reception determining method, and program therefor | |
| JP6595419B2 (en) | API providing apparatus and API request control method | |
| US20140189157A1 (en) | Energy management for communication network elements | |
| CN116760785B (en) | Bandwidth allocation method, device, electronic equipment and storage medium | |
| CN118714085A (en) | Flow transmission control method, apparatus, medium, electronic device, and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |