WO2017206749A1 - 一种自适应资源分配方法及装置 - Google Patents
一种自适应资源分配方法及装置 Download PDFInfo
- Publication number
- WO2017206749A1 WO2017206749A1 PCT/CN2017/085285 CN2017085285W WO2017206749A1 WO 2017206749 A1 WO2017206749 A1 WO 2017206749A1 CN 2017085285 W CN2017085285 W CN 2017085285W WO 2017206749 A1 WO2017206749 A1 WO 2017206749A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- statement
- time
- business
- business statement
- actual execution
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/283—Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/70—Admission control; Resource allocation
- H04L47/82—Miscellaneous aspects
- H04L47/826—Involving periods of time
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/24—Traffic characterised by specific attributes, e.g. priority or QoS
- H04L47/2425—Traffic characterised by specific attributes, e.g. priority or QoS for supporting services specification, e.g. SLA
- H04L47/2433—Allocation of priorities to traffic types
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/04—Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/24—Traffic characterised by specific attributes, e.g. priority or QoS
- H04L47/2408—Traffic characterised by specific attributes, e.g. priority or QoS for supporting different services, e.g. a differentiated services [DiffServ] type of service
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/24—Traffic characterised by specific attributes, e.g. priority or QoS
- H04L47/2416—Real-time traffic
Definitions
- the present application belongs to the field of information processing technologies, and in particular, to an adaptive resource allocation method and apparatus.
- the online transaction library generally refers to a database for storing online transaction OLTP (Online Transaction Processing) data.
- the online transaction library provides an online transaction service, which is sensitive to the response time of the request.
- the data warehouse generally refers to a database for providing online data analysis OLAP (Online Analytical Processing) and business intelligence (BI, Business Intelligence) functions. Compared with the online transaction database, the data warehouse requests access once. The amount of data is large and not very sensitive to response time.
- OLAP Online Analytical Processing
- BI Business Intelligence
- Database systems are often used to meet the business needs of two major areas: 1. Meet the needs of critical online transactional services, and need to support complete ACID (Atomicity, Consistency Consistency, Isolation, Durability) Feature transactions, very short request response time and large throughput; 2. Meet analytical business needs such as data mining and business intelligence. Analytical services have weak requirements for transactions, but the amount of data requested by a single request is very high. Large (hundreds of millions of lines, hundreds of millions of lines), the statement is complex, the resources consumed to execute a single request are many, the response time is relatively long, and it takes several minutes to several hours. Because the data in the data warehouse is usually imported from the online transaction library on a regular basis, the timeliness of the data warehouse is worse than the online transaction library.
- a widely adopted solution is to perform user resource management and statically set unused resource pools for different users.
- the resource pool includes a CPU (Central Processing Unit), memory, and IOPS (Input/Output Operations Per Second). Resources such as the number of seconds of read and write operations.
- CPU Central Processing Unit
- IOPS Input/Output Operations Per Second
- Resources such as the number of seconds of read and write operations.
- different users are configured for transactional and analytical services, and different resource pools are configured. Through this scheme, in a system that is better implemented, these two types of services can basically achieve the mutual use of resources.
- the preset resource quota is also not conducive to the full use of resources.
- the peak period of the type of business is usually during the day, and the late night is usually the low period; however, the analytical business is often done with periodic statistics, such as daily or monthly, and most of the peak period is at night.
- the purpose of the present application is to provide an adaptive resource allocation method and device, which can adaptively adjust resource quotas of different types of services in different time periods, so as to ensure that the core services can operate stably and fully utilize system resources. .
- the present application provides an adaptive resource allocation method and apparatus that are implemented as follows:
- An adaptive resource allocation method comprising: counting a request time, an actual execution time, and a response time of each business statement in a cycle; determining, according to the request time, the one to which each business statement belongs Different time periods of the cycle; determining the type of each business statement according to the actual execution time and the response time of each business statement in each time period, and adjusting different types of business statements in the same time period of the next cycle Resource quota.
- An adaptive resource allocation apparatus comprising: a statistics unit, configured to count a request time, an actual execution time, and a response time of each service statement in a cycle; and a dividing unit, configured to determine, according to the request time, each a different time period of the one cycle to which the business statement belongs; a type determining unit, configured to determine a type of each business statement according to the actual execution time and the response time of each business statement in each time period; The adjustment unit is used to adjust resource quotas of different types of business statements in the same period of the next period.
- the system can adaptively adjust resource ratios of different types of services in different time periods according to the type of service request received during operation and the load condition of different types of service requests. Achieve the goal of ensuring that core services (such as transactional services) can operate stably while fully utilizing system resources.
- core services such as transactional services
- FIG. 1 is a flowchart of a method of an embodiment of an adaptive resource allocation method proposed by the present application
- FIG. 2 is a schematic structural diagram of an embodiment of an adaptive resource allocation apparatus according to the present application.
- FIG. 3 is a schematic structural diagram of an embodiment of a type determining unit according to the present application.
- FIG. 4 is a schematic structural diagram of an embodiment of an adjusting unit according to the present application.
- FIG. 5 is a schematic structural diagram of a system to which an adaptive resource allocation scheme according to the present application is applied.
- the transaction type service and the analysis type service have better time period complementarity.
- the better solution is that the system can dynamically adjust the resource allocation according to the periodicity of the service.
- resources need to be more allocated to transactional services, limiting the total amount of resources used by analytic services; in the trough of transactional services, resources can be more configured for analytic services, It is beneficial to make full use of resources, complete analysis tasks quickly, and obtain analysis results for decision making. Based on this, the present application provides an adaptive resource allocation method and apparatus.
- FIG. 1 is a flowchart of a method of an embodiment of an adaptive resource allocation method proposed by the present application.
- the present application provides method operational steps or apparatus structures as shown in the following embodiments or figures, more or fewer operational steps or modular structures may be included in the method or apparatus based on conventional or no inventive labor. .
- the execution order of the steps or the module structure of the device is not limited to the execution order or the module structure provided by the embodiment of the present application.
- an embodiment of the adaptive resource allocation method provided by the present application may include:
- Step S102 counting the request time, the actual execution time, and the response time of each business statement in one cycle.
- Step S104 determining, according to the request time, different time periods of one cycle to which each business statement belongs; that is, determining which time period the request time of each business statement is located, determining that the business statement belongs to the time period.
- Step S106 Determine the type of each business statement according to the actual execution time and response time of each business statement in each time period, and adjust resource quotas of different types of business statements in the same time period of the next period.
- the system can adaptively adjust the resource ratio of different types of services in different time periods according to the type of the service request received during the operation and the load condition of different types of service requests, so as to ensure the core service. (for example, transactional business) can run stably and make full use of system resources.
- one cycle can be set to one day, divided into 24 time segments, and one time segment is one hour.
- a specific period of time and a corresponding time period may also be set based on actual requirements, which is not limited in this application. In general, the shorter the period and the duration of the corresponding time period, the higher the accuracy of the system adaptively allocating resources.
- the types of business statements may include: transactional business statements and analytical business statements. Of course, other types of business statements may also be included, which may be set based on actual needs. If the resource quota of other types of business statements needs to be adaptively adjusted, the above adaptive resource allocation method may also be adopted for the specific implementation.
- This application mainly describes two types of business statements: transaction-type business statements and analytical business statements.
- the type of each business statement can be determined according to the actual execution time of each business statement in each time period, which can be implemented by the following preferred embodiments: determining each time Whether the actual execution time of each business statement in the segment is greater than the execution time threshold; if yes, the business statement is determined to be an analytical business statement; if not, the business statement is determined to be a transactional business statement.
- the type of the business statement can be determined simply, quickly and accurately, and the value of the execution time threshold can be set or adjusted according to actual conditions.
- the resource quotas of different types of business statements in the same period of the next period can be adjusted according to the actual execution time and response time of each business statement in each time period.
- the following preferred implementations are implemented:
- the first step is to sum the actual execution time of all transaction-type business statements in each time period, and obtain the first actual execution time in each time period; the response time of all transaction-type business statements in each time period. Summing, obtaining the first response time sum in each time period;
- the actual execution time of all the analytical business statements in each time period is summed to obtain the second actual execution time in each time period;
- the third step according to the first actual execution time sum, the first response time sum, the second actual execution time, and the second response time sum, adjust the transaction type business statement and the analytical type business statement in the same time period of the next period Resource quota.
- the resource quota of the transaction type business statement Decrease to a maximum value greater than or equal to: the minimum resource quota for the default transactional business statement, the first actual execution time, and the actual consumed resource quota.
- the minimum resource quota of the analytical business statement and the minimum resource quota of the transaction type business statement may be set or adjusted according to actual needs.
- the system can adaptively adjust the resource quotas of different types of service statements according to the load conditions of different types of service statements, and the resource quota adjustment operation has high accuracy, and can fully utilize system resources to ensure different types of services. Run smoothly and avoid waste of resources.
- the present application provides a preferred embodiment for prioritizing transactional business statements and analytical business statements; wherein the transactional business statement has a higher priority than the analytical business statement; in a specific scenario, if If the request quantity of the transaction type business statement exceeds the threshold within the preset time, the resource quota of the analytical business statement is reduced to zero until the request quantity of the transaction type business statement is lower than the above threshold.
- the values of the above thresholds can be set or adjusted according to actual needs.
- different types of service statements can be prioritized based on the importance of different types of services, thereby ensuring priority operation of certain types of services in certain specific scenarios.
- the priority of transaction-type business statements and analytical business statements is set by the system administrator, and the administrator sorts the priorities according to the type of service that the system mainly supports.
- the present application provides a preferred embodiment, which prioritizes transaction-type business statements and analytical business statements; when system resources are lower than resource thresholds, priority is given to resource quotas for high-priority business statements. And/or, reduce the resource quota of the low priority business statement to zero.
- the present application provides an adaptive resource allocation apparatus, as described in the following embodiments. Since the principle of the problem is solved by the adaptive resource allocation device and the adaptive resource allocation method, the implementation of the adaptive resource allocation device can be referred to the implementation of the adaptive resource allocation method, and the repeated description is not repeated.
- FIG. 2 is a schematic structural diagram of an embodiment of an adaptive resource allocation apparatus according to the present application. As shown in FIG. 2, the apparatus may include:
- the statistical unit 10 is configured to count the request time, the actual execution time and the response of each business statement in a cycle. time.
- the statistic unit 10 is a part of the statistical resource allocation device that collects time information, and may be software, hardware, or a combination of the two. For example, it may be an input/output interface that completes the time information collecting function, a processing chip, and the like.
- the dividing unit 20 is configured to determine different time periods of one cycle to which each business statement belongs according to the request time.
- the dividing unit 20 is a time segment dividing operation part in the adaptive resource allocation device, and may be software, hardware, or a combination of the two. For example, it may be an input/output interface that completes the time segment dividing function, a processing chip, and the like.
- the type determining unit 30 is configured to determine the type of each business statement according to the foregoing actual execution time and the response time of each business statement in each time period.
- the type determining unit 30 is a time segment dividing operation portion in the adaptive resource allocation device, and may be software, hardware, or a combination of the two. For example, it may be an input/output interface that completes the time segment dividing function, a processing chip, and the like.
- the adjusting unit 40 is configured to adjust resource quotas of different types of business statements in the same period of time in the next period.
- the adjustment unit 40 is a resource quota adjustment part in the adaptive resource allocation apparatus, and may be software, hardware, or a combination of the two. For example, it may be an input/output interface that completes the resource quota adjustment function, and a processing chip.
- the system can adaptively adjust the resource ratio of different types of services in different time periods according to the type of the service request received during the operation and the load condition of different types of service requests, so as to ensure the core service. (for example, transactional business) can run stably and make full use of system resources.
- one cycle can be set to one day, divided into 24 time segments, and one time segment is one hour.
- one cycle may be set to one week, and one time period is one day; or, one cycle is one month, one time zone is one day; or one cycle is one year, and one time zone is one month.
- a specific period of time and a corresponding time period may also be set based on actual requirements, which is not limited in this application. In general, the shorter the period and the duration of the corresponding time period, the higher the accuracy of the system adaptively allocating resources.
- the types of business statements may include: transactional business statements and analytical business statements.
- other types of business statements can also be included, which can be set based on actual needs.
- the type of each service statement may be determined according to the actual execution time of each business statement in each time period, which may be implemented by the following preferred embodiments:
- the method may include: a threshold setting unit configured to set an execution time threshold.
- the value of the execution time threshold can be set or adjusted according to the actual situation.
- 3 is a schematic structural diagram of an embodiment of the type determining unit of the present application. As shown in FIG. 3, the type determining unit 30 may include: a determining module 31, configured to determine the actual state of each business statement in each time period.
- the first processing module 32 is configured to determine, in the case that the determination result of the determining module 31 is YES, that the business statement is an analytical business statement; the second processing module 33.
- the determination result of the determination module 31 is negative, it is determined that the business statement is a transaction type business statement.
- the type of business statement can be determined simply, quickly and accurately.
- the adjustment unit 40 includes: a first calculation module 41, configured to actually implement all transaction-type business statements in each time period. Performing time summation to obtain a first actual execution time sum in each time period; a second calculation module 42 for summing the response times of all transaction type business statements in each time period to obtain each time period The first response time sum; the third calculating module 43 is configured to sum the actual execution time of all the analytical business statements in each time period to obtain the second actual execution time and each time period; the fourth calculation The module 44 is configured to sum the response times of all the analysis type service statements in each time period to obtain a second response time sum in each time period; the resource adjustment module 45 is configured to use the first actual execution time sum, The first response time sum, the second actual execution time, and the second response time sum adjust resource quotas of the transaction type business statement and the analysis type business statement in the same time period of the next period.
- the resource adjustment module 45 may include: a first adjustment submodule, configured to determine whether the first response time and the first actual execution time are greater than, and if yes, determine that the resource quota of the transaction type business statement is too small, and the analysis is performed. The resource quota of the business statement is reduced.
- the resource quota of the analytical business statement is reduced to the minimum resource quota of the preset analytical business statement; and the second adjustment submodule is used to determine whether the second response time is greater than the first Second, the actual execution time and, if so, determine that the resource quota of the analytical business statement is too small, and reduce the resource quota of the transaction type business statement, specifically, reduce the resource quota of the transaction type business statement to be greater than or equal to the following Maximum value: the minimum resource quota of the default transaction type business statement, the first actual execution time, and the actual consumed resource quota.
- the minimum resource quota of the analytical business statement and the minimum resource quota of the transaction type business statement may be set or adjusted according to actual needs.
- the system can adaptively adjust the resource quotas of different types of service statements according to the load conditions of different types of service statements, and the resource quota adjustment operation has high accuracy, and can fully utilize system resources to ensure different types of services. Run smoothly and avoid waste of resources.
- the apparatus may further include: a first priority allocation unit, configured to prioritize the transaction type business statement and the analysis type business statement; wherein, the priority of the transaction type business statement Higher than the analytical business statement; in a specific scenario, if the request amount of the transaction type business statement exceeds the threshold within the preset time, the resource quota of the analytical business statement is reduced to zero until the request amount of the transaction type business statement is lower than Threshold.
- a first priority allocation unit configured to prioritize the transaction type business statement and the analysis type business statement; wherein, the priority of the transaction type business statement Higher than the analytical business statement; in a specific scenario, if the request amount of the transaction type business statement exceeds the threshold within the preset time, the resource quota of the analytical business statement is reduced to zero until the request amount of the transaction type business statement is lower than Threshold.
- the present application provides a preferred implementation manner, where the foregoing apparatus may further include: a second priority allocation unit, configured to prioritize the transaction type business statement and the analysis type business statement; when the system resource is lower than the resource threshold , preferentially allocate resource quotas for business statements with high priority, and/or reduce the resource quota of business statements with low priority to zero.
- a second priority allocation unit configured to prioritize the transaction type business statement and the analysis type business statement
- the adaptive resource allocation scheme of the present application mainly includes the following parts:
- the system administrator configures initial parameters for the system, including:
- an execution time threshold (large_query_threshold) may be preset based on actual conditions. If the actual execution time of the business statement is greater than the execution time threshold, the business statement is considered to be an analytical business statement, if the actual execution time of the business statement is less than the When the execution time threshold is executed, the business statement is considered to be a transactional business statement.
- the priority ordering may be performed according to the importance degree of different types of business statements, so that when the system resources are seriously insufficient, the business statements with higher priority and/or the business statements with lower active priority may be preferentially processed.
- transaction-oriented services are usually preferred.
- the execution status information mainly includes the following content: the time (T_start), the actual execution time (T_actual), and the response time (T_elapse) that the system receives the request for the business statement.
- the actual execution time (T_actual) refers to the total execution time allocated by the scheduler to the business statement. If the time allocated by the scheduler is not used, the execution ends.
- the actual execution time refers to the actual execution.
- Time, response time (T_elapse) refers to The time from the receipt of this business statement by the system to the completion of the execution.
- the ternary group (T_start, T_actual, T_elapse) is reported to the statistical analysis module, and the function of the statistical analysis module is equivalent to the statistical unit, the dividing unit, and the type determining unit in the adaptive resource allocation device.
- the foregoing scheduler refers to a module in the system that undertakes a request sent by a scheduling front end, and the scheduler makes a decision to start execution, suspension, or even termination of a certain task according to resource quota conditions of different types of tasks.
- the above-mentioned executor refers to a module that actually completes the front-end request task in the system. During the execution process, the execution is interrupted according to the instruction of the scheduler or the execution is terminated early. After the execution ends, the corresponding execution state information is obtained.
- the statistical analysis module counts the distribution of requests for each time period within a period (eg, every hour of the day). First, all the triples in a day are grouped according to the request time (T_start) to obtain an hourly request set ReqSet(h0), ReqSet(h1), ReqSet(h23); then, for each hour of the request set ReqSet( Hi) The triples inside, find all the tuples of the actual execution time (T_actual) ⁇ execution time threshold (large_query_threshold) value, and form a new set ReqSetOltp(hi), which is all the transactions in each time period. The set of business statements, the rest is the set of all analytical business statements ReqSetOlap (hi) in each time period.
- the actual execution time (T_actual) and response time (T_elapse) in the two tuples ReqSetOltp(hi) and ReqSetOlap(hi) are respectively summed to obtain the first actual execution time of four time values and TotalAtimeOltp(hi), First response time and TotalEtimeOltp(hi), second actual execution time and TotalAtimeOlap(hi), second response time, and TotalEtimeOlap(hi).
- the resource quota of a certain type of business statement can be reduced according to the priority level, or the resource quota of the two types of business statements can be reduced according to the same or different proportions.
- the resource quota AssignTimeOltp(hi), AssignTimeOlap(hi), assigned to the two types of business statements during the time period is for the same period of the next period (the same hour of the next day). Resource quotas are adjusted.
- the system calculates the acquired value based on the corresponding time period (in hours) from the previous cycle (for example, the first day), and adjusts the resource quota based on the policy described above. Set the proportion of resources for different types of services. At the same time, in the process of execution, the new load situation of the two types of business statements will be reported and analyzed.
- the system determines that the transaction continues for a certain period of time (for example, 1 minute) according to the real-time statistical information. Whether the request quantity of the transaction type business statement exceeds the threshold value may directly reduce or cancel the resource quota of the analytical business statement in the time period until the request quantity of the transaction type business statement returns to the normal level.
- the resources enjoyed by different types of business statements refer to the CPU (Central Processing Unit), memory, storage, and network resources required to execute the request.
- CPU Central Processing Unit
- This application mainly refers to CPU resources.
- adaptive adjustments to other system resources such as memory usage, IO usage, network resources, and the like, are also applicable to the technical solution of the present application.
- the present application provides a system administrator to initialize the parameters of the system (for example, the system is prioritized by the transaction type service), according to different types of business statements of the system in a certain period of time in the past.
- the resource usage adaptively adjusts the system's future allocation of resources during that time period. It not only ensures the stable operation of the core business, but also makes full use of system resources.
- the apparatus or module illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product having a certain function.
- the above devices are described as being separately divided into various modules by function.
- the functions of the various modules may be implemented in one or more software and/or hardware in the implementation of the present application.
- the controller can take, for example, a microprocessor or processor and storage can be processed by the (micro) Computer readable medium (eg, software or firmware) executed by a computer readable medium, logic gate, switch, Application Specific Integrated Circuit (ASIC), programmable logic controller, and embedded microcontroller,
- microcontrollers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, which may also be implemented as part of the control logic of the memory.
- the controller can be logically programmed by means of logic gates, switches, ASICs, programmable logic controllers, and embedding.
- modules in the apparatus described herein may be described in the general context of computer-executable instructions executed by a computer, such as program modules.
- program modules include routines, programs, objects, components, data structures, classes, and the like that perform particular tasks or implement particular abstract data types.
- the present application can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are connected through a communication network.
- program modules can be located in both local and remote computer storage media including storage devices.
- the present application can be implemented by means of software plus necessary hardware. Based on such understanding, the technical solution of the present application may be embodied in the form of a software product in essence or in the form of a software product, or may be embodied in the implementation process of data migration.
- the computer software product may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions for causing a computer device (which may be a personal computer, a mobile terminal, a server, or a network device) to execute the present application.
- a computer device which may be a personal computer, a mobile terminal, a server, or a network device
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
Description
Claims (18)
- 一种自适应资源分配方法,其特征在于,所述方法包括:统计一个周期内每个业务语句的请求时间、实际执行时间和响应时间;根据所述请求时间,确定每个业务语句所属的所述一个周期的不同时间段;根据每个时间段内每个业务语句的所述实际执行时间和所述响应时间,确定每个业务语句的类型,并调整下一个周期的同一时间段内不同类型业务语句的资源配额。
- 根据权利要求1所述的方法,其特征在于,业务语句的类型包括:交易型业务语句和分析型业务语句。
- 根据权利要求2所述的方法,其特征在于,所述方法还包括:设置执行时间阈值;根据每个时间段内每个业务语句的所述实际执行时间,确定每个业务语句的类型,包括:判断每个时间段内每个业务语句的所述实际执行时间是否大于所述执行时间阈值;如果是,则判定该业务语句是分析型业务语句;如果否,则判定该业务语句是交易型业务语句。
- 根据权利要求3所述的方法,其特征在于,根据每个时间段内每个业务语句的所述实际执行时间和所述响应时间,调整下一个周期的同一时间段内不同类型业务语句的资源配额,包括:将每个时间段内所有交易型业务语句的所述实际执行时间求和,得到每个时间段内的第一实际执行时间和;将每个时间段内所有交易型业务语句的所述响应时间求和,得到每个时间段内的第一响应时间和;将每个时间段内所有分析型业务语句的所述实际执行时间求和,得到每个时间段内的第二实际执行时间和;将每个时间段内所有分析型业务语句的所述响应时间求和,得到每个时间段内的第二响应时间和;根据所述第一实际执行时间和、所述第一响应时间和、所述第二实际执行时间和、所述第二响应时间和,调整下一个周期的同一时间段内所述交易型业务语句和所述分析型业务语句的资源配额。
- 根据权利要求4所述的方法,其特征在于,根据所述第一实际执行时间和、所 述第一响应时间和、所述第二实际执行时间和、所述第二响应时间和,调整下一个周期的同一时间段内所述交易型业务语句和所述分析型业务语句的资源配额,包括:判断所述第一响应时间和是否大于所述第一实际执行时间和,如果是,则判定所述交易型业务语句的资源配额过小,将所述分析型业务语句的资源配额减少;判断所述第二响应时间和是否大于所述第二实际执行时间和,如果是,则判定所述分析型业务语句的资源配额过小,将所述交易型业务语句的资源配额减少。
- 根据权利要求5所述的方法,其特征在于,将所述分析型业务语句的资源配额减少,包括:将所述分析型业务语句的资源配额降低为预设的分析型业务语句的最低资源配额;将所述交易型业务语句的资源配额减少,包括:将所述交易型业务语句的资源配额降低为大于或等于以下二者的最大值:预设的交易型业务语句的最低资源配额、所述第一实际执行时间和实际消耗的资源配额。
- 根据权利要求2所述的方法,其特征在于,所述方法还包括:对所述交易型业务语句和所述分析型业务语句进行优先级排序;其中,所述交易型业务语句的优先级高于所述分析型业务语句;在特定场景下,如果预设时间内所述交易型业务语句的请求量超出阈值,则将所述分析型业务语句的资源配额降为零,直至所述交易型业务语句的请求量低于所述阈值。
- 根据权利要求2所述的方法,其特征在于,所述方法还包括:对所述交易型业务语句和所述分析型业务语句进行优先级排序;在系统资源低于资源阈值时,优先为优先级高的业务语句分配资源配额,和/或,将优先级低的业务语句的资源配额降为零。
- 根据权利要求1所述的方法,其特征在于,所述一个周期是一天,一个时间段是一个小时。
- 一种自适应资源分配装置,其特征在于,所述装置包括:统计单元,用于统计一个周期内每个业务语句的请求时间、实际执行时间和响应时间;划分单元,用于根据所述请求时间,确定每个业务语句所属的所述一个周期的不同时间段;类型确定单元,用于根据每个时间段内每个业务语句的所述实际执行时间和所述响应时间,确定每个业务语句的类型;调整单元,用于调整下一个周期的同一时间段内不同类型业务语句的资源配额。
- 根据权利要求10所述的装置,其特征在于,业务语句的类型包括:交易型业务语句和分析型业务语句。
- 根据权利要求11所述的装置,其特征在于,所述装置还包括:阈值设置单元,用于设置执行时间阈值;所述类型确定单元包括:判断模块,用于判断每个时间段内每个业务语句的所述实际执行时间是否大于所述执行时间阈值;第一处理模块,用于在所述判断模块的判断结果为是的情况下,判定该业务语句是分析型业务语句;第二处理模块,用于在所述判断模块的判断结果为否的情况下,判定该业务语句是交易型业务语句。
- 根据权利要求12所述的装置,其特征在于,所述调整单元包括:第一计算模块,用于将每个时间段内所有交易型业务语句的所述实际执行时间求和,得到每个时间段内的第一实际执行时间和;第二计算模块,用于将每个时间段内所有交易型业务语句的所述响应时间求和,得到每个时间段内的第一响应时间和;第三计算模块,用于将每个时间段内所有分析型业务语句的所述实际执行时间求和,得到每个时间段内的第二实际执行时间和;第四计算模块,用于将每个时间段内所有分析型业务语句的所述响应时间求和,得到每个时间段内的第二响应时间和;资源调整模块,用于根据所述第一实际执行时间和、所述第一响应时间和、所述第二实际执行时间和、所述第二响应时间和,调整下一个周期的同一时间段内所述交易型业务语句和所述分析型业务语句的资源配额。
- 根据权利要求13所述的装置,其特征在于,所述资源调整模块包括:第一调整子模块,用于判断所述第一响应时间和是否大于所述第一实际执行时间和,如果是,则判定所述交易型业务语句的资源配额过小,将所述分析型业务语句的资源配额减少;第二调整子模块,用于判断所述第二响应时间和是否大于所述第二实际执行时间和,如果是,则判定所述分析型业务语句的资源配额过小,将所述交易型业务语句的资 源配额减少。
- 根据权利要求14所述的装置,其特征在于,所述第一调整子模块,具体用于将所述分析型业务语句的资源配额降低为预设的分析型业务语句的最低资源配额;所述第二调整子模块,具体用于将所述交易型业务语句的资源配额减少,包括:将所述交易型业务语句的资源配额降低为大于或等于以下二者的最大值:预设的交易型业务语句的最低资源配额、所述第一实际执行时间和实际消耗的资源配额。
- 根据权利要求11所述的装置,其特征在于,所述装置还包括:第一优先分配单元,用于对所述交易型业务语句和所述分析型业务语句进行优先级排序;其中,所述交易型业务语句的优先级高于所述分析型业务语句;在特定场景下,如果预设时间内所述交易型业务语句的请求量超出阈值,则将所述分析型业务语句的资源配额降为零,直至所述交易型业务语句的请求量低于所述阈值。
- 根据权利要求11所述的装置,其特征在于,所述装置还包括:第二优先分配单元,用于对所述交易型业务语句和所述分析型业务语句进行优先级排序;在系统资源低于资源阈值时,优先为优先级高的业务语句分配资源配额,和/或,将优先级低的业务语句的资源配额降为零。
- 根据权利要求10所述的装置,其特征在于,所述一个周期是一天,一个时间段是一个小时。
Priority Applications (6)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2018563444A JP7018554B2 (ja) | 2016-06-01 | 2017-05-22 | 適応型リソース割り当て方法及び装置 |
KR1020187037792A KR102281053B1 (ko) | 2016-06-01 | 2017-05-22 | 적응식 리소스 할당 방법 및 장치 |
EP21188934.0A EP3920490B1 (en) | 2016-06-01 | 2017-05-22 | Adaptive resource allocation method and apparatus |
EP17805694.1A EP3468121B1 (en) | 2016-06-01 | 2017-05-22 | Adaptive resource allocation method and apparatus |
US16/205,397 US11341164B2 (en) | 2016-06-01 | 2018-11-30 | Adaptive resource allocation method and apparatus |
US16/724,713 US11550821B2 (en) | 2016-06-01 | 2019-12-23 | Adaptive resource allocation method and apparatus |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201610382326.4A CN106911592B (zh) | 2016-06-01 | 2016-06-01 | 一种自适应资源分配方法及装置 |
CN201610382326.4 | 2016-06-01 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/205,397 Continuation US11341164B2 (en) | 2016-06-01 | 2018-11-30 | Adaptive resource allocation method and apparatus |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2017206749A1 true WO2017206749A1 (zh) | 2017-12-07 |
Family
ID=59206193
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2017/085285 WO2017206749A1 (zh) | 2016-06-01 | 2017-05-22 | 一种自适应资源分配方法及装置 |
Country Status (7)
Country | Link |
---|---|
US (2) | US11341164B2 (zh) |
EP (2) | EP3468121B1 (zh) |
JP (1) | JP7018554B2 (zh) |
KR (1) | KR102281053B1 (zh) |
CN (1) | CN106911592B (zh) |
TW (1) | TWI697787B (zh) |
WO (1) | WO2017206749A1 (zh) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111028931A (zh) * | 2019-12-11 | 2020-04-17 | 医渡云(北京)技术有限公司 | 医疗数据处理方法及装置、电子设备、存储介质 |
CN111858030A (zh) * | 2020-06-17 | 2020-10-30 | 北京百度网讯科技有限公司 | 作业的资源处理方法、装置、电子设备及可读存储介质 |
Families Citing this family (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106911592B (zh) | 2016-06-01 | 2020-06-12 | 创新先进技术有限公司 | 一种自适应资源分配方法及装置 |
CN108196959B (zh) * | 2018-02-07 | 2021-06-01 | 聚好看科技股份有限公司 | Etl系统的资源管理方法及装置 |
CN110874352B (zh) * | 2018-08-29 | 2023-04-11 | 阿里巴巴集团控股有限公司 | 数据库管理方法及系统 |
CN110890951B (zh) * | 2018-09-11 | 2022-11-22 | 中国移动通信有限公司研究院 | 用户业务时间的分配方法、装置及系统 |
CN109559178B (zh) * | 2018-10-19 | 2024-05-17 | 中国平安人寿保险股份有限公司 | 电子发票批量开票方法、电子装置及可读存储介质 |
CN109523184A (zh) * | 2018-11-28 | 2019-03-26 | 郑州云海信息技术有限公司 | 一种资源调度的方法及装置 |
CN110034946A (zh) * | 2019-01-03 | 2019-07-19 | 阿里巴巴集团控股有限公司 | 自适应服务降级方法和装置 |
CN111222161A (zh) * | 2019-12-31 | 2020-06-02 | 航天信息股份有限公司 | 一种基于权限控制的图片库管理方法及装置 |
CN113296874B (zh) * | 2020-05-29 | 2022-06-21 | 阿里巴巴集团控股有限公司 | 一种任务的调度方法、计算设备及存储介质 |
CN111798113A (zh) * | 2020-06-28 | 2020-10-20 | 北京小米松果电子有限公司 | 资源分配方法、装置、存储介质和电子设备 |
CN113296946B (zh) * | 2021-05-24 | 2023-11-17 | 南京大学 | 边端协同的并发实时流式数据分析任务处理方法及设备 |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090307352A1 (en) * | 2008-06-10 | 2009-12-10 | International Business Machines Corporation | Requester-Side Autonomic Governor |
CN103067297A (zh) * | 2013-01-25 | 2013-04-24 | 中国科学院声学研究所 | 一种基于资源消耗预测的动态负载均衡方法及装置 |
CN104360905A (zh) * | 2014-10-29 | 2015-02-18 | 中国建设银行股份有限公司 | 一种应用于it系统的自适应控制方法和装置 |
CN104809070A (zh) * | 2015-05-13 | 2015-07-29 | 上海新炬网络信息技术有限公司 | 基于数据库交易类中间件的性能分析方法 |
Family Cites Families (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2002082926A (ja) | 2000-09-06 | 2002-03-22 | Nippon Telegr & Teleph Corp <Ntt> | 分散アプリケーション試験・運用管理システム |
JP2002202959A (ja) | 2000-12-28 | 2002-07-19 | Hitachi Ltd | 動的な資源分配をする仮想計算機システム |
US7453801B2 (en) * | 2001-11-08 | 2008-11-18 | Qualcomm Incorporated | Admission control and resource allocation in a communication system supporting application flows having quality of service requirements |
CN1237767C (zh) * | 2004-07-09 | 2006-01-18 | 清华大学 | 一种共享资源访问的调度控制方法及装置 |
US20060200496A1 (en) | 2005-02-22 | 2006-09-07 | Transparency Software, Inc. | Organization action incidents |
US7899817B2 (en) * | 2005-10-05 | 2011-03-01 | Microsoft Corporation | Safe mode for inverse query evaluations |
US8447859B2 (en) * | 2007-12-28 | 2013-05-21 | International Business Machines Corporation | Adaptive business resiliency computer system for information technology environments |
FR2937439B1 (fr) * | 2008-10-17 | 2012-04-20 | Commissariat Energie Atomique | Procede d'execution deterministe et de synchronisation d'un systeme de traitement de l'information comportant plusieurs coeurs de traitement executant des taches systemes. |
CN102083208B (zh) * | 2009-11-30 | 2013-09-25 | 普天信息技术研究院有限公司 | 资源分配方法和装置 |
US8667385B1 (en) * | 2009-12-07 | 2014-03-04 | Google Inc. | Method and system for generating and sharing analytics annotations |
WO2012150885A1 (en) * | 2011-05-05 | 2012-11-08 | Telefonaktiebolaget L M Ericsson (Publ) | Method and apparatus for controlling charging of a service |
US9552231B2 (en) * | 2012-09-27 | 2017-01-24 | Adobe Systems Incorporated | Client classification-based dynamic allocation of computing infrastructure resources |
US9760847B2 (en) * | 2013-05-29 | 2017-09-12 | Sap Se | Tenant selection in quota enforcing request admission mechanisms for shared applications |
US10963810B2 (en) * | 2014-06-30 | 2021-03-30 | Amazon Technologies, Inc. | Efficient duplicate detection for machine learning data sets |
US10362099B2 (en) * | 2015-06-23 | 2019-07-23 | Amazon Technologies, Inc. | Multiple instance types serving a single workload or application |
CN106911592B (zh) | 2016-06-01 | 2020-06-12 | 创新先进技术有限公司 | 一种自适应资源分配方法及装置 |
CN107239479B (zh) * | 2017-03-28 | 2020-03-13 | 创新先进技术有限公司 | 一种基于区块链的数据存储以及查询的方法及装置 |
-
2016
- 2016-06-01 CN CN201610382326.4A patent/CN106911592B/zh active Active
-
2017
- 2017-04-12 TW TW106112255A patent/TWI697787B/zh active
- 2017-05-22 EP EP17805694.1A patent/EP3468121B1/en active Active
- 2017-05-22 KR KR1020187037792A patent/KR102281053B1/ko active IP Right Grant
- 2017-05-22 WO PCT/CN2017/085285 patent/WO2017206749A1/zh unknown
- 2017-05-22 EP EP21188934.0A patent/EP3920490B1/en active Active
- 2017-05-22 JP JP2018563444A patent/JP7018554B2/ja active Active
-
2018
- 2018-11-30 US US16/205,397 patent/US11341164B2/en active Active
-
2019
- 2019-12-23 US US16/724,713 patent/US11550821B2/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090307352A1 (en) * | 2008-06-10 | 2009-12-10 | International Business Machines Corporation | Requester-Side Autonomic Governor |
CN103067297A (zh) * | 2013-01-25 | 2013-04-24 | 中国科学院声学研究所 | 一种基于资源消耗预测的动态负载均衡方法及装置 |
CN104360905A (zh) * | 2014-10-29 | 2015-02-18 | 中国建设银行股份有限公司 | 一种应用于it系统的自适应控制方法和装置 |
CN104809070A (zh) * | 2015-05-13 | 2015-07-29 | 上海新炬网络信息技术有限公司 | 基于数据库交易类中间件的性能分析方法 |
Non-Patent Citations (1)
Title |
---|
See also references of EP3468121A4 |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111028931A (zh) * | 2019-12-11 | 2020-04-17 | 医渡云(北京)技术有限公司 | 医疗数据处理方法及装置、电子设备、存储介质 |
CN111028931B (zh) * | 2019-12-11 | 2023-08-22 | 医渡云(北京)技术有限公司 | 医疗数据处理方法及装置、电子设备、存储介质 |
CN111858030A (zh) * | 2020-06-17 | 2020-10-30 | 北京百度网讯科技有限公司 | 作业的资源处理方法、装置、电子设备及可读存储介质 |
CN111858030B (zh) * | 2020-06-17 | 2024-03-22 | 北京百度网讯科技有限公司 | 作业的资源处理方法、装置、电子设备及可读存储介质 |
Also Published As
Publication number | Publication date |
---|---|
EP3468121A4 (en) | 2019-06-19 |
JP7018554B2 (ja) | 2022-02-14 |
EP3920490A1 (en) | 2021-12-08 |
US11550821B2 (en) | 2023-01-10 |
JP2019520646A (ja) | 2019-07-18 |
EP3920490B1 (en) | 2023-01-18 |
CN106911592B (zh) | 2020-06-12 |
US20190102343A1 (en) | 2019-04-04 |
KR102281053B1 (ko) | 2021-07-23 |
US20200201885A1 (en) | 2020-06-25 |
US11341164B2 (en) | 2022-05-24 |
TWI697787B (zh) | 2020-07-01 |
KR20190015370A (ko) | 2019-02-13 |
EP3468121B1 (en) | 2021-08-04 |
CN106911592A (zh) | 2017-06-30 |
TW201743223A (zh) | 2017-12-16 |
EP3468121A1 (en) | 2019-04-10 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2017206749A1 (zh) | 一种自适应资源分配方法及装置 | |
US10606653B2 (en) | Efficient priority-aware thread scheduling | |
US11582166B2 (en) | Systems and methods for provision of a guaranteed batch | |
US10089142B2 (en) | Dynamic task prioritization for in-memory databases | |
AU2007261666C1 (en) | Method, system, and apparatus for scheduling computer micro-jobs to execute at non-disruptive times | |
WO2021233261A1 (zh) | 一种多任务动态资源调度方法 | |
US9870269B1 (en) | Job allocation in a clustered environment | |
JP2004213624A (ja) | 動的なスレッド・プール調整技法 | |
JP2005182796A (ja) | グリッド・システム資源の自律的制御のための装置、システム、および方法 | |
WO2013082119A1 (en) | Cloud provisioning accelerator | |
JP2004213625A (ja) | プログラムに基づいた応答時間ベースの作業負荷分配技法 | |
US10108459B2 (en) | System and method to dynamically allocate varying processing capacity entitlements based on workload importance | |
US7260728B2 (en) | Windows-based power management method and related portable device | |
US9817698B2 (en) | Scheduling execution requests to allow partial results | |
WO2023273502A1 (zh) | 作业处理的方法、装置、计算机设备以及存储介质 | |
WO2022252986A1 (zh) | 中断调度方法、电子设备及存储介质 | |
CN114816703A (zh) | 一种任务处理方法、装置、设备及介质 | |
US8572619B2 (en) | System and method for integrating software schedulers and hardware interrupts for a deterministic system | |
CN117827453A (zh) | 存储资源的订购方法、装置、设备及存储介质 | |
Halem et al. | A Hybrid Scheduling Algorithm for Data Intensive Workloads in a MapReduce Environment |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
ENP | Entry into the national phase |
Ref document number: 2018563444 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 17805694 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 20187037792 Country of ref document: KR Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 2017805694 Country of ref document: EP Effective date: 20190102 |