WO2017166643A1 - Method and device for quantifying task resources - Google Patents

Method and device for quantifying task resources Download PDF

Info

Publication number
WO2017166643A1
WO2017166643A1 PCT/CN2016/096967 CN2016096967W WO2017166643A1 WO 2017166643 A1 WO2017166643 A1 WO 2017166643A1 CN 2016096967 W CN2016096967 W CN 2016096967W WO 2017166643 A1 WO2017166643 A1 WO 2017166643A1
Authority
WO
WIPO (PCT)
Prior art keywords
task
processing device
resources
device group
task processing
Prior art date
Application number
PCT/CN2016/096967
Other languages
French (fr)
Chinese (zh)
Inventor
刘永华
王孝庆
Original Assignee
乐视控股(北京)有限公司
乐视云计算有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 乐视控股(北京)有限公司, 乐视云计算有限公司 filed Critical 乐视控股(北京)有限公司
Publication of WO2017166643A1 publication Critical patent/WO2017166643A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5083Techniques for rebalancing the load in a distributed system
    • G06F9/5088Techniques for rebalancing the load in a distributed system involving task migration
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5016Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00Indexing scheme relating to G06F9/00
    • G06F2209/50Indexing scheme relating to G06F9/50
    • G06F2209/503Resource availability

Definitions

  • the present disclosure relates to the field of data processing, for example, to a method and apparatus for quantifying task resources.
  • the performance of data processing equipment (hereinafter referred to as the device) has also been greatly improved.
  • the device can perform more and more tasks, and the number of executable tasks increases the device resources.
  • the utilization situation has attracted much attention.
  • a resource allocation scheme of the related art is to allocate a fixed number of tasks to the device, for example, allocating 12 tasks to the device.
  • Such resource allocation may cause the following two problems.
  • the 12 tasks may occupy a total of If there are few resources on the device, the remaining resources available in the device cannot be utilized, resulting in waste of resources.
  • the 12 tasks may occupy a lot of resources, and the available resources in the entire device are all 12 When the task is full, it will cause the device to run at high load, which will affect the performance of the device.
  • Embodiments of the present disclosure provide a method and apparatus for quantifying task resources, which are used to resolve existing resource points.
  • Embodiments of the present disclosure provide a method for quantifying task resources, including:
  • the task processing device group runs the assigned task, and records the evaluation parameter indicator processed by the task;
  • the theoretical quantity of resources occupied by each task is adjusted according to the preset amplitude, and the task is assigned to the task processing device group; the loop is stopped until the preset condition is met;
  • an optimal quantity of resources occupied by each task is determined; the optimal quantity is used for actual task distribution.
  • An embodiment of the present disclosure provides a device for quantifying task resources, including:
  • a task allocation unit configured to allocate a task according to a task flow sequence according to a task processing device group composed of the at least one task processing device according to a theoretical quantity of resources occupied by each task and a total resource amount of the single task processing device;
  • a recording unit configured to record an evaluation parameter indicator processed by the task when the task processing device group runs the assigned task
  • the adjusting unit is configured to adjust the theoretical quantity of resources occupied by each task according to the preset amplitude, and return to the task allocation unit; until the preset condition is met, the loop is stopped;
  • the determining unit is configured to determine an optimal amount of resources occupied by each task based on the optimal evaluation parameter indicator; the optimal amount is used for actual task distribution.
  • Embodiments of the present disclosure provide a non-transitory computer readable storage medium storing computer executable instructions that, when executed by an electronic device, cause the electronic device to perform the quantization method of the task resource described above.
  • Embodiments of the present disclosure provide a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions, when the program instructions are executed by a computer And causing the computer to perform the above-described quantification method of the task resource.
  • An embodiment of the present disclosure provides an electronic device including at least one processor and a memory communicatively coupled to the at least one processor, the memory for storing instructions executable by the at least one processor, the instructions being The at least one processor, when executed, causes the at least one processor to perform the quantization method of the task resource described above.
  • the method and device for quantifying task resources can continuously adjust the theoretical quantity of resources occupied by tasks, and distribute tasks according to the continuously adjusted theoretical quantity value to the task processing device group, and the task processing device group Run the distributed tasks and record the corresponding evaluation reference indicators to determine the optimal value of the resource value of each task corresponding to the optimal evaluation parameter indicator, so that the resources occupied by the task can be quantified.
  • the plurality of tasks that are distributed occupy the total amount of resources of the task processing device group, so that the waste of resources can be reduced, and the resources of the task processing device can be reasonably utilized; It can prevent the task from occupying all the resources of the task processing device, causing the device to run at a high load and ensuring the performance of the device.
  • Embodiment 1 is a flow chart of the steps of Embodiment 1 of a method for quantifying task resources according to the present disclosure
  • FIG. 2 is a schematic diagram of a broken line corresponding to an evaluation parameter indicator in an application example of an embodiment of the present disclosure
  • FIG. 3 is a flow chart of steps of a second embodiment of a method for quantifying task resources according to the present disclosure
  • Embodiment 4 is a schematic structural diagram of Embodiment 1 of a device for quantifying task resources according to the present disclosure
  • FIG. 5 is a schematic diagram of a hardware structure of an electronic device for quantifying task resources according to the present disclosure.
  • a flow chart of a first embodiment of a method for quantifying a task resource according to the present disclosure may be performed, and may include:
  • Step 101 According to a theoretical quantity of resources occupied by each task and a total resource quantity of a single task processing device, assign a task to the task processing device group composed of the at least one task processing device according to the task flow sequence;
  • the task type may include: transcoding, uploading, downloading, storing, etc.
  • the quantification method of the task resource may be used to quantify a type of task, and the one type of task may include multiple tasks. That is, the embodiments of the present disclosure may be applied to tasks such as transcoding, uploading, downloading, and storing of video and/or audio.
  • the embodiments of the present disclosure are mainly described by using transcoding as an example, and other task types may be cross-referenced. .
  • the foregoing resource type may include one or more of a memory, a CPU (Central Processing Unit), an I/O port (Input/Output), and a bandwidth.
  • a memory a central processing Unit
  • I/O port Input/Output
  • a bandwidth a bandwidth that is used by the resource type.
  • the embodiment mainly uses memory as an example, and other resource types can be referred to each other.
  • the theoretical quantity of resources occupied by each task can be occupied by each task.
  • the parameters such as the code rate and the number of CPU cores are calculated.
  • the process of calculating the theoretical value of the occupied memory by the code rate may include: Scencecut*Width*Height*3, where the above Width*Height refers to the resolution, and each pixel occupies the word.
  • the theoretical quantity value of each task occupying resources calculated by using the parameters such as the code rate and the number of CPU cores occupied by each task is only an example of a method for determining the theoretical quantity of the resource occupied by the task in the embodiment of the present disclosure. It is not understood as a limitation on the method for determining the theoretical quantity of the resource occupied by the task in the embodiment of the present disclosure. In fact, the embodiment of the present disclosure may also determine the optimal among the multiple running result data in the actual collection.
  • the resource size occupied by each task corresponding to the running result data is the theoretical quantity of the resource occupied by the task; the method of determining the theoretical quantity of the resource occupied by the task is not limited herein.
  • the task processing device group composed of the at least one task processing device may be assigned according to the task flow sequence.
  • the task is assigned to a single task processing device in the task processing device group according to the task flow sequence.
  • the total resource amount of all tasks assigned to the current task processing device is greater than or equal to the total resource amount of the current task processing device,
  • the percentage is preset, stop assigning tasks to the current task processing device and start assigning tasks to the next task processing device.
  • the occupied resources are memory
  • the theoretical quantities of the above five transcoding tasks occupy memory.
  • the values are A1 G, A2 G, A3 G, A4 G, and A5 G.
  • the current task processing device group consists of three task processing devices.
  • the total memory of each task processing device is X G, according to each transcoding.
  • the theoretical quantity of the resource occupied by the task and the total resource quantity of the single task processing device are the task processing device group composed of the at least one task processing device, and the process of assigning the task according to the task flow sequence is:
  • the task processing device 1 allocates A1 G memory for T1 and A3 G memory for T3.
  • T2 allocates the memory of A2G, and assumes that the ratio of the sum of the current A1, A3, and A2 to X is greater than or equal to the preset percentage W, then stops assigning tasks to the task processing device 1; continues to allocate A4G to the T4 in the task processing device 2.
  • Memory allocate A5 G memory for T5, allocate A2 G memory for T2, allocate A1 G memory for T1, assuming that the ratio of the current A4, A5, A2, A1 and X to the preset percentage W is stopped.
  • the task processing device 2 is assigned a task; in the task processing device 3, A4 G memory is allocated for T4, A3 G memory is allocated for T3, and A5 G memory is allocated for T5.
  • Step 102 The task processing device group runs the assigned task, and records the evaluation parameter indicator of the task processing.
  • the evaluation parameter indicator of the task processing may be used to evaluate the performance of the current task processing device group during operation, that is, when the evaluation parameter index is large, the performance of the current task processing device group is better.
  • the evaluation parameter indicator of the task processing may include: a throughput of the task processing device group, and/or a processing speed of the task processing device group,
  • the evaluation parameter index is taken as an example for example, and the parameter parameter is the cross-reference of the processing speed;
  • the task type is the evaluation parameter index of the task processing when uploading, downloading, and storing.
  • the method may include: an evaluation parameter indicator corresponding to the upload task type: an upload speed (file size/upload time); an evaluation parameter indicator corresponding to the download task type: download speed (file size/download time); upload (finished product storage) task type Corresponding evaluation parameter indicator: upload speed (finished product size / upload time).
  • Step 103 Adjust the theoretical quantity of resources occupied by each task according to the preset amplitude, and return to assign tasks to the task processing device group; stop the loop until the preset condition is met;
  • adjusting the theoretical quantity of resources occupied by each task according to the preset amplitude may include: increasing or decreasing the theoretical quantity of resources occupied by each task by a preset amplitude; occupying resources for each task
  • the process returns to step 101, that is, according to the adjusted theoretical quantity value, the task processing device group assigns tasks according to the task flow order.
  • the theoretical quantities of occupied resources are A1 and A2 respectively.
  • the preset amplitude is t
  • the theoretical quantities of resources occupied by tasks T and T2 are adjusted according to the preset amplitude.
  • the theoretical value of T occupied resources as A1 + t
  • adjust the theoretical value of T2 occupied resources as A2+t (or adjust the theoretical value of T occupied resources to A1-t, adjust the theoretical value of T2 occupied resources) A2-t)
  • the theoretical quantity of resources occupied by tasks T and T2 (A1+t, A2+t or A1-t, A2-t) and the total resource amount of a single task processing device, for the at least one task Processing a task processing device group composed of devices, and assigning tasks according to task flow order;
  • the task processing device group After the task processing device group runs the assigned task, after recording the evaluation parameter indicators processed by the task, the theoretical value of the resource occupied by the task T and T2 is continuously adjusted according to the preset amplitude, which may be: adjusting the theoretical value of the occupied resource of T A1+2 ⁇ t, the theoretical value of adjusting the resource occupied by T2 is A2+2 ⁇ t (or the theoretical value of adjusting the resource occupied by T is A1-2 ⁇ t, and the theoretical value of adjusting the resource occupied by T2 is A2-2 ⁇ t), according to the theoretical quantity of resources occupied by tasks T and T2 (A1+2 ⁇ t, A2+2 ⁇ t or A1-2 ⁇ t, A2-2 ⁇ t) and the total resources of a single task processing device,
  • the task processing device group composed of the at least one task processing device allocates tasks according to the task flow order; stops the loop until the preset condition is met, that is, stops adjusting the theoretical quantity of the resource occupied by the task when the preset condition is met.
  • the foregoing preset conditions may include:
  • the task processing device group running time meets a preset running time
  • the number of adjustments to the theoretical magnitude of the resources occupied by each task satisfies the preset number of adjustments.
  • the preset running time is three months
  • the theoretical quantity of the resource occupied by the task may be stopped; or, the preset running times are assumed.
  • the theoretical value of the resource occupied by the task can be stopped; or, if the preset adjustment number is 30 times, then When the number of adjustments of the theoretical amount of resources occupied by the task reaches 30 times, the theoretical value of the resource occupied by the task can be stopped.
  • the preset running time is three months, the preset running times are 100 times, and the preset adjustment times are 30 times, which are only used as the preset running time, the preset running times, and the preset adjustment times in the embodiment of the present disclosure.
  • An example is not to be construed as limiting the preset running time, the preset running times, and the preset adjustment times in the embodiment of the present disclosure.
  • the preset running time, the preset running times, and the preset adjustment times may be The technical personnel in the field determine the self-determination according to the business requirements.
  • the preset operating time, the preset running times, and the preset adjustment times are not limited herein.
  • Step 104 Determine an optimal quantity of resources occupied by each task based on an optimal evaluation parameter indicator; the optimal quantity is used for actual task distribution.
  • each time the task processing device group runs the evaluation parameter indicator of the task processing is recorded until the theoretical quantity of the task occupation resource is stopped, and the evaluation parameter indicator of the task processing recorded at this time includes multiple Data, from the plurality of data, selecting an optimal evaluation parameter indicator, and determining a theoretical quantity of resources occupied by each task corresponding to the evaluation parameter indicator as an optimal quantity of resources occupied by each task;
  • the task is a transcoding task for the video.
  • tasks T1, T2, and T3 exist, and the theoretical values of the corresponding task memory are 1G, 2G, and 2G, respectively, according to tasks T1, T2, and T3.
  • the theoretical amount of memory usage 1G, 2G, and 2G and the total memory of a single task processing device are task processing device groups, tasks are assigned according to task flow order, task processing device group runs assigned tasks, and task evaluation parameters are recorded.
  • A1 (in this example, the parameter is evaluated as throughput) is 1000 hours;
  • the theoretical values of the occupied memory of tasks T1, T2 and T3 are adjusted. Assuming the preset amplitude is 0.2G, the theoretical values of the occupied memory of the adjusted tasks T1, T2 and T3 are 1.2G, 2.2G and respectively. 2.2G, according to the theoretical values of the memory occupied by tasks T1, T2 and T3, 1.2G, 2.2G and 2.2G and the total memory of a single task processing device, for the task processing device group, assign tasks according to the task flow sequence, task processing device group Run the assigned task, record the throughput of the task processing a2 is 1300 hours;
  • T1 The theoretical values of the occupied memory of tasks T1, T2 and T3 are adjusted, and the theoretical values of the occupied memories of the adjusted tasks T1, T2 and T3 are respectively 1.4G, 2.4G and 2.4G, according to tasks T1 and T2.
  • T3 occupy the theoretical value of memory 1.4G, 2.4G and 2.4G and the total memory of a single task processing device, for the task processing device group, assign tasks according to the task flow sequence, task processing device group runs the assigned tasks, record tasks The throughput a3 of processing is 1800 hours;
  • the theoretical values of the occupied memory of tasks T1, T2 and T3 are adjusted, and the theoretical quantities of the occupied memories of the adjusted tasks T1, T2 and T3 are 1.6G, 2.6G and 2.6G, respectively, according to tasks T1 and T2.
  • T3 occupy the theoretical value of memory 1.6G, 2.6G and 2.6G and the total memory of a single task processing device, for the task processing device group, assign tasks according to the task flow sequence, task processing device group runs the assigned tasks, record tasks
  • the processing throughput a4 is 1400 hours. If the preset adjustment times are 3 times, the number of adjustments to the theoretical amount of memory occupied by each task satisfies the preset adjustment number, and the theoretical value of the task occupied memory is stopped. ;
  • FIG. 2 a schematic diagram of a broken line corresponding to an evaluation parameter indicator in an application example of an embodiment of the present disclosure is shown; it can be seen that among the evaluation parameter indicators a1 to a4 of the recorded task processing, the optimal evaluation parameter indicator is a3, a3 corresponding to The theoretical values of the occupied memory of T1, T2 and T3 are 1.4G, 2.4G and 2.4G, respectively. It is determined that the optimal values of the occupied memory of T1, T2 and T3 are 1.4G, 2.4G and 2.4G respectively, and after determining the optimal magnitude of the resources occupied by T1, T2 and T3, the task can be processed according to the above optimal amount.
  • the group performs the distribution of tasks.
  • the resource in the above example is only one example of the task type in the embodiment of the present disclosure.
  • the task type may further include one or more of a CPU, an I/O, and a bandwidth.
  • the example does not limit the type of resource.
  • the embodiment of the present disclosure is described based on the case where the resource occupied by each task is a fixed value. In fact, the embodiment of the present disclosure is also applicable to the case where the resource occupied by each task is a non-fixed value, and the process can be referred to each other. The embodiments of the present disclosure will not be described again.
  • a method for quantifying task resources may adjust a theoretical quantity of resources occupied by each task, and distribute tasks according to the continuously adjusted theoretical quantity value, and the task processing is performed by the task processing device group.
  • the device group runs the tasks distributed, and records the corresponding evaluation reference indicators, and determines the theoretical value of each resource occupied by the optimal evaluation parameter indicator as the optimal value, so that each task occupies
  • the resources are known. Therefore, when the tasks are distributed for the task processing device group, the distributed tasks occupy the total amount of resources of the task processing device, so that the waste of resources can be reduced, and the resources of the task processing device can be reasonably utilized.
  • the embodiment of the present disclosure can avoid the problem that the task fills up all the resources of the task processing device, causing the device to operate at a high load, and thus can ensure the performance of the device.
  • a flow chart of the steps of the second embodiment of the method for quantifying the task resource of the present disclosure may be included, which may include:
  • Step 301 According to a theoretical quantity of resources occupied by each task and a total resource quantity of a single task processing device, a task processing device group composed of the at least one task processing device, according to a task flow sequence Assignments;
  • Step 302 The task processing device group runs the assigned task, records the evaluation parameter indicator of the task processing, and the processing speed of each task;
  • Step 303 Determine that the processing speed is less than the processing speed empirical value of the task is a task to be adjusted
  • the actual running result data may be collected, and the running result data may include the processing speed of each task, and the optimal processing speed of each task is determined as the processing speed empirical value of each task. .
  • the processing speed of the recorded task when the processing speed of the recorded task is less than the processing speed empirical value of the task, indicating that the resource allocated for the current task is unreasonable, the theoretical quantity of the resource occupied by the task should be adjusted;
  • the processing speed of the recorded task is greater than or equal to the processing speed empirical value of the task, it indicates that the resource allocated for the current task is reasonable, and the theoretical quantity of the resource occupied may not be adjusted.
  • Step 304 Adjust a theoretical quantity value of each of the resources to be adjusted for the task to be adjusted according to a preset amplitude, and return a task for the task processing device group; stop the loop until the preset condition is met;
  • Step 305 Determine an optimal quantity of resources occupied by each task based on the optimal evaluation parameter indicator; the optimal quantity is used for actual task distribution.
  • step 303 and step 304 are performed.
  • the adjustment of the theoretical quantity of resources occupied by each task according to the preset amplitude is refined, so that when the theoretical value of the resource occupied by the task is adjusted, only the task that needs to be adjusted by the theoretical value can be adjusted. It is not necessary to adjust the theoretical magnitude of the resources occupied by all tasks.
  • FIG. 4 is a schematic structural diagram of Embodiment 1 of a device for quantifying a task resource according to the present disclosure, which may include: a task allocating unit 401, a recording unit 402, an adjusting unit 403, and a determining unit 404;
  • the task allocation unit 401 may be configured to allocate, according to the theoretical quantity of resources occupied by each task and the total resource amount of the single task processing device, the task processing device group composed of the at least one task processing device according to the task flow order. task;
  • the recording unit 402 may be configured to record an evaluation parameter indicator of the task processing when the task processing device group runs the assigned task;
  • the adjusting unit 403 may be configured to adjust the theoretical quantity of resources occupied by each task according to the preset amplitude, and return to the task allocation unit; and stop the loop until the preset condition is met;
  • the determining unit 404 may be configured to determine an optimal magnitude of resources occupied by each task based on the optimal evaluation parameter indicator; the optimal amount is used for actual task distribution.
  • the task type may include: transcoding, uploading, downloading, and storing, and the quantifying method of the task resource is used to quantize a type of task, in the one type of task. Contains a variety of tasks.
  • the preset condition may include:
  • the task processing device group running time meets a preset running time
  • the number of adjustments to the theoretical magnitude of the resources occupied by each task satisfies the preset number of adjustments.
  • the task type is transcoding
  • the evaluation parameter indicator of the task processing may include: a throughput of the task processing device group, and/or the task processing device The processing speed of the group.
  • the resource may include one or more of a memory, a CPU, an IO, and a bandwidth.
  • the recording unit 402 may be further configured to record the processing speed of each task
  • the adjusting unit 403 may include:
  • the theoretical quantity subunit is adjusted to be configured to adjust the theoretical quantity of resources occupied by each of the to-be-adjusted tasks according to a preset amplitude.
  • the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment.
  • the embodiment of the present application provides a non-volatile computer storage medium, where the computer storage medium stores computer executable instructions for performing any of the embodiments of the present application.
  • a quantitative method of task resources is provided.
  • FIG. 5 is a schematic diagram of a hardware structure of an electronic device for quantifying a task resource according to the present disclosure. As shown in FIG. 5, the electronic device includes:
  • processors 510 and memory 520 one processor 510 is taken as an example in FIG.
  • the apparatus for performing the quantization method of the task resource may further include: an input device 530 and an output device 540.
  • the processor 510, the memory 520, the input device 530, and the output device 540 may be connected by a bus or other means, as exemplified by a bus connection in FIG.
  • the memory 520 is a non-volatile computer readable storage medium, and is applicable to a non-volatile software program, a non-volatile computer-executable program, and a module, such as a method for quantifying task resources in the embodiment of the present application.
  • Program instructions/modules for example, task assignment unit 401, recording unit 402, adjustment unit 403, and determination unit 404 shown in FIG. 4).
  • the processor 510 executes various functional applications and data processing of the server by running non-volatile software programs, instructions, and modules stored in the memory 520, that is, a method for quantifying the task resources of the foregoing method embodiments.
  • the memory 520 may include a storage program area and an storage data area, wherein the storage program area may store an operating system, an application required for at least one function; the storage data area may store data created by use of a quantization device of the task resource, and the like. Further, the memory 520 may include a high speed random access memory, and may also include a nonvolatile memory such as at least one magnetic disk storage device, flash memory device, or other nonvolatile solid state storage device. In some embodiments, memory 520 can optionally include memory remotely located relative to processor 510, which can be connected to a quantification device of task resources over a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
  • Input device 530 can receive input numeric or character information and generate key signal inputs related to user settings and function control of the quantization device of the task resource.
  • Output device 540 can include a display Such as display devices.
  • the one or more modules are stored in the memory 520, and when executed by the one or more processors 510, perform a quantization method of task resources in any of the above method embodiments.
  • the electronic device of the embodiment of the present application exists in various forms, including but not limited to:
  • Mobile communication devices These devices are characterized by mobile communication functions and are mainly aimed at providing voice and data communication.
  • Such terminals include: smart phones (such as iPhone), multimedia phones, functional phones, and low-end phones.
  • Ultra-mobile personal computer equipment This type of equipment belongs to the category of personal computers, has computing and processing functions, and generally has mobile Internet access.
  • Such terminals include: PDAs, MIDs, and UMPC devices, such as the iPad.
  • Portable entertainment devices These devices can display and play multimedia content. Such devices include: audio, video players (such as iPod), handheld game consoles, e-books, and smart toys and portable car navigation devices.
  • the server consists of a processor, a hard disk, a memory, a system bus, etc.
  • the server is similar to a general-purpose computer architecture, but because of the need to provide highly reliable services, processing power and stability High reliability in terms of reliability, security, scalability, and manageability.
  • the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may be or It may not be a physical unit, that is, it may be located in one place, or it may be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of the embodiment. Those of ordinary skill in the art can understand and implement without deliberate labor.
  • the present invention solves the problem of waste of resources in the existing resource allocation and the problem that affects the performance of the device, can reasonably utilize resources, and can ensure the performance of the device.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Debugging And Monitoring (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

A method and device for quantifying task resources. The method comprises: assigning, according to a theoretical quantity value of resources occupied by each task and a total resource quantity of a single task processing device, a task to a task process device group consisting of at least one task processing device on the basis of a task chronological order (101); running, by the task processing device group, the assigned task, and recording an evaluation parameter index of task processing (102); adjusting the theoretical quantity value of resources occupied by each task according to a preset amplitude, returning to the step of assigning the task to the task processing device group, and stopping circulation until a preset condition is satisfied (103); and determining, on the basis of an optimum evaluation parameter index, an optimum quantity value of resources occupied by each task, wherein the optimum quantity value is used for actual task distribution (104). The method allows rational utilization of resources and can ensure the performance of a device.

Description

一种任务资源的量化方法和装置Method and device for quantifying task resources
本申请要求在2016年03月31日提交中国专利局、申请号为2016102016291、公开名称为“一种任务资源的量化方法和装置”的中国专利申请的优先权,该申请的全部内容通过引用结合在本申请中。This application claims the priority of the Chinese Patent Application entitled "Quantification Method and Apparatus for a Task Resource", filed on March 31, 2016 by the Chinese Patent Office, with the application number 2016102016291, the entire contents of which are incorporated by reference. In this application.
技术领域Technical field
本公开涉及数据处理领域,例如涉及一种任务资源的量化方法和装置。The present disclosure relates to the field of data processing, for example, to a method and apparatus for quantifying task resources.
背景技术Background technique
随着网络技术的飞速发展,数据处理设备(以下简称设备)的性能也有了大幅度的提升,由于设备性能提升,因此设备可以执行越来越多的任务,可执行任务的增多就使得设备资源的利用情况备受人们关注。With the rapid development of network technology, the performance of data processing equipment (hereinafter referred to as the device) has also been greatly improved. As the performance of the device is improved, the device can perform more and more tasks, and the number of executable tasks increases the device resources. The utilization situation has attracted much attention.
相关技术的一种资源分配方案是为设备分配固定数量的任务,例如:为设备分配12个任务,这样的资源分配会造成以下两方面的问题,一种情况,这12个任务可能总共占用了设备很少的资源,则该设备中剩余大量可用资源无法被利用,造成了资源的浪费;另一种情况,这12个任务可能占用的资源很多,整个设备中的可用资源全被这12个任务占满,则会使得设备高负荷运行,进而会对设备的性能造成影响。A resource allocation scheme of the related art is to allocate a fixed number of tasks to the device, for example, allocating 12 tasks to the device. Such resource allocation may cause the following two problems. In one case, the 12 tasks may occupy a total of If there are few resources on the device, the remaining resources available in the device cannot be utilized, resulting in waste of resources. In another case, the 12 tasks may occupy a lot of resources, and the available resources in the entire device are all 12 When the task is full, it will cause the device to run at high load, which will affect the performance of the device.
因此,现在厄需一种对任务的资源进行量化的方法,以对任务的资源占用情况进行量化,以使得可以根据量化结果对设备进行任务的分配,进而使得设备的资源得到合理利用的同时,能够保证设备的性能。Therefore, there is a need for a method for quantifying resources of a task to quantify the resource occupancy of the task, so that the task can be assigned according to the quantization result, so that the resources of the device are properly utilized. Can guarantee the performance of the device.
发明内容Summary of the invention
本公开实施例提供一种任务资源的量化方法和装置,用以解决现有资源分 配中资源的浪费问题及对设备性能造成影响的问题,以能够合理利用资源,且能够保证设备的性能。Embodiments of the present disclosure provide a method and apparatus for quantifying task resources, which are used to resolve existing resource points. The problem of wasting resources and affecting the performance of the equipment, so that resources can be rationally utilized and the performance of the equipment can be guaranteed.
本公开实施例提供一种任务资源的量化方法,包括:Embodiments of the present disclosure provide a method for quantifying task resources, including:
根据每种任务占用资源的理论量值及单个任务处理设备的总资源量,为所述至少一个任务处理设备组成的任务处理设备群,按照任务流水顺序分配任务;And assigning tasks to the task processing device group formed by the at least one task processing device according to the task flow order according to the theoretical quantity of resources occupied by each task and the total resource amount of the single task processing device;
所述任务处理设备群运行所分配的任务,记录任务处理的评估参数指标;The task processing device group runs the assigned task, and records the evaluation parameter indicator processed by the task;
按照预设幅度对每种任务占用资源的理论量值进行调整,返回为任务处理设备群分配任务;直至满足预设条件时停止循环;The theoretical quantity of resources occupied by each task is adjusted according to the preset amplitude, and the task is assigned to the task processing device group; the loop is stopped until the preset condition is met;
基于最优的评估参数指标,确定每种任务占用资源的最优量值;所述最优量值用于实际任务分发。Based on the optimal evaluation parameter indicator, an optimal quantity of resources occupied by each task is determined; the optimal quantity is used for actual task distribution.
本公开实施例提供一种任务资源的量化装置,包括:An embodiment of the present disclosure provides a device for quantifying task resources, including:
任务分配单元,配置为根据每种任务占用资源的理论量值及单个任务处理设备的总资源量,为所述至少一个任务处理设备组成的任务处理设备群,按照任务流水顺序分配任务;a task allocation unit configured to allocate a task according to a task flow sequence according to a task processing device group composed of the at least one task processing device according to a theoretical quantity of resources occupied by each task and a total resource amount of the single task processing device;
记录单元,配置为在所述任务处理设备群运行所分配的任务时,记录任务处理的评估参数指标;a recording unit configured to record an evaluation parameter indicator processed by the task when the task processing device group runs the assigned task;
调整单元,配置为按照预设幅度对每种任务占用资源的理论量值进行调整,返回任务分配单元;直至满足预设条件时停止循环;The adjusting unit is configured to adjust the theoretical quantity of resources occupied by each task according to the preset amplitude, and return to the task allocation unit; until the preset condition is met, the loop is stopped;
确定单元,配置为基于最优的评估参数指标,确定每种任务占用资源的最优量值;所述最优量值用于实际任务分发。The determining unit is configured to determine an optimal amount of resources occupied by each task based on the optimal evaluation parameter indicator; the optimal amount is used for actual task distribution.
本公开实施例提供一种非暂态计算机可读存储介质,存储有计算机可执行指令,所述计算机可执行指令被电子设备执行时,使得所述电子设备执行上述的任务资源的量化方法。 Embodiments of the present disclosure provide a non-transitory computer readable storage medium storing computer executable instructions that, when executed by an electronic device, cause the electronic device to perform the quantization method of the task resource described above.
本公开实施例提供一种计算机程序产品,所述计算机程序产品包括存储在非易失性计算机可读存储介质上的计算机程序,所述计算机程序包括程序指令,当所述程序指令被计算机执行时,使所述计算机执行上述的任务资源的量化方法。Embodiments of the present disclosure provide a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions, when the program instructions are executed by a computer And causing the computer to perform the above-described quantification method of the task resource.
本公开实施例提供一种电子设备,包括至少一个处理器和与所述至少一个处理器通信连接的存储器,所述存储器用于存储可被所述至少一个处理器执行的指令,所述指令被所述至少一个处理器执行时,使所述至少一个处理器执行上述的任务资源的量化方法。An embodiment of the present disclosure provides an electronic device including at least one processor and a memory communicatively coupled to the at least one processor, the memory for storing instructions executable by the at least one processor, the instructions being The at least one processor, when executed, causes the at least one processor to perform the quantization method of the task resource described above.
本公开实施例提供的一种任务资源的量化方法及装置,可以通过对任务占用资源的理论量值进行不断调整,并根据不断调整的理论量值为任务处理设备群分发任务,任务处理设备群运行所分发的任务,并记录对应的评估参考指标,确定最优的评估参数指标对应的每种任务占用资源的理论量值为最优量值,这样一来,由于任务占用的资源是可以量化的,因此在为任务处理设备群分发任务时,分发的多个任务占用任务处理设备群的资源总量即可知,因此可以减少资源的浪费,使得任务处理设备的资源得到合理的利用;同时,可以避免任务占满任务处理设备的全部资源,造成设备高负荷运行的问题,能够保证设备的性能。The method and device for quantifying task resources provided by the embodiments of the present disclosure can continuously adjust the theoretical quantity of resources occupied by tasks, and distribute tasks according to the continuously adjusted theoretical quantity value to the task processing device group, and the task processing device group Run the distributed tasks and record the corresponding evaluation reference indicators to determine the optimal value of the resource value of each task corresponding to the optimal evaluation parameter indicator, so that the resources occupied by the task can be quantified. Therefore, when the tasks are distributed for the task processing device group, the plurality of tasks that are distributed occupy the total amount of resources of the task processing device group, so that the waste of resources can be reduced, and the resources of the task processing device can be reasonably utilized; It can prevent the task from occupying all the resources of the task processing device, causing the device to run at a high load and ensuring the performance of the device.
附图概述BRIEF abstract
一个或多个实施例通过与之对应的附图中的图片进行示例性说明,这些示例性说明并不构成对实施例的限定,附图中具有相同参考数字标号的元件表示为类似的元件,除非有特别申明,附图中的图不构成比例限制。The one or more embodiments are exemplified by the accompanying drawings in the accompanying drawings, and FIG. The figures in the drawings do not constitute a scale limitation unless otherwise stated.
图1为本公开的一种任务资源的量化方法实施例一的步骤流程图;1 is a flow chart of the steps of Embodiment 1 of a method for quantifying task resources according to the present disclosure;
图2为本公开实施例的应用示例中评估参数指标对应的折线示意图;2 is a schematic diagram of a broken line corresponding to an evaluation parameter indicator in an application example of an embodiment of the present disclosure;
图3为本公开的一种任务资源的量化方法实施例二的步骤流程图;3 is a flow chart of steps of a second embodiment of a method for quantifying task resources according to the present disclosure;
图4为本公开的一种任务资源的量化装置实施例一的结构示意图;以及 4 is a schematic structural diagram of Embodiment 1 of a device for quantifying task resources according to the present disclosure;
图5为本公开的一种任务资源的量化的电子设备的硬件结构示意图。FIG. 5 is a schematic diagram of a hardware structure of an electronic device for quantifying task resources according to the present disclosure.
本发明的实施方式Embodiments of the invention
为使本公开实施例的技术方案和优点更加清楚,下面将结合本公开实施例中的附图,对本公开实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例是本公开一部分实施例,而不是全部的实施例。基于本公开中的实施例,本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例,都属于本公开保护的范围。The technical solutions in the embodiments of the present disclosure will be clearly and completely described in conjunction with the drawings in the embodiments of the present disclosure. Some embodiments are disclosed, rather than all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present disclosure without departing from the inventive scope are the scope of the disclosure.
方法实施例一 Method embodiment 1
参照图1,示出了本公开的一种任务资源的量化方法实施例一的步骤流程图,可以包括:Referring to FIG. 1 , a flow chart of a first embodiment of a method for quantifying a task resource according to the present disclosure may be performed, and may include:
步骤101、根据每种任务占用资源的理论量值及单个任务处理设备的总资源量,为所述至少一个任务处理设备组成的任务处理设备群,按照任务流水顺序分配任务;Step 101: According to a theoretical quantity of resources occupied by each task and a total resource quantity of a single task processing device, assign a task to the task processing device group composed of the at least one task processing device according to the task flow sequence;
本公开实施例中,任务类型可以包括:转码、上传、下载、存储等,所述任务资源的量化方法可以用于对一类任务进行量化,所述一类任务中可以包含多种任务,也即,本公开实施例可以应用于视频、和/或音频的转码、上传、下载、存储等任务过程中,本公开实施例主要以转码为例进行说明,其他任务类型相互参照即可。In the embodiment of the present disclosure, the task type may include: transcoding, uploading, downloading, storing, etc., the quantification method of the task resource may be used to quantify a type of task, and the one type of task may include multiple tasks. That is, the embodiments of the present disclosure may be applied to tasks such as transcoding, uploading, downloading, and storing of video and/or audio. The embodiments of the present disclosure are mainly described by using transcoding as an example, and other task types may be cross-referenced. .
本公开实施例中,上述资源类型可以包括:内存、CPU(中央处理器,Central Processing Unit)、I/O端口(输入/输出,Input/Output)、带宽中的一种或多种,本公开实施例主要以内存为例进行说明,其他资源类型相互参照即可。In the embodiment of the present disclosure, the foregoing resource type may include one or more of a memory, a CPU (Central Processing Unit), an I/O port (Input/Output), and a bandwidth. The present disclosure The embodiment mainly uses memory as an example, and other resource types can be referred to each other.
本公开实施例中,每种任务占用资源的理论量值可以通过每种任务所占用 的码率、CPU核数等参数计算出来,例如:通过码率计算占用内存的理论量值过程可以包括:Scencecut*Width*Height*3,其中上述Width*Height指分辨率,每个像素占用字节3B,例如:mp4_350码率的分辨率是:640*350,那么其占用内存的理论量值为:60*640*350*3=40320000=38.45M。In the embodiment of the present disclosure, the theoretical quantity of resources occupied by each task can be occupied by each task. The parameters such as the code rate and the number of CPU cores are calculated. For example, the process of calculating the theoretical value of the occupied memory by the code rate may include: Scencecut*Width*Height*3, where the above Width*Height refers to the resolution, and each pixel occupies the word. Section 3B, for example: mp4_350 code rate resolution is: 640 * 350, then its theoretical value of memory usage: 60 * 640 * 350 * 3 = 403200000 = 38.45M.
可以理解,上述通过每种任务所占用的码率、CPU核数等参数计算每种任务占用资源的理论量值仅作为本公开实施例中确定任务占用资源的理论量值的方法的一种示例,而不理解为是对本公开实施例中确定任务占用资源的理论量值的方法的一种限制,实际上,本公开实施例还可以通过采集实际中的多次运行结果数据,确定其中最优的运行结果数据对应的每种任务占用的资源大小为任务占用资源的理论量值;本公开实施例在此对确定任务占用资源的理论量值的方法不做限定。It can be understood that the theoretical quantity value of each task occupying resources calculated by using the parameters such as the code rate and the number of CPU cores occupied by each task is only an example of a method for determining the theoretical quantity of the resource occupied by the task in the embodiment of the present disclosure. It is not understood as a limitation on the method for determining the theoretical quantity of the resource occupied by the task in the embodiment of the present disclosure. In fact, the embodiment of the present disclosure may also determine the optimal among the multiple running result data in the actual collection. The resource size occupied by each task corresponding to the running result data is the theoretical quantity of the resource occupied by the task; the method of determining the theoretical quantity of the resource occupied by the task is not limited herein.
本公开实施例中,根据每种任务占用资源的理论量值及单个任务处理设备的总资源量,为所述至少一个任务处理设备组成的任务处理设备群,按照任务流水顺序分配任务的过程可以为:根据任务流水顺序依次为任务处理设备群中的单个任务处理设备分配任务,当分配给当前任务处理设备的所有任务的总资源量在当前任务处理设备的总资源量中的占比大于等于预置百分比时,停止为当前任务处理设备分配任务,开始为下一个任务处理设备分配任务。In the embodiment of the present disclosure, according to the theoretical quantity of resources occupied by each task and the total resource amount of the single task processing device, the task processing device group composed of the at least one task processing device may be assigned according to the task flow sequence. The task is assigned to a single task processing device in the task processing device group according to the task flow sequence. When the total resource amount of all tasks assigned to the current task processing device is greater than or equal to the total resource amount of the current task processing device, When the percentage is preset, stop assigning tasks to the current task processing device and start assigning tasks to the next task processing device.
在本公开实施例的一种应用示例中,假设当前有五种转码任务,分别为T1、T2、T3、T4和T5,占用的资源为内存,上述五种转码任务占用内存的理论量值分别为A1 G、A2 G、A3 G、A4 G和A5 G,当前的任务处理设备群由3个任务处理设备组成,每个任务处理设备的总内存为X G,则根据每种转码任务占用资源的理论量值及单个任务处理设备的总资源量,为所述至少一个任务处理设备组成的任务处理设备群,按照任务流水顺序分配任务的过程为: In an application example of the embodiment of the present disclosure, it is assumed that there are currently five types of transcoding tasks, namely T1, T2, T3, T4, and T5, and the occupied resources are memory, and the theoretical quantities of the above five transcoding tasks occupy memory. The values are A1 G, A2 G, A3 G, A4 G, and A5 G. The current task processing device group consists of three task processing devices. The total memory of each task processing device is X G, according to each transcoding. The theoretical quantity of the resource occupied by the task and the total resource quantity of the single task processing device are the task processing device group composed of the at least one task processing device, and the process of assigning the task according to the task flow sequence is:
假设任务流水顺序为:T1、T3、T2、T4、T5、T2、T1、T4、T3、T5,则在任务处理设备1中为T1分配A1 G的内存,为T3分配A3 G的内存,为T2分配A2 G的内存,假设当前A1、A3、A2的和与X的比值大于等于预置百分比W,则停止为任务处理设备1分配任务;继续在任务处理设备2中为T4分配A4 G的内存,为T5分配A5 G的内存,为T2分配A2 G的内存,为T1分配A1 G的内存,假设当前A4、A5、A2、A1的和与X的比值大于等于预置百分比W,则停止为任务处理设备2分配任务;在任务处理设备3中为T4分配A4 G的内存,为T3分配A3 G的内存,为T5分配A5 G的内存。Assuming that the task flow order is: T1, T3, T2, T4, T5, T2, T1, T4, T3, T5, the task processing device 1 allocates A1 G memory for T1 and A3 G memory for T3. T2 allocates the memory of A2G, and assumes that the ratio of the sum of the current A1, A3, and A2 to X is greater than or equal to the preset percentage W, then stops assigning tasks to the task processing device 1; continues to allocate A4G to the T4 in the task processing device 2. Memory, allocate A5 G memory for T5, allocate A2 G memory for T2, allocate A1 G memory for T1, assuming that the ratio of the current A4, A5, A2, A1 and X to the preset percentage W is stopped. The task processing device 2 is assigned a task; in the task processing device 3, A4 G memory is allocated for T4, A3 G memory is allocated for T3, and A5 G memory is allocated for T5.
步骤102、所述任务处理设备群运行所分配的任务,记录任务处理的评估参数指标。Step 102: The task processing device group runs the assigned task, and records the evaluation parameter indicator of the task processing.
本公开实施例中,上述任务处理的评估参数指标可以用于评估当前的任务处理设备群运行时的性能,也即评估参数指标大的时候,表示当前任务处理设备群的性能较好。In the embodiment of the present disclosure, the evaluation parameter indicator of the task processing may be used to evaluate the performance of the current task processing device group during operation, that is, when the evaluation parameter index is large, the performance of the current task processing device group is better.
本公开实施例中,所述任务类型为转码时,所述任务处理的评估参数指标可以包括:所述任务处理设备群的吞吐量、和/或所述任务处理设备群的处理速度,本公开实施例中主要以评估参数指标为吞吐量为例进行说明,评估参数指标为处理速度的情况相互参照即可;所述任务类型为上传、下载、存储时,所述任务处理的评估参数指标可以包括:上传任务类型对应的评估参数指标:上传速度(文件大小/上传耗时);下载任务类型对应的评估参数指标:下载速度(文件大小/下载耗时);上传(成品存储)任务类型对应的评估参数指标:上传速度(成品大小/上传耗时)。In the embodiment of the present disclosure, when the task type is transcoding, the evaluation parameter indicator of the task processing may include: a throughput of the task processing device group, and/or a processing speed of the task processing device group, In the disclosed embodiment, the evaluation parameter index is taken as an example for example, and the parameter parameter is the cross-reference of the processing speed; the task type is the evaluation parameter index of the task processing when uploading, downloading, and storing. The method may include: an evaluation parameter indicator corresponding to the upload task type: an upload speed (file size/upload time); an evaluation parameter indicator corresponding to the download task type: download speed (file size/download time); upload (finished product storage) task type Corresponding evaluation parameter indicator: upload speed (finished product size / upload time).
步骤103、按照预设幅度对每种任务占用资源的理论量值进行调整,返回为任务处理设备群分配任务;直至满足预设条件时停止循环; Step 103: Adjust the theoretical quantity of resources occupied by each task according to the preset amplitude, and return to assign tasks to the task processing device group; stop the loop until the preset condition is met;
本公开实施例中,按照预设幅度对每种任务占用资源的理论量值进行调整可以包括:将每种任务占用资源的理论量值按预设幅度增加或者减少;在对每种任务占用资源的理论量值进行调整后,返回步骤101,也即根据调整后的理论量值为任务处理设备群按照任务流水顺序分配任务。In the embodiment of the present disclosure, adjusting the theoretical quantity of resources occupied by each task according to the preset amplitude may include: increasing or decreasing the theoretical quantity of resources occupied by each task by a preset amplitude; occupying resources for each task After the theoretical quantity is adjusted, the process returns to step 101, that is, according to the adjusted theoretical quantity value, the task processing device group assigns tasks according to the task flow order.
例如:当前存在任务T和T2,占用资源的理论量值分别为A1和A2,假设预设幅度为t,则按照预设幅度对任务T和T2占用资源的理论量值进行调整,调整过程可以为:调整T占用资源的理论量值为A1+t,调整T2占用资源的理论量值为A2+t(或者调整T占用资源的理论量值为A1-t,调整T2占用资源的理论量值为A2-t);根据任务T和T2占用资源的理论量值(A1+t、A2+t或者A1-t、A2-t)及单个任务处理设备的总资源量,为所述至少一个任务处理设备组成的任务处理设备群,按照任务流水顺序分配任务;For example, there are currently tasks T and T2, and the theoretical quantities of occupied resources are A1 and A2 respectively. If the preset amplitude is t, the theoretical quantities of resources occupied by tasks T and T2 are adjusted according to the preset amplitude. To: adjust the theoretical value of T occupied resources as A1 + t, adjust the theoretical value of T2 occupied resources as A2+t (or adjust the theoretical value of T occupied resources to A1-t, adjust the theoretical value of T2 occupied resources) A2-t); the theoretical quantity of resources occupied by tasks T and T2 (A1+t, A2+t or A1-t, A2-t) and the total resource amount of a single task processing device, for the at least one task Processing a task processing device group composed of devices, and assigning tasks according to task flow order;
任务处理设备群运行所分配的任务,记录任务处理的评估参数指标后,继续按照预设幅度对任务T和T2占用资源的理论量值进行调整,可以为:调整T占用资源的理论量值为A1+2×t,调整T2占用资源的理论量值为A2+2×t(或者调整T占用资源的理论量值为A1-2×t,调整T2占用资源的理论量值为A2-2×t),根据任务T和T2占用资源的理论量值(A1+2×t、A2+2×t或者A1-2×t、A2-2×t)及单个任务处理设备的总资源量,为所述至少一个任务处理设备组成的任务处理设备群,按照任务流水顺序分配任务;直至满足预设条件时停止循环,也即当满足预设条件时停止对任务占用资源的理论量值进行调整。After the task processing device group runs the assigned task, after recording the evaluation parameter indicators processed by the task, the theoretical value of the resource occupied by the task T and T2 is continuously adjusted according to the preset amplitude, which may be: adjusting the theoretical value of the occupied resource of T A1+2×t, the theoretical value of adjusting the resource occupied by T2 is A2+2×t (or the theoretical value of adjusting the resource occupied by T is A1-2×t, and the theoretical value of adjusting the resource occupied by T2 is A2-2× t), according to the theoretical quantity of resources occupied by tasks T and T2 (A1+2×t, A2+2×t or A1-2×t, A2-2×t) and the total resources of a single task processing device, The task processing device group composed of the at least one task processing device allocates tasks according to the task flow order; stops the loop until the preset condition is met, that is, stops adjusting the theoretical quantity of the resource occupied by the task when the preset condition is met.
在本公开的一种可选实施例中,上述预设条件可以包括:In an optional embodiment of the disclosure, the foregoing preset conditions may include:
所述任务处理设备群运行时间满足预置运行时间;或者,The task processing device group running time meets a preset running time; or
所述任务处理设备群运行次数满足预置运行次数;或者,The number of times the task processing device group runs meets the preset running times; or
对每种任务占用资源的理论量值的调整的次数满足预置调整次数。 The number of adjustments to the theoretical magnitude of the resources occupied by each task satisfies the preset number of adjustments.
本公开实施例中,假设预置运行时间为三个月,则当任务处理设备群运行时间满足三个月,即可停止对任务占用资源的理论量值进行调整;或者,假设预置运行次数为100次,则当任务处理设备群运行所分配的任务的次数达到100次,即可停止对任务占用资源的理论量值进行调整;或者,假设预置调整次数为30次,则当对每种任务占用资源的理论量值的调整的次数达到30次时,即可停止对任务占用资源的理论量值进行调整。In the embodiment of the present disclosure, if the preset running time is three months, when the task processing device group running time meets three months, the theoretical quantity of the resource occupied by the task may be stopped; or, the preset running times are assumed. For 100 times, when the number of tasks assigned to the task processing device group reaches 100 times, the theoretical value of the resource occupied by the task can be stopped; or, if the preset adjustment number is 30 times, then When the number of adjustments of the theoretical amount of resources occupied by the task reaches 30 times, the theoretical value of the resource occupied by the task can be stopped.
可以理解,上述预置运行时间为三个月、预置运行次数为100次及预置调整次数为30次仅作为本公开实施例中预置运行时间、预置运行次数及预置调整次数的一个示例,而不理解为是对本公开实施例中预置运行时间、预置运行次数及预置调整次数的限制,实际上,上述预置运行时间、预置运行次数及预置调整次数可以由本领域技术人员根据业务需求自行确定,本公开实施例在此对预置运行时间、预置运行次数及预置调整次数不做限定。It can be understood that the preset running time is three months, the preset running times are 100 times, and the preset adjustment times are 30 times, which are only used as the preset running time, the preset running times, and the preset adjustment times in the embodiment of the present disclosure. An example is not to be construed as limiting the preset running time, the preset running times, and the preset adjustment times in the embodiment of the present disclosure. In fact, the preset running time, the preset running times, and the preset adjustment times may be The technical personnel in the field determine the self-determination according to the business requirements. The preset operating time, the preset running times, and the preset adjustment times are not limited herein.
步骤104、基于最优的评估参数指标,确定每种任务占用资源的最优量值;所述最优量值用于实际任务分发。Step 104: Determine an optimal quantity of resources occupied by each task based on an optimal evaluation parameter indicator; the optimal quantity is used for actual task distribution.
本公开实施例中,任务处理设备群每运行一次,即记录一次任务处理的评估参数指标,直至停止对任务占用资源的理论量值进行调整,此时记录的任务处理的评估参数指标包含多个数据,从上述多个数据中,选取最优的评估参数指标,并确定该评估参数指标对应的每种任务占用资源的理论量值为每种任务占用资源的最优量值;In the embodiment of the present disclosure, each time the task processing device group runs, the evaluation parameter indicator of the task processing is recorded until the theoretical quantity of the task occupation resource is stopped, and the evaluation parameter indicator of the task processing recorded at this time includes multiple Data, from the plurality of data, selecting an optimal evaluation parameter indicator, and determining a theoretical quantity of resources occupied by each task corresponding to the evaluation parameter indicator as an optimal quantity of resources occupied by each task;
为使本领域技术人员更好的理解本公开实施例,以下通过示例对本公开实施例加以说明:In order to make those skilled in the art better understand the embodiments of the present disclosure, the embodiments of the present disclosure are explained by way of example:
假设资源为内存,任务为对视频的转码任务,当前存在任务T1、T2及T3,对应的任务内存占用的理论量值分别为1G、2G及2G,根据任务T1、T2及T3 占用内存的理论量值1G、2G及2G及单个任务处理设备的总内存,为任务处理设备群,按照任务流水顺序分配任务,任务处理设备群运行所分配的任务,记录任务处理的评估参数指标a1(本示例中评估参数指标为吞吐量)为1000小时;Assuming that the resource is memory, the task is a transcoding task for the video. Currently, tasks T1, T2, and T3 exist, and the theoretical values of the corresponding task memory are 1G, 2G, and 2G, respectively, according to tasks T1, T2, and T3. The theoretical amount of memory usage 1G, 2G, and 2G and the total memory of a single task processing device are task processing device groups, tasks are assigned according to task flow order, task processing device group runs assigned tasks, and task evaluation parameters are recorded. A1 (in this example, the parameter is evaluated as throughput) is 1000 hours;
对任务T1、T2及T3的占用内存的理论量值进行调整,假设预设幅度为0.2G,则调整后的任务T1、T2及T3的占用内存的理论量值分别为1.2G、2.2G及2.2G,根据任务T1、T2及T3占用内存的理论量值1.2G、2.2G及2.2G及单个任务处理设备的总内存,为任务处理设备群,按照任务流水顺序分配任务,任务处理设备群运行所分配的任务,记录任务处理的吞吐量a2为1300小时;The theoretical values of the occupied memory of tasks T1, T2 and T3 are adjusted. Assuming the preset amplitude is 0.2G, the theoretical values of the occupied memory of the adjusted tasks T1, T2 and T3 are 1.2G, 2.2G and respectively. 2.2G, according to the theoretical values of the memory occupied by tasks T1, T2 and T3, 1.2G, 2.2G and 2.2G and the total memory of a single task processing device, for the task processing device group, assign tasks according to the task flow sequence, task processing device group Run the assigned task, record the throughput of the task processing a2 is 1300 hours;
对任务T1、T2及T3的占用内存的理论量值进行调整,则调整后的任务T1、T2及T3的占用内存的理论量值分别为1.4G、2.4G及2.4G,根据任务T1、T2及T3占用内存的理论量值1.4G、2.4G及2.4G及单个任务处理设备的总内存,为任务处理设备群,按照任务流水顺序分配任务,任务处理设备群运行所分配的任务,记录任务处理的吞吐量a3为1800小时;The theoretical values of the occupied memory of tasks T1, T2 and T3 are adjusted, and the theoretical values of the occupied memories of the adjusted tasks T1, T2 and T3 are respectively 1.4G, 2.4G and 2.4G, according to tasks T1 and T2. And T3 occupy the theoretical value of memory 1.4G, 2.4G and 2.4G and the total memory of a single task processing device, for the task processing device group, assign tasks according to the task flow sequence, task processing device group runs the assigned tasks, record tasks The throughput a3 of processing is 1800 hours;
对任务T1、T2及T3的占用内存的理论量值进行调整,则调整后的任务T1、T2及T3的占用内存的理论量值分别为1.6G、2.6G及2.6G,根据任务T1、T2及T3占用内存的理论量值1.6G、2.6G及2.6G及单个任务处理设备的总内存,为任务处理设备群,按照任务流水顺序分配任务,任务处理设备群运行所分配的任务,记录任务处理的吞吐量a4为1400小时,假设预置调整次数为3次,则对每种任务占用内存的理论量值的调整的次数满足预置调整次数,停止对任务占用内存的理论量值进行调整;The theoretical values of the occupied memory of tasks T1, T2 and T3 are adjusted, and the theoretical quantities of the occupied memories of the adjusted tasks T1, T2 and T3 are 1.6G, 2.6G and 2.6G, respectively, according to tasks T1 and T2. And T3 occupy the theoretical value of memory 1.6G, 2.6G and 2.6G and the total memory of a single task processing device, for the task processing device group, assign tasks according to the task flow sequence, task processing device group runs the assigned tasks, record tasks The processing throughput a4 is 1400 hours. If the preset adjustment times are 3 times, the number of adjustments to the theoretical amount of memory occupied by each task satisfies the preset adjustment number, and the theoretical value of the task occupied memory is stopped. ;
参照图2,示出了本公开实施例的应用示例中评估参数指标对应的折线示意图;可见,记录的任务处理的评估参数指标a1至a4中,最优的评估参数指标为a3,a3对应的T1、T2及T3的占用内存的理论量值分别为1.4G、2.4G及2.4G, 确定T1、T2及T3占用内存的最优量值分别为1.4G、2.4G及2.4G,确定T1、T2及T3占用资源的最优量值后,可以根据上述最优量值为任务处理设备群进行任务的分发。Referring to FIG. 2, a schematic diagram of a broken line corresponding to an evaluation parameter indicator in an application example of an embodiment of the present disclosure is shown; it can be seen that among the evaluation parameter indicators a1 to a4 of the recorded task processing, the optimal evaluation parameter indicator is a3, a3 corresponding to The theoretical values of the occupied memory of T1, T2 and T3 are 1.4G, 2.4G and 2.4G, respectively. It is determined that the optimal values of the occupied memory of T1, T2 and T3 are 1.4G, 2.4G and 2.4G respectively, and after determining the optimal magnitude of the resources occupied by T1, T2 and T3, the task can be processed according to the above optimal amount. The group performs the distribution of tasks.
可以理解,上述示例中资源为内存仅作为本公开实施例中任务类型的一种示例,实际上,任务类型还可以包括:CPU、I/O以及带宽中的一种或多种,本公开实施例对资源的类型不做限定。It can be understood that the resource in the above example is only one example of the task type in the embodiment of the present disclosure. In fact, the task type may further include one or more of a CPU, an I/O, and a bandwidth. The example does not limit the type of resource.
本公开实施例是基于每种任务所占资源为固定值的情况下加以说明的,实际上,本公开实施例对于每种任务所占资源为非固定值的情况同样适用,过程相互参照即可,本公开实施例对此不再赘述。The embodiment of the present disclosure is described based on the case where the resource occupied by each task is a fixed value. In fact, the embodiment of the present disclosure is also applicable to the case where the resource occupied by each task is a non-fixed value, and the process can be referred to each other. The embodiments of the present disclosure will not be described again.
综上,本公开实施例提供的一种任务资源的量化方法,可以通过对每种任务占用资源的理论量值进行调整,并根据不断调整的理论量值为任务处理设备群分发任务,任务处理设备群运行所分发的任务,并记录对应的评估参考指标,确定最优的评估参数指标对应的每种任务占用资源的理论量值为最优量值,这样一来,由于每种任务占用的资源是已知的,因此在为任务处理设备群分发任务时,分发的多个任务占用任务处理设备的资源总量即可知,因此可以减少资源的浪费,使得任务处理设备的资源得到合理的利用;同时,本公开实施例可以避免任务占满任务处理设备的全部资源,造成设备高负荷运行的问题,因此能够保证设备的性能。In summary, a method for quantifying task resources according to an embodiment of the present disclosure may adjust a theoretical quantity of resources occupied by each task, and distribute tasks according to the continuously adjusted theoretical quantity value, and the task processing is performed by the task processing device group. The device group runs the tasks distributed, and records the corresponding evaluation reference indicators, and determines the theoretical value of each resource occupied by the optimal evaluation parameter indicator as the optimal value, so that each task occupies The resources are known. Therefore, when the tasks are distributed for the task processing device group, the distributed tasks occupy the total amount of resources of the task processing device, so that the waste of resources can be reduced, and the resources of the task processing device can be reasonably utilized. At the same time, the embodiment of the present disclosure can avoid the problem that the task fills up all the resources of the task processing device, causing the device to operate at a high load, and thus can ensure the performance of the device.
方法实施例二Method embodiment two
参照图3,示出了本公开的一种任务资源的量化方法实施例二的步骤流程图,可以包括:Referring to FIG. 3, a flow chart of the steps of the second embodiment of the method for quantifying the task resource of the present disclosure may be included, which may include:
步骤301、根据每种任务占用资源的理论量值及单个任务处理设备的总资源量,为所述至少一个任务处理设备组成的任务处理设备群,按照任务流水顺序 分配任务;Step 301: According to a theoretical quantity of resources occupied by each task and a total resource quantity of a single task processing device, a task processing device group composed of the at least one task processing device, according to a task flow sequence Assignments;
步骤302、所述任务处理设备群运行所分配的任务,记录任务处理的评估参数指标及每种任务的处理速度;Step 302: The task processing device group runs the assigned task, records the evaluation parameter indicator of the task processing, and the processing speed of each task;
步骤303、确定所述处理速度小于任务的处理速度经验量值的任务为待调整任务;Step 303: Determine that the processing speed is less than the processing speed empirical value of the task is a task to be adjusted;
本公开实施例中,可以通过采集实际中的多次运行结果数据,运行结果数据中可以包含每种任务的处理速度,确定每种任务最优的处理速度为每种任务的处理速度经验量值。In the embodiment of the present disclosure, the actual running result data may be collected, and the running result data may include the processing speed of each task, and the optimal processing speed of each task is determined as the processing speed empirical value of each task. .
本公开实施例中,当记录的任务的处理速度小于任务的处理速度经验量值时,说明为当前的任务分配的资源是不合理的,则应当对该任务占用资源的理论量值进行调整;当记录的任务的处理速度大于或者等于任务的处理速度经验量值时,说明为当前的任务分配的资源是合理的,可以不对其占用资源的理论量值进行调整。In the embodiment of the present disclosure, when the processing speed of the recorded task is less than the processing speed empirical value of the task, indicating that the resource allocated for the current task is unreasonable, the theoretical quantity of the resource occupied by the task should be adjusted; When the processing speed of the recorded task is greater than or equal to the processing speed empirical value of the task, it indicates that the resource allocated for the current task is reasonable, and the theoretical quantity of the resource occupied may not be adjusted.
步骤304、按照预设幅度对每种所述待调整任务占用资源的理论量值进行调整,返回为任务处理设备群分配任务;直至满足预设条件时停止循环;Step 304: Adjust a theoretical quantity value of each of the resources to be adjusted for the task to be adjusted according to a preset amplitude, and return a task for the task processing device group; stop the loop until the preset condition is met;
本公开的一种应用示例中,假设当前存在任务T1、T2和T3,处理速度经验量值分别为30、35和40,按照任务T1、T2和T3占用资源的理论量值为任务处理设备群,按照任务流水顺序分配任务;任务处理设备群运行所分配的任务,记录的任务的处理速度分别为35、20和15,可见任务T2和T3的处理速度小于其处理速度经验量值,因此在对任务占用资源的理论量值进行调整时,可以仅对任务T2和T3进行调整即可。In an application example of the present disclosure, it is assumed that there are currently tasks T1, T2, and T3, and the processing speed empirical values are 30, 35, and 40, respectively, and the theoretical quantity of resources occupied according to tasks T1, T2, and T3 is a task processing device group. The tasks are assigned according to the task flow sequence; the task processing device group runs the assigned tasks, and the recorded tasks are processed at speeds of 35, 20, and 15, respectively. It can be seen that the processing speeds of the tasks T2 and T3 are less than the processing speed empirical values, so When the theoretical quantity of the resource occupied by the task is adjusted, only the tasks T2 and T3 can be adjusted.
步骤305、基于最优的评估参数指标,确定每种任务占用资源的最优量值;所述最优量值用于实际任务分发。 Step 305: Determine an optimal quantity of resources occupied by each task based on the optimal evaluation parameter indicator; the optimal quantity is used for actual task distribution.
相对于方法实施例一,本公开实施例在所对述任务处理设备群运行所分配的任务,记录任务处理的评估参数指标时,还记录每种任务的处理速度,并通过步骤303及步骤304对按照预设幅度对每种任务占用资源的理论量值进行调整进行了细化,以使得在调整任务占用资源的理论量值时,可以只对需要进行理论量值调整的任务进行调整,而不必对所有任务占用资源的理论量值都进行调整。With respect to the first embodiment of the method, in the embodiment of the present disclosure, when the task assigned to the task processing device group is run, and the evaluation parameter indicator of the task processing is recorded, the processing speed of each task is also recorded, and step 303 and step 304 are performed. The adjustment of the theoretical quantity of resources occupied by each task according to the preset amplitude is refined, so that when the theoretical value of the resource occupied by the task is adjusted, only the task that needs to be adjusted by the theoretical value can be adjusted. It is not necessary to adjust the theoretical magnitude of the resources occupied by all tasks.
对于方法实施例,为了简单描述,故将其都表述为一系列的动作组合,但是本领域技术人员应该知悉,本申请实施例并不受所描述的动作顺序的限制,因为依据本申请实施例,某些步骤可以采用其他顺序或者同时进行。其次,本领域技术人员也应该知悉,说明书中所描述的均为实施例,所涉及的动作并不一定是本申请所必须的。For the method embodiments, for the sake of brevity, they are all described as a series of action combinations, but those skilled in the art should understand that the embodiments of the present application are not limited by the described action sequence, because the embodiment according to the present application Some steps can be performed in other orders or at the same time. In the following, those skilled in the art should also understand that the descriptions in the specification are all examples, and the actions involved are not necessarily required by the present application.
装置实施例一 Device embodiment 1
参照图4,示出了本公开的一种任务资源的量化装置实施例一的结构示意图,可以包括:任务分配单元401、记录单元402、调整单元403及确定单元404;FIG. 4 is a schematic structural diagram of Embodiment 1 of a device for quantifying a task resource according to the present disclosure, which may include: a task allocating unit 401, a recording unit 402, an adjusting unit 403, and a determining unit 404;
其中,任务分配单元401,可以配置为根据每种任务占用资源的理论量值及单个任务处理设备的总资源量,为所述至少一个任务处理设备组成的任务处理设备群,按照任务流水顺序分配任务;The task allocation unit 401 may be configured to allocate, according to the theoretical quantity of resources occupied by each task and the total resource amount of the single task processing device, the task processing device group composed of the at least one task processing device according to the task flow order. task;
记录单元402,可以配置为在所述任务处理设备群运行所分配的任务时,记录任务处理的评估参数指标;The recording unit 402 may be configured to record an evaluation parameter indicator of the task processing when the task processing device group runs the assigned task;
调整单元403,可以配置为按照预设幅度对每种任务占用资源的理论量值进行调整,返回任务分配单元;直至满足预设条件时停止循环;及The adjusting unit 403 may be configured to adjust the theoretical quantity of resources occupied by each task according to the preset amplitude, and return to the task allocation unit; and stop the loop until the preset condition is met;
确定单元404,可以配置为基于最优的评估参数指标,确定每种任务占用资源的最优量值;所述最优量值用于实际任务分发。 The determining unit 404 may be configured to determine an optimal magnitude of resources occupied by each task based on the optimal evaluation parameter indicator; the optimal amount is used for actual task distribution.
在本公开的一种可选实施例中,所述任务类型可以包括:转码、上传、下载、存储,所述任务资源的量化方法用于对一类任务进行量化,所述一类任务中包含多种任务。In an optional embodiment of the present disclosure, the task type may include: transcoding, uploading, downloading, and storing, and the quantifying method of the task resource is used to quantize a type of task, in the one type of task. Contains a variety of tasks.
在本公开的一种可选实施例中,所述预设条件可以包括:In an optional embodiment of the disclosure, the preset condition may include:
所述任务处理设备群运行时间满足预置运行时间;或者,The task processing device group running time meets a preset running time; or
所述任务处理设备群运行次数满足预置运行次数;或者,The number of times the task processing device group runs meets the preset running times; or
对每种任务占用资源的理论量值的调整次数满足预置调整次数。The number of adjustments to the theoretical magnitude of the resources occupied by each task satisfies the preset number of adjustments.
在本公开的一种可选实施例中,所述任务类型为转码,则所述任务处理的评估参数指标可以包括:所述任务处理设备群的吞吐量、和/或所述任务处理设备群的处理速度。In an optional embodiment of the present disclosure, the task type is transcoding, and the evaluation parameter indicator of the task processing may include: a throughput of the task processing device group, and/or the task processing device The processing speed of the group.
在本公开的一种可选实施例中,所述资源可以包括:内存、CPU、IO、带宽中的一种或多种。In an optional embodiment of the disclosure, the resource may include one or more of a memory, a CPU, an IO, and a bandwidth.
在本公开的一种可选实施例中,所述记录单元402还可以配置为记录每种任务的处理速度;In an optional embodiment of the present disclosure, the recording unit 402 may be further configured to record the processing speed of each task;
则所述调整单元403,可以包括:The adjusting unit 403 may include:
确定待调整任务子单元,可以配置为确定所述处理速度小于任务的处理速度经验量值的任务为待调整任务;Determining the task subunit to be adjusted, and the task that is configured to determine that the processing speed is less than the processing speed empirical value of the task is a task to be adjusted;
调整理论量值子单元,可以配置为按照预设幅度对每种所述待调整任务占用资源的理论量值进行调整。The theoretical quantity subunit is adjusted to be configured to adjust the theoretical quantity of resources occupied by each of the to-be-adjusted tasks according to a preset amplitude.
对于装置实施例而言,由于其与方法实施例基本相似,所以描述的比较简单,相关之处参见方法实施例的部分说明即可。For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment.
本申请实施例提供了一种非易失性计算机存储介质,所述计算机存储介质存储有计算机可执行指令,该计算机可执行指令用于执行本申请实施例中任一 的任务资源的量化方法。The embodiment of the present application provides a non-volatile computer storage medium, where the computer storage medium stores computer executable instructions for performing any of the embodiments of the present application. A quantitative method of task resources.
图5是本公开的一种任务资源的量化的电子设备的硬件结构示意图,如图5所示,该电子设备包括:FIG. 5 is a schematic diagram of a hardware structure of an electronic device for quantifying a task resource according to the present disclosure. As shown in FIG. 5, the electronic device includes:
一个或多个处理器510以及存储器520,图5中以一个处理器510为例。One or more processors 510 and memory 520, one processor 510 is taken as an example in FIG.
执行任务资源的量化方法的设备还可以包括:输入装置530和输出装置540。The apparatus for performing the quantization method of the task resource may further include: an input device 530 and an output device 540.
处理器510、存储器520、输入装置530和输出装置540可以通过总线或者其他方式连接,图5中以通过总线连接为例。The processor 510, the memory 520, the input device 530, and the output device 540 may be connected by a bus or other means, as exemplified by a bus connection in FIG.
存储器520作为一种非易失性计算机可读存储介质,可用于存储非易失性软件程序、非易失性计算机可执行程序以及模块,如本申请实施例中的任务资源的量化方法对应的程序指令/模块(例如,附图4所示的任务分配单元401、记录单元402、调整单元403及确定单元404)。处理器510通过运行存储在存储器520中的非易失性软件程序、指令以及模块,从而执行服务器的各种功能应用以及数据处理,即实现上述方法实施例任务资源的量化方法。The memory 520 is a non-volatile computer readable storage medium, and is applicable to a non-volatile software program, a non-volatile computer-executable program, and a module, such as a method for quantifying task resources in the embodiment of the present application. Program instructions/modules (for example, task assignment unit 401, recording unit 402, adjustment unit 403, and determination unit 404 shown in FIG. 4). The processor 510 executes various functional applications and data processing of the server by running non-volatile software programs, instructions, and modules stored in the memory 520, that is, a method for quantifying the task resources of the foregoing method embodiments.
存储器520可以包括存储程序区和存储数据区,其中,存储程序区可存储操作系统、至少一个功能所需要的应用程序;存储数据区可存储根据任务资源的量化装置的使用所创建的数据等。此外,存储器520可以包括高速随机存取存储器,还可以包括非易失性存储器,例如至少一个磁盘存储器件、闪存器件、或其他非易失性固态存储器件。在一些实施例中,存储器520可选包括相对于处理器510远程设置的存储器,这些远程存储器可以通过网络连接至任务资源的量化装置。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。The memory 520 may include a storage program area and an storage data area, wherein the storage program area may store an operating system, an application required for at least one function; the storage data area may store data created by use of a quantization device of the task resource, and the like. Further, the memory 520 may include a high speed random access memory, and may also include a nonvolatile memory such as at least one magnetic disk storage device, flash memory device, or other nonvolatile solid state storage device. In some embodiments, memory 520 can optionally include memory remotely located relative to processor 510, which can be connected to a quantification device of task resources over a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
输入装置530可接收输入的数字或字符信息,以及产生与任务资源的量化装置的用户设置以及功能控制有关的键信号输入。输出装置540可包括显示屏 等显示设备。 Input device 530 can receive input numeric or character information and generate key signal inputs related to user settings and function control of the quantization device of the task resource. Output device 540 can include a display Such as display devices.
所述一个或者多个模块存储在所述存储器520中,当被所述一个或者多个处理器510执行时,执行上述任意方法实施例中的任务资源的量化方法。The one or more modules are stored in the memory 520, and when executed by the one or more processors 510, perform a quantization method of task resources in any of the above method embodiments.
上述产品可执行本申请实施例所提供的方法,具备执行方法相应的功能模块和有益效果。未在本实施例中详尽描述的技术细节,可参见本申请实施例所提供的方法。The above products can perform the methods provided by the embodiments of the present application, and have the corresponding functional modules and beneficial effects of the execution method. For technical details that are not described in detail in this embodiment, reference may be made to the method provided by the embodiments of the present application.
本申请实施例的电子设备以多种形式存在,包括但不限于:The electronic device of the embodiment of the present application exists in various forms, including but not limited to:
(1)移动通信设备:这类设备的特点是具备移动通信功能,并且以提供话音、数据通信为主要目标。这类终端包括:智能手机(例如iPhone)、多媒体手机、功能性手机,以及低端手机等。(1) Mobile communication devices: These devices are characterized by mobile communication functions and are mainly aimed at providing voice and data communication. Such terminals include: smart phones (such as iPhone), multimedia phones, functional phones, and low-end phones.
(2)超移动个人计算机设备:这类设备属于个人计算机的范畴,有计算和处理功能,一般也具备移动上网特性。这类终端包括:PDA、MID和UMPC设备等,例如iPad。(2) Ultra-mobile personal computer equipment: This type of equipment belongs to the category of personal computers, has computing and processing functions, and generally has mobile Internet access. Such terminals include: PDAs, MIDs, and UMPC devices, such as the iPad.
(3)便携式娱乐设备:这类设备可以显示和播放多媒体内容。该类设备包括:音频、视频播放器(例如iPod),掌上游戏机,电子书,以及智能玩具和便携式车载导航设备。(3) Portable entertainment devices: These devices can display and play multimedia content. Such devices include: audio, video players (such as iPod), handheld game consoles, e-books, and smart toys and portable car navigation devices.
(4)服务器:提供计算服务的设备,服务器的构成包括处理器、硬盘、内存、系统总线等,服务器和通用的计算机架构类似,但是由于需要提供高可靠的服务,因此在处理能力、稳定性、可靠性、安全性、可扩展性、可管理性等方面要求较高。(4) Server: A device that provides computing services. The server consists of a processor, a hard disk, a memory, a system bus, etc. The server is similar to a general-purpose computer architecture, but because of the need to provide highly reliable services, processing power and stability High reliability in terms of reliability, security, scalability, and manageability.
(5)其他具有数据交互功能的电子设备。(5) Other electronic devices with data interaction functions.
以上所描述的装置实施例仅仅是示意性的,其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是或者也 可以不是物理单元,即可以位于一个地方,或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案。本领域普通技术人员在不付出创造性的劳动的情况下,即可以理解并实施。The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may be or It may not be a physical unit, that is, it may be located in one place, or it may be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of the embodiment. Those of ordinary skill in the art can understand and implement without deliberate labor.
通过以上的实施方式的描述,本领域的技术人员可以清楚地了解到各实施方式可借助软件加通用硬件平台的方式来实现,当然也可以通过硬件。基于这样的理解,上述技术方案本质上或者说对相关技术做出贡献的部分可以以软件产品的形式体现出来,该计算机软件产品可以存储在计算机可读存储介质中,如ROM/RAM、磁碟、光盘等,包括若干指令用以使得一台计算机设备(可以是个人计算机,服务器,或者网络设备等)执行各个实施例或者实施例的某些部分所述的方法。Through the description of the above embodiments, those skilled in the art can clearly understand that the various embodiments can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on such understanding, the above technical solutions may be embodied in the form of software products in essence or in the form of software products, which may be stored in a computer readable storage medium such as a ROM/RAM or a disk. , an optical disk, etc., includes instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to perform the methods described in various embodiments or portions of the embodiments.
以上实施例仅用以说明本公开的技术方案,而非对其限制;尽管参照前述实施例对本公开进行了详细的说明,本领域的普通技术人员应当理解:其依然可以对前述各实施例所记载的技术方案进行修改,或者对其中部分技术特征进行等同替换;而这些修改或者替换,并不使相应技术方案的本质脱离本公开各实施例技术方案的范围。The above embodiments are only used to illustrate the technical solutions of the present disclosure, and are not intended to be limiting; although the present disclosure has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that The technical solutions are described as being modified, or equivalent to some of the technical features are replaced; and such modifications or substitutions do not depart from the scope of the technical solutions of the embodiments of the present disclosure.
工业实用性Industrial applicability
本申请解决了现有资源分配中资源的浪费问题及对设备性能造成影响的问题,能够合理利用资源,且能够保证设备的性能。 The present invention solves the problem of waste of resources in the existing resource allocation and the problem that affects the performance of the device, can reasonably utilize resources, and can ensure the performance of the device.

Claims (15)

  1. 一种任务资源的量化方法,应用于电子设备,所述方法包括:A method for quantifying task resources, applied to an electronic device, the method comprising:
    根据每种任务占用资源的理论量值及单个任务处理设备的总资源量,为所述至少一个任务处理设备组成的任务处理设备群,按照任务流水顺序分配任务;And assigning tasks to the task processing device group formed by the at least one task processing device according to the task flow order according to the theoretical quantity of resources occupied by each task and the total resource amount of the single task processing device;
    所述任务处理设备群运行所分配的任务,记录任务处理的评估参数指标;The task processing device group runs the assigned task, and records the evaluation parameter indicator processed by the task;
    按照预设幅度对每种任务占用资源的理论量值进行调整,返回为任务处理设备群分配任务;直至满足预设条件时停止循环;以及Adjusting the theoretical amount of resources occupied by each task according to the preset amplitude, returning to assign tasks to the task processing device group; stopping the loop until the preset condition is met;
    基于最优的评估参数指标,确定每种任务占用资源的最优量值;所述最优量值用于实际任务分发。Based on the optimal evaluation parameter indicator, an optimal quantity of resources occupied by each task is determined; the optimal quantity is used for actual task distribution.
  2. 根据权利要求1所述的方法,其中,所述任务类型包括转码、上传、下载、存储,所述任务资源的量化方法用于对一类任务进行量化,所述一类任务中包含多种任务。The method according to claim 1, wherein the task type comprises transcoding, uploading, downloading, and storing, and the quantifying method of the task resource is used for quantifying a type of task, wherein the one type of task includes multiple task.
  3. 根据权利要求1所述的方法,其中,所述预设条件包括:The method of claim 1 wherein said predetermined condition comprises:
    所述任务处理设备群运行时间满足预置运行时间;或者,The task processing device group running time meets a preset running time; or
    所述任务处理设备群运行次数满足预置运行次数;或者,The number of times the task processing device group runs meets the preset running times; or
    对每种任务占用资源的理论量值的调整次数满足预置调整次数。The number of adjustments to the theoretical magnitude of the resources occupied by each task satisfies the preset number of adjustments.
  4. 根据权利要求1所述的方法,其中,所述任务类型为转码,则所述任务处理的评估参数指标包括:所述任务处理设备群的吞吐量、和/或所述任务处理设备群的处理速度。The method according to claim 1, wherein the task type is transcoding, and the evaluation parameter indicator of the task processing comprises: throughput of the task processing device group, and/or the task processing device group Processing speed.
  5. 根据权利要求1所述的方法,其中,所述资源包括:内存、CPU、IO、带宽中的一种或多种。The method of claim 1, wherein the resources comprise one or more of a memory, a CPU, an IO, and a bandwidth.
  6. 根据权利要求1所述的方法,其中,所述任务处理设备群运行所分配的任务,记录任务处理的评估参数指标,还包括: The method according to claim 1, wherein the task processing device group runs the assigned task, records the evaluation parameter indicator of the task processing, and further includes:
    记录每种任务的处理速度;Record the processing speed of each task;
    则所述按照预设幅度对每种任务占用资源的理论量值进行调整的步骤,包括:The step of adjusting the theoretical quantity of resources occupied by each task according to the preset amplitude includes:
    确定所述处理速度小于任务的处理速度经验量值的任务为待调整任务;以及Determining that the processing speed is less than the processing speed empirical value of the task is the task to be adjusted;
    按照预设幅度对每种所述待调整任务占用资源的理论量值进行调整。The theoretical quantity of resources occupied by each of the to-be-adjusted tasks is adjusted according to a preset amplitude.
  7. 一种任务资源的量化装置,所述装置包括:A quantification device for task resources, the device comprising:
    任务分配单元,配置为根据每种任务占用资源的理论量值及单个任务处理设备的总资源量,为所述至少一个任务处理设备组成的任务处理设备群,按照任务流水顺序分配任务;a task allocation unit configured to allocate a task according to a task flow sequence according to a task processing device group composed of the at least one task processing device according to a theoretical quantity of resources occupied by each task and a total resource amount of the single task processing device;
    记录单元,配置为在所述任务处理设备群运行所分配的任务时,记录任务处理的评估参数指标;a recording unit configured to record an evaluation parameter indicator processed by the task when the task processing device group runs the assigned task;
    调整单元,配置为按照预设幅度对每种任务占用资源的理论量值进行调整,返回任务分配单元;直至满足预设条件时停止循环;以及The adjusting unit is configured to adjust the theoretical quantity of resources occupied by each task according to the preset amplitude, and return to the task allocation unit; until the preset condition is met, the loop is stopped;
    确定单元,配置为基于最优的评估参数指标,确定每种任务占用资源的最优量值;所述最优量值用于实际任务分发。The determining unit is configured to determine an optimal amount of resources occupied by each task based on the optimal evaluation parameter indicator; the optimal amount is used for actual task distribution.
  8. 根据权利要求7所述的装置,其中,所述任务类型包括转码、上传、下载、存储,所述任务资源的量化方法用于对一类任务进行量化,所述一类任务中包含多种任务。The apparatus according to claim 7, wherein the task type comprises transcoding, uploading, downloading, and storing, and the quantifying method of the task resource is used for quantifying a type of task, wherein the one type of task includes multiple task.
  9. 根据权利要求7所述的装置,其中,所述预设条件包括:The apparatus of claim 7, wherein the predetermined condition comprises:
    所述任务处理设备群运行时间满足预置运行时间;或者,The task processing device group running time meets a preset running time; or
    所述任务处理设备群运行次数满足预置运行次数;或者,The number of times the task processing device group runs meets the preset running times; or
    对每种任务占用资源的理论量值的调整次数满足预置调整次数。 The number of adjustments to the theoretical magnitude of the resources occupied by each task satisfies the preset number of adjustments.
  10. 根据权利要求7所述的装置,其中,所述任务类型为转码,则所述任务处理的评估参数指标包括:所述任务处理设备群的吞吐量、和/或所述任务处理设备群的处理速度。The apparatus according to claim 7, wherein the task type is transcoding, and the evaluation parameter indicator of the task processing comprises: a throughput of the task processing device group, and/or a task processing device group Processing speed.
  11. 根据权利要求7所述的装置,其中,所述资源包括:内存、CPU、IO、带宽中的一种或多种。The apparatus of claim 7, wherein the resources comprise one or more of a memory, a CPU, an IO, and a bandwidth.
  12. 根据权利要求7所述的装置,其中,所述记录单元还配置为记录每种任务的处理速度;The apparatus according to claim 7, wherein said recording unit is further configured to record a processing speed of each task;
    则所述调整单元,包括:Then the adjusting unit comprises:
    确定待调整任务子单元,配置为确定所述处理速度小于任务的处理速度经验量值的任务为待调整任务;以及Determining a task subunit to be adjusted, and the task configured to determine that the processing speed is less than the processing speed empirical value of the task is a task to be adjusted;
    调整理论量值子单元,配置为按照预设幅度对每种所述待调整任务占用资源的理论量值进行调整。The theoretical quantity subunit is adjusted to be configured to adjust the theoretical quantity of resources occupied by each of the to-be-adjusted tasks according to a preset amplitude.
  13. 一种非暂态计算机可读存储介质,存储有计算机可执行指令,所述计算机可执行指令被电子设备执行时,使得所述电子设备执行权利要求1-6任一项所述的方法。A non-transitory computer readable storage medium storing computer executable instructions that, when executed by an electronic device, cause the electronic device to perform the method of any of claims 1-6.
  14. 一种计算机程序产品,所述计算机程序产品包括存储在非易失性计算机可读存储介质上的计算机程序,所述计算机程序包括程序指令,当所述程序指令被计算机执行时,使所述计算机执行权利要求1-6任一项所述的方法。A computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions, when the program instructions are executed by a computer, causing the computer The method of any of claims 1-6.
  15. 一种电子设备,包括至少一个处理器和与所述至少一个处理器通信连接的存储器,所述存储器用于存储可被所述至少一个处理器执行的指令,所述指令被所述至少一个处理器执行时,使所述至少一个处理器执行权利要求1-6任一项所述的方法。 An electronic device comprising at least one processor and a memory communicatively coupled to the at least one processor, the memory for storing instructions executable by the at least one processor, the instructions being processed by the at least one When executed, the at least one processor is caused to perform the method of any of claims 1-6.
PCT/CN2016/096967 2016-03-31 2016-08-26 Method and device for quantifying task resources WO2017166643A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610201629.1 2016-03-31
CN201610201629.1A CN105843687A (en) 2016-03-31 2016-03-31 Method and device for quantifying task resource

Publications (1)

Publication Number Publication Date
WO2017166643A1 true WO2017166643A1 (en) 2017-10-05

Family

ID=56596526

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/096967 WO2017166643A1 (en) 2016-03-31 2016-08-26 Method and device for quantifying task resources

Country Status (2)

Country Link
CN (1) CN105843687A (en)
WO (1) WO2017166643A1 (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110084476A (en) * 2019-03-25 2019-08-02 平安科技(深圳)有限公司 Case method of adjustment, device, computer equipment and storage medium
CN111124644A (en) * 2019-12-19 2020-05-08 北京旷视科技有限公司 Method, device and system for determining task scheduling resources
CN111381960A (en) * 2018-12-29 2020-07-07 中兴通讯股份有限公司 Terminal task processing method and device, storage medium and electronic device
CN111813557A (en) * 2020-07-21 2020-10-23 北京千丁互联科技有限公司 Task processing device, method, terminal device and readable storage medium
CN112685180A (en) * 2020-12-29 2021-04-20 杭州海康威视数字技术股份有限公司 Computing unit distribution method and device and computing unit distribution system
CN115269014A (en) * 2022-09-26 2022-11-01 上海登临科技有限公司 Instruction scheduling method, chip and electronic equipment
CN115756773A (en) * 2022-11-01 2023-03-07 中电金信软件有限公司 Task scheduling method and device, electronic equipment and storage medium
CN116449935A (en) * 2023-06-02 2023-07-18 工业富联(佛山)创新中心有限公司 Cluster energy-saving management method, electronic equipment and computer storage medium

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105843687A (en) * 2016-03-31 2016-08-10 乐视控股(北京)有限公司 Method and device for quantifying task resource
CN106454402B (en) * 2016-08-31 2018-11-23 腾讯科技(深圳)有限公司 Transcoding method for scheduling task and device
CN107220115A (en) * 2017-05-25 2017-09-29 深信服科技股份有限公司 A kind of task bottleneck based on cloud platform determines method and device
CN111031350B (en) * 2019-12-24 2022-04-12 北京奇艺世纪科技有限公司 Transcoding resource scheduling method, electronic device and computer readable storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110225590A1 (en) * 2010-03-15 2011-09-15 Thomson Steven S System and method of executing threads at a processor
CN103903070A (en) * 2014-04-15 2014-07-02 广东电网公司信息中心 Resource demand measuring and calculating system for application system
CN104391749A (en) * 2014-11-26 2015-03-04 北京奇艺世纪科技有限公司 Resource allocation method and device
CN105843687A (en) * 2016-03-31 2016-08-10 乐视控股(北京)有限公司 Method and device for quantifying task resource

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPWO2007072544A1 (en) * 2005-12-20 2009-05-28 富士通株式会社 Information processing apparatus, computer, resource allocation method, and resource allocation program
US8887169B2 (en) * 2010-09-15 2014-11-11 Empire Technology Development Llc Task assignment in cloud computing environment
CN102360313B (en) * 2011-09-29 2014-03-26 中国科学技术大学苏州研究院 Performance acceleration method of heterogeneous multi-core computing platform on chip
CN103729246B (en) * 2013-12-31 2017-05-03 浪潮(北京)电子信息产业有限公司 Method and device for dispatching tasks
CN103986669B (en) * 2014-05-07 2017-04-19 华东师范大学 Assessment method of resource allocation strategy in cloud computing
CN103995749B (en) * 2014-05-21 2017-06-16 广东省电信规划设计院有限公司 The calculation task allocating method and system of cell cloud system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110225590A1 (en) * 2010-03-15 2011-09-15 Thomson Steven S System and method of executing threads at a processor
CN103903070A (en) * 2014-04-15 2014-07-02 广东电网公司信息中心 Resource demand measuring and calculating system for application system
CN104391749A (en) * 2014-11-26 2015-03-04 北京奇艺世纪科技有限公司 Resource allocation method and device
CN105843687A (en) * 2016-03-31 2016-08-10 乐视控股(北京)有限公司 Method and device for quantifying task resource

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111381960A (en) * 2018-12-29 2020-07-07 中兴通讯股份有限公司 Terminal task processing method and device, storage medium and electronic device
CN110084476A (en) * 2019-03-25 2019-08-02 平安科技(深圳)有限公司 Case method of adjustment, device, computer equipment and storage medium
CN111124644B (en) * 2019-12-19 2023-04-04 北京旷视科技有限公司 Method, device and system for determining task scheduling resources
CN111124644A (en) * 2019-12-19 2020-05-08 北京旷视科技有限公司 Method, device and system for determining task scheduling resources
CN111813557A (en) * 2020-07-21 2020-10-23 北京千丁互联科技有限公司 Task processing device, method, terminal device and readable storage medium
CN112685180A (en) * 2020-12-29 2021-04-20 杭州海康威视数字技术股份有限公司 Computing unit distribution method and device and computing unit distribution system
CN112685180B (en) * 2020-12-29 2023-10-10 杭州海康威视数字技术股份有限公司 Computing unit distribution method, computing unit distribution device and computing unit distribution system
CN115269014B (en) * 2022-09-26 2022-12-30 上海登临科技有限公司 Instruction scheduling method, chip and electronic equipment
CN115269014A (en) * 2022-09-26 2022-11-01 上海登临科技有限公司 Instruction scheduling method, chip and electronic equipment
CN115756773A (en) * 2022-11-01 2023-03-07 中电金信软件有限公司 Task scheduling method and device, electronic equipment and storage medium
CN115756773B (en) * 2022-11-01 2023-08-29 中电金信软件有限公司 Task scheduling method, device, electronic equipment and storage medium
CN116449935A (en) * 2023-06-02 2023-07-18 工业富联(佛山)创新中心有限公司 Cluster energy-saving management method, electronic equipment and computer storage medium
CN116449935B (en) * 2023-06-02 2023-11-21 工业富联(佛山)创新中心有限公司 Cluster energy-saving management method, electronic equipment and computer storage medium

Also Published As

Publication number Publication date
CN105843687A (en) 2016-08-10

Similar Documents

Publication Publication Date Title
WO2017166643A1 (en) Method and device for quantifying task resources
CN108776934B (en) Distributed data calculation method and device, computer equipment and readable storage medium
EP3129880B1 (en) Method and device for augmenting and releasing capacity of computing resources in real-time stream computing system
CN107239336B (en) Method and device for realizing task scheduling
US9081623B1 (en) Service resource allocation
US10783002B1 (en) Cost determination of a service call
CN107688492B (en) Resource control method and device and cluster resource management system
US8910128B2 (en) Methods and apparatus for application performance and capacity analysis
CN107295090B (en) Resource scheduling method and device
CN107851039A (en) System and method for resource management
WO2015101091A1 (en) Distributed resource scheduling method and device
JP2015517147A5 (en)
US11861410B2 (en) Cloud computing burst instance management through transfer of cloud computing task portions between resources satisfying burst criteria
US20170185454A1 (en) Method and Electronic Device for Determining Resource Consumption of Task
CN109189572B (en) Resource estimation method and system, electronic equipment and storage medium
CN112148468B (en) Resource scheduling method and device, electronic equipment and storage medium
US9742684B1 (en) Adaptive service scaling
CN108376103A (en) A kind of the equilibrium of stock control method and server of cloud platform
CN110599148A (en) Cluster data processing method and device, computer cluster and readable storage medium
CN105630604A (en) SLA based multi-tenant virtual machine resource allocation method
CN111124644B (en) Method, device and system for determining task scheduling resources
WO2023174037A1 (en) Resource scheduling method, apparatus and system, device, medium, and program product
CN108241535B (en) Resource management method and device and server equipment
US9501321B1 (en) Weighted service requests throttling
CN110891087B (en) Log transmission method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16896413

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 16896413

Country of ref document: EP

Kind code of ref document: A1