CN111104217A - Rendering farm intelligent flow scheduling method and system based on semantic analysis - Google Patents
Rendering farm intelligent flow scheduling method and system based on semantic analysis Download PDFInfo
- Publication number
- CN111104217A CN111104217A CN201911180826.XA CN201911180826A CN111104217A CN 111104217 A CN111104217 A CN 111104217A CN 201911180826 A CN201911180826 A CN 201911180826A CN 111104217 A CN111104217 A CN 111104217A
- Authority
- CN
- China
- Prior art keywords
- rendering
- scheduling
- semantic analysis
- farm
- rule
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/005—General purpose rendering architectures
Abstract
The invention discloses a semantic analysis-based rendering farm intelligent flow scheduling method and a semantic analysis-based rendering farm intelligent flow scheduling system, wherein the scheduling method sequentially comprises the steps of classifying and marking rendering tasks through semantic analysis; routing the marked rendering tasks to the corresponding scheduling agents; and scheduling the rendering tasks of the GPU servers of the same type in the rendering farm by the scheduling agent. The invention identifies the user intention from the request source, and then intelligently routes the rendering request to the most suitable rendering server for rendering, can avoid the waste of rendering resources to the maximum extent, saves the hardware cost and the operation and maintenance cost, provides better service and meets the real requirement of the user.
Description
Technical Field
The invention belongs to the field of digital media design, and particularly relates to an intelligent flow scheduling method for a rendering farm.
Background
Render farm (Renderfarm), school name: a distributed parallel cluster computing system is a super computer built by utilizing an off-the-shelf CPU, Ethernet and an operating system, and achieves or approaches the computing power of the super computer by using mainstream commercial computer hardware equipment. There are GPUs in various configurations, such as 1080Ti, 2080Ti, Titan V, etc., in a rendering farm, and there are also 4U towers, blades, etc., which are diversified in server types.
The utilization rate of the traditional rendering load balancing strategy for the GPU is low, and when a large number of rendering tasks are submitted to a rendering farm, the traditional rendering load balancing strategy cannot meet the requirements. Therefore, an intelligent traffic scheduling strategy is urgently needed to be provided, so that the utilization rate of the GPU cluster can be greatly improved, and all rendering tasks can be completed in the shortest time.
Disclosure of Invention
In order to solve the technical problems mentioned in the background art, the invention provides a rendering farm intelligent traffic scheduling method and system based on semantic analysis.
In order to achieve the technical purpose, the technical scheme of the invention is as follows:
a rendering farm intelligent flow scheduling method based on semantic analysis comprises the following steps:
step 1: classifying and marking the rendering task through semantic analysis;
step 2: routing the marked rendering tasks to the corresponding scheduling agents;
and step 3: and scheduling the rendering tasks of the GPU servers of the same type in the rendering farm by the scheduling agent.
Based on the preferable scheme of the technical scheme, in the step 1, an acoustic input device is adopted, a rendering request is initiated by sending a voice command, the intention of a user is identified through a semantic identifier, and a specific label is marked for a rendering task.
Based on the preferred scheme of the above technical solution, in step 1, the printed label includes a requirement for the number of cuda cores and a requirement for the performance of the GPU, and these requirements are determined according to the identified user intention.
Based on the preferable scheme of the technical scheme, in the step 2, a RuleEngine rule engine is adopted to distribute the marked rendering tasks to the corresponding scheduling agents.
Based on the preferred scheme of the above technical scheme, in step 3, the scheduling policy is generated by the rule escaping module and is issued to the scheduling agent through the routing policy issuing device, and the rule escaping module and the routing policy issuing device are responsible for the life cycle of the whole rule.
Based on the preferable scheme of the above technical solution, the rule escaping module generates a rule expression to execute the corresponding scheduling policy, where the rule expression is a group of servers with the same label.
Based on the preferable scheme of the technical scheme, the scheduling strategy adopts a rendering server backpressure mode, a queuing rendering queue of each rendering server is inquired before scheduling, the server with the least queue depth is the server needing to schedule the rendering task, and the server with the queue depth larger than a set threshold value stops scheduling.
A rendering farm intelligent traffic scheduling system based on semantic analysis comprises:
the acoustic input equipment is used for acquiring a voice instruction of a user;
the semantic recognizer is used for recognizing the intention of the user according to the voice command;
the RuleEngine rule engine is used for distributing the marked rendering tasks to the corresponding scheduling agents;
the rule escape module is used for generating a scheduling strategy;
and the routing strategy issuing device is used for issuing the generated scheduling strategy to the scheduling agent.
Based on the preferable scheme of the technical scheme, the acoustic input device can convert the audio signal into a byte stream or a character stream which can be recognized by a computer, and further convert the byte stream or the character stream into a command character string through an ASR service.
Based on the preferable scheme of the technical scheme, the acoustic input device comprises a headset with a microphone.
Adopt the beneficial effect that above-mentioned technical scheme brought:
the invention adopts a rendering flow scheduling mode based on semantic analysis, can identify the user intention from the request source, and then intelligently routes the rendering request to the most suitable rendering server for rendering, can avoid the waste of rendering resources to the maximum extent, saves the hardware cost and the operation and maintenance cost, provides better service, and meets the real requirements of designers.
Drawings
FIG. 1 is a rendering farm dispatch strategy diagram of a conventional load balancing strategy;
FIG. 2 is a basic flow diagram of the present invention;
fig. 3 is a schematic diagram of the system of the present invention.
Detailed Description
The technical scheme of the invention is explained in detail in the following with the accompanying drawings.
Fig. 1 is a schematic diagram of a conventional rendered farm scheduling policy employing a load balancing (e.g., Nginx) policy, and through production environment verification, usage rates of common scheduling policies such as round robin, random, hash, and least recently invoked principles for a GPU are all low.
In order to improve the utilization rate of a GPU cluster and improve the performance of a rendering farm, the invention designs a rendering farm intelligent flow scheduling method and a system based on semantic analysis, and FIG. 2 is a basic flow chart of the scheduling method, which comprises the following steps:
step 1: classifying and marking the rendering task through semantic analysis;
step 2: routing the marked rendering tasks to the corresponding scheduling agents;
and step 3: and scheduling the rendering tasks of the GPU servers of the same type in the rendering farm by the scheduling agent.
Specifically, in step 1, an acoustic input device is adopted, a rendering request is initiated by sending a voice command, the intention of the user is recognized through a semantic recognizer, and a specific label is marked for the rendering task.
The rendering tasks with high dpi requirement, high calculation power or high stability requirement are routed to the 4U tower servers configured by the corresponding professional cards, and the rendering tasks with low requirement are routed to the game cards at the low end, which is exemplified as follows:
the user speaks the following 4 voice instructions to the acoustic input device:
(1) please help me to render the high-definition picture;
(2) please help me to render a picture of 2k (or 4k, 8 k) dpi;
(3) please help me render the picture for the client to show;
(4) please help me render this picture and i want to see the effect.
As shown in the above instructions (1), (2), and (3), it can be obtained through semantic analysis that the user intends to render a high-quality image for the task, whether the high-resolution high-dpi image is required to be displayed to the client or a high-resolution high-dpi image is required to be displayed. For the common appeal of high-quality graphs to rendering servers, more cuda cores are needed, and more stable professional rendering cards such as p6000 and GV100 are required to achieve the goal, the role of the semantic analyzer is to identify these intents and to label the rendering task with corresponding labels: cuda nuclear >5000, gpu = professional card. However, for the instruction (4), the designer only wants to see the following effect, and can print a label with lower requirement: cuda core >2000, gpu = game card 1080 Ti.
Specifically, in step 2, a RuleEngine is adopted to distribute the marked rendering tasks to the corresponding scheduling agents.
Specifically, in step 3, the scheduling policy is generated by the rule escaping module and is issued to the scheduling agent through the routing policy issuing device, and the rule escaping module and the routing policy issuing device are responsible for the life cycle of the whole rule. The rule escaping module generates a rule expression to execute the corresponding scheduling policy, wherein the rule expression is a group of servers with the same label.
The scheduling strategy adopts a rendering server backpressure mode, a queuing rendering queue of each rendering server is inquired before scheduling, one server with the least queue depth is a server needing to schedule rendering tasks, and the server with the queue depth larger than a set threshold value stops scheduling. The renderers in the market are provided with rendering queues, and all rendering requests can be completed most quickly by adopting the scheduling strategy.
Fig. 3 is a schematic diagram of a scheduling system of the present invention, which includes:
a rendering farm intelligent traffic scheduling system based on semantic analysis comprises:
the acoustic input equipment is used for acquiring a voice instruction of a user;
the semantic recognizer is used for recognizing the intention of the user according to the voice command;
the RuleEngine rule engine is used for distributing the marked rendering tasks to the corresponding scheduling agents;
the rule escape module is used for generating a scheduling strategy;
and the routing strategy issuing device is used for issuing the generated scheduling strategy to the scheduling agent.
In particular, the acoustic input device is capable of converting audio signals into a byte stream or character stream that can be recognized by a computer, and then converted into a command string by an ASR service.
In particular, the acoustic input device comprises a headset with a microphone.
The embodiments are only for illustrating the technical idea of the present invention, and the technical idea of the present invention is not limited thereto, and any modifications made on the basis of the technical scheme according to the technical idea of the present invention fall within the scope of the present invention.
Claims (10)
1. A rendering farm intelligent flow scheduling method based on semantic analysis is characterized by comprising the following steps:
step 1: classifying and marking the rendering task through semantic analysis;
step 2: routing the marked rendering tasks to the corresponding scheduling agents;
and step 3: and scheduling the rendering tasks of the GPU servers of the same type in the rendering farm by the scheduling agent.
2. The intelligent traffic scheduling method for rendering farm based on semantic analysis according to claim 1, wherein in step 1, an acoustic input device is used to initiate a rendering request by sending out a voice command, and a semantic recognizer recognizes the user's intention and marks a rendering task with a specific label.
3. The semantic analysis based rendering farm intelligent traffic scheduling method according to claim 1, wherein in step 1, the requirement for the number of cuda cores and the requirement for the performance of the GPU are included in the marked tags, and the requirements are determined according to the identified user intention.
4. The intelligent traffic scheduling method for rendering farms based on semantic analysis as claimed in claim 1, wherein in step 2, a RuleEngine is adopted to distribute the marked rendering tasks to the corresponding scheduling agents.
5. The intelligent traffic scheduling method for rendering farms based on semantic analysis as claimed in claim 1, wherein in step 3, the scheduling policy is generated by a rule escaping module and is issued to the scheduling agent through a routing policy issuing device, and the rule escaping module and the routing policy issuing device are responsible for the life cycle of the whole rule.
6. The semantic analysis based rendering farm intelligent traffic scheduling method according to claim 5, wherein in one embodiment, the rule escaping module generates a rule expression to execute the corresponding scheduling policy, wherein the rule expression is a set of servers with the same label.
7. The intelligent traffic scheduling method for the rendering farm based on semantic analysis according to claim 5, wherein in one embodiment, the scheduling policy adopts a rendering server backpressure mode, a queued rendering queue of each rendering server is queried before scheduling, the server with the least queue depth is the server to which the rendering task needs to be scheduled, and the server with the queue depth larger than a set threshold value stops scheduling.
8. A render farm intelligent traffic scheduling system based on semantic analysis, comprising:
the acoustic input equipment is used for acquiring a voice instruction of a user;
the semantic recognizer is used for recognizing the intention of the user according to the voice command;
the RuleEngine rule engine is used for distributing the marked rendering tasks to the corresponding scheduling agents;
the rule escape module is used for generating a scheduling strategy;
and the routing strategy issuing device is used for issuing the generated scheduling strategy to the scheduling agent.
9. The semantic analysis based rendering farm intelligent traffic scheduling system of claim 8, wherein in one embodiment the acoustic input device is capable of converting audio signals into a computer recognizable byte stream or character stream, which is in turn converted into a command string by an ASR service.
10. The semantic analysis based rendering farm intelligent traffic scheduling system of claim 8, wherein in one embodiment the acoustic input device comprises a headset with a microphone.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911180826.XA CN111104217B (en) | 2019-11-27 | 2019-11-27 | Rendering farm intelligent flow scheduling method and system based on semantic analysis |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911180826.XA CN111104217B (en) | 2019-11-27 | 2019-11-27 | Rendering farm intelligent flow scheduling method and system based on semantic analysis |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111104217A true CN111104217A (en) | 2020-05-05 |
| CN111104217B CN111104217B (en) | 2021-06-25 |
Family
ID=70421668
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911180826.XA Active CN111104217B (en) | 2019-11-27 | 2019-11-27 | Rendering farm intelligent flow scheduling method and system based on semantic analysis |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111104217B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2538328A1 (en) * | 2011-06-21 | 2012-12-26 | Unified Computing Limited | A method of provisioning a cloud-based render farm |
| CN103942103A (en) * | 2014-04-16 | 2014-07-23 | 乐视致新电子科技（天津）有限公司 | Method and device for generating processor scheduling policy in multi-core system and scheduling system |
| CN105446810A (en) * | 2015-12-24 | 2016-03-30 | 赞奇科技发展有限公司 | Cost based multi-farm cloud rendering task distributing system and method |
| CN105976420A (en) * | 2015-08-28 | 2016-09-28 | 深圳市彬讯科技有限公司 | Online rendering method and system |
| CN107291545A (en) * | 2017-08-07 | 2017-10-24 | 星环信息科技（上海）有限公司 | The method for scheduling task and equipment of multi-user in computing cluster |
| CN107562546A (en) * | 2017-09-18 | 2018-01-09 | 上海量明科技发展有限公司 | Method for allocating tasks, device and JICQ |
| CN107807854A (en) * | 2017-10-25 | 2018-03-16 | 深圳市瑞云科技有限公司 | The method and rendering task processing method of a kind of Automatic dispatching Node station |
| CN109634748A (en) * | 2018-12-12 | 2019-04-16 | 深圳前海微众银行股份有限公司 | Cluster resource dispatching method, device, equipment and computer readable storage medium |
| CN109726008A (en) * | 2017-10-31 | 2019-05-07 | 阿里巴巴集团控股有限公司 | Resource allocation methods and equipment |
-
2019
- 2019-11-27 CN CN201911180826.XA patent/CN111104217B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2538328A1 (en) * | 2011-06-21 | 2012-12-26 | Unified Computing Limited | A method of provisioning a cloud-based render farm |
| CN103942103A (en) * | 2014-04-16 | 2014-07-23 | 乐视致新电子科技（天津）有限公司 | Method and device for generating processor scheduling policy in multi-core system and scheduling system |
| CN105976420A (en) * | 2015-08-28 | 2016-09-28 | 深圳市彬讯科技有限公司 | Online rendering method and system |
| CN105446810A (en) * | 2015-12-24 | 2016-03-30 | 赞奇科技发展有限公司 | Cost based multi-farm cloud rendering task distributing system and method |
| CN107291545A (en) * | 2017-08-07 | 2017-10-24 | 星环信息科技（上海）有限公司 | The method for scheduling task and equipment of multi-user in computing cluster |
| CN107562546A (en) * | 2017-09-18 | 2018-01-09 | 上海量明科技发展有限公司 | Method for allocating tasks, device and JICQ |
| CN107807854A (en) * | 2017-10-25 | 2018-03-16 | 深圳市瑞云科技有限公司 | The method and rendering task processing method of a kind of Automatic dispatching Node station |
| CN109726008A (en) * | 2017-10-31 | 2019-05-07 | 阿里巴巴集团控股有限公司 | Resource allocation methods and equipment |
| CN109634748A (en) * | 2018-12-12 | 2019-04-16 | 深圳前海微众银行股份有限公司 | Cluster resource dispatching method, device, equipment and computer readable storage medium |
Non-Patent Citations (2)
| Title |
|---|
| D. PETROU等: ""Scheduling speculative tasks in a compute farm"", 《SC "05: PROCEEDINGS OF THE 2005 ACM/IEEE CONFERENCE ON SUPERCOMPUTING》 * |
| 梁骥: ""基于分布式集群运算的渲染农场的研究与构建"", 《实验室研究与探索》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111104217B (en) | 2021-06-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110769278B (en) | Distributed video transcoding method and system | |
| WO2021151279A1 (en) | Method and apparatus for cloud monitoring based on edge computing, electronic device, and storage medium | |
| CN102915254B (en) | task management method and device | |
| CN110520853A (en) | The queue management of direct memory access | |
| CN109788315A (en) | Video transcoding method, apparatus and system | |
| CN110968423A (en) | Method and apparatus for distributing workload to accelerators using machine learning | |
| JP2005505833A5 (en) | ||
| CN107241305B (en) | Network protocol analysis system based on multi-core processor and analysis method thereof | |
| US20190065251A1 (en) | Method and apparatus for processing a heterogeneous cluster-oriented task | |
| WO2024060571A1 (en) | Heterogeneous computing power-oriented multi-policy intelligent scheduling method and apparatus | |
| CN111506434B (en) | Task processing method and device and computer readable storage medium | |
| WO2022048204A1 (en) | Image generation method and apparatus, electronic device, and computer readable storage medium | |
| KR20210080292A (en) | Scheduling method, device, equipment and medium of deep learning inference engine | |
| US11470336B2 (en) | Method for transcoding video and related electronic device | |
| CN111176806B (en) | Service processing method and device and computer readable storage medium | |
| CN105677763A (en) | Image quality evaluating system based on Hadoop | |
| US20210365285A1 (en) | Voice data procession method, apparatus, device and storage medium | |
| CN103677844B (en) | Lightweight class data processing method based on message chain | |
| CN111104217B (en) | Rendering farm intelligent flow scheduling method and system based on semantic analysis | |
| US9111009B2 (en) | Self-parsing XML documents to improve XML processing | |
| CN115759577A (en) | Dynamic task scheduling method and device, electronic equipment and storage medium | |
| CN209803659U (en) | Clock control system in GPU server | |
| CN103873953A (en) | Cloud computing-based implementation step of high-performance courseware transcoding | |
| CN106357676A (en) | Method for optimizing overhead of cloud service resource | |
| CN112769788A (en) | Charging service data processing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |