KR20240015021A - Anchor scheduler with super-resolution acceleration function and anchor scheduling method using the same - Google Patents

Abstract

Motivated by the limitations of neural network-enhanced live streaming, an anchor scheduler with super-resolution acceleration and an anchor scheduling method using the same are disclosed to reduce the cost of end-to-end neural network enhancement and maximize overall quality improvement in a computing cluster. An anchor scheduler with a super-resolution acceleration function includes a decoder that decodes an externally provided video stream; and a resource management module that distributes the load while optimally allocating computing resources to each video stream decoded by the decoder.

Description

Anchor scheduler with super-resolution acceleration function and anchor scheduling method using the same {ANCHOR SCHEDULER WITH SUPER-RESOLUTION ACCELERATION FUNCTION AND ANCHOR SCHEDULING METHOD USING THE SAME}

The present invention relates to an anchor scheduler with a super-resolution acceleration function and an anchor scheduling method using the same. More specifically, the present invention relates to super-resolution acceleration to reduce the cost of end-to-end neural network reinforcement and maximize overall quality improvement in a computing cluster. It relates to a functional anchor scheduler and an anchor scheduling method using the same.

The demand for live streaming has grown rapidly. Live video traffic is expected to account for 17% of Internet traffic by 2022. Current live streaming infrastructure relies on two key elements: Video is transmitted to the media server using a low-latency streaming protocol. On the distribution side, the client runs the Adaptive Bitrate (ABR) algorithm to select the highest quality video that can be streamed in real time.

Traditional live streaming is inadequate to consistently deliver high-quality video (e.g. 4K/8K), as ingestion video quality is unfortunately highly dependent on the streamer's uplink bandwidth. Congestion of the collection path directly reduces overall downstream video quality. However, video quality is the most important factor affecting user engagement in live streaming. For example, more than 50% of live viewers will abandon a video stream if video quality degrades for more than 90 seconds. This loss of viewers can have a huge impact on the revenue of live streaming providers.

Recent advances in neural enhanced streaming show great promise in leveraging computation in media servers to improve ingestion video quality. When the ingestion video quality deteriorates, the media server executes end-to-end neural enhancement, which consists of decoding the low-quality stream, applying a super-resolution Deep Neural Network (DNN), and super-resolved output encoding to recover high-quality video. . This provides dramatic quality improvement in downstream video.

However, neural enhancement is too expensive to support commercial-scale live streaming. For example, Twitch supports over 100,000 simultaneous live streams. Applying end-to-end neural enhancement in this setup requires tens of thousands of graphics processing units (GPUs) and costs more than $169,000 per hour in the public cloud. The cost breakdown shows that both video super-resolution and encoding are expensive. Neural super-resolution requires 100 to 1000 times more computation than DNNs used for discrimination tasks, and video encoding is up to 3.3 times slower than super-resolution.

Korean Patent No. 10-2313136 (2021. 10. 15.)

Accordingly, the technical problem of the present invention is focused on this point, and the purpose of the present invention is to reduce the cost of end-to-end neural network reinforcement by motivating the limitations of neural network-enhanced live streaming, and to reduce the cost of end-to-end neural network enhancement, Network) pairs and output high-resolution video streams to provide an anchor scheduler with super-resolution acceleration to maximize overall quality improvement in the computing cluster.

Another object of the present invention is to provide an anchor scheduling method using an anchor scheduler having the above-described super-resolution acceleration function.

In order to realize the object of the present invention described above, according to one embodiment, an anchor scheduler with a super-resolution acceleration function includes: a decoder for decoding an externally provided video stream; and a resource management module that distributes the load while optimally allocating computing resources to each video stream decoded by the decoder.

In one embodiment, the resource management module includes: a zero-guessing anchor frame selector running on a server to select the most advantageous anchor frame in all video streams; and an anchor-level load balancer that selects an anchor frame and then dynamically balances the load across compute instances at the anchor frame granularity.

In one embodiment, the zero-inference anchor frame selector includes a frame division unit that divides frames into key frame groups, alternative reference frame groups, and other frame groups according to the type of each video stream; an anchor gain estimation unit that iteratively selects the most advantageous frames for frames in the alternative reference frame group and the other frame groups and estimates anchor gains; a group merging and sorting unit for merging and sorting each of the key frame group, replacement reference frame group, and other frame groups into a key frame global group, a replacement reference frame global group, and other frame global groups; and an anchor frame selection unit that repeatedly selects an anchor frame from global groups sorted from the key frame global group to the alternative reference frame global group and the other frame global group.

In one embodiment, the anchor gain estimation unit includes a residual calculation module that calculates residual accumulated over the entire frame; a residual calculation module that calculates a residual amount by which each frame is reduced for each iteration; an anchor gain setting module that selects the frame that reduces the residual the most and sets the anchor gain to the amount of the reduced residual; and an update module that updates the accumulated residual of each frame.

In one embodiment, the remaining amount calculation module,

(Here, Res(F[j]) is the accumulated residual of the jth frame, and k is the index of the nearest frame where the residual is reset).

In one embodiment, the zero-inference anchor frame selector may periodically execute a zero-inference algorithm by selecting the maximum number of anchor frames that can be processed in real time by the computing cluster.

In one embodiment, the zero-inference anchor frame selector is:

(where AF is the anchor frame aligned with the anchor gain, TDNN is the DNN latency, Tintv is the anchor frame selection interval, which is a configurable parameter, and M is the number of compute instances in the cluster).

In order to realize another object of the present invention described above, an anchor scheduling method according to an embodiment includes the steps of decoding an externally provided video stream; and distributing the load while optimally allocating computing resources to each of the decoded video streams.

In one embodiment, the load balancing step includes: (i) executing on a server to select the most advantageous anchor frame from all video streams; and (ii) dynamically balancing load across compute instances at anchor frame granularity after selecting an anchor frame.

In one embodiment, the step of selecting the anchor frame includes: (i-1) dividing the frames into key frame groups, replacement reference frame groups, and other frame groups according to the type of each video stream; (i-2) estimating anchor gains for frames of the alternative reference frame group and the other frame groups; (i-3) merging and sorting each of the key frame group, replacement reference frame group, and other frame groups into a key frame global group, a replacement reference frame global group, and other frame global groups; and (i-4) repeatedly selecting anchor frames from global groups sorted from the key frame global group to the alternative reference frame global group and the other frame global group.

In one embodiment, priority is given to selecting the anchor frame in the order of the key frame group, the replacement frame group, and the other frame group.

In one embodiment, the anchor gain may be calculated based on accumulated residuals.

In one embodiment, the anchor gain may be estimated using residuals.

In one embodiment, estimating the anchor gain includes calculating residuals accumulated across frames; Iteratively selecting the most advantageous frames and estimating anchor gains; selecting the frame that reduces the residual the most and setting the anchor gain to the amount of the reduced residual; and updating the cumulative residual of each frame to reflect the influence of the selected frame.

According to the anchor scheduler with such a super-resolution acceleration function and the anchor scheduling method using the same, anchor frames are selected without neural inference through a zero-inference anchor frame selection algorithm, and frame-by-frame inference is executed while accelerating the encoding of selective super-resolution video. It can provide similar quality to the algorithm used. Additionally, the resource management module can execute global anchor frame selection across all video streams to select the optimal set of anchor frames, and distribute the load across instances on a per-anchor frame basis, allowing the overhead of anchor frames to be accurately predicted.

Figure 1 is a conceptual diagram to schematically explain existing streaming and neural network-enhanced streaming (adaptive streaming case).
Figure 2 is a conceptual diagram schematically explaining selective super-resolution.
Figure 3 is a conceptual diagram schematically explaining a neuroscaler according to an embodiment of the present invention.
Figure 4 is a conceptual diagram schematically explaining the deployment model of a neuroscaler for adaptive streaming.
FIG. 5 is a conceptual diagram schematically explaining the anchor scheduler shown in FIG. 3.
FIG. 6 is a conceptual diagram schematically explaining the anchor frame selector shown in FIG. 3.

Hereinafter, with reference to the attached drawings, an anchor scheduler with a super-resolution acceleration function according to the present invention and an anchor scheduling method using the same will be described in more detail. Since the present invention can be subject to various changes and can have various forms, specific embodiments will be illustrated in the drawings and described in detail in the text. However, this is not intended to limit the present invention to a specific disclosed form, and should be understood to include all changes, equivalents, and substitutes included in the spirit and technical scope of the present invention.

While describing each drawing, similar reference numerals are used for similar components. In the attached drawings, the dimensions of the structures are enlarged from the actual size for clarity of the present invention.

Terms such as first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, a first component may be referred to as a second component without departing from the scope of the present invention, and similarly, the second component may also be referred to as a first component. Singular expressions include plural expressions unless the context clearly dictates otherwise.

In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Additionally, unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by a person of ordinary skill in the technical field to which the present invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and unless explicitly defined in the present application, should not be interpreted in an ideal or excessively formal sense. No.

Hereinafter, the present invention will be described in detail with reference to the attached drawings.

Live streaming consists of collection and distribution aspects, as shown in Figure 1.

Figure 1 is a conceptual diagram to schematically explain existing streaming and neural network-enhanced streaming (adaptive streaming case).

Referring to Figure 1, on the collection side, the streamer captures live video and uploads it to a media server. In adaptive streaming, the streamer uploads a single video stream, and the media server transcodes it into multiple quality versions.

In multi-party video conferencing, broadcasters upload video streams of multiple qualities using simulcast or scalable video codecs (SVC), and media servers deliver the video streams to viewers. Next, on the distribution side, the client runs the Adaptive Bitrate (ABR) algorithm for video streaming quality control to select/download the highest quality video in the given network bandwidth. The ABR algorithm can be divided into heuristics-based methods and machine learning-based methods. Heuristic-based methods are classified into network bandwidth, client buffer occupancy, and hybrid based methods that combine the two.

A common limitation of traditional live streaming is that video quality is limited by the streamer's uplink bandwidth. Even if viewers have sufficient network bandwidth, limitations may deprive them of the opportunity to enjoy high-quality video.

Super-resolution is a type of technology that creates high-resolution images from low-resolution images. Recent research uses deep neural networks (DNNs) to learn mapping from low-resolution to high-resolution and shows dramatic quality improvements.

Neural-enhanced live streaming uses ultra-high-resolution DNNs to enhance the collected video stream, as shown in Figure 1. Unlike traditional live streaming, the media server implements end-to-end neural enhancement. When the video arrives, it is first decoded into raw frames. Then, the DNN is applied to all or some frames depending on the super-resolution method. The output is re-encoded into video before being delivered to the client.

To provide reliable quality improvement, content-aware DNNs are commonly used for each video stream and can also be updated during live streaming. Context switching between content-aware DNNs incurs two types of overhead.

First, while modern DNN compilers provide optimized inference for the target accelerator, this includes an upfront cost due to model optimization, which can take up to several minutes. Apply both graph-level and operator-level optimization to create a DNN tailored to your specific target.

Second, before executing inference, the DNN and frames must be loaded into the device memory of the target accelerator. Because neural accelerators are typically limited in memory size (for example, 16 GB for the NVIDIA T4), multiple content-aware DNNs, each requiring several GB of memory, must be replaced frequently.

Selective super-resolution (SR) reduces computing overhead by leveraging temporal redundancy across video frames. Neural inference applies only to selective frames, called anchor frames.

Figure 2 is a conceptual diagram schematically explaining selective super-resolution.

As can be seen in Figure 2, non-anchor frames are reconstructed by reusing previous super-resolution frames guided by codec level information (e.g., reference index, motion vector, residual). It includes lightweight bilinear interpolation and decoding, allowing real-time processing even on mobile devices.

Upscaling non-anchor frames by reusing previous super-resolution frames inevitably leads to quality loss (compared to frame-by-frame inference) due to time differences. This loss accumulates in successive non-anchor frames but is reset at the anchor frame. Therefore, it is important to select an informative set of anchor frames to maximize quality. To achieve this, the NEMO algorithm relies on expensive per-frame inference. Therefore, it cannot be used for live streaming where anchor frame selection must be performed online in real time.

Figure 3 is a conceptual diagram schematically explaining a neuroscaler according to an embodiment of the present invention.

Referring to FIG. 3, a NeuroScaler according to an embodiment of the present invention includes an anchor scheduler (100) and an anchor enhancer (200), and collects video streams and DNN ( Deep Neural Network) pairs are imported and output a high-resolution video stream. The DNN can also be dynamically updated through online learning, such as in LiveNAS.

Anchor scheduler 100 has a super-resolution acceleration function to decode the collected video stream and select the most advantageous anchor frame from the video stream. Here, the anchor frame refers to a frame in which the optional super-resolution is extended to accelerate video super-resolution, but super-resolution is applied in real time using codec-level information. Then, the anchor scheduler 100, for each video stream, transfers the DNN and the selected anchor frame to the anchor enhancer 200 equipped with a neural accelerator.

The anchor enhancer 200 preprocesses the DNN, applies the preprocessed DNN to the anchor frame, and re-encodes the super-resolution output.

In this embodiment, all processes of anchor scheduler 100 and anchor enhancer 200 are pipelined and parallelized to maximize overall throughput.

According to the neuroscaler according to the present invention, by motivating the limitations of neural network-enhanced live streaming, the end-to-end neural network enhancement cost can be reduced and the overall quality improvement in the computing cluster can be maximized.

Figure 4 is a conceptual diagram schematically explaining the deployment model of a neuroscaler for adaptive streaming.

Referring to Figure 4, when a low-quality video stream (e.g., 360p or 720p) is uploaded to a media server, the neuroscaler generates a high-resolution video stream using real-time super-resolution.

Lower resolution versions (e.g., 240p to 720p) are also created from the collected video stream by using an existing transcoding pipeline. Using Neuroscaler, clients can view high-resolution video even when the collection path is congested. Deploying Neuroscaler for video conferencing is similar to the process above, but does not require multi-resolution transcoding.

FIG. 5 is a conceptual diagram schematically explaining the anchor scheduler 100 shown in FIG. 3.

3 and 5, the anchor scheduler 100 includes a decoder 110 and a resource management module 120.

The decoder 110 decodes an externally provided video stream and provides it to the resource management module 120. The decoded video stream may be provided directly to the resource management module 120 or may be provided to the resource management module 120 via memory.

Resource management module 120 includes an anchor selector 122 and an anchor-level load balancer 124 to distribute load while optimally allocating computing resources to each video stream. In this embodiment, the resource management module 120 balances the load while optimally allocating computing resources to each video stream to maximize quality improvement when processing a large number of video streams. In particular, the resource management module 120 performs global anchor frame selection in all video streams to select an optimal set of anchor frames, and distributes the load to instances in units of anchor frames where the overhead of anchor frames can be accurately predicted. In this embodiment, the anchor scheduler 100 has been described as including an anchor selector 122 and an anchor-level load balancer 124, but this is only logically divided for convenience of explanation and not hardwarely divided. .

Anchor selector 122 runs on a centralized server to select the most advantageous anchor frames from all video streams. In particular, the anchor selector 122 selects the maximum number of anchor frames that can be processed in real time in the computing cluster as shown in Equation (1) below and periodically executes the zero-inference algorithm.

[Formula 1]

here, is the anchor frame aligned with the anchor gain, is the DNN latency, is the anchor frame selection interval, which is a configurable parameter; is the number of compute instances in the cluster.

Unless otherwise stated in this specification = Use 666ms (40 frames for 60fps video). This global anchor frame selection can alleviate the imbalance of anchor frames per video stream.

In general, using longer scheduling intervals can improve quality by selecting more influential anchor frames. Therefore, the spacing must be adjusted according to application requirements.

Anchor-level load balancer 124 selects an anchor frame and then dynamically balances the load across compute instances at anchor frame granularity. To this end, the anchor scheduler 100 divides the selected anchor frames into groups for each instance and the frames of each group are delivered to the corresponding computing instance for processing.

To meet real-time constraints, the anchor scheduler 100 determines that the total waiting time is the anchor frame selection interval ( ) Allocate a maximum number of anchor frames for each group that is less than . Through this anchor-level load balancing, the load imbalance for each instance can be alleviated.

FIG. 6 is a conceptual diagram schematically explaining the anchor selector 122 shown in FIG. 5.

5 and 6, the anchor selector 122 includes a frame splitting unit 122a, an anchor gain estimating unit 122b, a group merging and sorting unit 122c, and an anchor frame selection unit 122d, Codec-level information (e.g., frame type, residual) is utilized to select informative anchor frames. In this embodiment, the anchor selector 122 has been described as including a frame dividing unit 122a, an anchor gain estimating unit 122b, a group merging and sorting unit 122c, and an anchor frame selection unit 122d. For convenience of explanation, they are divided logically and not hardware-wise.

The frame division unit 122a divides frames into groups for each video stream according to the type of each video stream. That is, the frame division unit 122a divides frames into key frame groups (key frame groups) according to the type of each video stream. ), alternate reference frame group ( ) and other frame groups ( ) divided by Here, the group is , and There is a priority order for selecting anchor frames.

The anchor gain estimation unit 122b is an alternative reference frame group ( ) and other frame groups ( ), we estimate the benefit of using an anchor frame, called the anchor gain. The anchor gain is calculated based on the accumulated residuals using Algorithm 1, described later. Key frame group ( ), since there are only a few key frames and they are generally all selected as anchor frames, it is assumed to have an anchor gain equal to the overall performance.

The anchor gain estimation unit 122b includes a residual calculation module (not shown), a residual calculation module (not shown), an anchor gain setting module (not shown), and an update module (not shown). In this embodiment, the anchor gain estimation unit 122b has been described as including a residual calculation module, a residual amount calculation module, an anchor gain setting module, and an update module, but these are only logically divided for convenience of explanation and hardwarely divided. I didn't do it. The residual calculation module calculates the residual accumulated over the entire frame. The residual calculation module calculates the residual amount by which each frame is reduced for each iteration. The anchor gain setting module selects the frame that reduces the residual the most and sets the anchor gain to the amount of the reduced residual. The update module updates the cumulative residual of each frame.

The group merging and sorting unit 122c merges groups for each video stream into a global group. That is, the keyframe global group ( ), alternative reference frame global group ( ), and other frame global groups ( ) includes key frames, alternate reference frames, and regular frames of all video streams, respectively. Frames of the same group are sorted according to anchor gain.

The anchor frame selection unit 122d is a key frame global group ( ) in the alternate reference frame global group ( ) and other frame global groups ( ) Iteratively selects anchor frames from the global group sorted.

In this embodiment, the neuroscaler measures the latency of the DNN once and selects the maximum number of anchor frames whose total latency is less than the available computing time to meet real-time constraints.

Hereinafter, we will describe how the neuroscaler uses residuals to estimate anchor gains through Algorithm 1.

Algorithm 1 shows how the neuroscaler estimates the gain of the anchor frame.

Algorithm 1 first calculates the residuals accumulated across frames (Algorithm 1, line #2).

Then, the most advantageous frame is iteratively selected and the anchor gain is estimated. That is, for each repetition, the remaining amount by which each frame is reduced is calculated as shown in the following equation (2) (lines #6-8).

[Formula 2]

Here, Res(F[j]) is the cumulative residual of the jth frame, and k is the index of the nearest frame where the residual is reset. Residuals are cleared from key frames or frames where the anchor gain was estimated from the previous iteration. If such a frame does not exist within a given frame, the algorithm predicts that the residual is reset at the key frame of the next video chunk.

Next, the frame that reduces the residual the most is selected and the anchor gain is set to the amount of the reduced residual (lines #12, 13).

Finally, the cumulative residual for each frame is updated to reflect the impact of the selected frame (line #14). To quickly estimate the anchor gain, the total residual pixel value is approximated by the size of the encoded residual frame. Since both values are highly correlated, they have minimal impact on quality (=△0.05dB in PSNR).

As described above, according to the present invention, anchor frames are selected without neural inference through a zero-inference anchor frame selection algorithm, thereby accelerating the encoding of selective super-resolution video while providing quality similar to that of an algorithm that performs frame-by-frame inference. can do. Additionally, the resource management module can execute global anchor frame selection across all video streams to select the optimal set of anchor frames, and distribute the load across instances on a per-anchor frame basis, allowing the overhead of anchor frames to be accurately predicted.

Components according to embodiments of the present invention may be implemented in the form of software or hardware such as FPGA (Field Programmable Gate Array) or ASIC (Application Specific Integrated Circuit), and may perform certain roles. However, 'components' are not limited to software or hardware, and each component may be configured to reside on an addressable storage medium or may be configured to run on one or more processors.

Thus, as an example, a component may include components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, procedures, and sub-processes. Includes routines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables.

Components and the functionality provided within them may be combined into a smaller number of components or further separated into additional components. At this time, it will be understood that each block in the drawings can be executed by computer program instructions. These computer program instructions may be mounted on a processor of a general-purpose computer, special-purpose computer, or other programmable data processing equipment, so that the instructions performed through the processor of the computer or other programmable data processing equipment may perform the functions described in the block(s). It creates the means to carry out these tasks. These computer program instructions may be stored in a computer-readable memory or may be stored in a computer-readable memory that can be directed to a computer or other programmable data processing equipment to implement a function in a particular manner. Instructions stored in memory may also produce manufactured items containing instruction means to perform the functions described in the block(s). Computer program instructions can also be mounted on a computer or other programmable data processing equipment, so that a series of operational steps are performed on the computer or other programmable data processing equipment to create a process that is executed by the computer, thereby generating a process that is executed by the computer or other programmable data processing equipment. Instructions that perform processing equipment may also provide steps for executing the functions described in the block(s).

Additionally, each block may represent a module, segment, or portion of code that includes one or more executable instructions for executing specified logical function(s). Additionally, it should be noted that in some alternative execution examples it is possible for the functions mentioned in the blocks to occur out of order. For example, it is possible for two blocks shown in succession to be performed substantially at the same time, or it is possible for the blocks to be performed in reverse order depending on the corresponding function.

At this time, the term '~unit' used in this embodiment refers to software or hardware components such as FPGA or ASIC, and the '~unit' performs certain roles. However, '~part' is not limited to software or hardware. The '~ part' may be configured to reside in an addressable storage medium and may be configured to reproduce on one or more processors. Therefore, as an example, '~ part' refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, and procedures. , subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and 'parts' may be combined into a smaller number of components and 'parts' or may be further separated into additional components and 'parts'. Additionally, components and 'parts' may be implemented to regenerate one or more CPUs within a device or a secure multimedia card.

Although the above has been described with reference to examples, those skilled in the art can make various modifications and changes to the present invention without departing from the spirit and scope of the present invention as set forth in the claims below. You will understand.

120: Resource Management Module 122: Anchor Selector
124: anchor-level load balancer 122a: frame divider
122b: Anchor gain estimation unit 122c: Group merging and sorting unit
122d: anchor frame selection unit 210: pre-processor
220: Inference Engine 230: Encoder

Claims (14)

A decoder that decodes an externally provided video stream; and
An anchor scheduler with a super-resolution acceleration function, comprising a resource management module that distributes the load while optimally allocating computing resources to each video stream decoded by the decoder. The method of claim 1, wherein the resource management module,
A zero-inference anchor frame selector that runs on the server and selects the most advantageous anchor frame from any video stream; and
An anchor scheduler with super-resolution acceleration, comprising an anchor-level load balancer that selects an anchor frame and then dynamically balances the load across compute instances at anchor frame granularity. 3. The method of claim 2, wherein the zero-inference anchor frame selector comprises:
A frame divider that divides frames into key frame groups, alternate reference frame groups, and other frame groups according to the type of each video stream;
an anchor gain estimation unit that iteratively selects the most advantageous frames for frames in the alternative reference frame group and the other frame groups and estimates anchor gains;
a group merging and sorting unit for merging and sorting each of the key frame group, replacement reference frame group, and other frame groups into a key frame global group, a replacement reference frame global group, and other frame global groups; and
An anchor scheduler with a super-resolution acceleration function, comprising an anchor frame selection unit that repeatedly selects anchor frames from global groups sorted from the key frame global group to the alternative reference frame global group and the other frame global group. The method of claim 3, wherein the anchor gain estimation unit,
a residual calculation module that calculates residuals accumulated across frames;
a residual calculation module that calculates a residual amount by which each frame is reduced for each iteration;
an anchor gain setting module that selects the frame that reduces the residual the most and sets the anchor gain to the amount of the reduced residual; and
An anchor scheduler with a super-resolution acceleration function, comprising an update module that updates the accumulated residual of each frame. The method of claim 4, wherein the remaining amount calculation module,

(Here, Res(F[j]) is the accumulated residual of the jth frame, and k is the index of the nearest frame where the residual is reset) with a super-resolution acceleration function, characterized in that the residual is calculated using Anchor scheduler. The method of claim 2, wherein the zero-inference anchor frame selector has a super-resolution acceleration function, wherein the zero-inference anchor frame selector selects the maximum number of anchor frames that can be processed in real time in the computing cluster and periodically executes the zero-inference algorithm. Anchor scheduler. 7. The method of claim 6, wherein the zero-inference anchor frame selector comprises:

(where AF is the anchor frame aligned with the anchor gain, T _DNN is the DNN latency, Tintv is the anchor frame selection interval, which is a configurable parameter, and M is the number of compute instances in the cluster). An anchor scheduler with super-resolution acceleration. Decoding an externally provided video stream; and
An anchor scheduling method comprising distributing load while optimally allocating computing resources to each of the decoded video streams. The method of claim 8, wherein distributing the load comprises:
(i) running on the server to select the most advantageous anchor frame from all video streams; and
(ii) selecting an anchor frame and then dynamically balancing the load among computing instances at the anchor frame granularity. The method of claim 9, wherein selecting the anchor frame comprises:
(i-1) dividing frames into key frame groups, alternative reference frame groups and other frame groups according to the type of each video stream;
(i-2) estimating anchor gains for frames of the alternative reference frame group and the other frame groups;
(i-3) merging and sorting each of the key frame group, replacement reference frame group, and other frame groups into a key frame global group, a replacement reference frame global group, and other frame global groups; and
(i-4) An anchor scheduling method comprising repeatedly selecting anchor frames from global groups sorted from the key frame global group to the alternative reference frame global group and the other frame global group. The anchor scheduling method of claim 9, wherein the anchor frame is selected in priority order from the key frame group, the replacement frame group, and the other frame group. The anchor scheduling method of claim 9, wherein the anchor gain is calculated based on accumulated residuals. The anchor scheduling method of claim 9, wherein the anchor gain is estimated using residuals. The method of claim 9, wherein estimating the anchor gain comprises:
calculating residuals accumulated throughout the frame;
Iteratively selecting the most advantageous frames and estimating anchor gains;
selecting the frame that reduces the residual the most and setting the anchor gain to the amount of the reduced residual; and
An anchor scheduling method comprising updating the cumulative residual of each frame to reflect the influence of the selected frame.