KR20220095693A - Gop selection method based on reinforcement learning and image analysis apparatus - Google Patents

Abstract

A group of picture (GOP) selection method based on reinforcement learning includes the steps of: receiving an input image composed of a plurality of frames by an analysis device; determining, by the analysis device, a path of a binary tree for determining a GOP based on the plurality of frames; and selecting, by the analysis device, the GOP of the input image based on a leaf node of the binary tree. The analysis unit determines the path of the binary tree using reinforcement learning using the binary tree as the environment, whether the tree is branched as the behavior, and the coding efficiency of the selected GOP as the reward.

Description

GOP selection method and analysis device based on reinforcement learning

A technique to be described below relates to a technique for selecting a group of pictures (GOP) in video encoding.

Video encoding technology including HEVC (High Efficiency Video Coding)/H.265 (hereafter HEVC) divides an image in units of GOPs and performs encoding. In video encoding, an object to be currently encoded is encoded using information already encoded in the same frame or another frame.

HEVC uses I-frame, P-frame, and B-frame and uses only intra prediction in All Intra (AI) mode, low delay mode (LD: Low Delay), and random access mode (RA: Random Access) coding mode can be provided to selectively use the encoding and decoding structures according to the purpose of the application service. In particular, the random access mode can compress high-definition images into low bits using B-frames. HEVC is encoded with a reference structure limited into the GOP.

US Registered Patent No. US10523940

The GOP size is related to the temporal correlation between frames in the corresponding GOP. That is, the change in the size of the GOP results in a change in the reference hierarchy structure of the GOP and the reference frame of the frame to be currently encoded. For example, if the size of the GOP is large in the case of a video having a sudden scene change or a large motion change, texture information of a reference frame is different from a frame currently being encoded, and encoding efficiency may deteriorate.

The technique to be described below is intended to provide a technique for selecting the size of a GOP for encoding. The technique to be described below is intended to provide a technique for determining the GOP size based on a learning model.

The GOP selection method based on reinforcement learning comprises the steps of: an analysis device receiving an input image composed of a plurality of frames; determining, and selecting, by the analysis apparatus, a GOP of the input image based on leaf nodes of the binary tree.

An analysis device for selecting a GOP based on reinforcement learning is an input device that receives an input image composed of a plurality of frames, and a reinforcement learning model that determines a path of a binary tree for determining a GOP (Group of Picture) based on the frames. and a storage device for storing the plurality of frames and a calculation device for selecting a GOP of the input image based on leaf nodes of the binary tree by applying the plurality of frames to the binary tree.

In the reinforcement learning, the environment is the binary tree, the action is whether the tree is branched, and the reward is the encoding efficiency of the selected GOP.

The technique described below uses reinforcement learning to provide an optimal GOP size for the current image. Accordingly, the techniques described below contribute to maximizing the efficiency of video encoding.

1 shows a hierarchical coding structure in a random access mode.
2 is an example of a reinforcement learning environment for GOP selection.
3 is an example of an adaptive GOP tree structure.
4 is an example of a GOP selection scenario according to a GOP size.
5 is an example of a neural network model for determining a branch of a GOP binary tree.
6 is an example of QP determination using reinforcement learning.
7 is an example of an analysis device.

The technology to be described below can apply various changes and can have various embodiments, and specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the technology described below to specific embodiments, and it should be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the technology described below.

Terms such as first, second, A, and B may be used to describe various components, but the components are not limited by the above terms, and only for the purpose of distinguishing one component from other components. is used only as For example, a first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component without departing from the scope of the present invention. and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.

In terms of terms used herein, the singular expression should be understood to include the plural expression unless the context clearly dictates otherwise, and terms such as "comprises" include the described feature, number, step, operation, element. , parts or combinations thereof are to be understood, but not to exclude the possibility of the presence or addition of one or more other features or numbers, step operation components, parts or combinations thereof.

Prior to a detailed description of the drawings, it is intended to clarify that the classification of the constituent parts in the present specification is merely a division according to the main function each constituent unit is responsible for. That is, two or more components to be described below may be combined into one component, or one component may be divided into two or more for each more subdivided function. In addition, each of the constituent units to be described below may additionally perform some or all of the functions of other constituent units in addition to the main function it is responsible for. Of course, it can also be performed by being dedicated to it.

In addition, in performing the method or method of operation, each process constituting the method may occur differently from the specified order unless a specific order is clearly described in context. That is, each process may occur in the same order as specified, may be performed substantially simultaneously, or may be performed in the reverse order.

A technique to be described below is a technique for selecting a GOP using a learning model. In particular, a technique to be described below is a technique for selecting a GOP using reinforcement learning. First, reinforcement learning will be briefly described.

Supervised learning predicts values or categories for new data by learning from data with correct answers, and unsupervised learning appropriately groups data without correct answers or finds relationships between data. In contrast, reinforcement learning learns how an agent should behave in a given environment. Reinforcement learning is a methodology in which an agent defined in a certain environment recognizes the current state and selects an action or action sequence that maximizes a reward among selectable actions. Reinforcement learning aims to learn actions or sequences of actions so that the agent maximizes the reward.

Reinforcement learning mainly uses a probabilistic model called MDP (Markov Decision Process). MDP models the decision-making probability that the state at time t is affected only by the state at t-1, and shows a probability expression as shown in Equation 1 below.

The concept extended by adding a reward to the state in MDP is called the Markov reward process and is expressed in the form of a tuple such as (χ,A,p,q,p ₀ ). χ is the state, A is the action, p(•|x,a) is the probability of going to the next state x _{t +1} , q(•|x,a) is the probability of the reward R(x _t ,a _t ) for the action , p ₀ means the initial probability distribution.

Reinforcement learning takes action in consideration of how much reward you will receive for each action when you take action in the current state. A good action is selected by taking the actions that can receive the highest reward in succession. The sum of all rewards finally received is called a Q-value and is expressed as Equation 2 below. The sum of all rewards Q(s,a) that can be received when taking action a in the current state s can be calculated as the sum of the immediate reward for taking the current action and the maximum value of the future reward received in the future.

Here, r(s,a) represents the immediate reward received when taking action a in the current state s. s' is the next state reached by taking action a in the current state s. max _a Q(s',a) is the maximum value of the reward that can be received in the next state s'. γ is a value that adjusts the importance of the future value with the discount rate. The larger the discount rate value, the greater the value of future rewards, and the smaller the discount rate, the more important the immediate rewards. The goal of reinforcement learning is to choose the behavior that maximizes the value of Q(s,a).

The Q-table represents the action value function value for the action taken in the current state. The Q table is initially initialized to a random value and is updated as shown in Equation 3 below as learning proceeds.

A method of creating a Q table for actions and continuously updating this table is called Q-learning. If the Markov state assumption is valid, Equation 3 proves that the recursive property can propagate future rewards to the distant past. A model that receives the current state value as an input value without updating the Q table and predicts the Q value for actions that can be taken in the current state is called a Q-Network.

A method of learning Q networks using convolution networks is called DQN (Deep Q-Network). DQN can be effectively learned using a convolutional network when high-dimensional data (image, video, etc.) is given as a state as input. A detailed description of the operation of the DQN will be omitted. The structure of the DQN may vary. In the technology to be described below, a Q value for GOP determination can be predicted using DQN.

As described above, HEVC may selectively use any one of the encoding modes of the AI mode, the low-delay mode, and the random access mode according to the purpose of the application service. In the random access mode, encoding is performed with reference to both pictures encoded and decoded at a previous time and a subsequent time based on a picture to be encoded. Since hierarchical encoding is performed using reference pictures in two directions, it is possible to obtain higher compression performance than the low-delay mode using only reference pictures in the previous time direction.

1 shows a hierarchical coding structure in a random access mode. 1 is a case in which the GOP size is 8; If the GOP is set to 16 or 32, the hierarchical structure is changed accordingly, and the reference frame of the currently encoded frame is also changed.

In FIG. 1, the numbers in the rectangle indicate the encoding order. It can be seen that the I-frame is coded 0th first. Then, P (or B) frames are coded with reference to the I frames and the B frames in between are coded. These three frames are coded as QP=I, QP=I+1, and QP=I+2, respectively. Such a frame is said to have the lowest depth hierarchically. Or say Temporal ID = 0.

Next, frames between frames belonging to Temporal ID = 0 are coded. For example, a frame located in the middle of the 0th and 2nd coded frames is coded next. In addition, a frame located in the middle of the first and second coded frames is coded next. These frames are given Temporal ID = 1. In the same way, frames assigned Temporal ID = 2 are coded.

POC (Picture of Count) is an index given according to the time sequence in the GOP. It has values from POC = 0 to POC = 8 in order from the 0th first frame to the 8th frame.

In HEVC, the QP size for each frame is determined according to the encoding scenario, and encoding proceeds according to this. For example, in the case of the GOP 8 scenario as shown in FIG. 1, if the QP of the INTRA frame is I, the QP of the next encoded frame is encoded as I+1, and the QP of the next encoded frame is encoded as I+2.

In the following description, an apparatus for selecting a GOP is an apparatus for processing an image. Accordingly, the image processing apparatus selects the GOP. Furthermore, since the GOP is selected by analyzing the input image, the device for selecting the GOP may be called an analysis device. Hereinafter, it is assumed that the analyzer performs the process of selecting the GOP. The analysis device corresponds to a device capable of receiving, processing, and calculating an image. The analysis device may be a device such as a computer device, a smart device, or a server on a network. Meanwhile, the analysis device may be an encoder that encodes an image.

2 is an example of a reinforcement learning environment for GOP selection. In reinforcement learning, the agent and the environment exchange behaviors, states, and rewards.

In FIG. 2, the environment has an adaptive GOP binary tree structure. The GOP tree is composed of nodes in a tree structure. Each node n means encoding of one GOP or sub-GOP, and is defined in the form of n(S, L). Here, S is the start frame, and L is the number of frames configured in one GOP or sub GOP. 2 shows an example of a GOP tree for a video encoded from POC 0. As nodes branch, the depth of the tree increases. As the depth increases, a leaf node is reached. A corresponding leaf node represents a determined structure for one GOP or sub GOP. When one GOP or sub GOP is determined, the encoding order in the corresponding section is determined according to the hierarchical B structure.

In FIG. 2 , an action is composed of U (undetermined) and D (determined). Selection U branches from the current state node in the GOP tree and moves to the next depth state, and selection D determines the current node as a leaf node and is a frame of length L to be encoded from the start POC S frame [S:S+L] ] means to encode. If the encoding method is determined for all input frames, GOP selection for the input video is finished, and the reinforcement learning episode is ended.

In FIG. 2, the compensation uses encoding efficiency provided when the selected GOP is used. The encoding efficiency is used by calculating the rate-distortion (RD) cost J. The RD cost is defined as in Equation 4 below.

where D is the distortion, R is the required bit, and λ is a constant.

The analyzer grants a bonus point (r _t >0) if the selected GOP provides a cost J that is lower than the current cost. If the selected GOP provides a cost J greater than or equal to the current cost, the analysis device gives a deduction (r _t ≤ 0). In the case where the stop condition is satisfied and in the case where the stop condition is not satisfied, each y _t is determined as in Equation 5 below.

where Q is a value function with a weight θ as an action, r _t is a reward, and φ _t is a sequence in t steps.

DQN can obtain the greatest reward when the actual value function Q reaches the true value. The analysis device defines the cost function by Equation 6 below and updates the gradient.

3 is an example of an adaptive GOP tree structure. The adaptive GOP tree is used as an environment for determining the GOP of the input video using reinforcement learning. As shown in FIG. 3, the GOP tree is composed of nodes and paths, and each node is composed of an encoding start POC and the number of frames to be encoded. The adaptive GOP tree starts with the first n ₀ and is expressed as in Equation 7 below.

Here, S means the start POC of the input video and L means the length. For example, when S = 0 and L = 32, n ₀ = [0,32] and coding of 32 frames is considered based on POC 0. 3 is an example of a case where the minimum unit of a GOP is 8 and the maximum unit is 32. FIG. The tree structure composes a deeper tree as the minimum unit of GOP is smaller. When the depth d is increased at an arbitrary node n, the left node and the right node branch are connected based on the current node, and it is expressed as follows.

In this case, P _d (S) is the starting frame of the parent node of the current branch node. The left node follows the starting POC to be coded of the parent node and its length is set to 1/2 of L. The right node has the starting point of the POC moved by L/2 ¹ to the S of the parent node, and the length is set to 1/2 of L. For example, if S = 0, L = 32, then n _1,l = [0,16], and 16 frames of coding are considered based on POC 0. n _1,r = [16,16], and 16 frame coding is considered based on POC 16.

The length of a node cannot be divided smaller than the minimum unit. That is, a depth d that satisfies L/2 ^d = L _min becomes the depth of the adaptive tree.

In the adaptive GOP tree structure, a path includes a path that selects itself and a path that branches to the next depth. The path at each node is determined by the behavior of reinforcement learning. In the adaptive GOP tree structure, it consists of actions U (undetermined) and D (determined). When action D is selected, the analysis device determines the corresponding node as the final path, and when U is selected, it branches to the next depth and determines the path again. At this time, the branch consists of a right node and a left node, and an action is selected for each node.

When depth d = 0, since there is only one node n ₀ , we choose the action U/D from one node. If we choose U at depth d = 0, we choose action U or D from two nodes because there are two nodes n _r ,n _l at depth d = 1.

If the analysis device selects action D and chooses a path to select the node itself, then selects and codes a GOP that has a length up to the set length based on the starting POC of the node.

4 is an example of a GOP selection scenario according to a GOP size. 4 shows the number of GOP cases determined through the adaptive GOP tree structure when the minimum unit of the GOP is 8 and the maximum unit is 32. FIG. 4 is a case of branching based on the GOP tree of FIG.

(i) Case 1 is a case where action D is selected from n ₀ . (ii) Case 2 is a case where action U is selected in n ₀ , then action D is selected in n _l and action D is selected in n _r . (iii) Case 3 branches from a node and selects action D from leaf node n _1,l , selects action D from n ₁ , _r , selects action D from n _r,l , and selects action D from n r,l , n _r , _r This is the case in which action D is chosen. (iv) Case 4 takes action U at n ₀ , action U at n _l , then selects action D at leaf node n _1,l , selects action D at n ₁ , _r , and action U at n _r In case D is selected. (v) Case 5 is a case where action D is selected in n _l , action U is selected in n _r , action D is selected in n _r,l , and action D is selected in n _r , _r .

The action decision is complete when either D is selected in every branch, or the smallest L is encountered. When the action decision is completed, the analysis device determines the GOP according to the final branch path.

Convolutional Neural Network (CNN) can be used to predict behavior given an input video. 5 is an example of a neural network model 100 for determining a branch of a GOP binary tree. The neural network model 100 includes a first input terminal 110 , a second input terminal 120 , and an output terminal 130 . The first input terminal 110 and the second input terminal 120 have the same structure and are configured to extract feature values from the input image, respectively. The input terminals 110 and 120 may include a convolutional layer and a pooling layer. The output terminal 130 fusions the output of the first input terminal 110 and the output of the second input terminal 120 and outputs the final result in all connected layers. The neural network model 100 determines the behavior (whether to branch) at the current node of the GOP binary tree.

Fig. 5 shows a network that predicts the input video and its behavior when the depth is zero. The first input terminal 110 receives the first stream. The first stream is data obtained by concatenating three frames. The first stream is data obtained by combining frames n ₀ , n ₈ and n ₁₆ . The second input terminal 120 receives the second stream. The second stream is data obtained by combining three frames. The second stream is data obtained by combining frames n ₈ , n ₁₆ and n ₃₂ . The input frame input to the input terminal may be determined by Equation 9 below.

S is the starting frame number, L is the length of the frame set at that node, d is the tree depth, P _d (S) is the starting frame of the current node's parent node, and P _d (L) is the frame set at the current node's parent node. The length of n _d,l (S) is the starting frame number of the left child node of the parent node, and n _d,r (L) is the length of the frame set in the right child node of the parent node.

In Equation 9, (1) is input frames generating a first stream, and (2) is input frames generating a second stream.

The output terminal 130 fuses the output of the first input terminal 110 and the output of the second input terminal 120 , and finally outputs information on the action of the input data.

Meanwhile, the artificial neural network 100 may receive an optical flow expressing motion between frames, a motion vector, etc. in addition to the input image.

The output of the artificial neural network 100 comes out as a Q value for the action. The behavior is chosen with the larger Q value of U or D.

The learning process updates the network parameters in the direction that the reward is maximized. Adaptive GOP selection reinforcement learning is defined as one episode for the input video. The criterion for ending an episode is when the union interval of nodes whose branching is terminated due to behavior D in the adaptive GOP tree structure matches L. In this case, the end of the branch corresponds to the case in which action D or depth cannot be deepened due to L _min . L _min is the GOP minimum size. The reward is calculated at the end of the episode, and if the best GOP combination is set for the input video by selecting the maximum reward, the episode proceeds to the next video. The episode of the next video is executed initialized with n ₀ of the adaptive GOP tree, while the reward retains the value obtained from the previous episode. On the other hand, if points are deducted due to incorrect GOP prediction, the parameters of the network are updated through Equation 6, and the input video, reward, and state are all initialized and the reinforcement learning starts again with another video.

Hereinafter, a process of determining a QP (Quantization Parameter) for the determined frame of the GOP will be described. Previous QP determination has been described with reference to FIG. 1 . The technique described below may determine the QP using a technique similar to the above-described GOP selection technique. That is, it is a QP determination technique using reinforcement learning.

The environment may be a QP binary tree similar to a GOP binary tree. Alternatively, the environment may be in the form of a chain. A QP tree or QP chain is made up of state nodes according to actions. The status node means a frame currently being encoded. The action may select a value of α expressed as the difference between the QP size of the current node and the previous (or higher) node in the QP tree or chain. The compensation is the encoding efficiency provided when the selected QP is used. The encoding efficiency can be calculated from the RD cost. In a QP tree or QP chain, depth refers to the sequence of state nodes and successive actions within an episode. Until reinforcement learning ends, the depth increases with successive actions.

If the selected QP chooses a lower RD cost, additional points are awarded as compensation, otherwise, points are awarded. The encoding efficiency may use the cost J of Equation 4 described above.

The analysis device may select the size of the QP according to the input frame using a QP tree or QP chain through a reinforcement learning algorithm. 6 is an example of QP determination using reinforcement learning.

6 is an example of a QP chain in the case of 32 of the GOP. If the QP size of POC 0, which is an INTRA frame, is q, the QP size of the next frame, POC 32, becomes q + α. In the case of 1 person, POC No. 32 is q + 1, and POC No. 16 is also determined to have a size of q + 1 and is coded. In case 2, POC 32 is q + 1 and POC 16 is q + 2, which is the same possible combination as the fixed QP of the existing HEVC. Various values of α may be used.

The adaptive QP selection network may use a network having the same structure as the adaptive GOP selection network. As in the adaptive GOP selection network of FIG. 5, when an input video is given, the QP size is predicted by considering the inter-frame correlation. In training, the network parameters are updated in the direction that the reward is maximized.

If the size of the optimal GOP is determined, the analysis apparatus may determine the optimal QP size for each frame through the QP chain while maintaining the same hierarchical structure of the determined GOP size.

Alternatively, encoding may be performed using only the aforementioned adaptive GOP selection. In this case, the QP setting follows the fixed QP value of the encoding scenario. For example, if the video is determined to be GOP 16, the same QP size in GOP 16 as that of the existing HEVC may be used.

Furthermore, encoding may be performed using only adaptive QP selection. In this case, the GOP size uses a fixed value. For example, if a fixed GOP 8 is used, an optimal QP according to an input video frame may be determined while maintaining the hierarchical structure of the GOP 8.

7 is an example of the analysis device 200 . The analysis device 200 may be a dedicated device for determining only the GOP and/or GP. Alternatively, the analysis device 200 may be an image processing device that processes an input image. For example, the analysis device 200 may be an encoder. The analysis apparatus 200 may be implemented in the form of a computer device capable of processing and analyzing image data, a server of a network, a chipset in which a program is embedded, and the like.

The analysis device 200 includes a storage device 210 , a memory 220 , an arithmetic device 220 , and an interface device 230 . Furthermore, the analysis device 200 may include a communication device 250 .

The storage device 210 may store a GOP binary tree and a reinforcement learning model for determining a path of a binary tree for determining a GOP based on frames.

The storage device 210 may store an input or received input image.

The memory 220 may temporarily store information generated or necessary in the process of image processing and GOP selection.

The interface device 240 means a configuration for receiving data and commands. The interface device 240 may include a physical device and a communication protocol for internal communication. The interface device 240 may receive an input image. The interface device 240 may receive a command for analyzing the input image.

The communication device 250 may receive certain information from an external object through wired or wireless communication. The communication device 200 may receive an input image. The communication device 250 may transmit the determined GPO and/or GP to an external object.

The communication device 250 or the interface device 240 are devices that receive predetermined data or commands from the outside. The communication device 250 or the interface device 240 may be referred to as an input device to receive predetermined data.

The arithmetic unit 230 refers to a configuration for processing given data or information. The computing device 230 may be a device such as a processor, an AP, or a chip in which a program is embedded.

The arithmetic unit 230 inputs frames for an input image input from the input device to a root node of the GOP binary tree, and determines a path of the GOP binary tree.

The computing unit 230 may determine the branch of the GOP binary tree using the aforementioned reinforcement learning model.

The computing unit 230 may determine all paths of the GOP binary tree using the reinforcement learning model, and select the GOP based on the finally determined leaf node.

The computing device 230 may determine the behavior (U or D) of the corresponding node based on the Q value output by the reinforcement learning model receiving a plurality of frames among the frames.

The calculating unit 230 may determine the QP of the selected GOP by using reinforcement learning as described above.

In addition, the image processing method, the GOP selection method, or the QP determination method as described above may be implemented as a program (or application) including an executable algorithm that can be executed in a computer. The program may be provided by being stored in a temporary or non-transitory computer readable medium.

The non-transitory readable medium refers to a medium that stores data semi-permanently, rather than a medium that stores data for a short moment, such as a register, cache, memory, and the like, and can be read by a device. Specifically, the various applications or programs described above are CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM (read-only memory), PROM (programmable read only memory), EPROM (Erasable PROM, EPROM) Alternatively, it may be provided by being stored in a non-transitory readable medium such as an EEPROM (Electrically EPROM) or flash memory.

Temporarily readable media include Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (Enhanced) SDRAM, ESDRAM), Synchronous DRAM (Synclink DRAM, SLDRAM) and Direct Rambus RAM (Direct Rambus RAM, DRRAM) refers to a variety of RAM.

This embodiment and the drawings attached to this specification merely clearly show a part of the technical idea included in the above-described technology, and within the scope of the technical idea included in the specification and drawings of the above-described technology, those skilled in the art can easily It will be said that it is obvious that all inferred modified examples and specific embodiments are included in the scope of the above-described technology.

Claims (12)

receiving, by the analysis apparatus, an input image composed of a plurality of frames;
determining, by the analysis device, a path of a binary tree for determining a GOP (Group of Picture) based on the plurality of frames; and
Comprising the step of the analysis device selecting the GOP of the input image based on the leaf node of the binary tree,
In the analysis apparatus, the environment is the binary tree, the action is whether the tree is branched, and the reward is a GOP selection method based on reinforcement learning that determines the path of the binary tree using reinforcement learning using the encoding efficiency of the selected GOP. The method of claim 1,
A node of the binary tree represents a GOP candidate, and a GOP selection method based on reinforcement learning in which input frames are defined by a start number and length. According to claim 1,
The analysis apparatus is a GOP selection method based on reinforcement learning that determines whether to branch at the node based on a Q value output by an artificial neural network receiving a plurality of frames among frames input to a node of the binary tree. 4. The method of claim 3,
The artificial neural network
a first input terminal receiving data obtained by combining a plurality of frames among frames input to the node;
a second input terminal receiving data obtained by combining a plurality of frames among frames input to the node; and
and an output terminal fusion of the output of the first input terminal and the output of the second input terminal, and outputting the Q value based on the fused data. 4. The method of claim 3,
The artificial neural network is a GOP selection method based on reinforcement learning comprising an input layer that receives data combining frames determined by Equation (1) below, and an input layer that receives data combining frames determined by Equation (2) below. .

(P _d (S) is the starting frame of the parent node of the current node, L is the length set in the current node, P _d (L) is the length of the frame set in the parent node of the current node, n _d,l (S) is The starting frame number of the left child node of the parent node, n _d,r (L) is the length of the frame set in the right child node of the parent node) According to claim 1,
Further comprising the step of the analysis device determining the QP (Quantization Parameter) of the selected GOP using reinforcement learning,
In the reinforcement learning, a GOP selection method based on reinforcement learning, which is the encoding efficiency provided when the state node is the currently encoded frame, the behavior is the QP difference between the current node and the previous node, and the reward is the selected QP. an input device for receiving an input image composed of a plurality of frames;
a storage device for storing a reinforcement learning model for determining a path of a binary tree for determining a GOP (Group of Picture) based on frames; and
and an arithmetic unit for selecting a GOP of the input image based on leaf nodes of the binary tree by applying the plurality of frames to the binary tree,
The computing device is the binary tree, the action is whether the tree is branched, and the reward is the GOP selection based on reinforcement learning that determines the path of the binary tree using reinforcement learning using the encoding efficiency of the selected GOP. analysis device. 8. The method of claim 7,
A node of the binary tree represents a GOP candidate, and an analysis apparatus for selecting a GOP based on reinforcement learning defined by a start number and length of input frames. 8. The method of claim 7,
The arithmetic unit is an analysis device for selecting a GOP based on reinforcement learning that determines whether to branch at the node based on a Q value output by an artificial neural network receiving a plurality of frames among frames input to a node of the binary tree . 10. The method of claim 9,
The artificial neural network
a first input terminal receiving data obtained by combining a plurality of frames among frames input to the node;
a second input terminal receiving data obtained by combining a plurality of frames among frames input to the node; and
An analysis apparatus for selecting a GOP based on reinforcement learning, including an output terminal that fusions the output of the first input terminal and the output of the second input terminal, and outputs the Q value based on the fused data. 10. The method of claim 9,
The artificial neural network selects a GOP based on reinforcement learning including an input layer that receives data combining frames determined by Equation (1) below, and an input layer that receives data that combines frames determined by Equation (2) below. analysis device.

(P _d (S) is the starting frame of the parent node of the current node, L is the length set in the current node, P _d (L) is the length of the frame set in the parent node of the current node, n _d,l (S) is The starting frame number of the left child node of the parent node, n _d,r (L) is the length of the frame set in the right child node of the parent node) 8. The method of claim 7,
The arithmetic unit determines the QP (Quantization Parameter) of the selected GOP using reinforcement learning,
In the reinforcement learning, the state node is the currently encoded frame, the behavior is the QP difference between the current node and the previous node, and the compensation is an analysis device that selects the GOP based on reinforcement learning, which is the encoding efficiency provided when the selected QP is used.

Images