KR102562380B1 - 8K scalable video encoding method - Google Patents

Abstract

A video data encoding method and processing method, and an electronic device performing the method are disclosed. A video data encoding method according to an embodiment of the present invention includes the steps of calculating a complexity adjustment value of video data using video data according to a target bit rate related to encoding of video data, and based on the complexity adjustment value , adjusting the complexity of the video data, and encoding the video data whose complexity is adjusted.

Description

8K scalable video encoding method {8K scalable video encoding method}

It relates to a video data encoding method and processing method according to various embodiments, and an electronic device performing the method. A video data encoding method and processing method for calculating complexity of video data, processing video data according to a complexity adjustment value, and encoding the processed video data, and an electronic device performing the method.

The ATSC 3.0 method has been commercialized as a method for 4K UHD broadcasting service, and recently, SHVC encoding that supports backward-compatibility with the existing 4K UHD broadcasting service to provide 8K UHD broadcasting service that is clearer than 4K UHD. Research on an 8K UHD scalable broadcasting service method using the method is in progress.

When encoding is performed in a fixed bit rate mode in a scalable video data encoding method such as the SHVC encoding method, an encoding skip mode is used to forcibly partially copy the previous image frame data in a section where the complexity of the image is very high, and the output bit rate is Encode video data to lower .

According to various embodiments, a video data encoding method and processing method, and an electronic device performing the method, may occur when video data is encoded using an encoding skip mode in a section with high complexity of an image according to a fixed bit rate. Image quality deterioration can be improved.

A video data encoding method according to various embodiments includes the steps of calculating a complexity adjustment value of video data using the video data according to a target bit rate related to encoding of the video data, based on the complexity adjustment value, The method may include adjusting complexity of video data and encoding the video data having the adjusted complexity.

The method of encoding video data may further include downsampling the video data, and calculating the complexity adjustment value may calculate the complexity adjustment value using the downsampled video data.

The calculating of the complexity adjustment value may include calculating the spatial complexity of the video data based on the spatial distribution of the luminance component of the video data, and the video data based on the temporal distribution of the luminance component of the video data. The method may include calculating temporal complexity of data and determining the complexity adjustment value based on the target bit rate, the spatial complexity, and the temporal complexity.

The calculating of the spatial complexity may include calculating the spatial complexity based on the variance of the luminance component for each frame of the video data and for each preset region, and calculating the temporal complexity may include: Calculating the temporal complexity based on the difference between the luminance component of a frame and the next frame, and determining the complexity adjustment value may include the calculated spatial complexity and temporal complexity for each frame of the video data. Based on, The complexity adjustment value may be determined.

In the adjusting of the complexity of the video data, sharpness and frame of the video data may be adjusted based on the complexity adjustment value.

In the adjusting the complexity of the video data, the sharpness may be adjusted by filtering the video data, and the frame may be adjusted by sampling the video data.

The video data is 8K video data, and the encoding of the video data with the adjusted complexity may encode the video data with the adjusted complexity according to the SHVC encoding method.

A video data processing method according to various embodiments includes calculating a complexity adjustment value of the video data using the video data according to a target bit rate related to encoding of the video data, and based on the complexity adjustment value, It may include adjusting the complexity of video data.

The calculating of the complexity adjustment value may include calculating the spatial complexity of the video data based on the spatial distribution of the luminance component of the video data, and the video data based on the temporal distribution of the luminance component of the video data. The method may include calculating temporal complexity of data and determining the complexity adjustment value based on the target bit rate, the spatial complexity, and the temporal complexity.

The calculating of the spatial complexity may include calculating the spatial complexity based on the variance of the luminance component for each frame of the video data and for each preset region, and calculating the temporal complexity may include: Calculating the temporal complexity based on the difference between the luminance component of a frame and the next frame, and determining the complexity adjustment value may include the calculated spatial complexity and temporal complexity for each frame of the video data. Based on, The complexity adjustment value may be determined.

In the adjusting of the complexity of the video data, sharpness and frame of the video data may be adjusted based on the complexity adjustment value.

In the adjusting the complexity of the video data, the sharpness may be adjusted by filtering the video data, and the frame may be adjusted by sampling the video data.

An encoding device according to various embodiments includes a processor and a memory electrically connected to the processor and storing instructions executed by the processor, wherein the processor is related to encoding video data when the instructions are executed. According to the target bit rate, a complexity adjustment value of the video data is calculated using the video data, the complexity adjustment value of the video data is adjusted based on the complexity adjustment value, and the complexity-adjusted video data is encoded. can do.

The processor may down-sample the video data and calculate the complexity adjustment value using the down-sampled video data.

The processor calculates the spatial complexity of the video data based on the spatial distribution of the luminance component of the video data, and calculates the temporal complexity of the video data based on the temporal distribution of the luminance component of the video data; , The complexity adjustment value may be determined based on the target bit rate, the spatial complexity, and the temporal complexity.

The processor calculates the spatial complexity based on the variance of the luminance component for each frame of the video data and for each preset region, and calculates the spatial complexity based on a difference between the luminance component of a previous frame and a next frame of the video data. Temporal complexity may be calculated, and the complexity adjustment value may be determined based on the spatial complexity and the temporal complexity calculated for each frame of the video data.

The processor may adjust the sharpness and frame of the video data based on the complexity adjustment value.

The processor may adjust the sharpness by filtering the video data, and adjust the frame by sampling the video data.

The video data is 8K video data, and the processor may encode the video data whose complexity is adjusted according to the SHVC encoding method.

A video data encoding method according to various embodiments includes downsampling video data, based on a spatial distribution of a luminance component for each frame set in advance for each frame of the downsampled video data, the downsampled video data Calculating the spatial complexity of the downsampled video data based on the difference according to the temporal distribution of the luminance component of the previous frame and the next frame of the downsampled video data, Downsampling calculating a complexity based on the spatial complexity and the temporal complexity calculated for each frame of the video data; a complexity adjustment value including a spatial complexity adjustment value and a temporal complexity adjustment value based on the complexity and a target bit rate; Determining, based on the complexity adjustment value, filtering the video data to adjust the sharpness, based on the complexity adjustment value, sampling the video data to adjust the frame and the sharpness and the frame and encoding the adjusted video data.

According to an embodiment of the present invention, in a scalable video data encoding method, it is possible to prevent rapid picture quality deterioration due to an encoding skip mode according to a target bit rate in a section where the complexity of video data is very high.

According to an embodiment of the present invention, when encoding 8K video data according to the SHVC encoding method, the complexity of the video data is calculated, and the video data is processed using a complexity adjustment value calculated according to the complexity, thereby achieving a target bit rate. Accordingly, video data may be encoded, and quality degradation of restored video data may be improved.

1 is a diagram illustrating the operation of an encoding device according to various embodiments.
2 is a diagram illustrating an operation of performing a video data encoding method by an encoding device according to various embodiments.
3 is a diagram illustrating an operation of calculating a complexity adjustment value of video data by an encoding apparatus according to various embodiments.
4 is a diagram illustrating a preset region for an encoding device to calculate spatial complexity according to various embodiments.
5 is a diagram illustrating a preset unit area for calculating spatial complexity by an encoding apparatus according to various embodiments.
6 is a diagram illustrating an operation of calculating a complexity adjustment value of video data by an encoding apparatus according to various embodiments.
7 is a diagram illustrating an operation of outputting video data of which complexity is adjusted by an encoding device according to various embodiments.
8 is a diagram illustrating an operation of processing video data by an electronic device according to various embodiments.
9 is a diagram illustrating an operation of performing a video data processing method by an electronic device according to various embodiments.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, since various changes can be made to the embodiments, the scope of the patent application is not limited or limited by these embodiments. It should be understood that all changes, equivalents or substitutes to the embodiments are included within the scope of rights.

Terms used in the examples are used only for descriptive purposes and should not be construed as limiting. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as "include" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art to which the embodiment belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in the present application, they should not be interpreted in an ideal or excessively formal meaning. don't

In addition, in the description with reference to the accompanying drawings, the same reference numerals are given to the same components regardless of reference numerals, and overlapping descriptions thereof will be omitted. In describing the embodiment, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the embodiment, the detailed description will be omitted.

1 is a diagram illustrating an operation of an encoding device 100 according to various embodiments.

Referring to FIG. 1 , an encoding device 100 according to various embodiments may include at least one of a processor 120 and an encoder 110. For example, the encoding apparatus 100 may use the processor 120 to output video data whose complexity is adjusted from input video data. The encoding apparatus 100 may encode video data whose complexity is adjusted using the encoder 110 and output encoded video data, for example, a bitstream.

According to an embodiment, the encoding apparatus 100 may calculate complexity of video data. The encoding apparatus 100 may calculate a complexity adjustment value according to a target bit rate using the calculated complexity. The encoding apparatus 100 may process video data according to the complexity adjustment value and output video data with the complexity adjusted.

As shown in FIG. 1 , the encoding apparatus 100 may output video data whose complexity is adjusted based on the complexity of the video data, and encode the output video data whose complexity is adjusted. The encoding apparatus 100 may improve quality degradation of high complexity video data by processing high complexity video data.

For example, when the encoder 110 encodes video data according to the SHVC method and the complexity of the video data is high, an encoding skip mode in which video data of a previous frame is copied and used is used so that the output bit rate is the target bit rate. make it below The use of the encoding skip mode may cause deterioration in the quality of the video obtained by reconstructing the encoded video data.

The processor 120 may, for example, execute software (eg, a program) to control at least one other component (eg, hardware or software component) of the encoding device 100 connected to the processor 120. and can perform various data processing or calculations. According to an embodiment, as at least part of data processing or operation, the processor 120 stores commands or data received from other components (eg, the encoder 110) in a volatile memory (not shown), and Commands or data stored in may be processed, and resultant data may be stored in a non-volatile memory (not shown). According to an embodiment, the processor 120 may include a main processor (eg, a central processing unit or an application processor, or an auxiliary processor (eg, a graphic processing unit, a neural processing unit (NPU)) operable independently of or together with the main processor, image signal processor, sensor hub processor, or communication processor) For example, when the encoding device 100 includes a main processor and an auxiliary processor, the auxiliary processor uses less power than the main processor or It can be set to be function-specific: the secondary processor can be implemented separately from the main processor or as part of it.

A secondary processor is an encoding device, for example, on behalf of the main processor while the main processor is in an inactive (eg sleep) state, or together with the main processor while the main processor is in an active (eg application execution) state. It is possible to control at least some of functions or states related to at least one of the components of 100 (eg, the encoder 110). According to one embodiment, the auxiliary processor (eg, image signal processor or communication processor) may be implemented as part of other functionally related components.

The encoding apparatus 100 according to an embodiment may improve the quality of encoded video data by adjusting the complexity of video data having a high complexity. For example, video data input to the encoder 110 has complexity adjusted according to a target bit rate and a complexity adjustment value, and the encoder 110, even when the complexity of the video data input to the encoding device 100 is high, Encoded video data can be output by using less or less encoding skip mode.

2 is a diagram illustrating an operation of performing a video data encoding method by the encoding apparatus 100 according to various embodiments.

Referring to FIG. 2 , the encoding apparatus 100 according to various embodiments may calculate a complexity adjustment value of video data according to a target bit rate in operation 210 . For example, in operation 210, the encoding apparatus 100 may calculate a complexity adjustment value of video data according to a target bit rate related to video data encoding.

As an example, the target bit rate may mean a parameter related to encoding of video data. For example, the target bit rate may mean the maximum bit rate of encoded video data output from the encoder 110 of the encoding device 100. For example, the encoder 110 of the encoding device 100 may encode video data so that the output bit rate of the encoded video data is equal to or less than the target bit rate.

For example, the encoding apparatus 100 may calculate a complexity adjustment value of video data according to a target bit rate. For example, the target bit rate may have a correlation with the complexity adjustment value. For example, when the target bit rate is high, the encoding apparatus 100 may determine the complexity adjustment value lower than when the target bit rate is low. For example, the target bit rate may have a negative correlation with the complexity adjustment value. As the target bit rate is higher, the encoding apparatus 100 may determine a complexity adjustment value with a lower size for the same video data.

For example, in operation 220, the encoding apparatus 100 may adjust the complexity of video data based on the complexity adjustment value. As an example, the complexity adjustment value may mean a parameter for adjusting the complexity of video data. For example, as the complexity adjustment value increases, the encoding apparatus 100 can greatly reduce the complexity of video data. For example, as the complexity adjustment value decreases, the encoding apparatus 100 may reduce the complexity of video data.

For example, the complexity of video data may mean the temporal and/or spatial distribution of luminance components of the video data. For example, video data having a high spatial complexity may refer to video data having a large deviation in the spatial distribution of luminance components in each frame of the video data. For example, the deviation of the spatial distribution of the luminance component may mean a difference in luminance of each pixel in the same frame.

For example, video data having a high temporal complexity may refer to video data in which a temporal distribution of a luminance component of the video data has a large deviation. For example, the deviation of the temporal distribution of the luminance component may mean a difference in magnitude of the luminance component of the same pixel in the previous frame and the next frame.

For example, the encoding apparatus 100 may output video data whose complexity is adjusted in operation 220 . For example, the video data whose complexity is adjusted may refer to video data whose deviation in spatial distribution and/or temporal distribution of the luminance component of the video data is adjusted.

For example, the encoding apparatus 100 may output video data whose complexity is adjusted based on the complexity adjustment value in operation 220 . For example, when the complexity adjustment value is large, the encoding apparatus 100 can greatly reduce the deviation of the spatial distribution and/or temporal distribution of the luminance component of video data. For example, when the complexity adjustment value is small, the encoding apparatus 100 may reduce the deviation of the spatial distribution and/or the temporal distribution of the luminance component of video data to a small extent.

For example, the encoding apparatus 100 may encode video data whose complexity is adjusted in operation 230 . For example, the encoding device 100 may input video data whose complexity is adjusted to the encoder 110 and output encoded video data.

For example, the encoder 110 may output encoded video data based on at least one of a set target bit rate and a quantization parameter. When the complexity of the video data encoded by the encoder 110 is high, the bit rate of the output encoded video data may exceed the target bit rate even if the quantization parameter is adjusted. When the complexity of the input video data is high, the encoder 110 outputs encoded video data using an encoding skip mode, and the quality may be degraded.

For example, video data may be 8K video data. The encoding device 100 may adjust the complexity of 8K video data. The encoder 110 of the encoding device 100 may encode 8K video data whose complexity is adjusted. For example, the encoder 110 may encode 8K video data of which complexity is adjusted according to the SHVC encoding method.

When the complexity of the video data is high, the encoding apparatus 100 may improve quality when the encoded video data is reconstructed by inputting the complexity-adjusted video data to the encoder 110 .

According to an embodiment different from the embodiment shown in FIG. 2 , the encoding apparatus 100 may calculate a complexity adjustment value for each set frame of input video data according to a target bit rate. For example, the set frame may mean the length of video data. For example, when set frames are 30 frames, the encoding apparatus 100 may calculate a complexity adjustment value of video data every 30 frames.

For example, the encoding apparatus 100 may adjust the complexity of video data based on a complexity adjustment value for each set frame. For example, if the set frame is 30 frames, the complexity of video data of 31 to 60 frames may be adjusted without adjusting the complexity of video data of 1 to 30 frames.

The length of the frame set in the above example is illustrative, and the frame set differently from the above example may be set in various ways, such as 60 frames or 120 frames.

3 is a diagram illustrating an operation of calculating a complexity adjustment value of video data by the encoding apparatus 100 according to various embodiments.

Referring to FIG. 3 , the encoding device 100 may downsample video data in operation 310 . For example, the processor 120 of the encoding device 100 may downsample an input video. For example, when the input video data is 8K video data, the processor 120 may down-sample the input video data to 4K or 2K video data.

For example, the processor 120 of the encoding device 100 may down-sample input video data using a Bi-Linear sampling method or a BI-CUBIC sampling method. A method of downsampling input video data by the encoding device 100 is not limited to the above example, and the encoding device 100 may downsample video data by applying various known downsampling methods.

For example, the encoding apparatus 100 may calculate the spatial complexity of the video data based on the spatial distribution of the luminance component of the downsampled video data in operation 320 . For example, the spatial distribution of the luminance component may mean a deviation of the luminance component of pixels in each frame.

For example, the encoding apparatus 100 may calculate the variance of the luminance component for each pre-set area of each frame with respect to video data. For example, when the variance of the luminance component of each pixel in each frame is large, the spatial complexity value calculated by the encoding apparatus 100 may be large. For example, when the deviation of the luminance component of each pixel in each frame is small, the spatial complexity value calculated by the encoding apparatus 100 may be small.

For example, the encoding apparatus 100 may calculate the variance of the luminance component in each frame for each preset region. The encoding apparatus 100 may calculate the luminance component variance of the entire region by using the luminance component variance calculated for each region. For example, spatial complexity may mean a variance of luminance components of an entire region calculated for each frame.

For example, the encoding apparatus 100 may calculate the temporal complexity of the video data based on the temporal distribution of the luminance component of the down-sampled video data in operation 330 . For example, the temporal distribution of the luminance component may mean a difference between luminance components of each pixel in the previous frame and the next frame.

For example, the encoding apparatus 100 may calculate a mean absolute deviation (MAD) of a luminance component for each pixel in a previous frame and a next frame with respect to video data. For example, the temporal complexity may refer to a difference in average absolute values of luminance components calculated for each frame compared to a previous frame.

For example, in operation 340, the encoding apparatus 100 may determine a complexity adjustment value based on the target bit rate, spatial complexity, and temporal complexity. For example, the encoding apparatus 100 may calculate complexity of video data using spatial complexity and temporal complexity.

The encoding apparatus 100 may calculate a complexity adjustment value using complexity according to a target bit rate. For example, the encoding apparatus 100 may determine the complexity adjustment value as 0 when the complexity is lower than a predetermined threshold value according to the target bit rate. For example, the encoding apparatus 100 may determine the complexity adjustment value as a value greater than or equal to 0 when the complexity is higher than a predetermined threshold value according to the target bit rate.

For example, when the complexity adjustment value is 0, the encoding apparatus 100 may not adjust the complexity of video data. For example, as the complexity adjustment value increases, the encoding apparatus 100 can greatly reduce spatial complexity and/or temporal complexity of video data.

FIG. 4 is a diagram illustrating a preset region in which the encoding apparatus 100 according to various embodiments calculates spatial complexity.

4 is a diagram showing each frame of video data as a preset area. For example, each frame of video data may be divided into unit area A. In the example of FIG. 4 , each frame of video data is divided into x * y unit areas A along the horizontal axis and the vertical axis.

FIG. 5 is a diagram illustrating a preset unit area in which the encoding apparatus 100 according to various embodiments calculates spatial complexity.

Referring to FIG. 5 , a unit area of each frame of video data may include a preset number of pixels. For example, as shown in FIG. 5 , the unit area may include 16×16 pixels, a total of 256 pixels. For example, the unit area shown in FIG. 5 may represent the unit area A shown in FIG. 4 .

Referring to FIGS. 4 and 5 , the encoding apparatus 100 calculates the variance of the luminance component for each preset area (eg, unit area A) in each frame of video data, and uses the variance of the set area for the corresponding frame. The variance of the luminance component of can be calculated. The variance of the luminance component of the corresponding frame may mean the spatial complexity of the corresponding frame.

For example, the encoding apparatus 100 may calculate the variance of the luminance component in units of 16x16 blocks (eg, unit area A) for each frame of video data. The encoding apparatus 100 may calculate the variance of the luminance component for each frame using the average of the variance of the luminance component calculated in units of 16x16 blocks. The variance of the luminance component calculated for each frame may mean the spatial complexity of the corresponding frame.

6 is a diagram illustrating an operation of calculating a complexity adjustment value of video data by the encoding apparatus 100 according to various embodiments.

Referring to FIG. 6 , the encoding apparatus 100 according to various embodiments may calculate spatial complexity for each frame of video data in operation 610 based on the variance of the luminance component calculated in a preset area.

For example, as shown in FIGS. 4 and 5, each frame of video data may be divided into preset regions. The encoding apparatus 100 may calculate the variance of the luminance component for each unit area A of FIG. 4 . For example, the encoding apparatus 100 may calculate the variance of the luminance component for each unit area A as shown in Equations 1 and 2 below.

[Equation 1]

[Equation 2]

In Equations 1 and 2 above, Is the variance of the luminance component of the unit area at position (x, y) in frame t, is the luminance component of the (i, j) pixel in the unit area at the (x, y) position in frame t, Means the average of the luminance components of the unit area at the position (x,y) in the frame t.

For example, the unit area at the (x, y) position may refer to a unit area located at the x-th position of the horizontal axis and the y-th position of the vertical axis in FIG. 4 . For example, the (i,j) pixel may refer to a pixel located at an i-th position on a horizontal axis and a j-th position on a vertical axis in FIG. 5 .

For example, the encoding device 100 calculates the variance of the luminance component of each unit area calculated in Equations 1 and 2 above. The variance of the luminance component of the entire area can be calculated for each frame by using .

For example, the encoding apparatus 100 calculates the variance of the luminance component of the entire area in each frame as shown in Equation 3 below. can be calculated. Dispersion of the luminance component of the entire area for each frame may mean the spatial complexity of each frame.

[Equation 3]

For example, in operation 620, the encoding apparatus 100 may calculate temporal complexity based on a difference between a luminance component of a previous frame and a next frame of video data. For example, the encoding apparatus 100 may calculate a mean absolute deviation (MAD) of luminance components of a previous frame and a next frame of video data.

For example, the encoding apparatus 100 calculates the average absolute value difference between the luminance components of the previous frame and the next frame as shown in Equation 4 below. can be calculated. Average absolute value difference of the luminance component of the previous frame and the next frame may mean temporal complexity in frame t.

[Equation 4]

In Equation 4, N is the number of horizontal axes of all pixels, M is the number of vertical axes of all pixels, may mean the luminance component of the (i,j) pixel in frame t. for example, may mean a luminance component of an (i,j) pixel among all NxM pixels.

Spatial complexity and temporal complexity calculated according to Equations 3 and 4 above may mean statistical data for video data.

For example, the encoding apparatus 100 may determine a complexity adjustment value based on spatial complexity and temporal complexity calculated for each frame of video data in operation 630 .

For example, the encoding apparatus 100 may calculate complexity k as shown in Equation 5 below.

[Equation 5]

In Equation 5, T denotes a total frame of video data, Is the spatial complexity in frame t calculated according to Equation 3 above, May mean the temporal complexity in frame t calculated according to Equation 4 above.

For example, the encoding apparatus 100 may determine a complexity adjustment value according to a target bit rate using the calculated complexity. For example, the encoding apparatus 100 may determine a complexity adjustment value by comparing complexity with a threshold value determined according to a set target bit rate.

For example, the encoding apparatus 100 may analyze statistical data on temporal complexity and spatial complexity and output a complexity adjustment value suitable for a target bit rate.

For example, when the complexity is less than the determined threshold value, the encoding apparatus 100 may determine the complexity adjustment value as 0. When the complexity adjustment value is 0, the encoding apparatus 100 may not adjust the complexity of video data. When the complexity adjustment value is 0, the encoding device 100 may input the input video data to the encoder 110 and output encoded video data.

For example, when the complexity is greater than or equal to the determined threshold value, the encoding apparatus 100 may determine the complexity adjustment value as a value other than zero. As an example, the complexity adjustment value may mean a parameter for the encoding apparatus 100 to adjust the complexity of video data. For example, as the complexity adjustment value increases, the encoding apparatus 100 may process video data so that spatial complexity and/or temporal complexity of the video data are greatly reduced.

For example, the encoding apparatus 100 may determine a complexity adjustment value including a spatial complexity adjustment value and a temporal complexity adjustment value. For example, the encoding apparatus 100 may adjust the spatial complexity of video data using the spatial complexity adjustment value, and adjust the temporal complexity of video data using the temporal complexity adjustment value.

For example, the encoding apparatus 100 may calculate complexity for each set frame section of video data. For example, the encoding apparatus 100 may calculate complexities k1, k2, ..., kn for each of n frame sections of video data, as shown in Equation 6 below.

[Equation 6]

In Equation 6 above, Is the spatial complexity in frame t calculated according to Equation 3 above, May mean the temporal complexity in frame t calculated according to Equation 4 above.

The encoding apparatus 100 may determine a complexity adjustment value for each frame interval using the complexities k1, k2, ..., kn calculated for each frame interval. For example, the encoding apparatus 100 may adjust the complexity of video data using a complexity adjustment value calculated for each frame period. For example, when the complexity adjustment value of the first frame section is 0 and the complexity adjustment value of the second frame section is not 0, the encoding apparatus 100 does not adjust the complexity of the first frame section of video data, and the second frame section complexity can be adjusted.

The sequence of operations illustrated in FIG. 6 is exemplary, and the encoding apparatus 100 may calculate complexity and determine a complexity adjustment value differently from the sequence of operations illustrated in FIG. 6 . For example, the encoding apparatus 100 may calculate temporal complexity, calculate spatial complexity, and determine a complexity adjustment value.

7 is a diagram illustrating an operation of outputting video data having an adjusted complexity by the encoding apparatus 100 according to various embodiments.

Referring to FIG. 7 , the encoding apparatus 100 according to various embodiments may filter video data in operation 710 to adjust sharpness of the video data and output video data having an adjusted spatial complexity.

For example, the encoding apparatus 100 may adjust the spatial complexity of video data based on the spatial complexity adjustment value included in the complexity adjustment value. For example, the encoding apparatus 100 may adjust the spatial complexity of video data by performing low-pass filtering on video data according to the spatial complexity adjustment value.

For example, adjusting spatial complexity of video data may mean adjusting sharpness of video data. For example, video data whose spatial complexity is adjusted may refer to video data whose sharpness is lower than that of the original video data. When video data whose spatial complexity is adjusted is encoded by the encoder 110, it may be encoded with a higher compression rate than when original video data is encoded, and may be encoded with a smaller bit rate.

For example, in operation 720, the encoding apparatus 100 may sample video data of which spatial complexity is adjusted, adjust a frame of the video data, and output video data of which complexity is adjusted.

For example, the encoding apparatus 100 may adjust the temporal complexity of video data based on the temporal complexity adjustment value included in the complexity adjustment value. For example, the encoding apparatus 100 may adjust the temporal complexity of video data by performing frame sampling on video data according to the temporal complexity adjustment value.

For example, adjusting temporal complexity of video data may mean adjusting frame sampling of video data. For example, video data with temporal complexity adjusted may mean video data having a lower frame rate than original video data. When video data whose temporal complexity is adjusted is encoded by the encoder 110, it may be encoded with a higher compression rate than when original video data is encoded, and may be encoded with a smaller bit rate.

For example, the encoding apparatus 100 may output video data of a low frame rate (eg, 30 Hz, 15 Hz, etc.) by performing frame sampling on original video data of 60 Hz according to the temporal complexity adjustment value.

The operation sequence of operation 710 and operation 720 shown in FIG. 7 is exemplary, and the encoding apparatus 100 may output video data whose complexity is adjusted differently from the operation sequence shown in FIG. 7 . For example, the encoding apparatus 100 outputs video data whose temporal complexity is adjusted based on the temporal complexity adjustment value, adjusts the spatial complexity of the output video data whose temporal complexity is adjusted, and controls the video data whose complexity is adjusted. can output

8 is a diagram illustrating an operation of processing video data by an electronic device 800 according to various embodiments.

Referring to FIG. 8 , the processor 820 of the electronic device 800 according to various embodiments may adjust complexity of input video data. For example, the processor 820 may adjust the complexity of video data and output video data with the adjusted complexity.

For example, the electronic device 800 may input video data whose output complexity is adjusted to the encoder 810 . The encoder 810 may encode input video data whose complexity is adjusted.

9 is a diagram illustrating an operation of performing a video data processing method by an electronic device 800 according to various embodiments.

Referring to FIG. 9 , the electronic device 800 according to various embodiments may calculate a complexity adjustment value of video data according to a target bit rate in operation 910 .

For example, the electronic device 800 may calculate the spatial complexity of the video data based on the spatial distribution of the luminance component of the video data. For example, the electronic device 800 may calculate the temporal distribution of the video data based on the temporal distribution of the luminance component of the video data. The electronic device 800 may calculate complexity of video data based on spatial complexity and temporal complexity.

The electronic device 800 may determine a complexity adjustment value according to the complexity of video data. For example, the complexity adjustment value may include a spatial complexity adjustment value and/or a temporal complexity adjustment value.

For example, the electronic device 800 may adjust the complexity of video data based on the complexity adjustment value in operation 920 . For example, the electronic device 800 may adjust the complexity of video data and output video data with the adjusted complexity. Video data whose complexity is adjusted may be input to the encoder 810 and encoded.

For example, the electronic device 800 may adjust the spatial complexity of video data based on the spatial complexity adjustment value. For example, the electronic device 800 may adjust the sharpness of the video data by performing low-pass filtering on the video data.

For example, the electronic device 800 may adjust the temporal complexity of video data based on the temporal complexity adjustment value. For example, the electronic device 800 may perform frame sampling on video data to adjust the frame rate of video data.

Even if the content of the electronic device 800 described in FIGS. 8 and 9 is omitted, the description of the encoding device 100 described in FIGS. 1 to 7 can be applied.

For example, the electronic device 800 may determine the complexity adjustment value substantially the same as the operation of determining the complexity adjustment value by the encoding device 100 described in FIG. 6 . As another example, the electronic device 800 may output video data whose complexity is adjusted in substantially the same manner as the operation of outputting the video data whose complexity is adjusted by the encoding device 100 described in FIG. 7 .

Meanwhile, the method according to the present invention is written as a program that can be executed on a computer and can be implemented in various recording media such as magnetic storage media, optical reading media, and digital storage media.

Implementations of the various techniques described herein may be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or combinations thereof. Implementations may be a computer program product, i.e., an information carrier, e.g., a machine-readable storage, for processing by, or for controlling, the operation of a data processing apparatus, e.g., a programmable processor, computer, or plurality of computers. It can be implemented as a computer program tangibly embodied in a device (computer readable medium) or a radio signal. A computer program, such as the computer program(s) described above, may be written in any form of programming language, including compiled or interpreted languages, and may be written as a stand-alone program or in a module, component, subroutine, or computing environment. It can be deployed in any form, including as other units suitable for the use of. A computer program can be deployed to be processed on one computer or multiple computers at one site or distributed across multiple sites and interconnected by a communication network.

Processors suitable for processing a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from read only memory or random access memory or both. Elements of a computer may include at least one processor that executes instructions and one or more memory devices that store instructions and data. In general, a computer may include, receive data from, send data to, or both, one or more mass storage devices that store data, such as magnetic, magneto-optical disks, or optical disks. It can also be combined to become. Information carriers suitable for embodying computer program instructions and data include, for example, semiconductor memory devices, for example, magnetic media such as hard disks, floppy disks and magnetic tapes, compact disk read only memory (CD-ROM) ), optical media such as DVD (Digital Video Disk), magneto-optical media such as Floptical Disk, ROM (Read Only Memory), RAM (RAM) , Random Access Memory), flash memory, EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), and the like. The processor and memory may be supplemented by, or included in, special purpose logic circuitry.

In addition, computer readable media may be any available media that can be accessed by a computer, and may include both computer storage media and transmission media.

Although this specification contains many specific implementation details, they should not be construed as limiting on the scope of any invention or what is claimed, but rather as a description of features that may be unique to a particular embodiment of a particular invention. It should be understood. Certain features that are described in this specification in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable subcombination. Further, while features may operate in particular combinations and are initially depicted as such claimed, one or more features from a claimed combination may in some cases be excluded from that combination, and the claimed combination is a subcombination. or sub-combination variations.

Similarly, while actions are depicted in the drawings in a particular order, it should not be construed as requiring that those actions be performed in the specific order shown or in the sequential order, or that all depicted actions must be performed to obtain desired results. In certain cases, multitasking and parallel processing can be advantageous. Further, the separation of various device components in the embodiments described above should not be understood as requiring such separation in all embodiments, and the program components and devices described may generally be integrated together into a single software product or packaged into multiple software products. You have to understand that you can.

On the other hand, the embodiments of the present invention disclosed in this specification and drawings are only presented as specific examples to aid understanding, and are not intended to limit the scope of the present invention. In addition to the embodiments disclosed herein, it is obvious to those skilled in the art that other modified examples based on the technical idea of the present invention can be implemented.

100: encoding device
110, 810: encoder
120, 820: processor
800: electronic device

Claims (20)

In the video data encoding method,
calculating a complexity adjustment value of the video data using the video data according to a target bit rate related to encoding of the video data;
reducing the complexity of the video data based on the complexity adjustment value; and
Encoding the video data of which the complexity is reduced
including,
In the step of calculating the complexity adjustment value,
calculating a spatial complexity of the video data based on the spatial distribution of the luminance component of the video data, based on the variance of the luminance component for each frame and each preset region of the video data;
calculating a temporal complexity of the video data based on a temporal distribution of the luminance component of the video data and a difference between the luminance component of a previous frame and a next frame of the video data; and
determining the complexity adjustment value based on the target bit rate, the spatial complexity, and the temporal complexity;
Including, video data encoding method. According to claim 1,
Downsampling the video data
Including more,
In the step of calculating the complexity adjustment value,
The video data encoding method of calculating the complexity adjustment value using the downsampled video data. delete According to claim 1,
The step of determining the complexity adjustment value,
The video data encoding method of determining the complexity adjustment value based on the spatial complexity and the temporal complexity calculated for each frame of the video data. According to claim 1,
Reducing the complexity of the video data,
Based on the complexity adjustment value, the video data encoding method of adjusting the sharpness and frame rate of the video data. According to claim 5,
Reducing the complexity of the video data,
The video data encoding method of controlling the sharpness by filtering the video data and adjusting the frame rate by sampling the video data. According to claim 1,
The video data is 8K video data,
Encoding the video data of which the complexity is adjusted,
According to the SHVC encoding method, encoding the video data whose complexity is adjusted, video data encoding method. In the video data processing method,
calculating a complexity adjustment value of the video data using the video data according to a target bit rate related to encoding of the video data; and
reducing the complexity of the video data based on the complexity adjustment value;
including,
In the step of calculating the complexity adjustment value,
calculating a spatial complexity of the video data based on the spatial distribution of the luminance component of the video data, based on the variance of the luminance component for each frame and each preset region of the video data;
calculating a temporal complexity of the video data based on a temporal distribution of the luminance component of the video data and a difference between the luminance component of a previous frame and a next frame of the video data; and
determining the complexity adjustment value based on the target bit rate, the spatial complexity, and the temporal complexity;
Including, video data processing method. delete According to claim 8,
The step of determining the complexity adjustment value,
The video data processing method of determining the complexity adjustment value based on the spatial complexity and the temporal complexity calculated for each frame of the video data. According to claim 8,
Reducing the complexity of the video data,
Based on the complexity adjustment value, the video data processing method of adjusting the sharpness and frame rate of the video data. According to claim 11,
Reducing the complexity of the video data,
The method of processing video data, wherein the sharpness is adjusted by filtering the video data, and the frame rate is adjusted by sampling the video data. In the encoding device,
processor; and
A memory electrically connected to the processor and storing instructions executed by the processor
including,
the processor,
When the above command is executed,
calculating a complexity adjustment value of the video data using the video data according to a target bit rate related to encoding of the video data;
adjust the complexity of the video data based on the complexity adjustment value;
Encoding the video data whose complexity is adjusted;
the processor,
Calculate spatial complexity of the video data based on the spatial distribution of the luminance component of the video data, based on the variance of the luminance component for each frame and each preset region of the video data;
calculating a temporal complexity of the video data based on a temporal distribution of the luminance component of the video data and a difference between the luminance component of a previous frame and a next frame of the video data;
An encoding device for determining the complexity adjustment value based on the target bit rate, the spatial complexity, and the temporal complexity. According to claim 13,
the processor,
An encoding device that downsamples the video data and calculates the complexity adjustment value using the downsampled video data. delete According to claim 13,
the processor,
and determining the complexity adjustment value based on the spatial complexity and the temporal complexity calculated for each frame of the video data. According to claim 13,
the processor,
Based on the complexity adjustment value, the encoding device for adjusting the sharpness and frame rate of the video data. According to claim 17,
the processor,
An encoding device configured to adjust the sharpness by filtering the video data and to adjust the frame rate by sampling the video data. According to claim 13,
The video data is 8K video data,
the processor,
An encoding device for encoding the video data whose complexity is adjusted according to the SHVC encoding method. In the video data encoding method,
downsampling the video data;
According to the spatial distribution of the luminance component for each region of the downsampled video data, according to the spatial distribution of the luminance component for each region, which is downsampled for each frame of the video data. calculating spatial complexity;
According to the difference between the temporal distribution of the luminance components of the previous frame and the next frame of the downsampled video data, based on the difference between the luminance components of the previous frame and the next frame of the video data, the downsampled video data Calculating the temporal complexity of
calculating a complexity based on a target bit rate, the spatial complexity and the temporal complexity calculated for each frame of the downsampled video data;
determining a complexity adjustment value including a spatial complexity adjustment value and a temporal complexity adjustment value, based on the complexity and the target bit rate;
filtering the video data to reduce sharpness based on the complexity adjustment value;
reducing frames by sampling the video data based on the complexity adjustment value; and
Encoding the video data of which the sharpness and the frame are adjusted
Including, video data encoding method.

Images