CN108259902B

CN108259902B - Video data encoding and video encoder configured to perform video data encoding

Info

Publication number: CN108259902B
Application number: CN201711360590.9A
Authority: CN
Inventors: 田敏秀; 鲁圣昊
Original assignee: Samsung Electronics Co Ltd
Current assignee: Samsung Electronics Co Ltd
Priority date: 2016-12-28
Filing date: 2017-12-15
Publication date: 2022-07-05
Anticipated expiration: 2037-12-15
Also published as: KR20180076591A; CN108259902A; US10313669B2; US20180184084A1

Abstract

An apparatus that may implement a video encoder may include generating a video display that includes groups of coded pictures selected based on a comparison between different groups of coded pictures generated according to separate, independent coding schemes. The input picture may include a plurality of subgroups with one or more input pictures. Each sub-group of pictures may be encoded multiple times according to separate, independent encoding schemes to generate multiple candidate encoding sets associated with each sub-group. For each sub-group, one candidate encoding group may be selected as the encoded picture associated with the sub-group. The selected coded picture may be used to generate a video display including coded pictures associated with a plurality of subgroups.

Description

Video data encoding and video encoder configured to perform video data encoding

Cross Reference to Related Applications

This application claims priority from korean patent application No. 10-2016-.

Technical Field

Example embodiments relate generally to video processing and, more particularly, to a method of encoding video data to adaptively adjust a structure of a group of pictures (GOP), a video encoder performing the method, and an electronic system including the video encoder.

Background

MPEG (moving picture experts group) under ISO/IEC (international organization for standardization/international electrotechnical commission) and VCEG (video coding experts group) under ITU-T (international telecommunication union, telecommunication) are leading standards for video coding/decoding. For example, various international standards for video encoding/decoding, such as MPEG-1, MPEG-2, H.261, H.262 (or MPEG-2 part 2), H.263, MPEG-4, AVC (advanced video coding), HEVC (high efficiency video coding), etc., have been established and used. AVC is also known as H.264 or MPEG-4 part 10, and HEVC is also known as H.265 or MPEG-H part 2. In accordance with the increasing demand for high-resolution and high-quality video, such as High Definition (HD) video, Ultra High Definition (UHD) video, and the like, research is focused on video encoding for achieving improved compression performance.

Disclosure of Invention

Accordingly, the present disclosure is provided to substantially obviate one or more problems due to limitations and disadvantages of the related art.

At least one example embodiment of the present disclosure provides a method of encoding video data, which is capable of adaptively adjusting a structure of a group of pictures (GOP) based on characteristics of an input picture.

At least one example embodiment of the present disclosure provides a video encoder that performs a method of encoding video data.

At least one example embodiment of the present disclosure provides an electronic system including a video encoder.

According to some example embodiments, a method of encoding video data may comprise: a plurality of input pictures is received, the plurality of input pictures including a plurality of subgroups of one or more input pictures, each subgroup having a size that is less than a size of a group of pictures (GOP). The method may include: for each given sub-group, a plurality of candidate encoding groups associated with the given sub-group are generated by encoding one or more input pictures included in the given sub-group a plurality of times according to a separate, independent encoding scheme. The method may include: for each given sub-group, one of the candidate encoding groups associated with the given sub-group is selected as the plurality of encoded pictures associated with the given sub-group by comparing at least one of encoding costs and encoding quality of the candidate encoding groups associated with the given sub-group with each other. The method may include: generating a video display based on the selection, the video display including a plurality of coded pictures respectively associated with the plurality of subgroups.

According to some example embodiments, a video encoder may include a memory storing a program of instructions and a processor. The processor may be configured to execute the program of instructions to: receiving a plurality of input pictures; outputting the plurality of input pictures as a plurality of subgroups of one or more input pictures, each subgroup having a size smaller than a group of pictures (GOP); for each given sub-group, generating a plurality of candidate encoding groups associated with the given sub-group by encoding one or more input pictures included in the given sub-group a plurality of times according to a separate, independent encoding scheme; for each given sub-group, storing a plurality of candidate encoding groups associated with the given sub-group in a cache pool; selecting, for each given sub-group, one of a plurality of candidate encoding groups associated with the given sub-group as a plurality of encoded pictures associated with the given sub-group by comparing at least one encoding parameter of encoding costs and encoding quality of the plurality of candidate encoding groups associated with the given sub-group with each other; and generating a video display based on the selection, the video display including a plurality of coded pictures respectively associated with the plurality of subgroups.

According to some example embodiments, an electronic system may include a memory storing a program of instructions and a processor. The processor may be configured to execute the program of instructions to provide a plurality of input pictures and to encode the plurality of input pictures. The encoding may include: receiving the plurality of input pictures; outputting the plurality of input pictures as a plurality of subgroups with one or more input pictures, each subgroup having a size smaller than a group of pictures (GOP); for each given sub-group, generating a plurality of candidate encoding groups associated with the given sub-group by encoding one or more input pictures included in the given sub-group a plurality of times according to a separate, independent encoding scheme; storing the plurality of candidate encoding sets in a plurality of buffers; selecting, for each given sub-group, one of a plurality of candidate encoding groups associated with the given sub-group as a plurality of encoded pictures associated with the given sub-group by comparing at least one encoding parameter of encoding costs and encoding quality of the plurality of candidate encoding groups associated with the given sub-group with each other; and generating a video display based on the selection, the video display comprising a plurality of coded pictures respectively associated with the plurality of sub-groups.

According to some example embodiments, an electronic device may include a memory storing a program of instructions and a processor. The processor may be configured to execute the program of instructions to: receiving a plurality of input pictures, the plurality of input pictures comprising a plurality of subgroups of one or more pictures, each subgroup having a size less than a group of pictures (GOP); for each given sub-group, generating a plurality of candidate encoding groups associated with the given sub-group by encoding one or more input pictures included in the given sub-group a plurality of times according to a separate, independent encoding scheme; selecting, for each given sub-group, one of a plurality of candidate encoding groups associated with the given sub-group as a plurality of encoded pictures associated with the given sub-group by comparing at least one encoding parameter of encoding costs and encoding quality of the plurality of candidate encoding groups associated with the given sub-group with each other; and generating a video display based on the selection, the video display including a plurality of coded pictures respectively associated with the plurality of subgroups.

In a method of encoding video data and a video encoder according to some example embodiments, one of a plurality of candidate encoding groups (e.g., an optimal one, a most appropriate one) generated by encoding each sub-group a plurality of times may be selected. In other words, the structure of the GOP may not be fixed, but may vary. The optimal coded picture and the optimal picture type of the current picture can be adaptively selected, whereby coding efficiency and performance can be improved. Furthermore, the method of encoding video data according to some example embodiments may be effectively applicable to an example in which the characteristics of a picture change sharply or drastically (e.g., in the case of a scene change).

Drawings

Illustrative, non-limiting example embodiments will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings.

Fig. 1 is a flow chart illustrating a method of encoding video data according to some example embodiments.

Fig. 2 is a block diagram illustrating a video encoder according to some example embodiments.

Fig. 3 is a diagram illustrating an example of a GOP with a regular structure according to some example embodiments.

Fig. 4 is a flow chart illustrating a method of encoding video data according to some example embodiments.

Fig. 5 is a flowchart showing an example of generating the candidate encoding group of the first sub-group in fig. 4.

Fig. 6 is a flowchart showing an example of selecting the first sub-group of coded pictures in fig. 4.

Fig. 7 is a diagram for describing the method of fig. 4 for encoding video data.

Fig. 8 is a diagram for describing a method of encoding video data according to some example embodiments.

Fig. 9 is a flowchart showing another example of selecting the first sub-group of coded pictures in fig. 4.

Fig. 10A, 10B, and 10C are diagrams for describing the method of encoding video data of fig. 4.

Fig. 11 is a diagram for describing the method of fig. 4 for encoding video data.

Fig. 12 is a flowchart showing another example of generating the candidate encoding group of the first sub-group in fig. 4.

Fig. 13 and 14 are diagrams for describing the method of fig. 4 for encoding video data.

Fig. 15 is a block diagram illustrating an example of an encoding module included in a video encoder according to some example embodiments.

Fig. 16 is a block diagram illustrating a video decoder according to some example embodiments.

Fig. 17 is a block diagram illustrating a video encoding and decoding system according to some example embodiments.

Fig. 18 is a block diagram illustrating an electronic system according to some example embodiments.

Detailed Description

Various example embodiments will be described more fully with reference to the accompanying drawings, in which embodiments are shown. This disclosure may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In the present application, like reference numerals refer to like elements.

Fig. 1 is a flow chart illustrating a method of encoding video data, according to some example embodiments. The method shown in fig. 1 may be implemented in whole or in part by one or more video encoders, electronic systems and/or electronic devices as described herein, including the electronic systems and/or electronic devices shown and described with reference to fig. 18.

In some example embodiments, including the example embodiment shown in fig. 1, the video data includes a plurality of pictures, and the plurality of pictures are encoded based on a group of pictures (GOP) and a sub-group. As will be described with reference to fig. 3, the GOP is determined by designating separate sets of pictures of the multiple pictures as one or more intra pictures, each of which is encoded without reference to any other ("separate") pictures of the multiple input pictures (e.g., encoded independently of the other input pictures), and as one or more inter pictures, each of which is encoded with reference to one or more other ("separate") pictures of the multiple input pictures (e.g., encoded in dependence on the one or more other input pictures).

Referring to fig. 1, in a method of encoding video data according to some example embodiments, a plurality of input pictures including at least partially an instance of video data are provided and/or received in units of a plurality of subgroups (e.g., the plurality of input pictures includes a plurality of subgroups and each subgroup includes one or more input pictures) (step S100). The size of each subgroup (e.g., the number of input pictures in each subgroup) is smaller than the size of the GOP (e.g., the number of input pictures in the GOP). The size of a GOP may represent the number of pictures ("quantity") included in a single GOP, and the size of each subgroup may represent the number of pictures included in a single subgroup. For example, a single subgroup may include more than two pictures. The configuration of the GOPs and subgroups will be described with reference to fig. 3 and 7.

Video data may be encoded in units of pictures according to standards such as MPEG-2, h.261, h.262, h.263, MPEG-4, h.264, HEVC, and the like. For example, each picture may correspond to a frame in progressive scan form or a field in interlaced scan form.

A plurality of candidate encoding groups for each sub-group are generated based on encoding one or more input pictures included in each sub-group a plurality of times in an independent and different scheme (e.g., a separate, independent encoding scheme) (step S200). Restated, for each given sub-group, multiple candidate encoding groups associated with the given sub-group may be generated based on encoding one or more input pictures included in the given sub-group multiple times according to separate, independent encoding schemes. For example, a single sub-group may be encoded more than twice in separate and distinct schemes ("encoding schemes"), and then more than two candidate encoding sets (the candidate encoding sets associated with each sub-group) for the single sub-group may be generated. A plurality of (e.g., more than two) encoding operations may be sequentially performed in the unit of a sub group. In a plurality (e.g., more than two) of candidate encoding groups of a single subgroup ("associated with a single subgroup"), the picture type and picture order (e.g., arrangement order) of candidate encoded pictures included in one candidate encoding group may be different from those of candidate encoded pictures included in another candidate encoding group.

For each given sub-group, one of the candidate encoding groups associated with the given sub-group may be selected as the set of one or more encoded pictures associated with the input pictures included in the given sub-group based on comparing at least one of encoding costs and encoding quality of the candidate encoding groups associated with the given sub-group with each other. Accordingly, a plurality of coded pictures corresponding to a plurality of input pictures can be sequentially selected and output in units of subgroups (e.g., separate coded pictures can be selected for separate respective subgroups). Restated, for each given sub-group, one of the plurality of candidate encoding groups associated with the given sub-group may be selected as the plurality of encoded pictures associated with the given sub-group based on comparing at least one encoding parameter of encoding quality and encoding costs of the plurality of candidate encoding groups associated with the given sub-group with each other.

The above-described generation and selection (S200 and S300) may be performed to generate encoded video data including the selected plurality of encoded pictures respectively associated with the plurality of subgroups. In some example embodiments, a video display is generated based at least on the selection such that the video display (also referred to herein simply as "video") includes a plurality of coded pictures respectively associated with a plurality of sub-groups (S400). The video display may be a sequence of coded pictures including a plurality of coded pictures respectively associated with a plurality of sub-groups. Generating the video display may include: the sequence of coded pictures is displayed in a display interface ("input/output (I/O) device") and/or transmitted to a remote device (e.g., via a video stream) such that the sequence of coded pictures is displayed in the display interface of the remote device.

In a method of encoding video data according to some example embodiments, one of a plurality of candidate encoding groups (e.g., an optimal one, a most appropriate one) generated by encoding each sub-group a plurality of times may be selected. In other words, in the method of encoding video data according to some example embodiments, the structure of the GOP may not be fixed but may be variable. The optimal coded picture and the optimal picture type of the current picture can be adaptively selected, whereby coding efficiency and performance can be improved. As a result, the operational performance of a video encoder, electronic system, and/or electronic device configured to at least perform encoding of video data may be improved. Furthermore, the method of encoding video data according to some example embodiments may be effectively applicable to an example in which the characteristics of a picture change sharply or drastically (e.g., in the case of a scene change). As a result, video encoders, electronic systems, and/or electronic devices configured to perform at least encoding of video data may be configured to improve video encoding performance and efficiency in such situations. Thereby, the generation of a video display based on the encoding of the video data can be improved.

Fig. 2 is a block diagram illustrating a video encoder according to some example embodiments. The video encoder shown in fig. 2 may be implemented in whole or in part by one or more electronic systems and/or electronic devices as described herein, including the electronic systems and/or electronic devices shown and described with reference to fig. 18. For example, the video encoder shown in fig. 2 may be implemented in whole or in part by the processor 1030 of the electronic system 1000 of fig. 18, the processor 1030 executing a program of instructions stored on a storage device 1050 (e.g., memory) of the electronic system 1000.

Referring to fig. 2, the video encoder 10 includes an input buffer 100, an encoding module 200, a control module 400, and a buffer pool 500. The video encoder 10 may also include a context buffer 300 and an output buffer 600.

The input buffer 100 receives a plurality of input pictures INP and outputs the plurality of input pictures INP in units of subgroups. The size of each sub-group is smaller than the size of the GOP. For example, a single GOP may include N input pictures in the plurality of input pictures INP, where N is a natural number. Each of the plurality of subgroups SG may include M input pictures, where M is a natural number equal to or greater than 2 and less than N.

The encoding module 200 generates a plurality of candidate encoding groups CG for each of the plurality of subgroups SG by encoding the M input pictures included in each subgroup a plurality of times in an independent and different scheme. For example, each candidate encoding group may include M candidate encoded pictures for each sub-group. A detailed configuration of the encoding module 200 will be described with reference to fig. 15.

In some example embodiments, each of the M candidate coded pictures included in each candidate coding group may be an inter picture. In other words, the method and video encoder according to some example embodiments may be applied or employed only to encode inter pictures. An intra picture may not be encoded multiple times but may be encoded only once. Here, intra pictures that are encoded (e.g., encoded and/or decoded) without reference to other pictures may be referred to as I pictures and/or IDR (instantaneous decoding refresh) pictures, and inter pictures that are encoded with reference to other pictures may be referred to as P pictures (predicted pictures) and/or B pictures (bidirectionally predicted pictures). A P picture can be encoded with reference to a previous picture with respect to the P picture, and a B picture can be encoded with reference to both a previous picture and a following picture with respect to the B picture.

In some example embodiments, the encoding module 200 may perform an encoding operation based on the context value CTX. For example, the context value CTX may be a context representing the current state of the video encoder 10 and/or a context of the current picture or current sub-group to be encoded.

The context buffer 300 may store context values CTX. The context value CTX may be changed or updated after a single encoding operation is performed on the current picture or the current sub-group. Thus, if multiple encoding operations are to be performed on the same ("common") picture or subgroup (e.g., the current picture or current subgroup), the context values CTX should be stored in the context buffer 300 before the first encoding operation is performed. The encoding module 200 may perform a first encoding operation on the current picture or the current sub-group based on the context value CTX, and may perform a second encoding operation and an encoding operation subsequent to the second encoding operation on the current picture or the current sub-group based on the context value CTX loaded from the context buffer 300.

The control module 400 selects one of the plurality of candidate encoding groups CG of each of the plurality of subgroups SG as M encoded pictures of the M input pictures included in each subgroup by comparing at least one encoding parameter among the encoding costs CS and the encoding qualities QS of the plurality of candidate encoding groups CG of each subgroup with each other.

The control module 400 may include a cost/quality calculator (C/Q CAL)410, a cost/quality comparator (C/Q COMP)420, and a cost/quality policy determiner (C/Q POL) 430.

The cost/quality calculator 410 may generate coding costs CS and coding qualities QS for the plurality of candidate coding groups CG. For example, the cost/quality calculator 410 may calculate the encoding cost and encoding quality for each candidate encoding group.

In some example embodiments, the encoding cost may represent the number of bits ("number") of the M candidate coded pictures included in each candidate coded group. The encoding quality may represent at least one of a peak signal-to-noise ratio (PSNR) and a Structural Similarity Index (SSIM) associated with the M candidate coded pictures included in each candidate encoding group. In some example embodiments, the encoding cost and encoding quality may include at least one of various criteria for determining an optimal encoded picture.

The cost/quality policy determiner 430 may set a cost/quality policy for comparing the coding costs CS and the coding qualities QS of the plurality of candidate coding groups CG with each other, and may generate a policy signal PS corresponding to the cost/quality policy.

In some example embodiments, the cost/quality policy may be fixed. For example, the cost/quality policy may be determined at the time of manufacture of the video encoder 10 and may be stored in the video encoder 10. In some example embodiments, the cost/quality policy may be variable. For example, the cost/quality policy may change based on a user's control (e.g., based on a user setting signal) or based on the current picture (e.g., based on results of analyzing the current picture).

Cost/quality comparator 420 may generate selection signal SEL based on a cost/quality policy (e.g., based on policy signal PS) by: the encoding costs CS of the plurality of candidate encoding groups CG are compared with each other, the encoding qualities QS of the plurality of candidate encoding groups CG are compared with each other, or both the encoding costs CS and the encoding qualities QS of the plurality of candidate encoding groups CG are compared with each other. The selection signal SEL may be used to select one of the plurality of candidate encoding groups CG of each of the plurality of sub groups SG as M encoded pictures of the M input pictures included in each sub group.

In some example embodiments, the cost/quality comparator 420 may select a candidate encoding group including a candidate encoded picture having the smallest number of bits as the encoding sub-group. In some example embodiments, the cost/quality comparator 420 may select a candidate encoding group having the highest PSNR or the largest SSIM as the encoding sub-group. In other example embodiments, the cost/quality comparator 420 may comprehensively evaluate the number of bits, PSNR, SSIM, etc. of each candidate encoding group to score each candidate encoding group, and may select the candidate encoding group having the highest score as the encoding sub-group.

The buffer pool 500 may comprise a plurality of buffers (BUF1, BUFI), 510i storing a plurality of candidate encoding groups CG. The buffer pool 500 may select one of the plurality of candidate encoding groups CG for each sub-group based on a selection signal SEL, and may output the selected encoding group SELG.

The output buffer 600 may receive an output from the buffer pool 500 and may output a plurality of encoded pictures ENP corresponding to a plurality of input pictures INP. For example, as will be described with reference to fig. 7, the output buffer 600 may output all the coded pictures included in the selected coding group of each sub-group. For another example, as will be described with reference to fig. 10A, 10B, and 10C, the output buffer 600 may output all or some of the coded pictures included in the selected coding group based on the picture type of the first coded picture in the selected coding group of each sub-group.

In some example embodiments, the encoding operation of the encoding module 200, the storage operation of the buffer pool 500, the overall operation (e.g., calculation, comparison, and selection) of the control module 400, and the output operation of the output buffer 600 may be performed in units of subgroups, as will be described with reference to fig. 4.

Referring to fig. 3, the size of the GOP may be determined by the interval of the designated I pictures, and the structure of the GOP may be determined by the arrangement of the designated P pictures and/or B pictures. The number of bits of encoded data can be reduced by appropriate arrangement of P-pictures and/or B-pictures (i.e., inter-pictures encoded with reference to other pictures). Error propagation through successive inter pictures can be limited and/or prevented by limiting the size of the GOP, i.e., by specifying regularly or irregularly I pictures that are coded without reference to other pictures, such that the GOP includes an irregular arrangement of inter pictures.

Fig. 3 shows an example of GOP setting with a normal size N by regularly specifying I pictures. In other words, the size of the GOP may be fixed. The first to nth pictures designated as I pictures may form a first GOP1, and the (N +1) th to 2 nth pictures designated as the next I pictures may form a second GOP 2. In the same manner, N pictures from the (2N +1) th picture form a third GOP 3.

In the first example case 1 in fig. 3, only P pictures may be repeated in a single GOP. In the second exemplary case 2 in fig. 3, B pictures and P pictures may be alternately repeated in a single GOP. As will be described with reference to fig. 8, when a single GOP is regularly set as in example cases 1 and 2 (e.g., when a single GOP has a regular structure), the encoding efficiency may be reduced.

Although not shown in fig. 3, a single GOP with a regular structure may be determined differently. For example, two B pictures and one P picture may be alternately repeated in a single GOP (e.g., a "B-P" structure), or one B picture and two P pictures may be alternately repeated in a single GOP (e.g., a "B-P" structure or a "P-B-P" structure). For another example, B pictures and P pictures may be repeated in one of various modes in a single GOP.

Further, the picture numbers PN in fig. 3 may represent a display order, and the display order may be substantially the same as the encoding order (e.g., the same within manufacturing and/or material tolerances) or different depending on the structure of the GOP. For example, in the first example scenario 1, the display order may be substantially the same as the encoding order (e.g., the same within manufacturing and/or material tolerances). In the second example case 2, the display order may be different from the encoding order. In the second exemplary case 2, a third picture (e.g., a P picture) must be encoded before a second picture (e.g., a B picture), and then the second picture can be encoded with reference to the encoded third picture.

Hereinafter, example embodiments will be described based on an example in which an encoding operation is performed on a single GOP.

Fig. 4 is a flow chart illustrating a method of encoding video data according to some example embodiments. Fig. 5 is a flowchart showing an example of generating the candidate encoding group of the first sub-group in fig. 4. Fig. 6 is a flowchart showing an example of selecting a first sub-group of coded pictures in fig. 4. Fig. 7 is a diagram for describing the method of fig. 4 for encoding video data. One or more of the operations shown in fig. 4-7 may be implemented in whole or in part by one or more video encoders, electronic systems and/or electronic devices as described herein, including the electronic systems and/or electronic devices shown and described with reference to fig. 18.

Referring to fig. 2, 4 and 7, in a method of encoding video data according to some example embodiments, a plurality of input pictures INP may be received (step S110), and the plurality of input pictures INP may be output in units of subgroups SG (step S120). For example, the plurality of input pictures INP may include N input pictures T1, T2, T3, T4, T5,..., T (N-1), and TN included in a single GOP, where N is a natural number.

Steps S110 and S120 may be included in step S100 of fig. 1 and may be performed by the input buffer 100. From a different perspective, step S120 may be described as an operation of loading a plurality of input pictures INP in units of subgroups SG.

Although not shown in fig. 4, a step of outputting an input picture T1 and a step of generating an encoded picture I1 by encoding an input picture T1, which input picture T1 is arranged at the forefront of the GOP and designated as an intra picture, may also be included between step S110 and step S120.

After step S120, encoded pictures can be sequentially generated in units of subgroups by encoding the input pictures T2 through TN designated as inter pictures.

For example, the candidate encoding groups CG1 and CG2 of the first sub-group SG1 may be independently generated (step S210). The candidate encoding groups CG1 and CG2 may have different picture orders or different picture type arrangements. Further, the optimal coded pictures P2 and P31 of the first sub-group SG1 may be selected based on the candidate coding groups CG1 and CG2 (step S310). In the example of fig. 7, each subgroup may include two input pictures, and then the number of possible picture orders (e.g., the number of types of possible candidate encoding groups) may be 2.

Referring to fig. 2, 4, 5 and 7, in step S210, a context value CTX representing a current state may be stored in the context buffer 300 (step S211a) before a first encoding operation (e.g., arrow (r) in fig. 7) is performed on the input pictures T2 and T3 included in the first sub-group SG 1.

The first candidate encoding group CG1 may be generated by performing a first encoding operation on the input pictures T2 and T3 included in the first subgroup SG1 based on the context value CTX (step S213 a). The first candidate encoding group CG1 may include a first candidate encoded picture P2 and a second candidate encoded picture P31, and may have a first picture order. The first candidate coded picture P2 may be generated by encoding the input picture T2, and the second candidate coded picture P31 may be generated by encoding the input picture T3. The first candidate coded picture P2 and the second candidate coded picture P31 may have the same picture type ("common picture type"). For example, the first candidate coded picture P2 and the second candidate coded picture P31 may be inter pictures, and each of the first candidate coded picture P2 and the second candidate coded picture P31 may be a P picture. In other words, the first picture order may correspond to a "P-P" structure.

A first coding cost and a first coding quality of the first candidate coding group CG1 may be calculated (step S215 a). For example, the first encoding cost may represent the number of bits included in the first candidate coded picture P2 and the second candidate coded picture P31. The first encoding quality may represent at least one parameter of PSNR and SSIM associated with the first candidate coded picture P2 and the second candidate coded picture P31.

After steps S213a and S215a, the context value CTX may be loaded from the context buffer 300 (step S211b) before a second encoding operation (e.g., arrow (c) in fig. 7) is performed on the input pictures T2 and T3 included in the first subgroup SG 1.

The second candidate encoding group CG2 may be generated by performing a second encoding operation on the input pictures T2 and T3 included in the first subgroup SG1 based on the context value CTX loaded from the context buffer 300 (step S213 b). The first encoding operation and the second encoding operation may be performed independently, whereby the second encoding operation may be performed without information of the first encoding operation (e.g., statistical information associated with the first encoding operation). The second candidate encoding group CG2 may include the third candidate encoded picture B2 and the fourth candidate encoded picture P32, and may have a second picture order different from the first picture order. The third candidate coded picture B2 may be generated by encoding the input picture T2, and the fourth candidate coded picture P32 may be generated by encoding the input picture T3. The third candidate coded picture B2 and the fourth candidate coded picture P32 may have different picture types (e.g., may be associated with different picture types). For example, the third candidate coded picture B2 and the fourth candidate coded picture P32 may be inter pictures, the third candidate coded picture B2 may be a B picture, and the fourth candidate coded picture P32 may be a P picture. In other words, the second picture order may correspond to a "B-P" structure.

A second encoding cost and a second encoding quality of the second candidate encoding group CG2 may be calculated (step S215 b).

Steps S211a and S211b may be performed by the encoding module 200 and the context cache 300, steps S213a and S213b may be performed by the encoding module 200, and steps S215a and S215b may be performed by the cost/quality calculator 410.

Referring to fig. 2, 4, 6, and 7, in step S310, it may be determined which of the first encoding candidate group CG1 and the second encoding candidate group CG2 is better by comparing at least one of the first encoding cost and the first encoding quality of the first encoding candidate group CG1 with at least one of the second encoding cost and the second encoding quality of the second encoding candidate group CG2 (step S311). For example, a better or optimal candidate encoding set may be determined based on at least one of the various criteria described with reference to fig. 2. For example, it may be determined that coded pictures with lower cost (e.g., fewer bits) and higher quality (e.g., higher PSNR or higher SSIM) are better.

When it is determined that the first coding candidate group CG1 is better or better than the second coding candidate group CG2 (step S311: yes), the first and second candidate coded pictures P2 and P31 included in the first coding candidate group CG1 can be selected as the coded pictures of the input pictures T2 and T3 included in the first subgroup SG1 (step S313). When it is determined that the second encoding-candidate group CG2 is better or better than the first encoding-candidate group CG1 (step S311: no), the third encoded picture candidate B2 and the fourth encoded picture candidate P32 included in the second encoding-candidate group CG2 may be selected as the encoded pictures of the input pictures T2 and T3 included in the first sub-group SG1 (step S315). Fig. 7 shows an example of a coded picture in which the first candidate coded pictures P2 and P31 are selected as the input pictures T2 and T3.

Although not shown in fig. 6, the candidate code group that is not selected in step S313 or step S315 (e.g., the second candidate code group CG2 in fig. 7) may be deleted from the cache pool 500.

All the candidate coded pictures P2 and P31 included in the selected candidate coding group CG1 may be output ("selected") as coded pictures of the first sub-group SG1 (step S317). In other words, the coded pictures P2 and P31 may be output ("selected") as a result of an optimal encoding operation on the first subgroup SG 1.

Steps S311, S313 and S315 may be performed by the cost/quality comparator 420, and step S317 may be performed by the buffer pool 500 and the output buffer 600.

Referring again to fig. 2, 4 and 7, as with steps S210 and S310, the candidate coding groups CG3 and CG4 of the second subgroup SG2 following the first subgroup SG1 may be independently generated (step S220), and the optimal coded pictures B4 and P52 of the second subgroup SG2 may be selected based on the candidate coding groups CG3 and CG4 (step S320). In the same manner, the candidate coding groups CG (2K-1) and CG2K of the K-th sub-group SGK may be independently generated (step S230), and the optimal coded pictures B (N-1) and PN2 of the K-th sub-group SGK, where K is a natural number less than N, may be selected based on the candidate coding groups CG (2K-1) and CG2K (step S330).

Steps S220 and S230 may be substantially the same as step S210, respectively (e.g., the same within manufacturing and/or material tolerances), and steps S320 and S330 may be substantially the same as step S310, respectively (e.g., the same within manufacturing and/or material tolerances).

For example, in step S220, the context value CTX may be stored in the context buffer 300. The third candidate encoding group CG3 may be generated by performing a third encoding operation (e.g., arrow three in fig. 7) on the input pictures T4 and T5 included in the second sub-group SG2 based on the context value CTX. The third candidate coding group CG3 may include the fifth candidate coded picture P4 and the sixth candidate coded picture P51, and may have the first picture order. A third coding cost and a third coding quality for the third candidate coding group CG3 may be calculated. The context value CTX may be loaded from the context cache 300. The fourth candidate encoding group CG4 may be generated by performing a fourth encoding operation (e.g., arrow (r) in fig. 7) on the input pictures T4 and T5 based on the loaded context value CTX without information of the third encoding operation. The fourth candidate coding group CG4 may include the seventh candidate coded picture B4 and the eighth candidate coded picture P52, and may have the second picture order. A fourth encoding cost and a fourth encoding quality for the fourth candidate encoding group CG4 may be calculated.

In step S320, which of the third candidate encoding group CG3 and the fourth candidate encoding group CG4 is better may be determined by comparing at least one of the third encoding cost and the third encoding quality of the third candidate encoding group CG3 with at least one of the fourth encoding cost and the fourth encoding quality of the fourth candidate encoding group CG4, and then a better or best candidate encoding group may be selected. The seventh candidate coded picture B4 and the eighth candidate coded picture P52 included in the fourth candidate coded group CG4 may be selected as the coded pictures of the input pictures T4 and T5 included in the second subgroup SG2, and all the candidate coded pictures B4 and P52 included in the selected candidate coded group CG4 may be output as the coded pictures of the second subgroup SG 2.

In step S230, the context value CTX may be stored in the context buffer 300. The candidate coding group CG (2K-1) may be generated by performing a coding operation (e.g., arrow(s) c in fig. 7) on the input pictures T (N-1) and TN included in the kth sub-group SGK based on the context value CTX. The candidate coding group CG (2K-1) may include the candidate coded picture P (N-1) and PN1 and may have the first picture order. The coding cost and the coding quality of the candidate coding group CG (2K-1) may be calculated. The context value CTX may be loaded from the context cache 300. The candidate encoding group CG2K may be generated by performing an encoding operation (e.g., arrow (c) in fig. 7) on the input pictures T (N-1) and TN based on the loaded context value CTX without information of the encoding operation (c). The candidate coding group CG2K may include the candidate coded picture B (N-1) and PN2 and may have the second picture order. The coding cost and the coding quality of the candidate coding group CG2K may be calculated.

In step S330, which of the candidate encoding groups CG (2K-1) and CG2K is better may be determined by comparing at least one of the encoding cost and the encoding quality of the candidate encoding group CG (2K-1) with at least one of the encoding cost and the encoding quality of the candidate encoding group CG2K, and then a better or best candidate encoding group may be selected. The candidate coded pictures B (N-1) and PN2 included in the candidate coding group CG2K may be selected as the coded pictures of the input pictures T (N-1) and TN included in the K sub-group SGK, and all the candidate coded pictures B (N-1) and PN2 included in the selected candidate coding group CG2K may be output as the coded pictures of the K sub-group SGK.

In some example embodiments, as shown in fig. 7, if a single GOP includes N input pictures and K subgroups and if a single subgroup includes two input pictures (e.g., M2), then N may be substantially equal (e.g., the same within manufacturing and/or material tolerances) to 2K +1 (e.g., N2K + 1).

In the example of fig. 7, coding operations (r), (c), and (c) corresponding to a first picture order may be referred to as a first coding path, and coding operations (r), (c), and (c) corresponding to a second picture order may be referred to as a second coding path. In the method and the video encoder according to some example embodiments, the encoding operation with respect to the first encoding path and the encoding operation with respect to the second encoding path may not be alternately performed in units of GOPs, but may be alternately performed in units of subgroups. In other words, in the method and the video encoder according to some example embodiments, the encoding order may not be the order of (r), (c), and (c), but may be the order of (r), (c), and (c). Accordingly, the optimal coded pictures I1, P2, P31, B4, P52,. gtoreq, B (N-1), and PN2 can be sequentially provided or output in units of subgroups, coding efficiency and performance can be improved, and coding processing and response can be effectively adapted to a real-time situation.

Unlike the

example cases

1 and 2 in fig. 3, GOPs encoded according to some example embodiments may have an irregular structure. In other words, the coded pictures P2 through PN2 (e.g., inter pictures) may be arranged irregularly in a single GOP that is coded according to some example embodiments. However, although the GOP has an irregular structure, the size of the GOP may be fixed.

Although fig. 7 shows a single GOP encoded according to some example embodiments, each of multiple GOPs may be encoded according to some example embodiments as in the example of fig. 7, and thus may have different structures. Further, the arrangement of the candidate coded pictures and the coded pictures I1 through PN2 may represent a display order, and the display order may be substantially the same as the coding order (e.g., the same within manufacturing and/or material tolerances) or different, depending on the structure of the candidate coding group, as described with reference to fig. 3.

Fig. 8 is a diagram for describing a method of encoding video data according to some example embodiments. One or more of the operations shown in fig. 8 may be implemented in whole or in part by one or more video encoders, electronic systems, and/or electronic devices as described herein, including the electronic systems and/or electronic devices shown and described with reference to fig. 18. In fig. 8, case 1 and case 2 correspond to case 1 and case 2, respectively, in fig. 3, and case 3 may represent a coded picture generated according to some example embodiments. In fig. 8, the horizontal axis and the vertical axis represent the lapse of time t and the number # of bits, respectively.

Referring to fig. 8, input pictures T (X-2), T (X-1), TX, and T (X +1) may be sequentially provided, and a scene change may occur in an xth input picture TX, where X is a natural number.

In the first exemplary case 1 of encoding all input pictures into P pictures P (X-2), P (X-1), PX, and P (X + I), the number of bits included in the xth encoded picture PX of the xth input picture TX may be sharply or drastically increased. In the second exemplary case 2 where input pictures are alternately encoded into P pictures P (X-1) and P (X + I) and B pictures B (X-2) and BX, the number of bits included in the xth encoded picture BX of the xth input picture TX may be smaller than the number of bits included in the xth encoded picture PX in the first exemplary case 1, whereas the number of bits included in another encoded picture (e.g., the (X-2) th encoded picture B (X-2)) may be larger than the number of bits included in the corresponding encoded picture (e.g., the (X-2) th encoded picture P (X-2)) in the first exemplary case 1.

In the third exemplary case 3, in which an input picture is encoded to have an irregular structure according to some exemplary embodiments, the optimal picture type of the current picture may be adaptively selected in real time, whereby the optimal encoded pictures P (X-2), P (X-1), BX, and P (X +1) may be efficiently generated.

Fig. 9 is a flowchart showing another example of selecting the first sub-group of coded pictures in fig. 4. Fig. 10A, 10B, and 10C are diagrams for describing the method of encoding video data of fig. 4. One or more of the operations shown in fig. 9-10C may be implemented in whole or in part by one or more video encoders, electronic systems and/or electronic devices as described herein, including the electronic systems and/or electronic devices shown and described with reference to fig. 18.

Referring to fig. 2, 4, 9, and 10A, steps S110, S120, and S210 may be substantially the same as steps S110, S120, and S210 described with reference to fig. 4 and 5 (e.g., the same within manufacturing and/or material tolerances).

In step S310, steps S311, S313, and S315 of fig. 9 may be substantially the same as steps S311, S313, and S315 of fig. 6, respectively (e.g., the same within manufacturing and/or material tolerances).

All or some of the candidate coded pictures P2 and P31 included in the selected candidate coding group CG1 may be output as coded pictures of the first sub-group SG1, based on the picture type of the front-most coded picture P2 disposed in the selected candidate coding group CG1 (step S318). For example, as shown in fig. 10A, when the coded picture P2 disposed at the forefront of the selected coding group candidate CG1 is a P picture, the coded picture P2 may be output only as a coded picture of the first subgroup SG 1. For another example, as will be described with reference to fig. 10B, when the coded picture B3 disposed at the forefront of the selected candidate coding group CG4 'is a B picture, all the coded pictures B3 and P42 may be output as coded pictures of the second sub-group SG 2'.

Like step S317 in fig. 6, step S318 may be performed by the buffer pool 500 and the output buffer 600.

Referring to fig. 2, 4 and 10B, as with steps S210 and S310, candidate encoding groups CG3 'and CG 4' of the second sub-group SG2 'following the first sub-group SG1 may be independently generated (step S220), and optimal encoding pictures B3 and P42 of the second sub-group SG 2' may be selected based on the candidate encoding groups CG3 'and CG 4' (step S320). According to a result of encoding a previous subgroup (e.g., the first subgroup SG1), two adjacent subgroups (e.g., the first subgroup SG1 and the second subgroup SG 2') may collectively include at least one input picture (e.g., the input picture T3).

For example, when only the encoded picture P2 of the input picture T2 is output as a result of encoding the first subgroup SG1, the second subgroup SG 2' may be set to include two input pictures T3 and T4 following the input picture P2. In other words, the size of second subgroup SG2 'in fig. 10B may be substantially the same as the size of second subgroup SG2 in fig. 7 (e.g., the same within manufacturing and/or material tolerances), and the starting point of second subgroup SG 2' in fig. 10B may be different from the starting point of second subgroup SG2 in fig. 7.

The candidate coding groups CG3 ' and CG4 ' may be generated by sequentially and independently performing coding operations (e.g., arrows (c) and (r) in fig. 10B) on the input pictures T3 and T4 included in the second sub-group SG2 '. The candidate coding group CG 3' may include candidate coded pictures P33 and P41 and may have a first picture order. The candidate coding group CG 4' may include candidate coded pictures B3 and P42, and may have a second picture order. For example, the candidate coded picture P33 in fig. 10B may be substantially the same as the candidate coded picture P31 in fig. 10A (e.g., the same within manufacturing and/or material tolerances), and then the generation of the candidate coded picture P33 may be omitted.

A better or best candidate encoding set (e.g., candidate encoding set CG4 ') may be selected by calculating and comparing the encoding costs and encoding qualities of the candidate encoding sets CG3 ' and CG4 '. Since the coded picture B3 disposed at the forefront of the selected coding candidate group CG4 'is a B picture, all the coded pictures B3 and P42 can be output as coded pictures of the second subgroup SG 2'.

Referring to fig. 10C, as in steps S210 and S310, candidate encoding groups CG5 ' and CG6 ' of a third subgroup SG3 ' following the second subgroup SG2 ' may be independently generated, and optimal encoding pictures P5 and P61 of the third subgroup SG3 ' may be selected based on the candidate encoding groups CG5 ' and CG6 '.

When all the encoded pictures B3 and P42 of the input pictures T3 and T4 are output as a result of encoding the second subgroup SG2 ', the third subgroup SG 3' may be set to include two input pictures T5 and T6 following the input picture T4. There may be no input pictures commonly included in the second subgroup SG2 'and the third subgroup SG 3'.

The candidate encoding groups CG5 'and CG 6' may be generated by sequentially and independently performing encoding operations (e.g., arrows (C ') and (C) in fig. 10C) on the input pictures T5 and T6 included in the third sub-group SG 3'. The candidate coding group CG 5' may include candidate coded pictures P5 and P61 and may have a first picture order. The candidate coding group CG 6' may include candidate coded pictures B5 and P62, and may have a second picture order.

A better or best candidate encoding set (e.g., candidate encoding set CG5 ') may be selected by calculating and comparing the encoding costs and encoding qualities of the candidate encoding sets CG5 ' and CG6 '. Since the coded picture P5 disposed at the forefront of the selected encoding candidate group CG5 'is a P picture, it is possible to output only the coded pictures P5 as the coded pictures of the third subgroup SG 3'.

Unlike the example of fig. 7, the start point of each sub-group can also be adaptively changed in the examples of fig. 10A, 10B, and 10C, whereby coding efficiency and performance can be improved. In the examples of fig. 10A, 10B, and 10C, the number of subgroups included in a single GOP can be increased as compared with the example of fig. 7.

Fig. 11 is a diagram for describing the method of fig. 4 for encoding video data. One or more of the operations illustrated in fig. 11 may be implemented in whole or in part by one or more video encoders, electronic systems and/or electronic devices as described herein, including the electronic systems and/or electronic devices shown and described with reference to fig. 18.

The example of fig. 11 may be similar to the example of fig. 7, except that in the example of fig. 11a single subgroup includes three input pictures.

Referring to fig. 2, 4, 5, and 11, steps S110 and S120 may be substantially the same as steps S110 and S120 described with reference to fig. 4 (e.g., the same within manufacturing and/or material tolerances).

In step S210, an encoding operation (e.g., arrows in fig. 11) may be performed by sequentially and independently performing the input pictures T2, T3, and T4 included in the first subgroup SGA

And

) Candidate code groups CGA and CGB are generated (steps S211a, S213a, S211b, and S213 b). The candidate coding group CGA may include candidate coded pictures P2A, P3A, and P4A, and may have a picture order of "P-P". The candidate coding group CGB may include candidate coded pictures B2A, B3A, and P4B, and may have a picture order of "B-P". The coding cost and the coding quality of the candidate coding groups CGA and CGB may be calculated (steps S215a and S215 b).

In step S310, a better or best candidate coding group (e.g., candidate coding group CGB) may be selected by comparing the coding costs and the coding qualities of the candidate coding groups CGA and CGB.

In some example embodiments, all of the candidate coded pictures B2A, B3A, and P4B included in the selected candidate coding group CGB may be output as coded pictures of the first sub-group SGA. In some example embodiments, all or some of the candidate coded pictures B2A, B3A, and P4B included in the selected candidate coded group CGB may be output as coded pictures of the first sub-group SGA based on the picture type of the coded picture B2A disposed at the forefront of the selected candidate coded group CGB.

In some example embodiments, the encoding operations for each subgroup after the first subgroup SGA may be performed based on the example of fig. 7. In some example embodiments, the encoding operation for each sub-group subsequent to the first sub-group SGA may be performed based on the examples of fig. 10A, 10B, and 10C.

Fig. 12 is a flowchart showing another example of generating the candidate encoding group of the first sub-group in fig. 4. Fig. 13 and 14 are diagrams for describing the method of fig. 4 for encoding video data. One or more of the operations shown in fig. 12-14 may be implemented in whole or in part by one or more video encoders, electronic systems and/or electronic devices as described herein, including the electronic systems and/or electronic devices shown and described with reference to fig. 18.

Referring to fig. 2, 4, 12, and 13, steps S110 and S120 may be substantially the same as steps S110 and S120 described with reference to fig. 4 (e.g., the same within manufacturing and/or material tolerances), and steps S211a, S213a, S215a, S211b, S213b, and S215b may be substantially the same as steps S211a, S213a, S215a, S211b, S213b, and S215b described with reference to fig. 5 and 11 (e.g., the same within manufacturing and/or material tolerances).

After step S215b, the context value CTX may be loaded again from the context buffer 300 (step S211 c). The encoding operation (e.g., arrows in fig. 13) may be additionally performed on the input pictures T2, T3, and T4 included in the first sub-group SGA by based on the loaded context value CTX

) Generating candidatesThe group CGC is encoded (step S213 c). Can operate without coding

And

performs an encoding operation in the case of information of

The candidate coded group CGC may include candidate coded pictures B2B, P3B, and P4C, and may have a picture order "B-P" different from that of the candidate coded groups CGA and CGB. The coding cost and the coding quality of the candidate coding group CGC may be calculated (step S215 c).

In step S310, a better or best candidate coding group (e.g., candidate coding group CGC) may be selected by comparing the coding costs and coding qualities of the candidate coding groups CGA, CGB, and CGC.

In comparison with the example of fig. 11, including an encoding operation may also be included in the example of fig. 13

The third encoding path of (1). In the method and the video encoder according to some example embodiments, encoding operations with respect to the first, second, and third encoding paths may be alternately performed in units of subgroups, whereby optimally encoded pictures may be sequentially provided or output in units of subgroups.

Referring to fig. 2, 4, and 14, steps S110 and S120 may be substantially the same as steps S110 and S120 described with reference to fig. 4 (e.g., the same within manufacturing and/or material tolerances), and steps S211a, S213a, S215a, S211b, S213b, S215b, S211c, S213c, and S215c may be substantially the same as steps S211a, S213a, S215a, S211b, S213b, S215b, S211c, S213c, and S215c described with reference to fig. 12 and 13 (e.g., the same within manufacturing and/or material tolerances).

After step S215c, context values may be loaded again from the context cache 300CTX. The encoding operation (e.g., arrows in fig. 14) may be additionally performed on the input pictures T2, T3, and T4 included in the first sub-group SGA by based on the loaded context value CTX

) Candidate code groups CGD are generated. Can operate without coding

Performing an encoding operation in the case of the information of e

The candidate coded group CGD may include candidate coded pictures P2B, B3B, and P4D, and may have a picture order "P-B-P" different from that of the candidate coded groups CGA, CGB, and CGC. The coding cost and the coding quality of the candidate coding set CGD may be calculated.

In step S310, a better or best candidate coding group (e.g., candidate coding group CGD) may be selected by comparing the coding costs and the coding qualities of the candidate coding groups CGA, CGB, CGC, and CGD.

In contrast to the example of FIG. 13, the inclusion of an encoding operation may also be included in the example of FIG. 14

The fourth encoding path of (1). In the method and the video encoder according to some example embodiments, encoding operations with respect to the first, second, third, and fourth encoding paths may be alternately performed in units of subgroups, whereby optimally encoded pictures may be sequentially provided or output in units of subgroups.

In the examples of fig. 13 and 14, the encoding operation for each sub-group subsequent to the first sub-group SGA may be performed based on the example of fig. 7 or based on the examples of fig. 10A, 10B, and 10C.

In the examples of fig. 11, 13 and 14, each subgroup may include three input pictures, and then the number of possible picture orders (e.g., the number of types of possible candidate encoding groups) may be greater than 3. According to some example embodiments, the number of input pictures included in a single subgroup and/or the number of possible picture orders for a single subgroup may vary.

As will be appreciated by one skilled in the art, the present disclosure may be embodied as a system, method, computer program product, and/or computer program product embodied in one or more computer-readable media having computer-readable program code embodied thereon. The computer readable program code may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, the computer readable medium may be a non-transitory computer readable medium.

Fig. 15 is a block diagram illustrating an example of an encoding module included in a video encoder according to some example embodiments. The encoding module shown in fig. 15 may be implemented in whole or in part by one or more electronic systems and/or electronic devices as described herein, including the electronic systems and/or electronic devices shown and described with reference to fig. 18. For example, the coding modules shown in fig. 15 may be implemented in whole or in part by the processor 1030 of the electronic system 1000 of fig. 18, where the processor 1030 executes a program of instructions stored on a storage device 1050 (e.g., memory) of the electronic system 1000.

Referring to fig. 15, the encoding module 200 may include a mode decision block (MD)210, a compression block 220, an entropy Encoder (EC)230, a reconstruction block 240, and a storage block (STG) 250.

The mode decision block 210 may generate a predicted picture PRE based on the current picture TY and the reference picture REF, and may generate encoding information INF including a prediction mode according to a prediction operation, a result of the prediction operation, a syntax element, a context value, and the like. The mode decision block 210 may include a motion estimation unit (ME)212, a motion compensation unit (MC)214, and an intra prediction unit (INTP) 216. Motion estimation unit 212 may generate or obtain motion vectors. The motion compensation unit 214 may perform a compensation operation based on the motion vector. The intra prediction unit 216 may perform intra prediction. The motion estimation unit 212 and the motion compensation unit 214 may be referred to as an inter prediction unit that performs inter prediction.

The compression block 220 may encode the current picture TY to generate encoded data ECD. The compression block 220 may include a subtractor 222, a transform unit (T)224, and a quantization unit (Q) 226. The subtractor 222 may subtract the predicted picture PRE from the current picture TY to generate a residual picture RES. The transformation unit 224 and the quantization unit 226 may transform and quantize the residual picture RES to generate encoded data ECD.

The reconstruction block 240 may be used to generate a reconstructed picture TY' by inversely decoding the encoded data ECD (e.g., lossy encoded data). Reconstruction block 240 may include inverse quantization unit (Q-1)242, inverse transform unit (T-1)244, and adder 246. The inverse quantization unit 242 and the inverse transformation unit 244 may inverse-quantize and inverse-transform the encoded data ECD to generate a residual picture RES'. The adder 246 can add the residual picture RES 'to the predicted picture PRE to generate a reconstructed picture TY'.

The entropy encoder 230 may perform lossless encoding on the encoded data ECD and the encoding information INF to generate an encoded picture EY. The reconstructed picture TY' may be stored in the memory 250 and may be used as another reference picture for encoding other pictures.

Fig. 16 is a block diagram illustrating a video decoder according to some example embodiments. The video decoder shown in fig. 16 may be implemented in whole or in part by one or more electronic systems and/or electronic devices as described herein, including the electronic systems and/or electronic devices shown and described with reference to fig. 18. For example, the video decoder shown in fig. 16 may be implemented in whole or in part by the processor 1030 of the electronic system 1000 of fig. 18, where the processor 1030 executes a program of instructions stored on a storage device 1050 (e.g., memory) of the electronic system 1000.

Referring to fig. 16, the video decoder 700 may include an Entropy Decoder (ED)710, a prediction block 720, a reconstruction block 730, and a memory 740. The video decoder 700 may generate a reconstructed picture or a decoded picture by inversely decoding a picture encoded by an encoding module (e.g., the encoding module 200 of fig. 15) included in the video encoder.

The entropy decoder 710 may decode the encoded picture EY (e.g., may perform lossless decoding on the encoded picture EY) to generate encoded data ECD and encoding information INF.

The prediction block 720 may generate a prediction picture PRE' based on the reference picture REF and the encoding information INF. The prediction block 720 may include a motion compensation unit 722 and an intra-prediction unit 724, where the motion compensation unit 722 and the intra-prediction unit 724 are substantially the same as the motion compensation unit 214 and the intra-prediction unit 216, respectively, in fig. 15 (e.g., the same within manufacturing and/or material tolerances).

Reconstructed block 730 may include inverse quantization unit 732, inverse transform unit 734, and adder 736. Reconstruction block 730, inverse quantization unit 732, inverse transform unit 734, adder 736, and memory 740 may be substantially the same as reconstruction block 240, inverse quantization unit 242, inverse transform unit 244, adder 246, and memory 250, respectively, in fig. 15 (e.g., the same within manufacturing and/or material tolerances). The reconstructed picture TY' may be used as another reference picture for encoding other pictures or may be provided to a display device (e.g., display device 826 in fig. 17).

In some example embodiments, a single picture may be divided into a plurality of picture blocks, and the encoding module 200 of fig. 15 and the video decoder 700 of fig. 16 may perform encoding and decoding operations in units of picture blocks included in the single picture. For example, each picture block may be referred to as a macroblock in the h.264 standard. Alternatively, each picture block may be referred to as a Coding Unit (CU), a Prediction Unit (PU), or a Transform Unit (TU) in the HEVC standard.

Fig. 17 is a block diagram illustrating a video encoding and decoding system according to some example embodiments. The video encoding and decoding system shown in fig. 17 may be implemented in whole or in part by one or more electronic systems and/or electronic devices as described herein, including the electronic systems and/or electronic devices shown and described with reference to fig. 18. For example, the video encoding and decoding system shown in fig. 17 may be implemented in whole or in part by the processor 1030 of the electronic system 1000 of fig. 18, where the processor 1030 executes a program of instructions stored on a storage device 1050 (e.g., memory) of the electronic system 1000.

Referring to fig. 17, a video encoding and decoding system 800 may include a first device 810 and a second device 820. The first device 810 may communicate with the second device 820 via a channel 830. For example, channel 830 may include a wired channel and/or a wireless channel.

The first device 810 and the second device 820 may be referred to as a source device and a destination device, respectively. For ease of illustration, some elements of the first device 810 and the second device 820 that are not relevant to the operation of the video encoding and decoding system 800 are omitted in fig. 17.

The first device 810 may include a video Source (SRC)812, a video encoder 814, and a Transmitter (TR) 816. Video source 812 may provide video data. The video encoder 814 may encode the video data. The transmitter 816 may transmit the encoded video data to the second device 820 via the channel 813. The video encoder 814 may be a video encoder according to some example embodiments. The second device 820 may include a Receiver (RC)822, a video decoder 824, and a display Device (DISP) 826. Receiver 822 may receive encoded video data transmitted from first device 810. Video decoder 824 may decode the encoded video data. Display device 826 may display video or images based on the decoded video data.

In some example embodiments, a video encoder according to some example embodiments may be incorporated with a video decoder in the same integrated circuit and/or corresponding software, and then the incorporated device may be referred to as a video encoder/decoder (codec).

Fig. 18 is a block diagram illustrating an electronic system according to some example embodiments. In some example embodiments, the electronic system 1000 shown in fig. 18 may be an electronic device.

Referring to fig. 18, electronic system 1000 includes a video source 1010 and a video codec 1020. Electronic system 1000 may also include processor 1030, connection module 1040, storage device 1050, input/output (I/O) device 1060, and power supply 1070.

The video source 1010 provides a plurality of input pictures INP. For example, the video source 1010 may include a video pickup device, memory, and the like.

Video codec 1020 includes a video encoder and a video decoder according to some example embodiments. The video encoder may encode a plurality of input pictures INP.

Processor 1030 may perform various computing functions, such as specific computations and tasks. The connection module 1040 may communicate with external devices and may include a transmitter 1042 and/or a receiver 1044. The storage device 1050 may operate as a data memory for data processed by the electronic system 1000 or as a working memory. The I/O devices 1060 may include at least one input device (e.g., keypad, buttons, microphone, touch screen, etc.) and/or at least one output device (e.g., speaker, display device 1062, etc.). The power supply 1070 may supply power to the electronic system 1000. Processor 1030 may execute one or more programs of instructions stored in storage 1050 to perform some or all of the operations and/or devices illustrated and described herein.

The present disclosure may be applied to various devices and/or systems that encode video data. In particular, some example embodiments of the inventive concept may be applied to video encoders compatible with standards such as MPEG, h.261, h.262, h.263, and h.264. Some exemplary embodiments of the inventive concept may be employed in the following technical fields: such as CATV (cable tv over optical network, copper wire, etc.), DBS (direct broadcast satellite video service), DSL (digital subscriber line video service), DTTB (digital terrestrial television broadcast), ISM (interactive storage medium (optical disc, etc.)), MMM (multimedia mail), MSPN (multimedia service over packet network), RTC (real time session service (video conference, video phone, etc.)), RVS (remote video monitoring), SSM (serial storage medium (digital VTR, etc.)).

The foregoing is illustrative of example embodiments and is not to be construed as limiting thereof. Although a few example embodiments have been described, those skilled in the art will readily appreciate that many modifications are possible in the example embodiments without materially departing from the novel teachings and advantages of the present disclosure. Accordingly, all such modifications are intended to be included within the scope of this disclosure as defined in the claims. Therefore, it is to be understood that the foregoing is illustrative of various example embodiments and is not to be construed as limited to the specific example embodiments disclosed, and that modifications to the disclosed example embodiments, as well as other example embodiments, are intended to be included within the scope of the appended claims.

Claims

1. A video encoder, comprising:

a memory storing a program of instructions; and

a processor configured to execute the program of instructions to:

a plurality of input pictures are received and,

outputting the plurality of input pictures as a plurality of subgroups of one or more input pictures, wherein a size of each subgroup is smaller than a size of a group of pictures, GOP,

for each given sub-group, generating a plurality of candidate encoding groups associated with the given sub-group by encoding one or more input pictures included in the given sub-group a plurality of times according to a separate, independent encoding scheme,

for each given sub-group, storing the plurality of candidate encoding groups associated with the given sub-group in a cache pool,

for each given sub-group, selecting one of the candidate encoding groups associated with the given sub-group as the plurality of encoded pictures associated with the given sub-group by comparing at least one of encoding costs and encoding quality of the candidate encoding groups associated with the given sub-group with each other, and

generating a video display based on the selection, the video display including a plurality of coded pictures respectively associated with the plurality of subgroups.

2. The video encoder of claim 1, wherein, for each given sub-group, generating a plurality of candidate encoding groups associated with the given sub-group comprises:

generating a first candidate encoding group by performing a first encoding operation on a first input picture and a second input picture included in a first sub-group, the first candidate encoding group including a first candidate encoded picture and a second candidate encoded picture, the first candidate encoding group having a first picture order, an

Generating a second candidate encoding group by performing a second encoding operation on the first input picture and the second input picture without using information associated with the first encoding operation, the second candidate encoding group including a third candidate encoding picture and a fourth candidate encoding picture, the second candidate encoding group having a second picture order different from the first picture order.

3. The video encoder of claim 2, the processor further configured to execute the program of instructions to:

storing a context value in a context buffer prior to performing the first encoding operation,

performing the first encoding operation based on the context value, an

Performing the second encoding operation based on the context value.

4. The video encoder of claim 2, wherein the processor is further configured to execute the program of instructions to:

calculating a first encoding cost and a first encoding quality associated with the first candidate encoding group,

calculating a second encoding cost and a second encoding quality associated with the second candidate encoding group,

comparing at least one first coding parameter of the first coding cost and the first coding quality with a corresponding second coding parameter of the second coding cost and the second coding quality, respectively, based on a cost/quality policy, and

selecting one of the first and second candidate encoding groups as first and second encoded pictures associated with the first and second input pictures based on a result of the comparison.

5. The video encoder of claim 4, the processor further configured to execute the program of instructions to:

outputting the first input picture and the second input picture transmitted from the buffer pool based on the comparison.

6. The video encoder of claim 4, the processor further configured to execute the program of instructions to:

outputting both the first coded picture and the second coded picture transmitted from the buffer pool, or outputting only the first coded picture transmitted from the buffer pool, based on the comparison and also based on a picture type of the first coded picture.

7. The video encoder of claim 4, wherein the cost/quality policy is variable.

8. An electronic system, comprising:

a memory storing a program of instructions; and

a processor configured to execute the program of instructions to:

providing a plurality of input screens, an

Encoding the plurality of input pictures, the encoding comprising:

receiving the plurality of input pictures, wherein the input pictures are displayed,

outputting the plurality of input pictures as a plurality of subgroups of one or more input pictures, wherein each subgroup has a size smaller than a size of a group of pictures, GOP,

storing the plurality of candidate encoding sets in a plurality of buffers,

generating a video display based on the selection, the video display comprising a plurality of coded pictures respectively associated with the plurality of subgroups.

9. The electronic system of claim 8, the processor further configured to execute the program of instructions to generate, for each given sub-group, a plurality of candidate encoding groups associated with the given sub-group based on:

Generating a second candidate encoding group including a third candidate encoding picture and a fourth candidate encoding picture by performing a second encoding operation on the first input picture and the second input picture without using information associated with the first encoding operation, the second candidate encoding group having a second picture order different from the first picture order.

10. The electronic system of claim 9, the processor further configured to execute the program of instructions to:

performing the first encoding operation based on the context value, an

Performing the second encoding operation based on the context value.

11. The electronic system of claim 9, the processor further configured to execute the program of instructions to:

comparing at least one first encoding parameter of the first encoding cost and the first encoding quality with a corresponding second encoding parameter of the second encoding cost and the second encoding quality, respectively, based on a cost/quality policy, and

based on the result of the comparison, one candidate encoding group having the first candidate encoded picture and the second candidate encoded picture or having the third candidate encoded picture and the fourth candidate encoded picture is selected as the first encoded picture and the second encoded picture associated with the first input picture and the second input picture.

12. The electronic system of claim 11, the processor further configured to execute the program of instructions to:

based on the comparison, the first input picture and the second input picture sent from a buffer pool are selected.

13. The electronic system of claim 11, the processor further configured to execute the program of instructions to:

outputting both the first coded picture and the second coded picture transmitted from a buffer pool, or outputting only the first coded picture transmitted from a buffer pool, based on the comparing and also based on a picture type of the first coded picture.

14. An electronic device, comprising:

a memory storing a program of instructions; and

a processor configured to execute the program of instructions to:

receiving a plurality of input pictures, the plurality of input pictures comprising a plurality of subgroups of one or more input pictures, each subgroup having a size less than the size of the group of pictures, GOP,

for each given sub-group, generating a plurality of candidate encoding groups associated with the given sub-group based on encoding one or more input pictures included in the given sub-group a plurality of times according to a separate, independent encoding scheme,

15. The electronic device of claim 14, wherein the processor is further configured to execute the program of instructions to generate, for each given sub-group, a plurality of candidate encoding groups associated with the given sub-group based on:

16. The electronic device of claim 15, the processor further configured to execute the program of instructions to:

performing the first encoding operation based on the context value, an

Performing the second encoding operation based on the context value.

17. The electronic device of claim 15, the processor further configured to execute the program of instructions to:

selecting one of the first or second candidate encoding groups as first and second encoded pictures associated with the first and second input pictures based on a result of the comparison.

18. The electronic device of claim 17, the processor further configured to execute the program of instructions to:

outputting the first input picture and the second input picture transmitted from a buffer pool based on the comparison.

19. The electronic device of claim 17, the processor further configured to execute the program of instructions to:

outputting both the first and second coded pictures sent from a buffer pool, or outputting only the first coded picture sent from a buffer pool, based on the comparing and also based on a picture type of the first coded picture.

20. The electronic device of claim 17, wherein the cost/quality policy is variable.