JP2015188249A - Video coding device and video coding method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a video coding device and a video coding method that further improve coding efficiency.SOLUTION: A video coding device comprises control means. The control means generates a B picture by using an inter-picture prediction structure that allows reference from a reference B picture in one GOP (Group Of Picture) to another reference B picture in the GOP.

Description

  The present embodiment relates to a moving image encoding apparatus and a moving image encoding method.

H. is one of the moving image encoding methods. H.264 can refer to a plurality of reference pictures by introducing DPB (Decoded Picture Buffer). The introduction of DPB This contributes to the improvement of encoding efficiency in the H.264 specification. Although the DPB has a restriction on the number of reference pictures due to the upper limit of the size, by using a decoded picture marking process or the like, not only a picture that is close in time to the decoded picture but also a picture that is far away can be referred to.

H. In a moving picture coding system such as H.264, there are an I picture, a P picture, and a B picture. Generally, the generated code amount decreases in the order of I picture, P picture, and B picture. Therefore, the more B pictures, the smaller the code amount of the stream,
Coding efficiency is improved.

In MPEG-2, which is one of the moving image coding systems, the time distance to the picture referred to by the B picture increases as the number of B pictures increases. For this reason, it has been known that the MPEG-2 specification makes it difficult to predict a B picture, resulting in a deterioration in encoding efficiency. Therefore, H.H. H.264 improves coding efficiency by introducing a reference B picture, that is, a picture that enables reference from a B picture to a B picture.

ARIB standard H.264 In the H.264 specification, in order to enable random access reproduction, high-speed reproduction, and the like in broadcasting, distribution, etc., restrictions on the inter-screen prediction structure (GOP (Group of Pictures) structure) are defined as follows. The decoding order of the non-reference B picture and the reference B picture is immediately after the I picture or P picture whose display order is immediately after. Here, the I picture or the P picture is a picture in the same GOP as the non-reference B picture or the reference B picture. A non-reference B picture is either (a) a frame or field pair of an I picture or P picture immediately before or immediately after the display order, or (b) a display order closer to the I picture or P picture immediately before or after the display order. Refer only to the frame or field pair of the reference B picture that is immediately before or after. Reference B
A picture refers to (a) a frame or field pair of an I picture or P picture immediately before or after in the display order, or (b) only a field of a reference B picture constituting the same frame.

The reference relationship between B pictures according to the restriction of the inter-screen prediction structure can take a hierarchical structure in which only reference from the upper hierarchy to the lower hierarchy is possible. As a result, a picture belonging to a certain hierarchy can be decoded without fail if a picture of a hierarchy below itself is decoded. This hierarchical relationship can be used for high-speed playback.

ARIB STD-B32 Part 1 Appendix 2 Chapter 3 3.6

However, it is impossible to refer from a non-reference B picture to a reference B picture under the constraints of the current inter-screen prediction structure. FIG. 9 shows the current ARIB standard H.264. It is a figure which shows the inter-screen prediction structure of each picture contained in GOP used as an example by H.264 specification. According to the reference relationship between pictures, I0 and P4 are the 0th layer, B2 is the 1st layer, and b1 and b3 are the 2nd layer. The 0th layer is composed of an I picture or a P picture. The first layer is composed of reference B pictures. The second layer is composed of non-reference B pictures. For this reason, the reference relationship between the B pictures only has a two-layer structure as shown in FIG. Under the constraints of the current inter-screen prediction structure, when the frame rate of the input image signal is increased, I included in the unit time
The number of pictures or P pictures increases in proportion to the frame rate. As a result, the coding efficiency decreases. Therefore, even if the frame rate of the input image signal is increased, B
If the number of pictures can be increased, encoding efficiency can be further improved.

An object of the present invention is to provide a moving picture coding apparatus and a moving picture coding method that further improve coding efficiency.

According to the embodiment, the moving image encoding apparatus includes a control unit. The control means performs control so that a B picture is generated using an inter-screen prediction structure that enables reference from a reference B picture in one GOP to another reference B picture in the GOP.

The block diagram which shows the structural example of the moving image encoder which becomes an example which concerns on embodiment. The figure which shows the inter-screen prediction structure of the reference B picture used as an example which concerns on embodiment. The figure which shows the inter-screen prediction structure of the non-reference B picture used as an example which concerns on embodiment. The figure which shows the inter-screen prediction structure of each picture contained in GOP used as an example which concerns on embodiment. The figure for demonstrating the high-speed reproduction used as an example which concerns on embodiment. The figure for demonstrating the high-speed reproduction used as an example which concerns on embodiment. The figure for demonstrating the high-speed reproduction used as an example which concerns on embodiment. The figure for demonstrating the change of the reproduction speed used as an example which concerns on embodiment. ARIB standard H.264 The figure which shows the inter-screen prediction structure of each picture contained in GOP used as an example by H.264 specification.

Hereinafter, this embodiment will be described with reference to the drawings.
FIG. 1 is a block diagram illustrating a configuration example of a moving image encoding apparatus as an example according to the embodiment. The moving image encoding apparatus 10 is an apparatus for generating an encoded bit string (encoded data) 260 from an input image signal (image data) 200. The moving image coding apparatus 10 includes a control unit (control means) 101, a subtractor 102, an orthogonal transformer 103, a quantizer 104, an inverse quantizer 105, an inverse orthogonal transformer 106, an adder 107, a loop filter 108, a frame A memory 109, a predicted image generation unit 110, and an entropy encoder 111 are provided.

The control unit 101 controls the operation of each element included in the video encoding device 10.
The subtracter 102 is supplied with an input image signal 200 from the outside, and a predicted image generator 1 described later.
The predicted image signal 250 is also supplied from 10. The subtracter 102 receives the input image signal 200 from
The prediction error signal 210 is obtained by subtracting the prediction image signal 250. The subtracter 102 outputs the prediction error signal 210 to the orthogonal transformer 102.
The orthogonal transformer 103 performs discrete cosine transform, for example, and orthogonally transforms the prediction error signal 210 to obtain orthogonal transform coefficient information 220. The orthogonal transformer 103 includes orthogonal transform coefficient information 220.
Is output to the quantizer 303.
The quantizer 104 quantizes the orthogonal transform coefficient information 220 to obtain quantized orthogonal transform coefficient information (quantized data) 230. The quantizer 104 outputs the quantized orthogonal transform coefficient information 230 to the inverse quantizer 105 and the entropy encoder 111.

The inverse quantizer 105 and the inverse orthogonal transformer 106 locally decode the quantized orthogonal transform coefficient information 230. The inverse orthogonal transformer 106 outputs the locally decoded quantized orthogonal transform coefficient information 230 to the adder 107.
The adder 107 adds the predicted image signal 250 to the locally decoded quantized orthogonal transform coefficient information 230.
To obtain a locally decoded image signal 240. The adder 107 outputs the locally decoded image signal 240 to the loop filter 108. The locally decoded image signal 240 is supplied to the frame memory 308 via the loop filter 306.

The frame memory 109 converts the stored local decoded image signal 240 into the predicted image generator 11.
Supply to zero.
The predicted image generator 110 obtains a predicted image signal 250 based on the locally decoded image signal 240. The predicted image generator 110 outputs the predicted image signal 250 to the subtracter 301 and the adder 315.
The entropy encoding unit 111 obtains an encoded bit string 260 by encoding the quantized orthogonal transform coefficient information 230. The entropy encoding unit 111 includes an encoded bit string 260.
Is output to the outside.

The moving picture encoding apparatus 10 generates an I picture, a P picture, and a B picture with the above configuration, and generates a GOP composed of a plurality of pictures including at least one I picture as an encoded bit string 260. Note that an I picture is a picture generated by being encoded only on the screen. A P picture is a picture generated by being encoded by unidirectional prediction. A B picture is a picture generated by being encoded by bidirectional prediction. B picture
There are two types: B pictures that can be referenced from other pictures (hereinafter referred to as reference B pictures) and B pictures that are not referenced from other pictures (hereinafter referred to as non-reference B pictures).

Next, restrictions on the inter-picture prediction structure relating to the B picture defined in this embodiment will be described. The control unit 101 performs control so as to generate a B picture using at least one of the restrictions on the inter-picture prediction structure for the five B pictures shown in (1) to (5) below. In addition,
An I picture or P picture described below indicates a picture in the same GOP as a non-reference B picture or a reference B picture.

(1) An inter-screen prediction structure that enables reference from a reference B picture to a reference B picture. That is, this inter-screen prediction structure enables reference from a reference B picture in one GOP to another reference B picture in this GOP. The reference from the non-reference B picture to the reference B picture can be performed as in the past (ARIB standard H.264 specification).
(2) An inter-screen prediction structure that enables reference from the B picture to the previous I or P picture in display order. That is, this inter-screen prediction structure is obtained from the first B picture in the GOP.
It is possible to refer to an I picture or P picture whose display order is earlier than the first B picture. If the display order of the B picture is the previous I or P picture, it is possible to refer to an I or P picture other than the previous I or P picture whose display order has been accepted in the past.
(3) An inter-screen prediction structure that disables reference from a B picture to a B picture that is farther than the previous P picture in display order. That is, this inter-screen prediction structure is the first B in the GOP.
Reference from a picture to a second B picture farther than the I picture or P picture immediately before the first B picture is disabled.
(4) An inter-screen prediction structure in which reference from a B picture to a P picture farther than the P picture immediately after the display order is impossible. That is, this inter-screen prediction structure is the first B in the GOP.
Reference from the picture to another I picture or another P picture farther than the I picture or P picture immediately after the first B picture is disabled. In other words, this inter-screen prediction structure is based on the fact that the I picture or the P picture whose display order is later than the first B picture in the GOP starts from the first B picture. Reference is made only to the immediately following I picture or P picture.
(5) An inter-screen prediction structure that refers only to a reference B picture closer to the I picture or P picture immediately before or immediately after the B picture. In other words, this inter-screen prediction structure is related to the reference B picture included in the GOP, in the GOP, from the first B picture, closer to the I picture or P picture immediately before or immediately after the first B picture. It is possible to refer to only the reference B picture.
It should be noted that frames of consecutive B pictures (non-reference B pictures or reference B pictures),
Alternatively, the maximum number of field pairs is set to 7 as an example, unlike conventional restrictions.

FIG. 2 is a diagram illustrating an inter-screen prediction structure of a reference B picture as an example according to the embodiment.
Here, the inter-screen prediction structure will be described using the reference B picture 301 as an example. In FIG. 2, “I” means an I picture, “P” means a picture, “B” means a reference B picture, and “b” means a non-reference B picture. In FIG. 2, the pictures included in one GOP are arranged in the display order. Solid arrows indicate an example of the relationship between the reference B picture 301 and other pictures that can be referred to by the above restrictions ((1), (2), (4), and (5) above). The symbol “◯” shown together with the solid line arrow indicates the AR. It shows that it can be referred to in the H.264 specification. A symbol “」 ”shown together with a solid arrow indicates that the reference can be made due to the restriction defined in the present embodiment. Dashed arrows indicate an example of the relationship between the reference B picture 301 that cannot be referred to due to the restrictions (above (3) and (4)) and other pictures. The numbers shown with the arrows correspond to the numbers of the constraints that apply. Note that an “x” symbol shown with an arrow indicates that reference is impossible.

FIG. 3 is a diagram illustrating an inter-screen prediction structure of a non-reference B picture as an example according to the embodiment. Here, the inter-screen prediction structure will be described using the non-reference B picture 302 as an example. Note that FIG.
“I”, “P”, “B”, and “b” in FIG. 2 mean pictures similar to those in FIG. In FIG. 3, the pictures included in one GOP are arranged in the display order. Solid arrows indicate an example of the relationship between the non-reference B picture 302 and other pictures that can be referred to by the above restrictions ((2), (4), and (5) above). The “O” symbol shown with the solid arrow is the ARI.
B standard H.264 It shows that it can be referred to in the H.264 specification. A symbol “」 ”shown together with a solid arrow indicates that the reference can be made due to the restriction defined in the present embodiment. The dashed arrow
An example of the relationship between the non-reference B picture 301 that cannot be referred to by the above restrictions (above (3) and (4)) and other pictures is shown. The numbers shown with the arrows correspond to the numbers of the constraints that apply. Note that an “x” symbol shown with an arrow indicates that reference is impossible.

As shown in FIGS. 2 and 3, a picture that can be referred to from a reference B picture and a picture that cannot be referred to are the same as a picture that can be referred to from a non-reference B picture and a picture that cannot be referred to.

FIG. 4 is a diagram illustrating an inter-screen prediction structure of each picture included in an exemplary GOP according to the embodiment. In FIG. 4, the pictures included in one GOP are arranged in the display order. The arrows indicate the reference relationship between the pictures according to the restrictions (1) to (5). According to the reference relationship between the pictures, I0 and P8 are the 0th layer, B4 is the 1st layer, B2 and B6 are 2 layers.
Hierarchies b1, b3, b5 and b7 are the third hierarchy. Layer 0 is I picture or P
Consists of pictures. The first layer and the second layer are composed of reference B pictures. The third layer is composed of non-reference B pictures. That is, one GOP can have an inter-screen prediction structure with three or more layers between B pictures. The reference relationship between the B pictures in accordance with the restrictions (1) to (5) described above can take a hierarchical structure of three or more hierarchies in which only reference from the upper hierarchy to the lower hierarchy is possible.

The decoder can decode each picture based on the exemplary inter-screen prediction structure shown in FIG. 4 and display it on the display in the display order. The decoder performs normal reproduction by decoding and displaying all the pictures located in the 0th to 3rd layers included in one GOP shown in FIG. Note that the decoder can perform high-speed playback at 2n times the normal playback speed described with reference to FIG. 4 by decoding only the minimum necessary pictures. 5 to 7 are diagrams for explaining high-speed playback as an example in the hierarchical structure shown in FIG. 5 to 7 arrange the pictures included in one GOP in the display order as in FIG. The arrows indicate the reference relationship between the pictures according to the restrictions (1) to (5). 5 to 7 indicate pictures used for high-speed playback and their reference relations, and broken lines shown in FIGS. 5 to 7 indicate pictures not used for high-speed playback and their reference relations. The high speed reproduction shown in FIG. 5 is a reproduction in which only the picture located in the 0th layer is decoded and displayed. The high speed reproduction shown in FIG. 6 is a reproduction in which only the pictures located in the 0th layer and the 1st layer are decoded and displayed. The high-speed playback shown in FIG. 7 is playback for decoding and displaying only pictures located in the 0th layer to the 2nd layer. The playback speed varies depending on the number of pictures to be decoded and displayed. Therefore, the reproduction speed increases in the order of normal reproduction shown in FIG. 4, high-speed reproduction shown in FIG. 7, high-speed reproduction shown in FIG. 6, and high-speed reproduction shown in FIG.

FIG. 8 is a diagram for explaining an example of changing the playback speed. FIG. 8 is a diagram in which the pictures included in one GOP are arranged in the display order. Here, with regard to B10, a part of the reference relationship that conforms to the above constraint is indicated by an arrow. A solid arrow indicates an example of a relationship between B10 that can be referred to and another picture. “O” sign indicates that reference is possible. A broken arrow indicates an example of a relationship between B10 that cannot be referred to and another picture. The “x” symbol indicates that reference is impossible. As an example, it is assumed that the decoder processes from I0 to B10 before high-speed playback that decodes only pictures located only in the 0th hierarchy described with reference to FIG. Assume that the playback speed is switched to the normal playback speed for playing back the pictures located in the 0th layer to the 3rd layer described with reference to FIG. 4 before B10. B
10 cannot refer to B4 due to the restriction of (3). Therefore, the decoder does not need to decode B4 that has not been decoded in order to decode B10. On the other hand, B10 can refer to not only P8 decoded by high-speed playback but also I0 due to the restriction (2). Since the decoder does not need to decode an undecoded picture only for decoding B10, the playback speed can be easily switched.

According to the restrictions (1) to (5) above, an inter-screen prediction structure having three or more layers can be taken between B pictures. Mainly according to the restrictions (1), (2), and (5), the encoding efficiency can be maintained or improved as much as possible. Mainly according to the restrictions (3) and (4) above, the encoded bit string can be reproduced at a high speed of 2n times on the decoder side, and the reproduction speed can be easily changed. Therefore, according to the present embodiment, the moving picture coding apparatus 10 can improve the coding efficiency without increasing the number of I pictures or P pictures included per unit time even if the frame rate of the input picture signal increases. As a result, it is possible to generate a coded bit string that can realize high-speed reproduction on the decoder side.

Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 10 ... Moving image coding apparatus, 101 ... Control part (control means), 102 ... Subtractor, 103 ... Orthogonal transformer, 104 ... Quantizer, 105 ... Inverse quantizer, 106 ... Inverse orthogonal transformer, 107 ... Addition 108, loop filter, 109, frame memory, 110, predicted image generation unit, 11
1: Entropy encoder.

Claims (2)

Control means for performing control for generating the encoded data in accordance with restrictions when generating encoded data having a picture hierarchy structure of 0th to 3rd layers;
The encoded data is distributed by broadcast waves,
One GOP (Group of Pictures) included in the encoded data includes a plurality of pictures,
The types of pictures included in the encoded data include at least an I picture, a P picture, and a B picture,
The 0th layer has the I picture and the P picture, the 1st to 3rd layers have the B picture,
The B picture is encoded with reference to a plurality of other pictures that are bidirectional in display time,
And a reference B picture that is referred to for encoding the other B picture, and a non-reference B picture that is not referred to for encoding the other B picture,
The P picture of the 0th layer is a unidirectional picture in display time and is encoded with reference to other pictures of the 0th layer,
The encoded data includes normal reproduction for decoding the 0th to 3rd layer pictures, (1) the 1st to 3rd layer pictures, (2) the 2nd and 3rd layer pictures, or (3) A format capable of switching between high-speed playback that omits decoding of the picture of the third layer and in the middle of the encoded data,
The constraints are
Enabling reference from the reference B picture in the one GOP to another reference B picture;
While prohibiting reference to other B pictures in the B picture that are earlier in display time than the I picture or P picture immediately before in display time and in the hierarchy higher than the first layer. , A moving picture that enables reference to the I picture or the P picture immediately before the display time, and to another picture that is before the I picture or the P picture immediately before and in the 0th layer Image encoding device. When generating encoded data having a picture hierarchy structure of 0th to 3rd layers using the first encoding method, control is performed to generate the encoded data according to restrictions, and the encoded data is , Delivered by broadcast waves,
One GOP (Group of Pictures) included in the encoded data includes a plurality of pictures,
The types of pictures included in the encoded data include at least an I picture, a P picture, and a B picture,
The 0th layer has the I picture and the P picture, the 1st to 3rd layers have the B picture,
The B picture is encoded with reference to a plurality of other pictures that are bidirectional in display time,
And a reference B picture that is referred to for encoding the other B picture, and a non-reference B picture that is not referred to for encoding the other B picture,
The P picture of the 0th layer is a unidirectional picture in display time and is encoded with reference to other pictures of the 0th layer,
The encoded data includes normal reproduction for decoding the 0th to 3rd layer pictures, (1) the 1st to 3rd layer pictures, (2) the 2nd and 3rd layer pictures, or (3) A format capable of switching between high-speed playback that omits decoding of the picture of the third layer and in the middle of the encoded data,
The constraints are
Enabling reference from the reference B picture in the one GOP to another reference B picture;
While prohibiting reference to other B pictures in the B picture that are earlier in display time than the I picture or P picture immediately before in display time and in the hierarchy higher than the first layer. , A moving picture that enables reference to the I picture or the P picture immediately before the display time, and to another picture that is before the I picture or the P picture immediately before and in the 0th layer Image coding method.

Images