JP4542064B2 - Inter-layer prediction method, apparatus, program, and recording medium thereof - Google Patents

Description

  The present invention relates to an inter-layer prediction method, apparatus, program, and recording medium for improving intra prediction when performing single loop decoding (SLD) in which reference of inter-layer prediction is limited to intra prediction blocks.

  In recent years, scalable coding to deal with diversifying network environments and terminal environments has attracted attention. In scalable coding, an image signal is divided hierarchically and coding is performed for each layer. As an encoding method corresponding to time, space, and SNR scalability, Scalable Video Coding (SVC) is attracting attention (see Non-Patent Document 1 and Non-Patent Document 2).

  The Joint Scalable Video Model (JSVM) shown in Non-Patent Document 1 and Non-Patent Document 2 is a coding method based on AVC, and is a one-way prediction / bi-directional prediction in the time direction, and a spatial prediction in a frame. , Inter-layer prediction using interpolated signals of lower layers is used. The SVC-compliant encoded stream corresponds to the encoding result of signals having different time, space, and image quality for each layer.

In SVC, the amount of calculation at the time of decoding tends to increase as compared to AVC. This is because inter-layer prediction is performed, so that when decoding a signal of a specific layer, it is also necessary to decode a signal of a lower layer. In particular, the calculation amount of processing related to reverse motion compensation accounts for a large proportion of the total calculation amount. In view of this, a method called single loop decoding (SLD) that does not perform inverse motion compensation is proposed outside the corresponding layer. In SLD, the reference from the lower layer is limited to only the intra-coded block, thereby making it possible to omit reverse motion compensation in the lower layer. If the reference pixel at the time of intra prediction cannot be used because it belongs to an inter-coded block, the processing is proceeding with the reference pixel value uniformly 127.
J. Reichel, M. Wien and H. Schwarz, “Working Draft 4 of ISO / IEC 14496-10: 200x Amd. 3 Scalable Video Coding” ISO / IEC JTC1 / SC29 / WG11 N7555, Octover 2005, Nice, France. J. Reichel, M. Wien and H. Schwarz, “Joint Scalable Video Model (JSVM) 4” ISO / IEC JTC1 / SC29 / WG11 N7556, Octover 2005, Nice, France.

  As described above, intra prediction in SLD conforms to AVC, and is a process closed to each layer. For this reason, when applied to a signal composed of layers having different temporal and spatial resolutions, it is not designed so that the correlation between layers can be used. That is, it is not optimized for encoding multi-level signals, leaving room for improvement in improving encoding efficiency.

  The present invention has been made in view of such circumstances, and design of an intra predictor that generates a reference pixel value by using a lower layer signal in a case where a reference pixel cannot be used in an intra prediction process for an SLD. The purpose is to establish the law.

First, symbols used in the description of the present invention are organized. The pixel value at the coordinate (x, y) in the frame at time t in the j-th layer is defined as f _j (x, y, t), and the decoded signal for f _j (x, y, t) is represented as g _j (x, y). , T). f _j (x, y, t) has a spatial resolution twice that of f _j-1 (x, y, t). For example, if f ₀ (x, y, t) is a QCIF size, f ₁ (x, y, t) is a CIF size. Let the block size in inter-frame prediction be L × L.

  The positional relationship between hierarchies in intra prediction used in SLD will be described with reference to FIG. FIGS. 2A, 2B, and 2C show frames of the k-th layer, the k-1th layer, and the k-2th layer, respectively. The black area in FIG. 2A represents a block that uses prediction from the immediately lower hierarchy (k-1 hierarchy). In the case of SLD, the lower layer area (black area in FIG. 2B) corresponding to the block is limited to the block encoded by intra coding. In the case of intra coding, a prediction signal is generated using a neighboring pixel as a reference pixel as in the shaded area in FIG.

  In the present invention, when a reference pixel (shaded area) for intra prediction in the (k-1) -th layer cannot be used because it belongs to an inter-coded block, the reference pixel is used by using a signal in the (k-2) -th layer. Give a way to generate. In the following, an intra block in the immediately lower hierarchy (k-1 hierarchy in the above example) referred to in the SLD is referred to as an SLD reference block. Also, as shown in FIG. 5C, a block at a corresponding position in the hierarchy immediately below the SLD reference block (k-2 hierarchy in the above example) is called a block immediately below the SLD reference block.

  FIG. 1 is a diagram showing a configuration example of an inter-layer prediction apparatus according to the present invention. In FIG. 1, reference numeral 1 denotes an inter-layer prediction apparatus that performs inter-layer prediction of an encoding or decoding target block. In the present invention, a reference pixel intra / inter determination unit 11, a lower layer presence / absence determination unit 12, a default reference pixel value A setting unit 13, a reference pixel generation unit 14, a reference block generation unit 17, and an inter-layer prediction unit 18 are provided. The reference pixel generation unit 14 includes a lower layer block reference unit 15 or a lower layer block neighborhood reference unit 16.

  The inter-layer prediction device 1 is a predictor that performs inter-layer prediction reference only to intra prediction blocks in moving image scalable coding or moving image scalable decoding for layers with different spatial resolutions.

  The reference pixel intra / inter determination unit 11 determines whether or not the reference pixel at the time of intra prediction of the reference block cannot be used because it belongs to the inter prediction block. When the reference pixel is an intra prediction block and can be used, the reference pixel of the intra prediction block is referenced and used for inter-layer prediction of the encoding target block or the decoding target block by the reference block generation unit 17. Generate a reference block.

  As a result of the determination by the reference pixel intra / inter determination unit 11, when the reference pixel is an inter prediction block, in the conventional SLD, the value of the reference pixel is uniformly 127, but in the present invention, it is directly below the reference block. Reference pixels are generated by referring to these blocks. Therefore, the lower layer presence / absence determination unit 12 determines whether or not a lower layer block exists in the reference block. When there is no lower layer block, the default reference pixel value setting unit 13 sets the reference pixel value to 127.

  When the lower layer block exists, the reference pixel generation unit 14 refers to the pixel value in the intra prediction block located at the spatially corresponding coordinate in the lower layer of the reference block by the lower layer block reference unit 15, A reference pixel is generated. When the reference pixel in the lower layer is not an intra prediction block, the reference pixel generation unit 14 uses the lower layer block neighborhood reference unit 16 to locate the intra prediction block located in the vicinity of the spatially corresponding coordinate in the lower layer of the reference block. A reference pixel is generated with reference to the pixel value in the.

  The reference block generation unit 17 performs intra prediction with reference to the generated or set reference pixels to generate a reference block. The inter-layer prediction unit 18 refers to the pixel value of the reference block generated by the reference block generation unit 17, performs inter-layer prediction of the encoding target block or the decoding target block, and generates an inter-layer prediction signal.

  According to the present invention, the reference pixel of the intra prediction block in the reference layer in the SLD can be adaptively changed according to the local property of the same layer. This leads to an improvement in prediction performance and consequently an improvement in coding efficiency.

[First Embodiment]
As shown in FIG. 3, the average pixel value of the block immediately below the SLD reference block in the k-2th layer (the black region in FIG. 3B) is calculated, and this average pixel value is used as the SLD reference block in the k-1th layer. This is used as a reference pixel value for intra prediction in (black area in FIG. 3A).

[Second Embodiment]
Consider a case where the size of the SLD reference block in the (k−1) th layer is 4 × 4. As shown in FIG. 4A, the intra prediction reference pixel in the (k−1) -th layer is divided into three partial areas. Assuming that the coordinate value of the upper left corner of the intra prediction block is (x ₀ , y ₀ ), the partial area is the left oblique line area (x, y) ∈ {x = x ₀ −1, y ₀ −1 ≦ y ≦ y ₀ +3}, right hatched area (x, y) ∈ {x ₀ ≤ x ≤ x ₀ +3, y = y ₀ -1}, shaded area (x, y) ∈ {x ₀ +4 ≤ x ≤ x ₀ +7, y = y ₀ −1}. For each partial region, the pixel value in the reference region is set based on the k-2th layer signal.

(1) In the case of the left hatched area ((x, y) ε {x = x ₀ −1, y ₀ −1 ≦ y ≦ y ₀ +3}) in FIG.
FIG left hatched area of 4 (b) (x, y ) ∈ {x = [x 0/2], [y 0/2] ≦ y ≦ [y 0/2] +1} based on the value of the following formula Value.

  Here, [r] is a function that returns the maximum integer that does not exceed the real number r.

(2) In the case of the right oblique line region ((x, y) ε {x ₀ ≦ x ≦ x ₀ +3, y = y ₀ −1}) in FIG.
Based on the value of the right shaded region (x, y) ∈ {[ x 0/2] ≦ x ≦ [x 0/2] + 1, y = y 0 -1} of FIG. 4 (c), the value of the following formula To do.

f _k-1 (x, y ₀ -1) = f _k-2 ([x / 2], [y _0/2 ]) (2)
(3) In the case of the shaded area ((x, y) ε {x ₀ + 4 ≦ x ≦ x ₀ +7, y = y ₀ −1}) in FIG.
f _k-1 (x, y ₀ −1) = f _k−2 ([x _0/2 + 1], [y _0/2 ]) (3)
Alternatively, the value of the following equation can be used.

[Third Embodiment]
Consider a case where the size of the SLD reference block in the (k−1) th hierarchy is 16 × 16. The reference pixel is divided into the following two areas with the coordinate value of the upper left corner of the intra prediction block as (x ₀ , y ₀ ).

(X, y) ε {x = x ₀ −1, y ₀ −1 ≦ y ≦ y ₀ +15}
(X, y) ε {x ₀ ≦ x ≦ x ₀ +15, y = y ₀ −1}
(1) When (x, y) ε {x = x ₀ −1, y ₀ −1 ≦ y ≦ y ₀ +15}:

  Alternatively, the value of the following equation can be used. In the following, m = 0, 1,..., 6 and m ′ = 0, 1,.

(2) When (x, y) ε {x ₀ ≦ x ≦ x ₀ +15, y = y ₀ −1}:
f _k-1 (x, y ₀ -1) = f _k-2 ([x / 2], [y _0/2 ]) (7)
Alternatively, the value of the following equation can be used. In the following, m = 0, 1,..., 6 and m ′ = 0, 1,.

[Fourth Embodiment]
Consider a case where the size of the SLD reference block in the (k−1) th layer is 4 × 4. The k-2th layer region (black region in FIG. 5 (b)) at the position corresponding to the k-1th layer intra prediction block (black region in FIG. 5 (a)) is not included in the intra prediction block. When any of the neighboring blocks of the same block (the shaded area in FIG. 5B) is included in the intra prediction block, the following processing is performed.

(1) When a neighboring block immediately above becomes an intra prediction block:
(1-1) Case of (x, y) ∈ {x = x ₀ -1, y ₀ ≤ y ≤ y ₀ +3} If the condition of (2-1) described later is satisfied, (2- The same processing as in 1) is performed. Otherwise, the average pixel value of a block that is an intra block among the four neighboring blocks is used as a reference pixel value for intra prediction in the (k−1) -th layer.

(1-2) In the case of (x, y) ∈ {x ₀ ≤ x ≤ x ₀ +3, y = y ₀ -1} Reference pixel (in Fig. 6 (a)) for intra prediction in the k-1th layer The value of the following equation is set as the value of (shaded area). This is a pixel value corresponding to the shaded portion in FIG. 6B.

f _k−1 (x, y ₀ −1) = f _k−2 ([x / 2], [y ₀ / 2-1]) (9)
(1-3) In the case of (x, y) ∈ {x ₀ + 4 ≦ x ≦ x ₀ +7, y = y ₀ −1} Reference pixel for intra prediction in the k−1th layer (FIG. 6 (e)) The value of the following equation is set as the value of (shaded area). This uses the pixel value corresponding to the shaded portion in FIG.

f _k-1 (x, y ₀ -1) = f _k-2 ([(x ₀ +3) / 2], [y ₀ / 2-1])
(Ten)
(2) When the immediate left neighboring block is an intra prediction block:
(2-1) In the case of (x, y) ε {x = x ₀ −1, y ₀ −1 ≦ y ≦ y ₀ +3} Reference pixel for intra prediction in the k−1th layer (FIG. 6C The value of the following equation is set as the value of () shaded area). This uses the pixel value corresponding to the shaded portion in FIG.

(2-2) Case of (x, y) ∈ {x ₀ ≤ x ≤ x ₀ +3, y = y ₀ -1} When the above condition (1-2) is satisfied, (1- The same processing as in 2) is performed. Otherwise, the average pixel value of a block that is an intra block among the four neighboring blocks is used as a reference pixel value for intra prediction in the (k−1) -th layer.

(2-3) Case of (x, y) ∈ {x ₀ + 4 ≦ x ≦ x ₀ +7, y = y ₀ −1} When the above condition (1-3) is satisfied, (1 -3) The same processing is performed. Otherwise, the average pixel value of a block that is an intra block among the four neighboring blocks is used as a reference pixel value for intra prediction in the (k−1) -th layer.

(3) When the neighboring block immediately below and right is an intra prediction block:
The average value of the corresponding neighboring blocks is used as the value of the reference pixel (the shaded area in FIGS. 6A, 6C, and 6E) for intra prediction in the (k-1) th layer.

[Fifth Embodiment]
Consider a case where the size of the SLD reference block in the (k−1) th hierarchy is 16 × 16.

(1) When a neighboring block immediately above becomes an intra prediction block:
(1-1) Case of (x, y) ∈ {x = x ₀ −1, y ₀ −1 ≦ y ≦ y ₀ +15} When the condition of (2-1) described later is satisfied, The same processing as in 2-1) is performed. Otherwise, the average pixel value of a block that is an intra block among the four neighboring blocks is used as a reference pixel value for intra prediction in the (k−1) -th layer.

(1-2) In the case of (x, y) ∈ {x ₀ ≤ x ≤ x ₀ +15, y = y ₀ -1} As the value of the reference pixel for intra prediction in the k-1th layer, Set the value.

f _k-1 (x, y ₀ -1) = f _k-2 ([x / 2], [y ₀ / 2-1]) (12)
Alternatively, the value of the following equation can be used. In the following, m = 0, 1,..., 6 and m ′ = 0, 1,.

(2) When the immediate left neighboring block is an intra prediction block:
(2-1) In the case of (x, y) ε {x = x ₀ −1, y ₀ −1 ≦ y ≦ y ₀ +15} Reference pixel for intra prediction in the k−1th layer (FIG. 6C The value of the following equation is set as the value of () shaded area).

  Alternatively, the value of the following equation can be used. In the following, m = 0, 1,..., 6 and m ′ = 0, 1,.

(2-2) Case of (x, y) ∈ {x ₀ ≤ x ≤ x ₀ +15, y = y ₀ -1} When the above condition (1-2) is satisfied, (1- The same processing as in 2) is performed. Otherwise, the average pixel value of a block that is an intra block among the four neighboring blocks is used as a reference pixel value for intra prediction in the (k−1) -th layer.

(3) When a neighboring block immediately below or right is an intra block:
Of the neighboring blocks immediately below or right, the average value of blocks that are intra blocks (both, if both are intra blocks, the average value of both blocks) is the value of the reference pixel for intra prediction in the (k-1) th layer. Used as

[Process flow]
An embodiment of the processing of the present invention will be described with reference to the drawing (FIG. 7). It is assumed that the reference pixel used in the intra prediction of the reference block cannot be used because it belongs to the inter-coded block. When a reference pixel is available, decoding is performed using the reference pixel as in the conventional technique.

  Step S101: Read the level of the decoding target layer signal and write the level to the register.

  Step S102: Using the level written to the register in step S101 as an input, it is determined whether or not there are two or more layers below. If true, the process proceeds to step S104, and if false, , The process proceeds to step S103. When the current hierarchy level k is 2 or more, the hierarchy of hierarchy levels 1 and 0 exists in the lower order. Specifically, it is determined whether k is 2 or more.

  Step S103: The encoded stream is input, SLD decoding processing defined in JSVC is performed, and a decoded signal is output.

  Step S104: If there are two or more layers below the decoding target layer, the size of the SLD reference block is read and written to the register.

  Step S105: The size of the SLD reference block is input, and a determination process is performed to determine whether or not the size is 4 × 4. If true, the process proceeds to step S106. If false, the process proceeds to step S110. Move on to processing.

  Step S106: The processing of the block at the corresponding position in the lower hierarchy (the block immediately below the SLD reference block) is input, and it is determined whether or not the mode is the intra mode. If true, the processing of step S107 is performed. If it is false, the process proceeds to step S108.

  Step S107: The block immediately below the SLD reference block is input, a process for generating a reference pixel value of the SLD reference block is performed, and the reference pixel value is output. Details of this processing will be described later.

  Step S108: The mode of the neighboring block of the block immediately below the SLD reference block is input, and it is determined whether or not at least one of the neighboring blocks is the intra mode. If true, the process proceeds to step S109. If false, the process proceeds to step S114.

  Step S109: A process of generating a reference pixel value of the SLD reference block is performed with a neighboring block immediately below the SLD reference block as input, and the reference pixel value is output. Details of this processing will be described later.

  Step S110: The mode of the block immediately below the SLD reference block is input, and it is determined whether or not the mode is the intra mode. If true, the process proceeds to step S111. The process proceeds to S112.

  Step S111: A block immediately below the SLD reference block is input, a process of generating a reference pixel value of the SLD reference block is performed, and the reference pixel value is output. Details of this processing will be described later.

  Step S112: The mode of the neighboring block of the block immediately below the SLD reference block is input, and it is determined whether or not at least one of the neighboring blocks is the intra mode. If true, the process proceeds to step S113. If false, the process proceeds to step S114.

  Step S113: A process of generating a reference pixel value of the SLD reference block is performed by using a neighboring block immediately below the SLD reference block as an input, and the reference pixel value is output. Details of this processing will be described later.

  Step S114: A default value 127 is set as a reference pixel.

  Step S115: The reference pixel value of the SLD reference block output from steps S107, S109, S111, S113, and S114 is input, the SLD reference block is decoded, and the decoded signal of the SLD reference block is output.

  Step S116: For all macroblocks in the frame, the process returns to step S104 until the decoding process is completed, and the above process is repeated.

  Next, details of the processing in step S107 will be described with reference to FIG.

  Step S201: A block immediately below the SLD reference block is input, a process for generating a reference pixel immediately to the left of the SLD reference block is performed, and the reference pixel value is written to the register. The specific calculation method follows equation (1).

  Step S202: The block immediately below the SLD reference block is input, a process of generating a reference pixel immediately above the SLD reference block is performed, and the reference pixel value is written to the register. The specific calculation method follows equation (2).

  Step S203: With the block immediately below the SLD reference block as an input, processing for generating a reference pixel immediately above the SLD reference block is performed, and the reference pixel value is written to the register. The specific calculation method follows Equation (3) or Equation (4).

  Step S204: Output all reference pixel values generated in steps S201, S202, and S203.

  Details of the processing in step S109 will be described with reference to FIG.

  Step S301: Using the mode of the block near the left of the block immediately below the SLD reference block as an input, it is determined whether or not the mode of the neighboring block is the intra mode. If true, the process proceeds to step S302. If false, the process proceeds to step S303.

  Step S302: The pixel value of the left neighboring block of the block immediately below the SLD reference block is input, a process of generating a reference pixel value immediately to the left of the SLD reference block is performed, and the reference pixel value is output. The specific calculation method follows equation (11).

  Step S303: The pixel value of the block encoded in the intra mode among the four neighboring blocks immediately below the SLD reference block is input, the average value of the pixel value is calculated, and the average value is output.

  Step S304: The average value output in step S303 is input, and the average value is output as the reference pixel value immediately to the left of the SLD reference block.

  Step S305: The mode of the block immediately above the block immediately below the SLD reference block is input, and it is determined whether or not the mode of the block is the intra mode. If true, the process proceeds to step S306. If false, the process proceeds to step S308.

  Step S306: The pixel value of the upper neighboring block immediately below the SLD reference block is input, a reference pixel value immediately above the SLD reference block is generated, and the reference pixel value is output. The specific calculation method follows equation (9).

  Step S307: The pixel value of the upper neighboring block immediately below the SLD reference block is input, a process of generating a reference pixel value immediately above the SLD reference block is performed, and the reference pixel value is output. The specific calculation method follows equation (10).

  Step S308: Same as step S303.

  Step S309: The average value output in step S308 is input, and the average value is output as the reference pixel values immediately above and right immediately above the SLD reference block.

  Step S310: All reference pixel values generated in steps S302, S304, S306, S307, and S309 are read and output as reference pixel values of the SLD reference block.

  Next, details of the processing in step S111 will be described with reference to FIG.

  Step S401: A block immediately below the SLD reference block is input, a process for generating a reference pixel immediately to the left of the SLD reference block is performed, and the reference pixel value is written to the register. The specific calculation method follows Equation (5) or Equation (6).

  Step S402: A block immediately below the SLD reference block is input, a process of generating a reference pixel immediately above the SLD reference block is performed, and the reference pixel value is written to the register. The specific calculation method follows Equation (7) or Equation (8).

  Step S403: Output all reference pixel values generated in steps S401 and S402.

  Next, details of the processing in step S113 will be described with reference to FIG.

  Step S501: The mode of the left neighboring block of the block immediately below the SLD reference block is input, and it is determined whether or not the mode of the neighboring block is the intra mode. If true, the process proceeds to step S502. If false, the process proceeds to step S503.

  Step S502: The pixel value of the left neighboring block of the block immediately below the SLD reference block is input, processing for generating a reference pixel value immediately to the left of the SLD reference block is performed, and the reference pixel value is output. The specific calculation method follows formula (14) or formula (15).

  Step S503: The pixel value of the block encoded in the intra mode among the four neighboring blocks immediately below the SLD reference block is input, the average value of the pixel value is calculated, and the average value is output.

  Step S504: The average value output in step S503 is input, and the average value is output as the reference pixel value immediately to the left of the SLD reference block.

  Step S505: The mode of the block immediately above the block immediately below the SLD reference block is input, and it is determined whether or not the mode of the neighboring block is the intra mode. If true, the process proceeds to step S506. If false, the process proceeds to step S507.

  Step S506: The pixel value of the upper neighboring block immediately below the SLD reference block is input, a process of generating a reference pixel value immediately above the SLD reference block is performed, and the reference pixel value is output. The specific calculation method follows formula (12) or formula (13).

  Step S507: Same as step S503.

  Step S508: The average value output in step S507 is input, and the average value is output as a reference pixel value immediately above the SLD reference block.

  Step S509: All the reference pixel values generated in steps S502, S504, S506, and S508 are read and output as reference pixel values of the SLD reference block.

  The example in the case where the present invention is applied to the moving picture scalable decoding method has been described above. However, the present invention can be similarly applied to the inter-layer prediction method in the moving picture scalable coding method. Absent.

[Processing equipment]
An example of a decoding device using the present invention is shown in FIG. This figure shows an example of a decoder that decodes a scalable encoded stream consisting of three layers. However, if the same devices as the layered prediction signal decoding processing unit 103 and the layered signal decoding processing unit 104 are added, the number of layers is three. The above encoded stream can be supported. Similarly, the present invention can be applied to an encoder.

  Separator 101: An encoded stream output from the scalable encoder is input, and the stream is separated into a first layer encoded stream, a second layer encoded stream, and a third layer encoded stream, The first layer encoded stream, the second layer encoded stream, and the third layer encoded stream are respectively provided as input signals to the layer signal decoding processing unit 102, the layer signal decoding processing unit 104, and the layer signal decoding processing unit 106. .

  Hierarchical signal decoding processing unit 102: Inputs the first layer encoded stream, performs decoding processing, and outputs a decoded image of the first layer and an inter-layer prediction signal used in decoding of the upper layer. Details of this decoding process are shown in FIG.

  Hierarchical signal decoding processing unit 104: Inputs the second layer encoded stream, performs decoding processing, and outputs a decoded signal and an inter-layer prediction signal used in upper layer decoding. Details of this decoding process are shown in FIG.

  Hierarchical signal decoding processing unit 106: Inputs the third hierarchical coded stream, performs decoding processing, and outputs a decoded signal. Details of this decoding process are shown in FIG.

  Hierarchical prediction signal decoding processing sections 103 and 105: Input a prediction signal from a lower hierarchy, decode the same signal, and output a decoded signal. In the case of SLD, the target of inter-layer prediction is limited to blocks in which the immediately lower layer is intra-coded. For this reason, the above-described decoding process is the same as the decoding process of the intra prediction encoding of JSVC except for the reference pixel generation method. Details of this processing will be described later with reference to FIG.

  Details of the hierarchical signal decoding processing units 102, 104, and 106 will be described with reference to FIG.

  Encoded stream storage unit 201: The input encoded stream of each layer is written.

  Entropy decoding unit 202: The encoded stream read from the encoded stream storage unit 201 is input, entropy decoding processing is performed, and the decoded quantized value is written to the quantized value storage unit 203.

  Inverse quantization unit 204: The quantized value read from the quantized value storage unit 203 is input, an inverse quantization process is performed, and the result is written to the transform coefficient storage unit 205.

  Inverse transform unit 206: The quantized value read from the transform coefficient storage unit 205 is input, an inverse transform process is performed, and the result is written to the decoded differential signal storage unit 207.

  Decoded signal storage unit 208: The addition value of the decoded difference signal read from the decoded difference signal storage unit 207 and the output of the prediction signal storage unit 213 is written.

  Deblocking processing unit 209: The decoded signal read from the decoded signal storage unit 208 is input, deblocking processing is performed, and the processing result is written to the deblocking signal storage unit 210. When the deblocking process is not performed on the encoder side, this process is omitted, and the signal read from the decoded signal storage unit 208 is written in the delay unit 211 as it is.

  Predicted signal determination unit 212: Inputs information for identifying a prediction mode in the encoded stream, and predicts either a decoded signal of the same layer delayed by the delay unit 211 or a signal stored in the inter-layer predicted signal storage unit 214 Processing for determining whether to use the signal is performed, and the result is written in the predicted signal storage unit 213. Note that the prediction mode indicates any one of inter-frame prediction, intra-frame prediction, and inter-layer prediction.

  Details of the hierarchy prediction signal decoding processing units 103 and 105 in FIG. 12 will be described with reference to FIG.

  Hierarchy level storage unit 301: Reads the level of the decoding target layer signal and stores the level.

  Lower hierarchy number determination unit 302: Using the hierarchy level read from the hierarchy level storage unit 301 as an input, it is determined whether or not there are two or more layers below, and if true, the SLD reference block size reading unit The process proceeds to 304, and in the case of false, the process proceeds to the process of the default SLD reference block reference pixel value generation unit 303. When the current hierarchy level k is 2 or more, the hierarchy of the hierarchy levels 1 and 0 exists at the lower level. Specifically, it is determined whether or not k is 2 or more.

  Default SLD reference block reference pixel value generation unit 303: Generates a reference pixel value of the SLD reference block in accordance with the process defined in JSVC, and writes the pixel value to the SLD reference block reference pixel value storage unit 313.

  SLD reference block size reading unit 304: The encoded stream is input, the size of the SLD reference block is extracted from the stream, and the size is written to the SLD reference block size storage unit 305.

  SLD reference block size determination unit 306: The SLD reference block size read from the SLD reference block size storage unit 305 is input and a determination process is performed to determine whether or not the size is 4 × 4. The process proceeds to the process of the block intra determination unit 307 immediately below the reference block, and in the case of false, the process proceeds to the process of the block intra determination unit 308 immediately below the SLD reference block.

  Block intra-determining unit 307 immediately below the SLD reference block: The mode of the block immediately below the SLD reference block is input, and it is determined whether or not the mode is the intra mode. If true, the SLD reference block reference pixel generation process is performed. Then, the processing of the unit 309 is started. In the case of false, the mode of the neighboring block is input, and determination processing is performed to determine whether or not at least one of the neighboring blocks is in the intra mode. If true, the SLD reference block reference pixel generation processing unit 310 If the result is false, the process proceeds to the process of the default SLD reference block reference pixel value generation unit 303.

  SLD reference block reference pixel generation processing unit 309: Inputs a block immediately below the SLD reference block, performs a process of generating a reference pixel value of the SLD reference block, and writes the reference pixel value to the SLD reference block reference pixel value storage unit 313. Details of this processing will be described later.

  SLD reference block reference pixel generation processing unit 310: Performs processing to generate a reference pixel value of an SLD reference block by using a neighboring block immediately below the SLD reference block as an input, and stores the reference pixel value as an SLD reference block reference pixel value storage unit 313. Export to Details of this processing will be described later.

  Block intra block determination unit 308 immediately below the SLD reference block: The mode of the block at the corresponding position in the lower hierarchy (the block immediately below the SLD reference block) is input, and it is determined whether or not the mode is the intra mode. First, the processing of the SLD reference block reference pixel generation processing unit 311 is performed. In the case of false, the mode of the neighboring block is input and determination processing is performed to determine whether or not at least one of the neighboring blocks is in the intra mode. If true, the SLD reference block reference pixel generation processing unit 312 If the result is false, the process proceeds to the process of the default SLD reference block reference pixel value generation unit 303.

  SLD reference block reference pixel generation processing unit 311: Inputs a block immediately below the SLD reference block, performs a process of generating a reference pixel value of the SLD reference block, and writes the reference pixel value to the SLD reference block reference pixel value storage unit 313. Details of this processing will be described later.

  SLD reference block reference pixel generation processing unit 312: Performs processing to generate a reference pixel value of an SLD reference block by using a neighboring block immediately below the SLD reference block as an input, and stores the reference pixel value as an SLD reference block reference pixel value storage unit 313. Export to Details of this processing will be described later.

  SLD reference block decoding processing unit 314: The reference pixel value of the SLD reference block read from the SLD reference block reference pixel value storage unit 313 is input, and the SLD reference block decoding process is performed according to the JSVC decoding process. Write to the reference block decoded value storage unit 315.

  Inter-layer prediction signal decoding processing unit 316: The SLD reference block decoded value read from the SLD reference block decoded value storage unit 315 is input, and the inter-layer prediction signal decoding process is performed according to the JSVC decoding process. Write to the predicted signal decoded value storage unit 317. The output of this processing is a signal stored in the inter-layer prediction signal storage unit 214 shown in FIG.

    Details of the SLD reference block reference pixel generation processing unit 309 in FIG. 14 will be described with reference to FIG.

  Block storage unit 401 immediately below the SLD reference block: stores the pixel value of the block immediately below the SLD reference block.

  Immediately left reference pixel generation unit 402: The pixel value of the block immediately below the SLD reference block read from the block storage unit 401 immediately below the SLD reference block is input, and a process is performed to generate a reference pixel value immediately to the left of the SLD reference block. The pixel value is written in the left-left reference pixel value storage unit 403. The specific calculation method follows equation (1).

  Immediately above reference pixel generation unit 404: The pixel value of the block immediately below the SLD reference block read from the block storage unit 401 immediately below the SLD reference block is input, and the reference pixel value immediately above the SLD reference block is generated. Are written in the reference pixel value storage unit 405 directly above. The specific calculation method follows equation (2).

  Directly upper right reference pixel generation unit 406: The pixel value of the block immediately below the SLD reference block read from the block storage unit 401 immediately below the SLD reference block is input, and a process of generating a reference pixel immediately above the SLD reference block is performed. The value is written in the right upper reference pixel value storage unit 407. The specific calculation method follows Equation (3) or Equation (4).

  SLD reference block reference pixel value storage unit 313: The left reference pixel value storage unit 403, the upper reference pixel value storage unit 405, the reference pixel value read from the upper right reference pixel value storage unit 407 is input, and the SLD reference pixel value is ,Store.

  Details of the SLD reference block reference pixel generation processing unit 310 in FIG. 14 will be described with reference to FIG.

  Neighboring block mode storage unit 501: Stores the four modes of the block immediately below the SLD reference block in association with the position information of the neighboring block.

  Immediate left block intra determination unit 502: Processing for determining whether or not the mode of the block near the left of the block immediately below the SLD reference block read from the block block storage unit 501 is the intra mode If true, the process proceeds to the process of the left-left reference pixel generation unit 503. If false, the process proceeds to the intra-block average reference pixel value generation unit 504.

  Immediate left reference pixel generation unit 503: Inputs the pixel value of the left neighboring block of the block immediately below the SLD reference block, performs a process of generating a reference pixel value immediately to the left of the SLD reference block, and uses the reference pixel value as the immediate left reference pixel Write to the value storage unit 505. The specific calculation method follows equation (11).

  Intra block average reference pixel value generation unit 504: among the four neighboring blocks immediately below the SLD reference block, the pixel value of the block encoded in the intra mode is input, and the average value of the pixel value is calculated. The average value is written in the right-left reference pixel value storage unit 505.

  Immediately above block intra determination unit 506: Inputs the mode of the block immediately above the block immediately below the SLD reference block read from the block block storage unit 501, and determines whether or not the mode of the block is the intra mode If it is true, the process proceeds to the process of the immediately upper reference pixel generation unit 507 and the process of the right upper reference pixel generation unit 510. If the result is false, the process proceeds to the process of the intra block average reference pixel value generation unit 508.

  Immediately above reference pixel generation unit 507: The pixel value of the upper neighboring block immediately below the SLD reference block is input, a process of generating a reference pixel value immediately above the SLD reference block is performed, and the reference pixel value is stored as an immediately above reference pixel value storage unit. Write to 509. The specific calculation method follows equation (9).

  Right upper reference pixel generation unit 510: Inputs the pixel value of the upper neighboring block immediately below the SLD reference block, performs processing to generate the right upper reference pixel value of the SLD reference block, and uses the reference pixel value as the right upper reference pixel. Write to the value storage unit 511. The specific calculation method follows equation (10).

  Intra block average reference pixel value generation unit 508: among the four neighboring blocks immediately below the SLD reference block, the pixel value of the block encoded in the intra mode is input, and the average value of the same pixel value is calculated. The average value is written in the upper right reference pixel value storage unit 509 and the upper right reference pixel value storage unit 511.

  SLD reference block reference pixel value storage unit 313: the left reference pixel value storage unit 505, the upper reference pixel value storage unit 509, and the reference pixel read from the upper right reference pixel value storage unit 511 as input, and the SLD reference pixel value as Store.

  Details of the SLD reference block reference pixel generation processing unit 311 in FIG. 14 will be described with reference to FIG.

  Block storage unit 601 immediately below the SLD reference block: The pixel value of the block immediately below the SLD reference block is stored.

  Immediately left reference pixel generation unit 602: Inputs the pixel value of the block immediately below the SLD reference block read from the block storage unit 601 immediately below the SLD reference block, and performs processing to generate a reference pixel immediately to the left of the SLD reference block. The value is written in the right-left reference pixel value storage unit 603. The specific calculation method follows Equation (5) or Equation (6).

  Immediately above reference pixel generation unit 604: The pixel value of the block immediately below the SLD reference block read from the block storage unit 601 immediately below the SLD reference block is input, and a process of generating a reference pixel immediately above the SLD reference block is performed. The data is written in the immediately above reference pixel value storage unit 605. The specific calculation method follows Equation (7) or Equation (8).

  SLD reference block reference pixel value storage unit 313: The left reference pixel value storage unit 603, the reference pixel read from the upper reference pixel value storage unit 605 is input and stored as an SLD reference pixel value.

  Details of the SLD reference block reference pixel generation processing unit 312 in FIG. 14 will be described with reference to FIG.

  Neighboring block mode storage unit 701: The four modes of the block immediately below the SLD reference block are stored in association with the position information of the neighboring block.

  Immediate left neighboring block intra determination unit 702: Processing for determining whether the mode of the left neighboring block of the block immediately below the SLD reference block read from the neighboring block mode storage unit 701 is input and whether the mode of the neighboring block is the intra mode If true, the process proceeds to the process of the left-left reference pixel generation unit 703. If false, the process proceeds to the intra block average reference pixel value generation unit 704.

  Immediate left reference pixel generation unit 703: Inputs the pixel value of the left neighboring block of the block immediately below the SLD reference block, performs a process of generating a reference pixel value immediately to the left of the SLD reference block, and uses the reference pixel value as the immediate left reference pixel Write to the value storage unit 705. The specific calculation method follows formula (14) or formula (15).

  Intra block average reference pixel value generation unit 704: among four neighboring blocks of the block immediately below the SLD reference block, a pixel value of a block encoded in the intra mode is input, and an average value of the same pixel value is calculated. The average value is written in the right-left reference pixel value storage unit 705.

  Immediately above block intra-determination unit 706: The process of determining whether or not the mode of the block near the SLD reference block immediately above the SLD reference block read from the neighborhood block mode storage unit 701 is the intra mode is input. If true, the process proceeds to the process of the immediately above reference pixel generation unit 707. If false, the process proceeds to the process of the intra block average reference pixel value generation unit 708.

  Immediately above reference pixel generation unit 707: The pixel value of the upper neighboring block immediately below the SLD reference block is input, a process of generating a reference pixel value immediately above the SLD reference block is performed, and the reference pixel value is stored as an immediately above reference pixel value storage unit. Write to 709. The specific calculation method follows formula (12) or formula (13).

  Intra block average reference pixel value generation unit 708: Among the four neighboring blocks immediately below the SLD reference block, a pixel value of a block encoded in the intra mode is input, and an average value of the same pixel value is calculated. The average value is written in the reference pixel value storage unit 709 directly above.

  SLD reference block reference pixel value storage unit 313: The left reference pixel value storage unit 705, the reference pixel read from the upper reference pixel value storage unit 709 is input and stored as an SLD reference pixel value.

  The above-described inter-layer prediction process in the video scalable coding or video scalable decoding can be realized by a computer and a software program, and the program can be recorded on a computer-readable recording medium and provided. It can also be provided through a network.

It is a figure which shows the structural example of the intra prediction apparatus which concerns on this invention. It is a figure which shows the relationship between hierarchy. It is a figure which shows the relationship between an intra block and a reference pixel. It is inter-layer prediction explanatory drawing of the reference pixel of a SLD reference block. It is a figure which shows the vicinity block of a SLD reference block and a block immediately under a SLD reference block. It is inter-layer prediction explanatory drawing of the reference pixel of a SLD reference block. It is a flowchart of the decoding process which concerns on embodiment of this invention. It is a figure which shows the detail of the process of step S107 of FIG. It is a figure which shows the detail of the process of step S109 of FIG. It is a figure which shows the detail of the process of step S111 of FIG. It is a figure which shows the detail of the process of step S113 of FIG. 1 is an overall view of a decoding device according to an embodiment of the present invention. It is a decoding apparatus figure of a hierarchical signal. It is a decoding apparatus figure of an inter-layer prediction signal. It is a figure which shows the detail of the SLD reference block reference pixel generation process part 309. It is a figure which shows the detail of the SLD reference block reference pixel generation process part 310. It is a figure which shows the detail of the SLD reference block reference pixel generation process part. It is a figure which shows the detail of the SLD reference block reference pixel generation process part.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Inter-layer prediction apparatus 11 Reference pixel intra / inter determination part 12 Lower hierarchy presence determination part 13 Default reference pixel value setting part 14 Reference pixel generation part 15 Lower hierarchy block reference part 16 Lower hierarchy block vicinity reference part 17 Reference block generation part 18 Inter-layer prediction section

Claims (9)

In a video scalable coding or video scalable decoding for a hierarchy of different spatial resolutions, an inter-layer prediction method in which inter-layer prediction is limited to intra prediction blocks,
When pixel values of surrounding reference pixels used in the intra prediction of the reference block that is the intra prediction block cannot be obtained by the intra prediction,
A reference pixel generation process for generating peripheral reference pixels used in the intra prediction using a pixel value in an intra prediction block located at a spatially corresponding coordinate in a lower hierarchy of the reference block;
A reference block generation process for generating a reference pixel in the reference block using the generated peripheral reference pixels;
An inter-layer prediction method, comprising: performing a prediction process for performing inter-layer prediction of a block to be encoded or decoded using a reference pixel in a generated reference block. In a video scalable coding or video scalable decoding for a hierarchy of different spatial resolutions, an inter-layer prediction method in which inter-layer prediction is limited to intra prediction blocks,
When pixel values of surrounding reference pixels used in the intra prediction of the reference block that is the intra prediction block cannot be obtained by the intra prediction,
A reference pixel generation process for generating peripheral reference pixels used in the intra prediction by using pixel values in an intra prediction block located in the vicinity of spatially corresponding coordinates in a lower hierarchy of the reference block;
A reference block generation process for generating a reference pixel in the reference block using the generated peripheral reference pixels;
An inter-layer prediction method, comprising: performing a prediction process for performing inter-layer prediction of a block to be encoded or decoded using a reference pixel in a generated reference block. In the reference pixel generation process,
When generating reference pixels used for intra prediction of a reference block having a block size of 4 × 4 pixels,
From the blocks in the lower hierarchy of the reference block, generate three classes of reference pixels immediately to the left of the reference block, immediately above and to the upper right.
In the reference block generation process,
The inter-layer prediction method according to claim 1 or 2, wherein a reference pixel in a reference block is generated from the three classes of reference pixels. In the reference pixel generation process,
When generating reference pixels used for intra prediction of a reference block having a block size of 16 × 16 pixels,
Generate two classes of reference pixels directly to the left and directly above the reference block from the blocks in the lower hierarchy of the reference block.
In the reference block generation process,
The inter-layer prediction method according to claim 1 or 2, wherein a reference pixel in a reference block is generated from the two classes of reference pixels. When there is no lower layer of the reference block, or when there is no intra prediction block in the corresponding position in the lower layer of the reference block and its neighboring blocks, reference pixels used for intra prediction of the reference block are previously stored. The inter-layer prediction method according to any one of claims 1 to 4, wherein a predetermined default value is used. An intra prediction apparatus that performs inter-layer prediction reference only to intra prediction blocks in video scalable coding or video scalable decoding for different spatial resolution layers,
When pixel values of surrounding reference pixels used in the intra prediction of the reference block that is the intra prediction block cannot be obtained by the intra prediction,
Reference pixel generation means for generating peripheral reference pixels used in the intra prediction using pixel values in an intra prediction block located at spatially corresponding coordinates in a lower hierarchy of the reference block;
Reference block generating means for generating reference pixels in the reference block using the generated peripheral reference pixels;
An inter-layer prediction apparatus comprising: a prediction unit that performs inter-layer prediction of a block to be encoded or decoded using a reference pixel in a generated reference block. An intra prediction apparatus that performs inter-layer prediction reference only to intra prediction blocks in video scalable coding or video scalable decoding for different spatial resolution layers,
When pixel values of surrounding reference pixels used in the intra prediction of the reference block that is the intra prediction block cannot be obtained by the intra prediction,
Reference pixel generating means for generating peripheral reference pixels used in the intra prediction using pixel values in an intra prediction block located in the vicinity of spatially corresponding coordinates in a lower hierarchy of the reference block;
Reference block generating means for generating reference pixels in the reference block using the generated peripheral reference pixels;
An inter-layer prediction apparatus comprising: a prediction unit that performs inter-layer prediction of a block to be encoded or decoded using a reference pixel in a generated reference block.   An inter-layer prediction program for causing a computer to execute the inter-layer prediction method according to any one of claims 1 to 5. A computer-readable recording medium recording an inter-layer prediction program for causing a computer to execute the inter-layer prediction method according to any one of claims 1 to 5.

Images