JP2016042717A - Image processor and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processor capable of enhancing the encoding efficiency by utilizing correlation in movements of an image between the layers which are subject to scalable encoding.SOLUTION: The image processor includes: an information acquisition section 191 that acquires a piece of setting information relevant to a motion vector to be set to a first prediction unit, which is a piece of setting information for setting a motion vector, to a second prediction unit in a second layer corresponding to a first prediction unit in a first layer of an image including the first layer and the second layer upper than the first layer, which is subjected to scalable coding; and a motion vector setting section 192 that sets a motion vector to the second prediction unit by using the setting information acquired by the information acquisition section.SELECTED DRAWING: Figure 18

Description

  The present disclosure relates to an image processing apparatus and an image processing method.

  Conventionally, the purpose of efficiently transmitting or storing digital images is to compress the amount of information of an image using redundancy unique to the image. Compression techniques such as the 26x (ITU-T Q6 / 16 VCEG) standard and the Moving Picture Experts Group (MPEG) -y standard are widespread. In Joint Model of Enhanced-Compression Video Coding as part of MPEG4 activities, It is possible to realize a higher compression ratio by incorporating new functions based on the 26x standard. An international standard named H.264 and MPEG-4 Part 10 (Advanced Video Coding; AVC) has been developed.

  One of the important techniques in these image coding methods is inter-frame prediction. In inter-frame prediction, the content of an image to be encoded is predicted using a reference image, and only the difference between the predicted image and the actual image is encoded. Thereby, the compression of the code amount is realized. However, when an object moves greatly in a series of images, the difference between the predicted image and the actual image becomes large, and a high compression rate cannot be obtained by simple inter-frame prediction. Therefore, by recognizing the motion of the object as a motion vector and compensating the pixel value of the region where the motion appears according to the motion vector, the prediction error in the inter-frame prediction can be reduced. Such a method is called motion compensation.

  H. In HEVC (High Efficiency Video Coding) HEVC, which is being standardized as a next-generation image encoding method following H.264 / AVC, each encoding unit (CU: Coding Unit) in an image further includes one or more prediction units. (PU: Prediction Unit), and a motion vector can be set for each prediction unit. The size and shape of the prediction unit of HEVC is H.264. It is more diverse than the H.264 / AVC block, and the motion of the object can be reflected more accurately in motion compensation (see Non-Patent Document 1 below). Non-Patent Document 2 below predicts a motion vector using spatial correlation or temporal correlation of motion in order to reduce the amount of code of the motion vector, and calculates only the difference between the predicted motion vector and the actual motion vector. A coding technique is proposed. Non-Patent Document 3 below proposes to reduce the amount of code of motion information by merging (merging) blocks having common motion information among adjacent blocks in an image.

Another important technique in the above-described image coding scheme is scalable coding (also referred to as SVC (Scalable Video Coding)). Scalable encoding refers to a technique for hierarchically encoding a layer that transmits a coarse image signal and a layer that transmits a fine image signal. Typical attributes hierarchized in scalable coding are mainly the following three types.
Spatial scalability: Spatial resolution or image size is layered.
-Time scalability: Frame rate is layered.
-Signal to noise ratio (SNR) scalability: SN ratio is hierarchized.
In addition, bit depth scalability and chroma format scalability are also discussed, although not yet adopted by the standard.

JCTVC-B205, “Test Model under Consideration”, Joint Collaborative Team on Video Coding meeting: Geneva, CH, 21-28 July, 2010 VCEG-AI22, “Motion Vector Coding with Optimal PMV Selection”, Jungyoup Yang, et al, July, 2008 JCTVC-A116, “Video Coding Technology Proposal by Fraunhofer HHI”, M. Winken, et al, April, 2010

  The method proposed by Non-Patent Document 2 and the method proposed by Non-Patent Document 3 do not presuppose scalable coding. If these existing methods are applied to each layer of an image to be scalable encoded, a certain amount of code reduction can be expected. However, depending on the type of scalable coding, the correlation of motion between layers is significant. Therefore, it is beneficial to increase the coding efficiency by utilizing such correlation of motion between layers.

  Therefore, the technology according to the present disclosure aims to increase the encoding efficiency by utilizing the correlation of motion between layers of an image to be scalable encoded.

  According to the present disclosure, the second corresponding to the first prediction unit in the first layer of the image to be scalable decoded including the first layer and the second layer higher than the first layer. Setting information for setting a motion vector in a second prediction unit in the layer, and an information acquisition unit for acquiring the setting information related to the motion vector set in the first prediction unit; There is provided an image processing apparatus comprising: a motion vector setting unit that sets a motion vector in the second prediction unit using the setting information acquired by the information acquisition unit.

  The image processing apparatus can typically be realized as an image decoding apparatus that decodes an image.

  Also, according to the present disclosure, the first layer and the first prediction unit in the first layer of the image to be scalable decoded including the second layer higher than the first layer correspond to the first prediction unit in the first layer. Setting information for setting a motion vector in the second prediction unit in the second layer, the setting information relating to the motion vector set in the first prediction unit; And setting a motion vector in the second prediction unit using the set information thus provided.

  Also, according to the present disclosure, the first layer and the first prediction unit in the first layer of the image to be scalable decoded including the second layer higher than the first layer correspond to the first prediction unit in the first layer. An information generation unit configured to generate setting information related to a motion vector set in the first prediction unit, which is setting information for setting a motion vector in a second prediction unit in the second layer; And an encoding unit that encodes the setting information generated by the information generation unit.

  The image processing apparatus can typically be realized as an image encoding apparatus that encodes an image.

  Also, according to the present disclosure, the first layer and the first prediction unit in the first layer of the image to be scalable decoded including the second layer higher than the first layer correspond to the first prediction unit in the first layer. Generation of setting information for setting a motion vector in a second prediction unit in the second layer, the setting information relating to the motion vector set in the first prediction unit; An image processing method including encoding the setting information.

  According to the present disclosure, encoding efficiency is further improved by utilizing the correlation of motion between layers of an image to be scalable encoded.

It is a block diagram which shows an example of a structure of the image coding apparatus which concerns on one Embodiment. It is explanatory drawing for demonstrating space scalability. It is explanatory drawing for demonstrating SNR scalability. It is a block diagram which shows an example of a detailed structure of the motion search part which concerns on a 1st Example. It is the 1st explanatory view for explaining an example of a predictor candidate for prediction of a motion vector. It is the 2nd explanatory view for explaining an example of a predictor candidate for prediction of a motion vector. It is a flowchart which shows an example of the flow of the motion search process by the motion search part which concerns on a 1st Example. It is a block diagram which shows an example of a detailed structure of the motion search part which concerns on a 2nd Example. It is explanatory drawing for demonstrating an example of the predictor between layers. It is a flowchart which shows an example of the flow of the motion search process by the motion search part which concerns on a 2nd Example. It is a block diagram which shows an example of a detailed structure of the motion search part which concerns on a 3rd Example. It is explanatory drawing which shows the 1st example of merge information. It is explanatory drawing which shows the 2nd example of merge information. It is explanatory drawing which shows the 3rd example of merge information. It is a flowchart which shows an example of the flow of the motion search process by the motion search part which concerns on a 3rd Example. It is a block diagram which shows an example of a detailed structure of the motion search part which concerns on a 4th Example. It is explanatory drawing which shows the 4th example of merge information. It is explanatory drawing which shows the 5th example of merge information. It is explanatory drawing which shows the 6th example of merge information. It is a flowchart which shows an example of the flow of the motion search process by the motion search part which concerns on a 4th Example. It is a block diagram which shows an example of a structure of the image decoding apparatus which concerns on one Embodiment. It is a block diagram which shows an example of a detailed structure of the motion compensation part which concerns on a 1st Example. It is a flowchart which shows an example of the flow of the motion compensation process by the motion compensation part which concerns on 1st Example. It is a block diagram which shows an example of a detailed structure of the motion compensation part which concerns on a 2nd Example. It is a flowchart which shows an example of the flow of the motion compensation process by the motion compensation part which concerns on 2nd Example. It is a block diagram which shows an example of a detailed structure of the motion compensation part which concerns on a 3rd Example. It is a flowchart which shows an example of the flow of the motion compensation process by the motion compensation part which concerns on a 3rd Example. It is a block diagram which shows an example of a detailed structure of the motion compensation part which concerns on a 4th Example. It is a flowchart which shows an example of the flow of the motion compensation process by the motion compensation part which concerns on a 4th Example. It is a block diagram which shows an example of a schematic structure of a television apparatus. It is a block diagram which shows an example of a schematic structure of a mobile telephone. It is a block diagram which shows an example of a schematic structure of a recording / reproducing apparatus. It is a block diagram which shows an example of a schematic structure of an imaging device.

  Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

The description will be given in the following order.
1. 1. Configuration example of image encoding device 2. Detailed configuration example of motion search unit 2-1. First Example 2-2. Second Example 2-3. Third Example 2-4. 4. Fourth embodiment 3. Configuration example of image decoding apparatus Detailed configuration example of motion compensation unit 4-1. First Example 4-2. Second Example 4-3. Third Example 4-4. 4. Fourth embodiment Application example 6. Summary

<1. Configuration Example of Image Encoding Device>
FIG. 1 is a block diagram illustrating an example of a configuration of an image encoding device 10 according to an embodiment. Referring to FIG. 1, an image encoding device 10 includes an A / D (Analogue to Digital) conversion unit 11, a rearrangement buffer 12, a subtraction unit 13, an orthogonal transformation unit 14, a quantization unit 15, a lossless encoding unit 16, The accumulation buffer 17, rate control unit 18, inverse quantization unit 21, inverse orthogonal transform unit 22, addition unit 23, deblock filter 24, frame memory 25, selectors 26 and 27, intra prediction unit 30, and motion search unit 40 are provided. Prepare.

  The A / D conversion unit 11 converts an image signal input in analog format into image data in digital format, and outputs a series of digital image data to the rearrangement buffer 12.

  The rearrangement buffer 12 rearranges images included in a series of image data input from the A / D conversion unit 11. The rearrangement buffer 12 rearranges the images according to the GOP (Group of Pictures) structure related to the encoding process, and then outputs the rearranged image data to the subtraction unit 13, the intra prediction unit 30, and the motion search unit 40. To do.

  The subtraction unit 13 is supplied with image data input from the rearrangement buffer 12 and predicted image data input from the intra prediction unit 30 or the motion search unit 40 described later. The subtraction unit 13 calculates prediction error data that is the difference between the image data input from the rearrangement buffer 12 and the predicted image data, and outputs the calculated prediction error data to the orthogonal transform unit 14.

  The orthogonal transform unit 14 performs orthogonal transform on the prediction error data input from the subtraction unit 13. The orthogonal transform executed by the orthogonal transform unit 14 may be, for example, discrete cosine transform (DCT) or Karoonen-Loeve transform. The orthogonal transform unit 14 outputs transform coefficient data acquired by the orthogonal transform process to the quantization unit 15.

  The quantization unit 15 is supplied with transform coefficient data input from the orthogonal transform unit 14 and a rate control signal from a rate control unit 18 described later. The quantizing unit 15 quantizes the transform coefficient data and outputs the quantized transform coefficient data (hereinafter referred to as quantized data) to the lossless encoding unit 16 and the inverse quantization unit 21. Further, the quantization unit 15 changes the bit rate of the quantized data input to the lossless encoding unit 16 by switching the quantization parameter (quantization scale) based on the rate control signal from the rate control unit 18. Let

  The lossless encoding unit 16 performs a lossless encoding process on the quantized data input from the quantization unit 15 to generate an encoded stream. The lossless encoding by the lossless encoding unit 16 may be variable length encoding or arithmetic encoding, for example. Further, the lossless encoding unit 16 multiplexes information related to intra prediction or information related to inter prediction input from the selector 27 in the header region of the encoded stream. Then, the lossless encoding unit 16 outputs the generated encoded stream to the accumulation buffer 17.

  The accumulation buffer 17 temporarily accumulates the encoded stream input from the lossless encoding unit 16. Then, the accumulation buffer 17 outputs the accumulated encoded stream to a transmission unit (not shown) (for example, a communication interface or a connection interface with a peripheral device) at a rate corresponding to the bandwidth of the transmission path.

  The rate control unit 18 monitors the free capacity of the accumulation buffer 17. Then, the rate control unit 18 generates a rate control signal according to the free capacity of the accumulation buffer 17 and outputs the generated rate control signal to the quantization unit 15. For example, the rate control unit 18 generates a rate control signal for reducing the bit rate of the quantized data when the free capacity of the storage buffer 17 is small. For example, when the free capacity of the accumulation buffer 17 is sufficiently large, the rate control unit 18 generates a rate control signal for increasing the bit rate of the quantized data.

  The inverse quantization unit 21 performs an inverse quantization process on the quantized data input from the quantization unit 15. Then, the inverse quantization unit 21 outputs transform coefficient data acquired by the inverse quantization process to the inverse orthogonal transform unit 22.

  The inverse orthogonal transform unit 22 restores the prediction error data by performing an inverse orthogonal transform process on the transform coefficient data input from the inverse quantization unit 21. Then, the inverse orthogonal transform unit 22 outputs the restored prediction error data to the addition unit 23.

  The adding unit 23 generates decoded image data by adding the restored prediction error data input from the inverse orthogonal transform unit 22 and the predicted image data input from the intra prediction unit 30 or the motion search unit 40. . Then, the addition unit 23 outputs the generated decoded image data to the deblock filter 24 and the frame memory 25.

  The deblocking filter 24 performs a filtering process for reducing block distortion that occurs when an image is encoded. The deblocking filter 24 removes block distortion by filtering the decoded image data input from the adding unit 23, and outputs the decoded image data after filtering to the frame memory 25.

  The frame memory 25 stores the decoded image data input from the adder 23 and the decoded image data after filtering input from the deblock filter 24 using a storage medium.

  The selector 26 reads filtered decoded image data used for inter prediction from the frame memory 25 and supplies the read decoded image data to the motion search unit 40 as reference image data. In addition, the selector 26 reads out the decoded image data before filtering used for intra prediction from the frame memory 25 and supplies the read out decoded image data to the intra prediction unit 30 as reference image data.

  In the inter prediction mode, the selector 27 outputs predicted image data as a result of inter prediction output from the motion search unit 40 to the subtraction unit 13 and outputs information related to inter prediction to the lossless encoding unit 16. In addition, in the intra prediction mode, the selector 27 outputs the prediction image data as a result of the intra prediction output from the intra prediction unit 30 to the subtraction unit 13 and outputs information related to the intra prediction to the lossless encoding unit 16. . The selector 27 switches between the inter prediction mode and the intra prediction mode according to the size of the cost function value output from the intra prediction unit 30 and the motion search unit 40.

  The intra prediction unit 30 is set in the image based on the image data to be encoded (original image data) input from the rearrangement buffer 12 and the decoded image data as reference image data supplied from the frame memory 25. Intra prediction processing is performed for each block to be processed. Then, the intra prediction unit 30 outputs information related to intra prediction including prediction mode information indicating an optimal prediction mode, a cost function value, and predicted image data to the selector 27.

  The motion search unit 40 performs a motion search process for inter prediction (interframe prediction) based on the original image data input from the rearrangement buffer 12 and the decoded image data supplied via the selector 26. The motion search process by the motion search unit 40 according to the present embodiment can be realized by extending the method described in Non-Patent Document 2 or the method described in Non-Patent Document 3. In the extension of the method described in Non-Patent Document 2, the motion search unit 40 can generate predictor information indicating an optimal predictor for each prediction unit. Further, in the extension of the technique described in Non-Patent Document 3, the motion search unit 40 can generate merge information indicating an optimal merge mode for each prediction unit. Then, the motion search unit 40 outputs information related to inter prediction including predictor information or merge information, motion vector information, and reference image information, a cost function value, and predicted image data to the selector 27. In the next section, four examples of the detailed configuration of the motion search unit 40 will be described.

  The image encoding device 10 repeats the series of encoding processes described here for each of a plurality of layers of an image to be scalable encoded. The layer encoded first is a layer that expresses the coarsest image, called a base layer. The base layer coded stream may be decoded independently without decoding the other layer coded streams. A layer other than the base layer is a layer called an enhancement layer (enhancement layer) that represents a finer image. The enhancement layer coded stream is coded using information included in the base layer coded stream in order to increase coding efficiency. Accordingly, in order to reproduce the enhancement layer image, both the base layer and enhancement layer encoded streams are decoded. There may be three or more layers handled in scalable coding. In this case, the lowest layer is the base layer, and the remaining layers are enhancement layers. The higher enhancement layer encoded stream may be encoded and decoded using information contained in the lower enhancement layer or base layer encoded stream. In this specification, of at least two layers having a dependency relationship, a layer on which the dependency is made is referred to as a lower layer, and a layer on which the dependency is concerned is referred to as an upper layer.

  When scalable coding is performed by the image coding device 10, motion correlation between layers is used to efficiently code information related to inter prediction. That is, in the inter prediction block, the motion vector is set to the upper layer based on the setting information related to the motion vector set to the lower layer. Specifically, the motion search unit 40 illustrated in FIG. 1 has a buffer for temporarily storing information obtained in inter prediction in a lower layer, and uses information stored in the buffer. To set a motion vector in the upper layer. The correlation of motion between layers can be particularly noticeable in scalable coding based on spatial scalability or SNR scalability.

  FIG. 2 is an explanatory diagram for explaining an example of spatial scalability. In FIG. 2, three layers L1, L2 and L3 to be scalable encoded are shown. Layer L1 is a base layer, and layers L2 and L3 are enhancement layers. The ratio of the spatial resolution of the layer L2 to the layer L1 is 2: 1. The ratio of the spatial resolution of layer L3 to layer L1 is 4: 1. Even if the resolutions are different from each other in this way, the motion that appears in the prediction unit B1 in the layer L1 may appear in the same way in the corresponding prediction unit B2 in the layer L2 and the corresponding prediction unit B3 in the layer L3. Is expensive. This is the correlation of motion between layers in spatial scalability.

  FIG. 3 is an explanatory diagram for explaining an example of SNR scalability. In FIG. 3, three layers L1, L2 and L3 to be scalable encoded are shown. Layer L1 is a base layer, and layers L2 and L3 are enhancement layers. The spatial resolutions of the layers L1, L2 and L3 are equal to each other. However, as an example, the minimum quantization scale of the layer L1 is 25, and the bit rate of the encoded stream is suppressed to about 2 Mbps by quantization of the orthogonal transform coefficient. On the other hand, for example, the minimum quantization scale of the layer L2 is 12, and the bit rate of the encoded stream is about 5 Mbps. For example, the minimum quantization scale of the layer L3 is 0, and the bit rate of the encoded stream is about 10 Mbps. Thus, even if the bit rates are different from each other, the motion appearing in the prediction unit B1 in the layer L1 can appear in the same way in the corresponding prediction unit B2 in the layer L2 and the corresponding prediction unit B3 in the layer L3. High nature. This is the correlation of motion between layers in SNR scalability.

  The image encoding device 10 according to the present embodiment efficiently uses the correlation of motion between layers as described above to efficiently encode information related to inter prediction.

  Note that the lower layer prediction unit corresponding to the upper layer prediction unit is, for example, a lower layer prediction unit having a pixel corresponding to a pixel at a predetermined position (for example, upper left) of the upper layer prediction unit. It's okay. With such a definition, even when there is an upper layer prediction unit that integrates a plurality of lower layer prediction units, the lower layer prediction unit corresponding to the upper layer prediction unit can be uniquely determined. . Instead, the prediction unit of the lower layer corresponding to the prediction unit of the upper layer is, for example, the largest overlap among the prediction units in the lower layer that overlap with the prediction unit of the upper layer (sharing pixels at the same position). It may be a prediction unit (the largest number of shared pixels). With such a definition, it is possible to determine the prediction unit in which the correlation of motion is most likely to appear as the “corresponding prediction unit”.

<2. Detailed configuration example of motion search unit>
In this section, four examples of the detailed configuration of the motion search unit 40 shown in FIG. 1 will be described. Of the four examples, the first and second examples are examples of the extension of the technique described in Non-Patent Document 2. On the other hand, the third and fourth examples are examples of the extension of the technique described in Non-Patent Document 3.

[2-1. First Example]
FIG. 4 is a block diagram illustrating an example of a detailed configuration of the motion search unit 40 according to the first embodiment. Referring to FIG. 4, the motion search unit 40 includes a search control unit 141, a motion vector calculation unit 142, a motion vector prediction unit 143, a motion vector buffer 144, a mode selection unit 145, an information generation unit 146, and a predictor information buffer 147. .

(1) Base Layer In the base layer motion search process, the search control unit 141 arranges one or more prediction units in a coding unit, and causes the motion vector calculation unit 142 to calculate a motion vector for each prediction unit. The motion vector calculated by the motion vector calculation unit 142 is output to the motion vector prediction unit 143 and stored in the motion vector buffer 144. The motion vector prediction unit 143 generates a predicted motion vector using a motion vector (referred to as a reference motion vector) of another block stored in the motion vector buffer 144 according to each of the plurality of predictor candidates. Then, the motion vector prediction unit 143 calculates a differential motion vector that is a difference between the motion vector calculated by the motion vector calculation unit 142 and the predicted motion vector. The mode selection unit 145 generates predicted image data using the motion vector calculated by the motion vector calculation unit 142, and evaluates the cost function value calculated based on the comparison between the generated predicted image data and the original image data. To do. Then, the mode selection unit 145 selects an optimal prediction unit arrangement that minimizes the cost function value and an optimal predictor for each prediction unit. The information generation unit 146 generates information related to inter prediction including predictor information indicating the optimal predictor selected for each prediction unit and differential motion vector information indicating the corresponding differential motion vector. For example, the predictor information may include an index that identifies the reference motion vector. Further, the predictor information may include a parameter that specifies a prediction formula. Then, the information generation unit 146 outputs the generated inter prediction information, cost function value, and predicted image data to the selector 27. Further, the predictor information generated by the information generation unit 146 is temporarily stored in the predictor information buffer 147 for processing in the upper layer.

  5 and 6 are explanatory diagrams for explaining examples of predictor candidates for motion vector prediction that can be used in such inter prediction. Referring to FIG. 5, one prediction unit PTe that is a prediction target and a predicted motion vector PMVe of the prediction unit PTe are illustrated. The prediction motion vector PMVe of the prediction unit PTe can be predicted using, for example, the motion vectors MVa, MVb, and MVc of the prediction unit adjacent to the prediction unit PTe as reference motion vectors. The reference motion vector MVa is a motion vector set in a prediction unit adjacent to the left of the prediction unit PTe. The reference motion vector MVb is a motion vector set in a prediction unit adjacent to the prediction unit PTe. The reference motion vector MVc is a motion vector set in a prediction unit adjacent to the upper right of the prediction unit PTe. The predicted motion vector PMVe can be generated according to the following prediction formula using these reference motion vectors MVa, MVb, and MVc.

  Formula (1) is a prediction formula based on the spatial correlation of motion. In the formula (1), med represents a median operation. That is, according to the equation (1), the predicted motion vector PMVe is a vector whose components are the median value of the horizontal components and the median value of the vertical components of the reference motion vectors MVa, MVb and MVc. The predicted motion vector PMVe generated according to Expression (1) is an example of a predictor candidate. A predicted motion vector calculated by a prediction formula based on such spatial correlation of motion is referred to as a spatial predictor.

  Equation (1) is only an example of a prediction equation. For example, if any of the motion vectors MVa, MVb, or MVc does not exist because the prediction unit to be predicted is located at the end of the image, the non-existing motion vector may be omitted from the median operation argument. Good. In addition, simpler spatial predictors may also be used as predictor candidates, as in the following formulas (2) to (4).

  Meanwhile, a temporal predictor that is a predicted motion vector calculated by a prediction formula based on a temporal correlation of motion may also be used as a predictor candidate. Referring to FIG. 6, an image IM01 including a prediction unit PTe to be predicted and a reference image IM02 are shown. The block Bcol in the reference image IM02 is a collocated block of the prediction unit PTe. The prediction formula using temporal correlation of motion uses, for example, a motion vector set in the collocated block Bcol or a block adjacent to the collocated block Bcol as a reference motion vector.

  For example, the motion vector set in the collocated block Bcol is MVcol. In addition, the motion vectors set in the upper, left, lower, right, upper left, lower left, lower right, and upper right blocks of the collocated block Bcol are MVt0 to MVt7, respectively. Then, the predicted motion vector PMVe can be generated from the reference motion vectors MVcol and MVt0 to MVt7 using, for example, the following prediction formula (5) or (6).

  The motion vector predicting unit 143 generates a predicted motion vector PMVe for each of the plurality of predictor candidates, and then, as in the following equation, the difference between the motion vector MVe calculated by the motion vector calculating unit 142 and the predicted motion vector PMVe The difference motion vector MVDe representing is calculated.

  Then, an optimal predictor (for example, a predictor with the highest prediction accuracy) for each prediction unit is selected by the mode selection unit 145, and a difference motion vector corresponding to the predictor information indicating the optimal predictor is displayed by the information generation unit 146. Difference motion vector information is generated. For a prediction unit in which the motion vector is not predicted, motion vector information indicating the motion vector calculated by the motion vector calculation unit 142 may be generated instead of the difference motion vector information. The information generated in this way can be encoded by the lossless encoding unit 16 as information regarding inter prediction. The predictor information is temporarily stored in the predictor information buffer 147 for processing in an upper layer.

(2) Enhancement Layer In the enhancement layer motion search process, a motion vector is predicted based on the predictor information of the lower layer stored in the predictor information buffer 147.

  First, the search control unit 141 causes the motion vector calculation unit 142 to calculate a motion vector for each prediction unit arranged in the coding unit. Then, the search control unit 141 causes the motion vector prediction unit 143 to generate a predicted motion vector for each prediction unit. The motion vector predictor 143 in the enhancement layer generates the motion vector predictor using predictor information that is setting information stored in the predictor information buffer 147. More specifically, for example, when the predictor information indicates a spatial predictor as shown in Expression (1) for a prediction unit in a lower layer corresponding to a prediction unit in a certain upper layer, a motion vector prediction unit 143 obtains the reference motion vector of the adjacent prediction unit in the higher layer from the motion vector buffer 144. Then, the motion vector predicting unit 143 substitutes the acquired reference motion vector into Expression (1) to generate a predicted motion vector. For example, when the predictor information indicates a temporal predictor as shown in Expression (5) for a prediction unit in a lower layer corresponding to a prediction unit in a certain upper layer, the motion vector prediction unit 143 refers to The reference motion vector of the collocated block in the image and the adjacent block of the collocated block is obtained from the motion vector buffer 144. Then, the motion vector predicting unit 143 substitutes the acquired reference motion vector into Expression (5) to generate a predicted motion vector. Furthermore, the motion vector prediction unit 143 calculates a difference motion vector that represents the difference between the motion vector calculated by the motion vector calculation unit 142 and the predicted motion vector. The mode selection unit 145 generates predicted image data using the motion vector calculated by the motion vector calculation unit 142, and calculates a cost function value. The information generation unit 146 generates differential motion vector information indicating the differential motion vector calculated for each prediction unit. Then, the information generation unit 146 outputs information regarding inter prediction including the difference motion vector information, the cost function value, and predicted image data to the selector 27.

(3) Process Flow FIG. 7 is a flowchart illustrating an example of the flow of the motion search process by the motion search unit 40 according to the present embodiment. Referring to FIG. First, the motion search unit 40 performs a base layer motion search process (step S110). As a result, the arrangement of prediction units within each coding unit is determined, and the optimal predictor for each prediction unit is selected. The predictor information buffer 147 buffers predictor information indicating an optimal predictor of each prediction unit as setting information.

  The processing in steps S111 to S117 is enhancement layer motion search processing. Among these processes, the processes of steps S111 to S116 are repeated for each prediction unit (hereinafter referred to as a focused PU) of each enhancement layer. In the following description, “upper layer” is a prediction target layer, and “lower layer” is a lower layer of the prediction target layer.

  First, the motion vector calculation unit 142 calculates a motion vector for one target PU in the upper layer based on the pixel value of the original image and the pixel value of the reference image input from the frame memory 25 (step S111). . Then, the motion vector calculation unit 142 outputs the calculated motion vector to the motion vector prediction unit 143 and the motion vector buffer 144.

  Next, the motion vector predicting unit 143 uses the predictor information for the corresponding PU in the lower layer stored in the predictor information buffer 147 and the reference motion vector acquired according to the predictor information, for the attention PU. A predicted motion vector is generated (step S112). Next, the motion vector prediction unit 143 calculates a differential motion vector by subtracting the predicted motion vector from the motion vector (step S113). Then, the motion vector prediction unit 143 outputs the motion vector and the difference motion vector for the attention PU to the mode selection unit 145.

  Next, the mode selection part 145 produces | generates the prediction image data and cost function value about attention PU (step S114). Further, the information generation unit 146 generates differential motion vector information indicating the differential motion vector for the attention PU (step S115).

  Thereafter, when an unprocessed PU remains in the prediction target layer, the process returns to step S111 (step S116). On the other hand, if there is no unprocessed PU remaining, it is determined whether there is a remaining layer (a higher layer) (step S117). Here, when there are remaining layers, the processing after step S111 is repeated with the previous prediction target layer as the lower layer and the next layer as the upper layer. Predictor information indicating the predictor selected for the lower layer is subsequently buffered by the predictor information buffer 147. If there are no remaining layers, the motion search process in FIG. 7 ends. The prediction image data generated here and information regarding inter prediction (which may include differential motion vector information) may be output to the subtraction unit 13 and the lossless encoding unit 16 via the switch 27, respectively.

  As described above, in the first embodiment, the predictor information is not encoded as the information related to the inter prediction of the upper layer, and the predictor information of the lower layer is reused, so that the code amount of the information related to the inter prediction is reduced. Can do.

[2-2. Second embodiment]
FIG. 8 is a block diagram illustrating an example of a detailed configuration of the motion search unit 40 according to the second embodiment. Referring to FIG. 8, the motion search unit 40 includes a search control unit 241, a motion vector calculation unit 242, a motion vector prediction unit 243, a motion vector buffer 244, a mode selection unit 245, and an information generation unit 246.

(1) Base Layer The base layer motion search process according to the present embodiment may be the same as the base layer motion search process according to the first embodiment described above. However, in this embodiment, the predictor information of the base layer does not have to be buffered, and the motion vector information of the base layer is buffered across the layers. In the base layer motion search process, the search control unit 241 arranges one or more prediction units in the encoding unit, and causes the motion vector calculation unit 242 to calculate a motion vector for each prediction unit. The motion vector calculated by the motion vector calculation unit 242 is output to the motion vector prediction unit 243 and stored in the motion vector buffer 244. The motion vector prediction unit 243 generates a predicted motion vector using the reference motion vector stored in the motion vector buffer 244 according to each of the plurality of predictor candidates. Then, the motion vector prediction unit 243 calculates a differential motion vector that is a difference between the motion vector calculated by the motion vector calculation unit 242 and the predicted motion vector. The mode selection unit 245 generates predicted image data using the motion vector calculated by the motion vector calculation unit 242, and evaluates the cost function value calculated based on the comparison between the generated predicted image data and the original image data. To do. Then, the mode selection unit 245 selects an optimal prediction unit arrangement that minimizes the cost function value and an optimal predictor for each prediction unit. The information generation unit 246 generates information related to inter prediction including predictor information indicating the optimal predictor selected for each prediction unit and differential motion vector information indicating the corresponding differential motion vector. Then, the information generation unit 246 outputs information regarding the generated inter prediction, a cost function value, and predicted image data to the selector 27.

(2) Enhancement Layer Predictor candidates searched in the base layer motion search process according to the present embodiment may include one or both of the spatial predictor and the temporal predictor described above. Furthermore, in the enhancement layer motion search process according to the present embodiment, additional predictor candidates are introduced. The predictor candidate introduced here is a predictor candidate using a motion vector set as a corresponding prediction unit of the lower layer as a reference motion vector. Such a predictor is referred to as an inter-layer predictor in this specification.

  FIG. 9 is an explanatory diagram for describing an example of an inter-layer predictor. Referring to FIG. 9, a prediction unit PTe in the upper layer L12 and a prediction motion vector PMVe of the prediction unit PTe are shown. The prediction unit PTbase in the lower layer L11 is a prediction unit corresponding to the prediction unit PTe. The reference motion vector MVbase is a motion vector set in the prediction unit PTbase. The inter-layer predictor can be expressed by the following equation (8), for example.

  When the spatial resolution differs between the lower layer and the upper layer, the motion vector expanded as shown in the following equation according to the spatial resolution ratio N between the lower layer and the upper layer is used as an inter-layer predictor. May be used. In this case, the values of the vertical component and the horizontal component of the inter-layer predictor can be rounded to match the accuracy of the motion vector of the upper layer (for example, ¼ pixel accuracy).

  Unlike the first embodiment, in this embodiment, the optimal predictor is selected from a plurality of predictor candidates in the motion search process of the enhancement layer.

  First, the search control unit 241 causes the motion vector calculation unit 242 to calculate a motion vector for each prediction unit in the coding unit. The motion vector calculated by the motion vector calculation unit 242 is output to the motion vector prediction unit 243 and stored in the motion vector buffer 244. The motion vector buffer 244 also stores a motion vector (reference motion vector) calculated for each prediction unit of the lower layer. The motion vector prediction unit 243 generates a predicted motion vector using the reference motion vector stored in the motion vector buffer 244 according to each of the plurality of predictor candidates. The plurality of predictor candidates here include the above-described inter-layer predictors. Then, the motion vector prediction unit 243 calculates a differential motion vector that is a difference between the motion vector calculated by the motion vector calculation unit 242 and the predicted motion vector. The mode selection unit 245 generates predicted image data using the motion vector calculated by the motion vector calculation unit 242, and evaluates the cost function value calculated based on the comparison between the generated predicted image data and the original image data. To do. And the mode selection part 245 selects the optimal predictor about each prediction unit. The information generation unit 246 generates information related to inter prediction including predictor information indicating the optimal predictor selected for each prediction unit and differential motion vector information indicating the corresponding differential motion vector. When the above-described inter-layer predictor is selected as the optimal predictor, the predictor information may include an index that identifies the reference motion vector of the lower layer. Then, the information generation unit 246 outputs information regarding the generated inter prediction, a cost function value, and predicted image data to the selector 27.

(3) Process Flow FIG. 10 is a flowchart illustrating an example of a flow of motion search processing by the motion search unit 40 according to the present embodiment. Referring to FIG. First, the motion search unit 40 performs a base layer motion search process (step S120). As a result, the arrangement of prediction units within each coding unit is determined, and the optimal predictor for each prediction unit is selected. The motion vector buffer 244 buffers the motion vector calculated for each prediction unit.

  The processing in steps S121 to S127 is enhancement layer motion search processing. Of these processes, the processes of steps S121 to S126 are repeated for each attention PU of each enhancement layer. In the following description, “upper layer” is a prediction target layer, and “lower layer” is a lower layer of the prediction target layer.

  First, the motion vector calculation unit 242 calculates a motion vector for one target PU in the upper layer based on the pixel value of the original image and the pixel value of the reference image input from the frame memory 25 (step S121). . Then, the motion vector calculation unit 242 outputs the calculated motion vector to the motion vector prediction unit 243 and the motion vector buffer 244.

  Next, the motion vector prediction unit 243 generates a predicted motion vector for the attention PU using the reference motion vector stored in the motion vector buffer 244 according to each of the plurality of predictor candidates (step S122). The plurality of predictor candidates here include inter-layer predictors. Next, the motion vector prediction unit 243 calculates a difference motion vector for each of the plurality of predictor candidates (step S123). Then, the motion vector prediction unit 243 outputs the motion vector and the difference motion vector for each predictor candidate to the mode selection unit 245.

  Next, the mode selection unit 245 generates predicted image data for each predictor candidate, and selects an optimal predictor by evaluating the cost function value (step S124). Then, the information generating unit 246 generates predictor information indicating the selected optimal predictor and differential motion vector information indicating the corresponding differential motion vector (step S125).

  Thereafter, when an unprocessed PU remains in the prediction target layer, the process returns to step S121 (step S126). On the other hand, if there is no unprocessed PU remaining, it is determined whether there is a remaining layer (higher layer) (step S127). Here, if there is a remaining layer, The processes after step S121 are repeated with the prediction target layer as the lower layer and the next layer as the upper layer. The motion vector calculated for each attention PU of the lower layer is buffered by the motion vector buffer 244. If there are no remaining layers, the motion search process in FIG. 10 ends. The prediction image data generated here and information regarding inter prediction (which may include predictor information and differential motion vector information) may be output to the subtraction unit 13 and the lossless encoding unit 16 via the switch 27, respectively.

  Thus, in the second embodiment, predictor information indicating that an inter-layer predictor based on a motion vector set in a lower layer should be used as information related to inter prediction in the upper layer. Therefore, it is possible to predict a motion vector based on a corresponding prediction unit of a lower layer having a significant motion correlation. Therefore, the accuracy of motion vector prediction is improved, and as a result, the code amount of the difference motion vector can be reduced.

  Note that the lossless encoding unit 16 that encodes the predictor information may assign the smallest code number to the inter-layer predictor among the plurality of predictor candidates in the encoding of the predictor information of the higher layer. Usually, the motion correlation between layers is stronger than the spatial correlation of motion and the temporal correlation of motion. Therefore, by assigning the minimum code number to the inter-layer predictor, it is possible to use more shorter codewords in the encoded stream after variable length encoding, thereby further reducing the code amount.

[2-3. Third Example]
FIG. 11 is a block diagram illustrating an example of a detailed configuration of the motion search unit 40 according to the third embodiment. Referring to FIG. 11, the motion search unit 40 includes a search control unit 341, a motion vector calculation unit 342, a motion vector buffer 344, a mode selection unit 345, an information generation unit 346, and a merge information buffer 347.

(1) Base Layer In the base layer motion search process, the search control unit 341 arranges one or more prediction units in a coding unit, and causes the motion vector calculation unit 342 to calculate a motion vector for each prediction unit. The motion vector calculated by the motion vector calculation unit 342 is output to the mode selection unit 345 and stored in the motion vector buffer 344. The mode selection unit 345 merges these prediction units when the motion vector calculated by the motion vector calculation unit 342 for a certain prediction unit is common to the reference motion vector set in one or more adjacent prediction units. Decide that. In the method proposed by Non-Patent Document 3, a certain prediction unit can be merged with a prediction unit adjacent on the top or a prediction unit adjacent on the left. That is, for example, the mode selection unit 345 can select, as the merge mode, one of merge with the prediction unit adjacent on the upper side, merge with the prediction unit adjacent on the left, or no merge. Further, the mode selection unit 345 generates predicted image data for each prediction unit, and calculates a cost function value based on a comparison between the generated predicted image data and original image data. The information generation unit 346 generates information related to inter prediction including merge information indicating the merge mode for each prediction unit and motion vector information for a prediction unit that is not merged with other prediction units. Then, the information generation unit 346 outputs information regarding the generated inter prediction, a cost function value, and predicted image data to the selector 27.

  The merge information generated in the present embodiment may include, for example, two flags “MergeFlag” and “MergeLeftFlag”. MergeFlag is a flag indicating whether or not the motion vector of the attention PU is common to the motion vector of at least one adjacent PU. For example, when MergeFlag = 1, the motion vector of the attention PU is common to the motion vector of at least one adjacent PU. When MergeFlag = 0, the motion vector of the attention PU is different from the motion vector of any adjacent PU. When MergeFlag = 0, MergeLeftFlag is not encoded, and instead, a motion vector (and motion information such as reference image information) is encoded for the attention PU. Even when MergeFlag = 1 and the motion vectors of two adjacent PUs are common, MergeLeftFlag may not be encoded.

  MergeLeftFlag is a flag indicating whether the motion vector of the attention PU is common to the motion vector of the left adjacent PU. For example, when MergeLeftFlag = 1, the motion vector of the attention PU is the same as the motion vector of the left adjacent PU. When MergeLeftFlag = 0, the motion vector of the attention PU is different from the motion vector of the left adjacent PU and is common to the motion vector of the upper adjacent PU.

  12A to 12C respectively show examples of merge information that can be generated in the present embodiment. In these three figures, the prediction unit B20 which is the attention PU in the layer L21 is shown. Prediction units B21 and B22 are adjacent to the left and top of prediction unit B20, respectively. The motion vector MV20 is a motion vector calculated by the motion vector calculation unit 342 for the prediction unit B20. The motion vectors MV21 and MV22 are reference motion vectors set in the prediction units B21 and B22, respectively.

  In the example of FIG. 12A, the motion vector MV20 is common to both the reference motion vectors MV21 and MV22. In this case, the information generation unit 346 generates MergeFlag = 1 as merge information. MergeLeftFlag is not included in merge information. The decoding side that has received such merge information can set a motion vector that is common to the motion vector set in the prediction unit B21 or B22 in the prediction unit B20 without decoding MergeLeftFlag.

  In the example of FIG. 12B, the motion vector MV20 is common to the reference motion vector MV21 and is different from the reference motion vector MV22. In this case, the information generation unit 346 generates MergeFlag = 1 and MergeLeftFlag = 1 as merge information. The decoding side that has received such merge information can set a motion vector common to the motion vector set in the prediction unit B21 in the prediction unit B20.

  In the example of FIG. 12C, the motion vector MV20 is common to the reference motion vector MV22 and is different from the reference motion vector MV21. In this case, the information generation unit 346 generates MergeFlag = 1 and MergeLeftFlag = 0 as merge information. The decoding side that has received such merge information can set a motion vector common to the motion vector set in the prediction unit B22 in the prediction unit B20.

(2) Enhancement Layer In the enhancement layer motion search process, a motion vector is set for each prediction unit using lower layer merge information stored in the merge information buffer 347.

  First, the search control unit 341 acquires merge information about the prediction unit in the lower layer corresponding to each prediction unit in the encoding unit of the upper layer from the merge information buffer 347. Then, when the acquired merge information indicates no merge (for example, MergeFlag = 0), the search control unit 341 causes the motion vector calculation unit 342 to calculate a motion vector for the prediction unit of the upper layer. The motion vector calculated by the motion vector calculation unit 342 is output to the mode selection unit 345 and stored in the motion vector buffer 344. On the other hand, when the acquired merge information indicates merging with another prediction unit, the search control unit 341 does not cause the motion vector calculation unit 342 to calculate a motion vector for the prediction unit of the higher layer. Instead, the mode selection unit 345 obtains a motion vector (for a prediction unit to be merged) (for example, MergeLeftFlag = 1) acquired from the motion vector buffer 344 for a prediction unit to be merged with another prediction unit. Prediction image data is generated using a motion vector of a prediction unit adjacent to the left), and a cost function value is calculated. On the other hand, the mode selection unit 345 generates prediction image data using a motion vector input from the motion vector calculation unit 342 and calculates a cost function value for a prediction unit that is not merged with another prediction unit. The information generation unit 346 generates information related to inter prediction including motion vector information about prediction units that are not merged with other prediction units. Then, the information generation unit 346 outputs information regarding the generated inter prediction, a cost function value, and predicted image data to the selector 27.

(3) Process Flow FIG. 13 is a flowchart illustrating an example of the flow of the motion search process by the motion search unit 40 according to the present embodiment. Referring to FIG. First, the motion search unit 341 performs base layer motion search processing (step S130). As a result, the arrangement of the prediction units in each coding unit is determined, and the merge mode of each prediction unit is selected. The motion vector buffer 344 buffers the motion vector calculated for each prediction unit. The merge information buffer 347 buffers merge information indicating the merge mode selected for each prediction unit as setting information.

  The processes in steps S131 to S136 are enhancement layer motion search processes. Among these processes, the processes of steps S131 to S135 are repeated for each attention PU of each enhancement layer. In the following description, “upper layer” is a prediction target layer, and “lower layer” is a lower layer of the prediction target layer.

  First, the search control unit 341 refers to one merged PU stored in the merge information buffer 347 to determine whether the PU of the corresponding lower layer is merged with another PU for one attention PU of the upper layer. Determination is made (step S131). Here, when the PU of the corresponding lower layer is merged with other PUs, the attention PU is also merged with other PUs, and therefore the subsequent process of step S132 is skipped.

  In step S132, the motion vector calculation unit 342 calculates a motion vector for the attention PU that is not merged with other PUs based on the pixel value of the original image and the pixel value of the reference image input from the frame memory 25 ( Step S132). Then, the motion vector calculation unit 342 outputs the calculated motion vector to the mode selection unit 345 and the motion vector buffer 344.

  Next, the mode selection unit 345 generates predicted image data using the motion vector calculated by the motion vector calculation unit 342 or acquired from the motion vector buffer 344, and calculates a cost function value (step S133). Then, the information generation unit 346 generates motion vector information for the attention PU that is not merged with other PUs (step S134).

  Thereafter, when an unprocessed PU remains in the prediction target layer, the process returns to step S131 (step S135). On the other hand, if there is no unprocessed PU remaining, it is determined whether there is a remaining layer (higher layer) (step S136). Here, if there is a remaining layer, The processes after step S131 are repeated with the previous prediction target layer as the lower layer and the next layer as the upper layer. The motion vector calculated for each attention PU of the lower layer is buffered by the motion vector buffer 344. The merge information is continuously buffered by the merge information buffer 347. If there are no remaining layers, the motion search process in FIG. 13 ends. Information about the predicted image data and the inter prediction generated here can be output to the subtraction unit 13 and the lossless encoding unit 16 via the switch 27, respectively.

  As described above, in the third embodiment, the merge information is not encoded as the information related to the inter prediction of the upper layer, and the merge information of the lower layer is reused, so that the code amount of the information related to the inter prediction is reduced. Can do.

[2-4. Fourth Example]
FIG. 14 is a block diagram illustrating an example of a detailed configuration of the motion search unit 40 according to the fourth embodiment. Referring to FIG. 14, the motion search unit 40 includes a search control unit 441, a motion vector calculation unit 442, a motion vector buffer 444, a mode selection unit 445, and an information generation unit 446.

(1) Base Layer The base layer motion search process according to the present embodiment may be the same as the base layer motion search process according to the third embodiment described above. However, in this embodiment, the base layer merge information may not be buffered. In the base layer motion search process, the search control unit 441 arranges one or more prediction units in the encoding unit, and causes the motion vector calculation unit 442 to calculate a motion vector for each prediction unit. The motion vector calculated by the motion vector calculation unit 442 is output to the mode selection unit 445 and stored in the motion vector buffer 444. The mode selection unit 445 merges these prediction units when the motion vector calculated by the motion vector calculation unit 442 for a certain prediction unit is the same as the reference motion vector set in one or more adjacent prediction units. Decide that. The mode selection unit 445 generates predicted image data for each prediction unit, and calculates a cost function value based on a comparison between the generated predicted image data and original image data. The information generation unit 346 generates information related to inter prediction including merge information indicating the merge mode for each prediction unit and motion vector information for a prediction unit that is not merged with other prediction units. Then, the information generation unit 346 outputs information regarding the generated inter prediction, a cost function value, and predicted image data to the selector 27.

(2) Enhancement Layer The merge information generated in the base layer motion search process according to the present embodiment may include two flags “MergeFlag” and “MergeLeftFlag” similar to those in the third embodiment. In contrast, the merge information generated in the motion search process of the enhancement layer may additionally include a new flag “MergeBaseFlag”. MergeBaseFlag is a flag indicating whether or not the motion vector of the attention PU is common to the motion vector of the PU of the corresponding lower layer. For example, when MergeBaseFlag = 1, the motion vector of the attention PU is common to the motion vector of the PU of the corresponding lower layer.

  15A to 15C respectively show examples of merge information that can be generated in the present embodiment. In these three figures, the prediction unit B30 which is the attention PU in the higher layer L30 is shown. The prediction units B31 and B32 are adjacent to the left and above the prediction unit B30, respectively. The motion vector MV30 is a motion vector calculated by the motion vector calculation unit 442 for the prediction unit B30. The motion vectors MV31 and MV32 are reference motion vectors set in the prediction units B31 and B32, respectively. A prediction unit B20 that is a PU corresponding to the attention PU in the lower layer 21 is also shown. The motion vector MV20 is a reference motion vector buffered for the prediction unit B20.

  In the example of FIG. 15A, the motion vector MV30 is common to all of the reference motion vectors MV31, MV32, and MV20. In this case, the information generation unit 446 generates MergeFlag = 1 as merge information. MergeBaseFlag and MergeLeftFlag are not included in the merge information. The decoding side that has received such merge information can set a motion vector common to the prediction unit B20, B31, or B32 in the prediction unit B30 without decoding MergeBaseFlag and MergeLeftFlag.

  In the example of FIG. 15B, the motion vector MV30 is common to the reference motion vector MV20 and is different from the reference motion vectors MV31 and MV32. In this case, the information generation unit 446 generates MergeFlag = 1 and MergeBaseFlag = 1 as merge information. The decoding side that has received such merge information can set a motion vector common to the motion vector set in the prediction unit B20 in the lower layer L21 to the prediction unit B30 in the upper layer L30.

  In the example of FIG. 15C, the motion vector MV30 is common to the reference motion vector MV31 and is different from the reference motion vectors MV20 and MV32. In this case, the information generation unit 446 generates MergeFlag = 1, MergeBaseFlag = 0, and MergeLeftFlag = 1 as merge information. The decoding side that has received such merge information can set a motion vector common to the motion vector set in the prediction unit B31 in the prediction unit B30.

  In the enhancement layer motion search process, the search control unit 441 causes the motion vector calculation unit 442 to calculate a motion vector for each prediction unit in the coding unit. The motion vector calculated by the motion vector calculation unit 442 is output to the mode selection unit 445 and stored in the motion vector buffer 444. The motion vector buffer 444 also stores a motion vector (reference motion vector) calculated for each prediction unit of the lower layer. When the motion vector calculated by the motion vector calculation unit 442 for a certain prediction unit is common to the adjacent prediction unit or the reference motion vector set to the corresponding prediction unit in the lower layer, these modes are selected. Decide to merge prediction units. That is, for example, the mode selection unit 445 selects, as the merge mode, one of merge with the lower layer, merge with the prediction unit adjacent to the upper layer, merge with the prediction unit adjacent to the left, or no merge. obtain. Further, the mode selection unit 445 generates predicted image data for each prediction unit, and calculates a cost function value based on a comparison between the generated predicted image data and original image data. The information generation unit 346 generates information related to inter prediction including merge information indicating the merge mode for each prediction unit and motion vector information for a prediction unit that is not merged with other prediction units. Then, the information generation unit 346 outputs information regarding the generated inter prediction, a cost function value, and predicted image data to the selector 27.

(3) Process Flow FIG. 16 is a flowchart illustrating an example of a flow of motion search processing by the motion search unit 40 according to the present embodiment. Referring to FIG. First, the motion search unit 441 performs base layer motion search processing (step S140). As a result, the arrangement of the prediction units in each coding unit is determined, and the merge mode of each prediction unit is selected. The motion vector buffer 444 buffers the motion vector calculated for each prediction unit.

  The processing in steps S141 to S146 is enhancement layer motion search processing. Among these processes, the processes of steps S141 to S145 are repeated for each attention PU of each enhancement layer. In the following description, “upper layer” is a prediction target layer, and “lower layer” is a lower layer of the prediction target layer.

  First, the motion vector calculation unit 442 calculates a motion vector for one target PU in the upper layer, based on the pixel value of the original image and the pixel value of the reference image input from the frame memory 25 (step S141). . Then, the motion vector calculation unit 442 outputs the calculated motion vector to the mode selection unit 445 and the motion vector buffer 444.

  Next, the mode selection unit 445 selects the merge mode by comparing the motion vector calculated by the motion vector calculation unit 442 with the reference motion vector stored in the motion vector buffer 444 (step S142). For example, if the motion vector calculated for the attention PU is common to the reference motion vector buffered for the corresponding PU in the lower layer, merging with the lower layer may be selected.

  Next, the mode selection unit 445 generates predicted image data using a motion vector for the attention PU, and calculates a cost function value (step S144). Then, the information generation unit 446 generates setting information including merge information about the attention PU (and motion vector information for the attention PU that is not merged with other PUs) (step S144).

  Thereafter, when an unprocessed PU remains in the prediction target layer, the process returns to step S141 (step S145). On the other hand, if there is no unprocessed PU remaining, it is further determined whether there is a remaining layer (higher layer) (step S146). Here, if there is a remaining layer, The processes after step S141 are repeated with the previous prediction target layer as the lower layer and the next layer as the upper layer. The motion vector calculated for each attention PU in the lower layer is buffered by the motion vector buffer 444. If there are no remaining layers, the motion search process in FIG. 16 ends. Information about the predicted image data and the inter prediction generated here can be output to the subtraction unit 13 and the lossless encoding unit 16 via the switch 27, respectively.

  As described above, in the fourth example, merge information indicating that the attention PU is merged with the corresponding PU in the lower layer (a common motion vector is set) as information related to inter prediction of the upper layer. Can be encoded. Therefore, the prediction unit can be merged with the lower layer where the correlation of the motion is remarkable, and the motion vector is not encoded for the prediction unit in the upper layer to be merged, so that the code amount is effectively reduced. Can do.

<3. Configuration Example of Image Decoding Device>
FIG. 17 is a block diagram illustrating an example of the configuration of the image decoding device 60 according to an embodiment. Referring to FIG. 17, the image decoding device 60 includes an accumulation buffer 61, a lossless decoding unit 62, an inverse quantization unit 63, an inverse orthogonal transform unit 64, an addition unit 65, a deblock filter 66, a rearrangement buffer 67, a D / A (Digital to Analogue) conversion unit 68, frame memory 69, selectors 70 and 71, intra prediction unit 80, and motion compensation unit 90 are provided.

  The accumulation buffer 61 temporarily accumulates the encoded stream input via the transmission path.

  The lossless decoding unit 62 decodes the encoded stream input from the accumulation buffer 61 according to the encoding method used at the time of encoding. In addition, the lossless decoding unit 62 decodes information multiplexed in the header area of the encoded stream. The information multiplexed in the header area of the encoded stream can include, for example, the information related to inter prediction and the information related to intra prediction described above. The lossless decoding unit 62 outputs information related to inter prediction to the motion compensation unit 90. Further, the lossless decoding unit 62 outputs information related to intra prediction to the intra prediction unit 80.

  The inverse quantization unit 63 inversely quantizes the quantized data decoded by the lossless decoding unit 62. The inverse orthogonal transform unit 64 generates prediction error data by performing inverse orthogonal transform on the transform coefficient data input from the inverse quantization unit 63 according to the orthogonal transform method used at the time of encoding. Then, the inverse orthogonal transform unit 64 outputs the generated prediction error data to the addition unit 65.

  The adding unit 65 adds the prediction error data input from the inverse orthogonal transform unit 64 and the predicted image data input from the selector 71 to generate decoded image data. Then, the addition unit 65 outputs the generated decoded image data to the deblock filter 66 and the frame memory 69.

  The deblock filter 66 removes block distortion by filtering the decoded image data input from the adder 65 and outputs the decoded image data after filtering to the rearrangement buffer 67 and the frame memory 69.

  The rearrangement buffer 67 generates a series of time-series image data by rearranging the images input from the deblocking filter 66. Then, the rearrangement buffer 67 outputs the generated image data to the D / A conversion unit 68.

  The D / A converter 68 converts the digital image data input from the rearrangement buffer 67 into an analog image signal. Then, the D / A conversion unit 68 displays an image by outputting an analog image signal to a display (not shown) connected to the image decoding device 60, for example.

  The frame memory 69 stores the decoded image data before filtering input from the adding unit 65 and the decoded image data after filtering input from the deblocking filter 66 using a storage medium.

  The selector 70 switches the output destination of the image data from the frame memory 69 between the intra prediction unit 80 and the motion compensation unit 90 for each block in the image according to the mode information acquired by the lossless decoding unit 62. . For example, when the intra prediction mode is designated, the selector 70 outputs the decoded image data before filtering supplied from the frame memory 69 to the intra prediction unit 80 as reference image data. Further, when the inter prediction mode is designated, the selector 70 outputs the decoded image data after filtering supplied from the frame memory 69 to the motion compensation unit 90 as reference image data.

  The selector 71 switches the output source of the predicted image data to be supplied to the adding unit 65 between the intra prediction unit 80 and the motion compensation unit 90 according to the mode information acquired by the lossless decoding unit 62. For example, the selector 71 supplies the prediction image data output from the intra prediction unit 80 to the adding unit 65 when the intra prediction mode is designated. Further, when the inter prediction mode is designated, the selector 71 supplies the predicted image data output from the motion compensation unit 90 to the adding unit 65.

  The intra prediction unit 80 performs intra prediction processing based on the information related to intra prediction input from the lossless decoding unit 62 and the reference image data from the frame memory 69, and generates predicted image data. Then, the intra prediction unit 80 outputs the generated predicted image data to the selector 71.

  The motion compensation unit 90 performs motion compensation processing based on the information related to inter prediction input from the lossless decoding unit 62 and the reference image data from the frame memory 69, and generates predicted image data. The motion compensation processing by the motion compensation unit 90 according to the present embodiment can be realized by extending the method described in Non-Patent Document 2 or the method described in Non-Patent Document 3. Then, the motion compensation unit 90 outputs predicted image data generated as a result of the motion compensation process to the selector 71. In the next section, four examples of the detailed configuration of the motion compensation unit 90 will be described.

  The image decoding device 60 repeats the series of decoding processes described here for each of a plurality of layers of a scalable encoded image. The layer that is decoded first is the base layer. After the base layer is decoded, one or more enhancement layers are decoded. When decoding the enhancement layer, information obtained by decoding the lower layer, which is the base layer or another enhancement layer, is used.

  In scalable decoding by the image decoding device 60, a motion vector is set in a prediction unit in a certain upper layer using setting information related to a motion vector set in a corresponding prediction unit in a lower layer. The setting information may include, for example, the above-described predictor information, merge information, or motion vector information.

<4. Detailed configuration example of motion compensation unit>
In this section, four examples of the detailed configuration of the motion compensation unit 90 shown in FIG. 17 will be described. The four examples correspond to the four examples of the motion search unit 40 of the image encoding device 10 described above. The first and second embodiments are embodiments for extending the method described in Non-Patent Document 2 above. On the other hand, the third and fourth examples are examples of the extension of the technique described in Non-Patent Document 3.

[4-1. First Example]
FIG. 18 is a block diagram illustrating an example of a detailed configuration of the motion compensation unit 90 according to the first embodiment. Referring to FIG. 18, the motion compensation unit 90 includes an information acquisition unit 191, a motion vector setting unit 192, a predictor information buffer 193, a motion vector buffer 194, and a compensation unit 195.

(1) Base Layer In the base layer motion compensation process, the information acquisition unit 191 acquires information related to inter prediction decoded from the encoded stream by the lossless decoding unit 62. In the present embodiment, the information related to inter prediction may include predictor information and differential motion vector information (motion vector information for a prediction unit in which motion vector prediction is not performed). The predictor information acquired here indicates, for example, the predictor selected for each prediction unit at the time of encoding among the various predictor candidates described above. The motion vector setting unit 192 sets a motion vector for each prediction unit. The motion vector set for each prediction unit by the motion vector setting unit 192 is output to the compensation unit 195 and stored in the motion vector buffer 194. Also, the predictor information for each prediction unit is temporarily stored in the predictor information buffer 193 for processing in the upper layer. The setting of the motion vector by the motion vector setting unit 192 can be performed using the predictor indicated by the predictor information and the differential motion vector indicated by the differential motion vector information for each prediction unit. For example, when the predictor information for a certain prediction unit indicates a spatial predictor as shown in Expression (1), the motion vector setting unit 192 uses the reference motion vector of the prediction unit adjacent to the prediction unit as a motion vector buffer. Obtained from 194. Then, the motion vector setting unit 192 generates the predicted motion vector by substituting the acquired reference motion vector into Expression (1). Furthermore, the motion vector setting unit 192 reconstructs the motion vector by adding the difference motion vector to the generated predicted motion vector. The motion vector reconstructed in this way can be set for each prediction unit. The compensation unit 195 generates predicted image data for each prediction unit using the motion vector set for each prediction unit by the motion vector setting unit 192 and the reference image data input from the frame memory 69. Then, the compensating unit 195 outputs the generated predicted image data to the adding unit 65 via the switch 71.

(2) Enhancement Layer In the enhancement layer motion compensation process, motion vectors are predicted based on the predictor information of the lower layer stored in the predictor information buffer 193.

  First, the information acquisition unit 191 acquires information on inter prediction decoded from the encoded stream by the lossless decoding unit 62. In the present embodiment, the information related to inter prediction of the enhancement layer may include differential motion vector information (motion vector information for a prediction unit in which motion vector prediction is not performed). In addition, the information acquisition unit 191 includes a predictor that indicates a predictor used when predicting a motion vector of a corresponding prediction unit in the lower layer as setting information for setting a motion vector in each prediction unit in the upper layer. Information is obtained from the predictor information buffer 193. The predictor information acquired here indicates, for example, one of the above-described spatial predictor or temporal predictor. The motion vector setting unit 192 reconstructs a motion vector using the difference motion vector information and the predictor information acquired by the information acquisition unit 191, and sets the reconstructed motion vector in each prediction unit. The motion vector set for each prediction unit by the motion vector setting unit 192 is output to the compensation unit 195 and stored in the motion vector buffer 194. The compensation unit 195 generates predicted image data for each prediction unit using the motion vector set for each prediction unit by the motion vector setting unit 192 and the reference image data input from the frame memory 69. Then, the compensating unit 195 outputs the generated predicted image data to the adding unit 65 via the switch 71.

(3) Processing Flow FIG. 19 is a flowchart illustrating an example of the flow of motion compensation processing by the motion compensation unit 90 according to the present embodiment. Referring to FIG. First, the motion compensation unit 90 performs base layer motion compensation processing (step S210). At that time, the predictor information buffer 193 buffers predictor information indicating the predictor selected at the time of encoding for each prediction unit as setting information.

  The processing in steps S211 to S218 is enhancement layer motion compensation processing. Of these processes, the processes of steps S211 to S217 are repeated for each attention PU of each enhancement layer. In the following description, “upper layer” is a prediction target layer, and “lower layer” is a lower layer of the prediction target layer.

  First, the information acquisition unit 191 uses one PU in the upper layer as the attention PU, and acquires predictor information about the PU in the lower layer corresponding to the attention PU from the predictor information buffer 193 (step S211). Further, the information acquisition unit 191 acquires differential motion vector information for the attention PU (step S212). The motion vector setting unit 192 decodes the difference motion vector information (step S213).

  Next, the motion vector setting unit 192 generates a predicted motion vector for the attention PU using the predictor information and the reference motion vector acquired by the information acquisition unit 191 (step S214). Next, the motion vector setting unit 192 reconstructs the motion vector by adding the difference motion vector to the generated predicted motion vector (step S215). The motion vector reconstructed in this way is set as the attention PU. Further, the reconstructed motion vector is temporarily stored in the motion vector buffer 194 for processing in the upper layer. Note that for a prediction unit in which a motion vector is not predicted, motion vector information may be acquired from the encoded stream instead of the difference motion vector information, and the motion vector may be decoded from the motion vector information.

  Next, the compensation unit 195 generates predicted image data of the attention PU using the motion vector set to the attention PU by the motion vector setting unit 192 and the reference image data input from the frame memory 69 (step S216). ).

  Thereafter, when an unprocessed PU remains in the prediction target layer, the process returns to step S211 (step S217). On the other hand, when there is no unprocessed PU remaining, it is determined whether there is a remaining layer (a higher layer) (step S218). Here, if there are remaining layers, the processing from step S211 is repeated with the previous prediction target layer as the lower layer and the next layer as the upper layer. Predictor information indicating the predictor selected for the lower layer is subsequently buffered by the predictor information buffer 193. If there are no remaining layers, the motion compensation process in FIG. 19 ends. The predicted image data generated here can be output to the adding unit 65 via the switch 71.

  As described above, in the first embodiment, the predictor information of the lower layer is reused at the time of decoding of the upper layer. Therefore, the predictor information does not need to be redundantly encoded for the upper layer. Therefore, it is possible to reduce the code amount of information related to inter prediction.

[4-2. Second embodiment]
FIG. 20 is a block diagram illustrating an example of a detailed configuration of the motion compensation unit 90 according to the second embodiment. Referring to FIG. 20, the motion compensation unit 90 includes an information acquisition unit 291, a motion vector setting unit 292, a motion vector buffer 294, and a compensation unit 295.

(1) Base Layer The base layer motion compensation processing according to the present embodiment may be the same as the base layer motion compensation processing according to the first embodiment described above. However, in this embodiment, the predictor information of the base layer does not have to be buffered, and the motion vector information of the base layer is buffered across the layers. In the motion compensation process of the base layer, the information acquisition unit 291 acquires information related to inter prediction decoded from the encoded stream by the lossless decoding unit 62. In the present embodiment, the information related to inter prediction may include predictor information and differential motion vector information (motion vector information for a prediction unit in which motion vector prediction is not performed). The predictor information acquired here indicates, for example, the predictor selected for each prediction unit at the time of encoding among the predictor candidates that may include the spatial predictor and the temporal predictor described above. The motion vector setting unit 292 sets a motion vector for each prediction unit. The motion vector set for each prediction unit by the motion vector setting unit 292 is output to the compensation unit 295 and stored in the motion vector buffer 294. The setting of the motion vector by the motion vector setting unit 292 can be performed using the predictor indicated by the predictor information and the differential motion vector indicated by the differential motion vector information for each prediction unit. The compensation unit 295 generates predicted image data for each prediction unit using the motion vector set for each prediction unit by the motion vector setting unit 292 and the reference image data input from the frame memory 69. Then, the compensating unit 295 outputs the generated predicted image data to the adding unit 65 via the switch 71.

(2) Enhancement Layer In the enhancement layer motion compensation process, motion vector prediction using an inter-layer predictor based on lower layer reference motion vectors stored in the motion vector buffer 294 may be performed.

  First, the information acquisition unit 291 acquires information regarding inter prediction decoded from the encoded stream by the lossless decoding unit 62. In this embodiment, the information related to the inter prediction of the enhancement layer includes, as setting information, predictor information indicating a predictor selected at the time of encoding from a plurality of predictor candidates including inter-layer predictors in addition to the difference motion vector information. obtain. Note that the predictor information indicating that the inter-layer predictor has been selected may be assigned the smallest code number among the plurality of predictor candidates. The motion vector setting unit 292 reconstructs a motion vector using the difference motion vector information and the predictor information acquired by the information acquisition unit 291 and sets the reconstructed motion vector in each prediction unit. In addition, when the predictor information indicates the inter-layer predictor, the motion vector setting unit 292 displays the reference motion vector expanded according to the spatial resolution ratio between layers as in the above-described equation (9). The predicted motion vector may be used. At that time, the motion vector setting unit 292 may round the predicted motion vector according to the motion vector accuracy. The motion vector set for each prediction unit by the motion vector setting unit 292 is output to the compensation unit 295 and stored in the motion vector buffer 294. The compensation unit 295 generates predicted image data for each prediction unit using the motion vector set for each prediction unit by the motion vector setting unit 292 and the reference image data input from the frame memory 69. Then, the compensating unit 295 outputs the generated predicted image data to the adding unit 65 via the switch 71.

(3) Processing Flow FIG. 21 is a flowchart illustrating an example of the flow of motion compensation processing by the motion compensation unit 90 according to the present embodiment. Referring to FIG. First, the motion compensation unit 90 performs base layer motion compensation processing (step S220). At that time, the motion vector buffer 294 buffers the motion vector set for each prediction unit.

  The processes in steps S221 to S228 are enhancement layer motion compensation processes. Among these processes, the processes of steps S221 to S227 are repeated for each attention PU of each enhancement layer. In the following description, “upper layer” is a prediction target layer, and “lower layer” is a lower layer of the prediction target layer.

  First, the information acquisition unit 291 uses one PU in the upper layer as a target PU, and acquires differential motion vector information and predictor information for the target PU from the encoded stream (step S221). The motion vector setting unit 292 decodes the difference motion vector information (step S222). Also, the motion vector setting unit 292 uses the predictor information to identify a predictor to be used when generating the predicted motion vector of the attention PU (step S223).

  Next, the motion vector setting unit 292 generates a predicted motion vector for the attention PU using the reference motion vector buffered by the motion vector buffer 294 in accordance with the specified predictor (step S224). For example, when the specified predictor is an inter-layer predictor, the motion vector set for the PU in the lower layer corresponding to the target PU is the reference motion vector MVbase of the above-described Expression (8) or Expression (9). used. Next, the motion vector setting unit 292 reconstructs the motion vector by adding the difference motion vector to the generated predicted motion vector (step S225). The motion vector reconstructed in this way is set as the attention PU. In addition, the reconstructed motion vector is temporarily stored in the motion vector buffer 294 for processing in an upper layer. Note that for a prediction unit in which a motion vector is not predicted, motion vector information may be acquired from the encoded stream instead of the difference motion vector information, and the motion vector may be decoded from the motion vector information.

  Next, the compensation unit 295 generates predicted image data of the attention PU using the motion vector set as the attention PU by the motion vector setting unit 292 and the reference image data input from the frame memory 69 (step S226). ).

  Thereafter, when an unprocessed PU remains in the prediction target layer, the process returns to step S221 (step S227). On the other hand, if there is no unprocessed PU remaining, it is determined whether or not there are remaining layers (higher layers) (step S228). Here, if there are remaining layers, the processing from step S221 onward is repeated with the previous prediction target layer as the lower layer and the next layer as the upper layer. If there are no remaining layers, the motion compensation process in FIG. 21 ends. The predicted image data generated here can be output to the adding unit 65 via the switch 71.

  As described above, in the second embodiment, the motion vector used in the upper layer motion compensation can be predicted according to the inter-layer predictor based on the motion vector set in the lower layer. Therefore, the accuracy of motion vector prediction is improved, and as a result, the code amount of the difference motion vector can be reduced.

[4-3. Third Example]
FIG. 22 is a block diagram illustrating an example of a detailed configuration of the motion compensation unit 90 according to the third embodiment. Referring to FIG. 22, the motion compensation unit 90 includes an information acquisition unit 391, a motion vector setting unit 392, a merge information buffer 393, a motion vector buffer 394, and a compensation unit 395.

(1) Base Layer In the base layer motion compensation process, the information acquisition unit 391 acquires information on inter prediction decoded from the encoded stream by the lossless decoding unit 62. In the present embodiment, the information related to inter prediction may include merge information and motion vector information. The merge information acquired here includes, for example, the MergeFlag and the MergeLeftFlag described with reference to FIGS. 12A to 12C, and the merge mode selected for each prediction unit at the time of encoding among a plurality of merge mode candidates. Can show. The motion vector setting unit 392 sets a motion vector for each prediction unit. The motion vector set for each prediction unit by the motion vector setting unit 392 is output to the compensation unit 395 and stored in the motion vector buffer 394. The merge information for each prediction unit is stored in the merge information buffer 393 for processing in the upper layer. For example, when the merge information indicates that a certain prediction unit is merged with an adjacent prediction unit adjacent to the prediction unit (a motion vector common to these prediction units is set), the motion vector setting unit 392 First, the motion vector set as the adjacent prediction unit is acquired from the motion vector buffer 394, and the acquired motion vector is set as the prediction unit. On the other hand, when the merge information indicates that a certain prediction unit is not merged with other prediction units, the motion vector setting unit 392 reconstructs by decoding the motion vector information acquired by the information acquisition unit 391. The motion vector to be performed is set as the prediction unit. The compensation unit 395 generates predicted image data for each prediction unit using the motion vector set for each prediction unit by the motion vector setting unit 392 and the reference image data input from the frame memory 69. Then, the compensating unit 395 outputs the generated predicted image data to the adding unit 65 via the switch 71.

(2) Enhancement Layer In the enhancement layer motion compensation process, a motion vector is set for each prediction unit according to the lower layer merge information stored in the merge information buffer 393.

  First, the information acquisition unit 391 uses, as the setting information for setting a motion vector in each prediction unit in the upper layer, merge information about the prediction unit in the lower layer corresponding to each prediction unit from the merge information buffer 393. get. In addition, the information acquisition unit 391 acquires motion vector information included in the information related to inter prediction for prediction units that are not merged with other prediction units. The merge information acquired by the information acquisition unit 391 may include, for example, MergeFlag and MergeLeftFlag described with reference to FIGS. 12A to 12C. The motion vector setting unit 392 sets a motion vector for each prediction unit according to the merge information acquired by the information acquisition unit 391. The motion vector set for each prediction unit by the motion vector setting unit 392 is output to the compensation unit 395 and stored in the motion vector buffer 394. For a prediction unit that is not merged with another prediction unit, the motion vector setting unit 392 may set a motion vector reconstructed by decoding the motion vector information as the prediction unit. The compensation unit 395 generates predicted image data for each prediction unit using the motion vector set for each prediction unit by the motion vector setting unit 392 and the reference image data input from the frame memory 69. Then, the compensating unit 395 outputs the generated predicted image data to the adding unit 65 via the switch 71.

(3) Processing Flow FIG. 23 is a flowchart illustrating an example of the flow of motion compensation processing by the motion compensation unit 90 according to the present embodiment. Referring to FIG. First, the motion compensation unit 90 performs base layer motion compensation processing (step S230). At that time, the merge information buffer 393 buffers merge information indicating the merge mode selected at the time of encoding for each prediction unit as setting information.

  The processing in steps S231 to S238 is enhancement layer motion compensation processing. Among these processes, the processes of steps S231 to S237 are repeated for each attention PU of each enhancement layer. In the following description, “upper layer” is a prediction target layer, and “lower layer” is a lower layer of the prediction target layer.

  First, the information acquisition unit 391 uses one PU in the upper layer as the attention PU, and acquires merge information about the PU in the lower layer corresponding to the attention PU from the merge information buffer 393 (step S231). Next, the information acquisition unit 391 determines whether or not the attention PU is merged with another PU from the acquired merge information (step S232). For example, if the corresponding PU in the lower layer is merged with the left adjacent PU, it can be determined that the attention PU is also merged with the left adjacent PU. Similarly, if the corresponding PU in the lower layer is merged with the upper neighboring PU, it can be determined that the attention PU is also merged with the upper neighboring PU. In these cases, the process proceeds to step S233. On the other hand, if the corresponding PU in the lower layer is not merged with the adjacent PU, it may be determined that the attention PU is not merged with other PUs. In this case, the process proceeds to step S234.

  In step S233, the motion vector setting unit 392 acquires the motion vector specified according to the merge information from the motion vector buffer 394, and sets the acquired motion vector as the attention PU (step S233). On the other hand, in step S234, the information acquisition unit 391 acquires motion vector information about the attention PU (step S234). Then, the motion vector setting unit 392 decodes the motion vector from the acquired motion vector information, and sets the decoded motion vector as the attention PU (step S235).

  Next, the compensation unit 395 generates predicted image data of the attention PU using the motion vector set to the attention PU by the motion vector setting unit 392 and the reference image data input from the frame memory 69 (step S236). ).

  Thereafter, when an unprocessed PU remains in the prediction target layer, the process returns to step S231 (step S237). On the other hand, if there is no unprocessed PU remaining, it is determined whether or not there are remaining layers (higher layers) (step S238). Here, when there are remaining layers, the processing after step S231 is repeated with the previous prediction target layer as the lower layer and the next layer as the upper layer. Merge information indicating the merge mode selected for the lower layer is continuously buffered by the merge information buffer 393. If there is no remaining layer, the motion compensation process in FIG. 23 ends. The predicted image data generated here can be output to the adding unit 65 via the switch 71.

  As described above, in the third embodiment, since the merge information of the lower layer is reused when decoding the upper layer, the merge information need not be encoded redundantly for the upper layer. Therefore, it is possible to reduce the code amount of information related to inter prediction.

[4-4. Fourth Example]
FIG. 24 is a block diagram illustrating an example of a detailed configuration of the motion compensation unit 90 according to the fourth embodiment. Referring to FIG. 24, the motion compensation unit 90 includes an information acquisition unit 491, a motion vector setting unit 492, a motion vector buffer 494, and a compensation unit 495.

(1) Base Layer In the base layer motion compensation process, the information acquisition unit 491 acquires information on inter prediction decoded from the encoded stream by the lossless decoding unit 62. In the present embodiment, the information related to inter prediction may include merge information and motion vector information. The merge information acquired for the base layer includes, for example, MergeFlag and MergeLeftFlag described with reference to FIGS. 12A to 12C, and the merge mode selected for each prediction unit at the time of encoding among a plurality of merge mode candidates Can be shown. The motion vector setting unit 492 sets a motion vector for each prediction unit. The motion vector set for each prediction unit by the motion vector setting unit 492 is output to the compensation unit 495 and stored in the motion vector buffer 494. For example, when the merge information indicates that a certain prediction unit is merged with an adjacent prediction unit adjacent to the prediction unit, the motion vector setting unit 492 uses the motion vector set to the adjacent prediction unit as a motion vector. The motion vector acquired from the buffer 494 is set as the prediction unit. On the other hand, when the merge information indicates that a certain prediction unit is not merged with another prediction unit, the motion vector setting unit 492 reconstructs by decoding the motion vector information acquired by the information acquisition unit 491. The motion vector to be performed is set as the prediction unit. The compensation unit 495 generates predicted image data for each prediction unit using the motion vector set for each prediction unit by the motion vector setting unit 492 and the reference image data input from the frame memory 69. Then, the compensating unit 495 outputs the generated predicted image data to the adding unit 65 via the switch 71.

(2) Enhancement Layer In the enhancement layer motion compensation process, merge information including MergeBaseFlag indicating a merge with a corresponding prediction unit in the lower layer can be used.

  First, the information acquisition unit 491 acquires information related to inter prediction decoded from the encoded stream by the lossless decoding unit 62. Information regarding inter prediction for the enhancement layer may include merge information and motion vector information. The merge information includes, for example, MergeFlag, MergeBaseFlag, and MergeLeftFlag described with reference to FIGS. 15A to 15C, and may indicate the merge mode selected for each prediction unit during encoding from among a plurality of merge mode candidates. The motion vector setting unit 492 sets a motion vector for each prediction unit according to the merge information acquired by the information acquisition unit 491. Note that, when prediction units are merged between layers, the motion vector setting unit 492 converts the buffered motion vector according to the spatial resolution ratio between layers as in Equation (9) described above. After enlarging, an enlarged motion vector may be set. At this time, the motion vector setting unit 492 may round the enlarged motion vector according to the motion vector accuracy. The motion vector set for each prediction unit by the motion vector setting unit 492 is output to the compensation unit 495 and stored in the motion vector buffer 494. For a prediction unit that is not merged with another prediction unit, the motion vector setting unit 492 may set a motion vector reconstructed by decoding the motion vector information as the prediction unit. The compensation unit 495 generates predicted image data for each prediction unit using the motion vector set for each prediction unit by the motion vector setting unit 492 and the reference image data input from the frame memory 69. Then, the compensating unit 495 outputs the generated predicted image data to the adding unit 65 via the switch 71.

(3) Processing Flow FIG. 25 is a flowchart illustrating an example of the flow of motion compensation processing by the motion compensation unit 90 according to the present embodiment. Referring to FIG. First, the motion compensation unit 90 performs base layer motion compensation processing (step S240). At that time, the motion vector buffer 494 buffers the motion vector set for each prediction unit.

  The processing in steps S241 to S248 is enhancement layer motion compensation processing. Among these processes, the processes of steps S241 to S247 are repeated for each attention PU of each enhancement layer. In the following description, “upper layer” is a prediction target layer, and “lower layer” is a lower layer of the prediction target layer.

  First, the information acquisition unit 491 acquires merge information for one attention PU of an upper layer (step S241). Next, the information acquisition unit 491 determines whether or not to merge the attention PU with another PU from the acquired merge information (step S242). For example, the attention PU may be merged with a corresponding PU in a lower layer or a neighboring PU in an upper layer. If the attention PU is merged with another PU, the process proceeds to step S243. On the other hand, when the attention PU is not merged with another PU, the process proceeds to step S244.

  In step S243, the motion vector setting unit 492 acquires the motion vector specified according to the merge information from the motion vector buffer 494, and sets the acquired motion vector as the attention PU (step S243). On the other hand, in step S244, the information acquisition unit 491 acquires motion vector information about the attention PU (step S244). Then, the motion vector setting unit 492 decodes the motion vector from the acquired motion vector information, and sets the decoded motion vector as the attention PU (step S245).

  Next, the compensation unit 495 generates predicted image data of the attention PU using the motion vector set to the attention PU by the motion vector setting unit 492 and the reference image data input from the frame memory 69 (step S246). ).

  Thereafter, when an unprocessed PU remains in the prediction target layer, the process returns to step S241 (step S247). On the other hand, if no unprocessed PU remains, it is determined whether or not there is a remaining layer (higher layer) (step S248). Here, if there are remaining layers, the processes after step S241 are repeated with the previous prediction target layer as the lower layer and the next layer as the upper layer. The motion vector set for each prediction unit of the lower layer is buffered by the motion vector buffer 494. If there are no remaining layers, the motion compensation process in FIG. 25 ends. The predicted image data generated here can be output to the adding unit 65 via the switch 71.

  Thus, in the fourth embodiment, a motion vector is used for each prediction unit of the enhancement layer using merge information indicating a merge mode selected from a plurality of merge mode candidates including a merge of prediction units between layers. Is set. Therefore, since the motion vector is not encoded for the prediction unit in the upper layer merged with the corresponding prediction unit in the lower layer where the correlation of the motion is significant, the code amount can be effectively reduced.

<5. Application example>
The image encoding device 10 and the image decoding device 60 according to the above-described embodiments are a transmitter or a receiver in satellite broadcasting, cable broadcasting such as cable TV, distribution on the Internet, and distribution to terminals by cellular communication, The present invention can be applied to various electronic devices such as a recording device that records an image on a medium such as an optical disk, a magnetic disk, and a flash memory, or a playback device that reproduces an image from these storage media. Hereinafter, four application examples will be described.

[5-1. First application example]
FIG. 26 illustrates an example of a schematic configuration of a television device to which the above-described embodiment is applied. The television apparatus 900 includes an antenna 901, a tuner 902, a demultiplexer 903, a decoder 904, a video signal processing unit 905, a display unit 906, an audio signal processing unit 907, a speaker 908, an external interface 909, a control unit 910, a user interface 911, And a bus 912.

  The tuner 902 extracts a signal of a desired channel from a broadcast signal received via the antenna 901, and demodulates the extracted signal. Then, the tuner 902 outputs the encoded bit stream obtained by the demodulation to the demultiplexer 903. In other words, the tuner 902 serves as a transmission unit in the television apparatus 900 that receives an encoded stream in which an image is encoded.

  The demultiplexer 903 separates the video stream and audio stream of the viewing target program from the encoded bit stream, and outputs each separated stream to the decoder 904. Further, the demultiplexer 903 extracts auxiliary data such as EPG (Electronic Program Guide) from the encoded bit stream, and supplies the extracted data to the control unit 910. Note that the demultiplexer 903 may perform descrambling when the encoded bit stream is scrambled.

  The decoder 904 decodes the video stream and audio stream input from the demultiplexer 903. Then, the decoder 904 outputs the video data generated by the decoding process to the video signal processing unit 905. In addition, the decoder 904 outputs audio data generated by the decoding process to the audio signal processing unit 907.

  The video signal processing unit 905 reproduces the video data input from the decoder 904 and causes the display unit 906 to display the video. In addition, the video signal processing unit 905 may cause the display unit 906 to display an application screen supplied via a network. Further, the video signal processing unit 905 may perform additional processing such as noise removal on the video data according to the setting. Furthermore, the video signal processing unit 905 may generate a GUI (Graphical User Interface) image such as a menu, a button, or a cursor, and superimpose the generated image on the output image.

  The display unit 906 is driven by a drive signal supplied from the video signal processing unit 905, and displays a video or an image on a video screen of a display device (for example, a liquid crystal display, a plasma display, or an OLED).

  The audio signal processing unit 907 performs reproduction processing such as D / A conversion and amplification on the audio data input from the decoder 904 and outputs audio from the speaker 908. The audio signal processing unit 907 may perform additional processing such as noise removal on the audio data.

  The external interface 909 is an interface for connecting the television device 900 to an external device or a network. For example, a video stream or an audio stream received via the external interface 909 may be decoded by the decoder 904. That is, the external interface 909 also has a role as a transmission unit in the television apparatus 900 that receives an encoded stream in which an image is encoded.

  The control unit 910 includes a processor such as a CPU (Central Processing Unit) and a memory such as a RAM (Random Access Memory) and a ROM (Read Only Memory). The memory stores a program executed by the CPU, program data, EPG data, data acquired via a network, and the like. The program stored in the memory is read and executed by the CPU when the television device 900 is activated, for example. The CPU controls the operation of the television device 900 according to an operation signal input from the user interface 911, for example, by executing the program.

  The user interface 911 is connected to the control unit 910. The user interface 911 includes, for example, buttons and switches for the user to operate the television device 900, a remote control signal receiving unit, and the like. The user interface 911 detects an operation by the user via these components, generates an operation signal, and outputs the generated operation signal to the control unit 910.

  The bus 912 connects the tuner 902, the demultiplexer 903, the decoder 904, the video signal processing unit 905, the audio signal processing unit 907, the external interface 909, and the control unit 910 to each other.

  In the television device 900 configured as described above, the decoder 904 has the function of the image decoding device 60 according to the above-described embodiment. Therefore, encoding efficiency can be further improved by utilizing the correlation of motion between layers in scalable decoding of an image in the television apparatus 900.

[5-2. Second application example]
FIG. 27 shows an example of a schematic configuration of a mobile phone to which the above-described embodiment is applied. A cellular phone 920 includes an antenna 921, a communication unit 922, an audio codec 923, a speaker 924, a microphone 925, a camera unit 926, an image processing unit 927, a demultiplexing unit 928, a recording / reproducing unit 929, a display unit 930, a control unit 931, an operation A portion 932 and a bus 933.

  The antenna 921 is connected to the communication unit 922. The speaker 924 and the microphone 925 are connected to the audio codec 923. The operation unit 932 is connected to the control unit 931. The bus 933 connects the communication unit 922, the audio codec 923, the camera unit 926, the image processing unit 927, the demultiplexing unit 928, the recording / reproducing unit 929, the display unit 930, and the control unit 931 to each other.

  The mobile phone 920 has various operation modes including a voice call mode, a data communication mode, a shooting mode, and a videophone mode, and is used for sending and receiving voice signals, sending and receiving e-mail or image data, taking images, and recording data Perform the action.

  In the voice call mode, an analog voice signal generated by the microphone 925 is supplied to the voice codec 923. The audio codec 923 converts an analog audio signal into audio data, A / D converts the compressed audio data, and compresses it. Then, the audio codec 923 outputs the compressed audio data to the communication unit 922. The communication unit 922 encodes and modulates the audio data and generates a transmission signal. Then, the communication unit 922 transmits the generated transmission signal to a base station (not shown) via the antenna 921. In addition, the communication unit 922 amplifies a radio signal received via the antenna 921 and performs frequency conversion to acquire a received signal. Then, the communication unit 922 demodulates and decodes the received signal to generate audio data, and outputs the generated audio data to the audio codec 923. The audio codec 923 expands the audio data and performs D / A conversion to generate an analog audio signal. Then, the audio codec 923 supplies the generated audio signal to the speaker 924 to output audio.

  Further, in the data communication mode, for example, the control unit 931 generates character data that constitutes an e-mail in response to an operation by the user via the operation unit 932. In addition, the control unit 931 causes the display unit 930 to display characters. In addition, the control unit 931 generates e-mail data in response to a transmission instruction from the user via the operation unit 932, and outputs the generated e-mail data to the communication unit 922. The communication unit 922 encodes and modulates email data and generates a transmission signal. Then, the communication unit 922 transmits the generated transmission signal to a base station (not shown) via the antenna 921. In addition, the communication unit 922 amplifies a radio signal received via the antenna 921 and performs frequency conversion to acquire a received signal. Then, the communication unit 922 demodulates and decodes the received signal to restore the email data, and outputs the restored email data to the control unit 931. The control unit 931 displays the content of the electronic mail on the display unit 930 and stores the electronic mail data in the storage medium of the recording / reproducing unit 929.

  The recording / reproducing unit 929 has an arbitrary readable / writable storage medium. For example, the storage medium may be a built-in storage medium such as a RAM or a flash memory, or an externally mounted storage medium such as a hard disk, a magnetic disk, a magneto-optical disk, an optical disk, a USB memory, or a memory card. May be.

  In the shooting mode, for example, the camera unit 926 captures an image of a subject, generates image data, and outputs the generated image data to the image processing unit 927. The image processing unit 927 encodes the image data input from the camera unit 926 and stores the encoded stream in the storage medium of the recording / playback unit 929.

  Further, in the videophone mode, for example, the demultiplexing unit 928 multiplexes the video stream encoded by the image processing unit 927 and the audio stream input from the audio codec 923, and the multiplexed stream is the communication unit 922. Output to. The communication unit 922 encodes and modulates the stream and generates a transmission signal. Then, the communication unit 922 transmits the generated transmission signal to a base station (not shown) via the antenna 921. In addition, the communication unit 922 amplifies a radio signal received via the antenna 921 and performs frequency conversion to acquire a received signal. These transmission signal and reception signal may include an encoded bit stream. Then, the communication unit 922 demodulates and decodes the received signal to restore the stream, and outputs the restored stream to the demultiplexing unit 928. The demultiplexing unit 928 separates the video stream and the audio stream from the input stream, and outputs the video stream to the image processing unit 927 and the audio stream to the audio codec 923. The image processing unit 927 decodes the video stream and generates video data. The video data is supplied to the display unit 930, and a series of images is displayed on the display unit 930. The audio codec 923 decompresses the audio stream and performs D / A conversion to generate an analog audio signal. Then, the audio codec 923 supplies the generated audio signal to the speaker 924 to output audio.

  In the mobile phone 920 configured as described above, the image processing unit 927 has the functions of the image encoding device 10 and the image decoding device 60 according to the above-described embodiment. Therefore, encoding efficiency can be further improved by utilizing the correlation of motion between layers in scalable encoding and decoding of an image with the mobile phone 920.

[5-3. Third application example]
FIG. 28 shows an example of a schematic configuration of a recording / reproducing apparatus to which the above-described embodiment is applied. For example, the recording / reproducing device 940 encodes audio data and video data of a received broadcast program and records the encoded data on a recording medium. In addition, the recording / reproducing device 940 may encode audio data and video data acquired from another device and record them on a recording medium, for example. In addition, the recording / reproducing device 940 reproduces data recorded on the recording medium on a monitor and a speaker, for example, in accordance with a user instruction. At this time, the recording / reproducing device 940 decodes the audio data and the video data.

  The recording / reproducing apparatus 940 includes a tuner 941, an external interface 942, an encoder 943, an HDD (Hard Disk Drive) 944, a disk drive 945, a selector 946, a decoder 947, an OSD (On-Screen Display) 948, a control unit 949, and a user interface. 950.

  The tuner 941 extracts a signal of a desired channel from a broadcast signal received via an antenna (not shown), and demodulates the extracted signal. Then, the tuner 941 outputs the encoded bit stream obtained by the demodulation to the selector 946. That is, the tuner 941 has a role as a transmission unit in the recording / reproducing apparatus 940.

  The external interface 942 is an interface for connecting the recording / reproducing apparatus 940 to an external device or a network. The external interface 942 may be, for example, an IEEE 1394 interface, a network interface, a USB interface, or a flash memory interface. For example, video data and audio data received via the external interface 942 are input to the encoder 943. That is, the external interface 942 serves as a transmission unit in the recording / reproducing device 940.

  The encoder 943 encodes video data and audio data when the video data and audio data input from the external interface 942 are not encoded. Then, the encoder 943 outputs the encoded bit stream to the selector 946.

  The HDD 944 records an encoded bit stream in which content data such as video and audio is compressed, various programs, and other data on an internal hard disk. Also, the HDD 944 reads out these data from the hard disk when playing back video and audio.

  The disk drive 945 performs recording and reading of data with respect to the mounted recording medium. The recording medium mounted on the disk drive 945 may be, for example, a DVD disk (DVD-Video, DVD-RAM, DVD-R, DVD-RW, DVD + R, DVD + RW, etc.) or a Blu-ray (registered trademark) disk. .

  The selector 946 selects an encoded bit stream input from the tuner 941 or the encoder 943 when recording video and audio, and outputs the selected encoded bit stream to the HDD 944 or the disk drive 945. In addition, the selector 946 outputs the encoded bit stream input from the HDD 944 or the disk drive 945 to the decoder 947 during video and audio reproduction.

  The decoder 947 decodes the encoded bit stream and generates video data and audio data. Then, the decoder 947 outputs the generated video data to the OSD 948. The decoder 904 outputs the generated audio data to an external speaker.

  The OSD 948 reproduces the video data input from the decoder 947 and displays the video. Further, the OSD 948 may superimpose a GUI image such as a menu, a button, or a cursor on the video to be displayed.

  The control unit 949 includes a processor such as a CPU and memories such as a RAM and a ROM. The memory stores a program executed by the CPU, program data, and the like. The program stored in the memory is read and executed by the CPU when the recording / reproducing apparatus 940 is activated, for example. The CPU controls the operation of the recording / reproducing device 940 according to an operation signal input from the user interface 950, for example, by executing the program.

  The user interface 950 is connected to the control unit 949. The user interface 950 includes, for example, buttons and switches for the user to operate the recording / reproducing device 940, a remote control signal receiving unit, and the like. The user interface 950 detects an operation by the user via these components, generates an operation signal, and outputs the generated operation signal to the control unit 949.

  In the recording / reproducing apparatus 940 configured as described above, the encoder 943 has the function of the image encoding apparatus 10 according to the above-described embodiment. The decoder 947 has the function of the image decoding device 60 according to the above-described embodiment. Therefore, encoding efficiency can be further improved by utilizing the correlation of motion between layers in scalable encoding and decoding of an image in the recording / reproducing apparatus 940.

[5-4. Fourth application example]
FIG. 29 illustrates an example of a schematic configuration of an imaging apparatus to which the above-described embodiment is applied. The imaging device 960 images a subject to generate an image, encodes the image data, and records it on a recording medium.

  The imaging device 960 includes an optical block 961, an imaging unit 962, a signal processing unit 963, an image processing unit 964, a display unit 965, an external interface 966, a memory 967, a media drive 968, an OSD 969, a control unit 970, a user interface 971, and a bus. 972.

  The optical block 961 is connected to the imaging unit 962. The imaging unit 962 is connected to the signal processing unit 963. The display unit 965 is connected to the image processing unit 964. The user interface 971 is connected to the control unit 970. The bus 972 connects the image processing unit 964, the external interface 966, the memory 967, the media drive 968, the OSD 969, and the control unit 970 to each other.

  The optical block 961 includes a focus lens and a diaphragm mechanism. The optical block 961 forms an optical image of the subject on the imaging surface of the imaging unit 962. The imaging unit 962 includes an image sensor such as a CCD or a CMOS, and converts an optical image formed on the imaging surface into an image signal as an electrical signal by photoelectric conversion. Then, the imaging unit 962 outputs the image signal to the signal processing unit 963.

  The signal processing unit 963 performs various camera signal processes such as knee correction, gamma correction, and color correction on the image signal input from the imaging unit 962. The signal processing unit 963 outputs the image data after the camera signal processing to the image processing unit 964.

  The image processing unit 964 encodes the image data input from the signal processing unit 963, and generates encoded data. Then, the image processing unit 964 outputs the generated encoded data to the external interface 966 or the media drive 968. The image processing unit 964 also decodes encoded data input from the external interface 966 or the media drive 968 to generate image data. Then, the image processing unit 964 outputs the generated image data to the display unit 965. In addition, the image processing unit 964 may display the image by outputting the image data input from the signal processing unit 963 to the display unit 965. Further, the image processing unit 964 may superimpose display data acquired from the OSD 969 on an image output to the display unit 965.

  The OSD 969 generates a GUI image such as a menu, a button, or a cursor, and outputs the generated image to the image processing unit 964.

  The external interface 966 is configured as a USB input / output terminal, for example. The external interface 966 connects the imaging device 960 and a printer, for example, when printing an image. Further, a drive is connected to the external interface 966 as necessary. For example, a removable medium such as a magnetic disk or an optical disk is attached to the drive, and a program read from the removable medium can be installed in the imaging device 960. Further, the external interface 966 may be configured as a network interface connected to a network such as a LAN or the Internet. That is, the external interface 966 has a role as a transmission unit in the imaging device 960.

  The recording medium mounted on the media drive 968 may be any readable / writable removable medium such as a magnetic disk, a magneto-optical disk, an optical disk, or a semiconductor memory. Further, a recording medium may be fixedly attached to the media drive 968, and a non-portable storage unit such as a built-in hard disk drive or an SSD (Solid State Drive) may be configured.

  The control unit 970 includes a processor such as a CPU and memories such as a RAM and a ROM. The memory stores a program executed by the CPU, program data, and the like. The program stored in the memory is read and executed by the CPU when the imaging device 960 is activated, for example. The CPU controls the operation of the imaging device 960 according to an operation signal input from the user interface 971, for example, by executing the program.

  The user interface 971 is connected to the control unit 970. The user interface 971 includes, for example, buttons and switches for the user to operate the imaging device 960. The user interface 971 detects an operation by the user via these components, generates an operation signal, and outputs the generated operation signal to the control unit 970.

  In the imaging device 960 configured as described above, the image processing unit 964 has the functions of the image encoding device 10 and the image decoding device 60 according to the above-described embodiment. Therefore, encoding efficiency can be further improved by utilizing the correlation of motion between layers in scalable encoding and decoding of an image in the imaging device 960.

<6. Summary>
Up to this point, the four examples of the image encoding device 10 and the image decoding device 60 according to an embodiment have been described with reference to FIGS. According to these embodiments, setting information for setting a motion vector in a second prediction unit in an upper layer corresponding to a first prediction unit in a lower layer upon scalable coding and decoding of an image is provided. A motion vector is set in the second prediction unit using setting information related to the motion vector set in the first prediction unit. Therefore, it is possible to set a motion vector for each prediction unit of the upper layer by utilizing the correlation of motion between layers. Therefore, since redundant encoding of motion vector information, difference motion vector information, predictor information, or merge information is avoided, encoding efficiency can be improved.

  For example, according to the first embodiment, the predictor information indicating the predictor used when predicting the motion vector of the prediction unit in the lower layer is reused when the motion vector of the prediction unit in the upper layer is predicted. Is done. Therefore, redundant encoding of predictor information can be avoided.

  Also, for example, according to the second embodiment, an inter-layer predictor based on a motion vector set for a corresponding prediction unit in the lower layer for a prediction unit in the upper layer is introduced as a new predictor candidate. Is done. Therefore, it is possible to improve the accuracy of motion vector prediction for the prediction unit in the upper layer, and to reduce the amount of code required for encoding the difference motion vector information.

  For example, according to the third embodiment, the merge information indicating the merge mode selected for the prediction unit in the lower layer is reused for the prediction unit in the upper layer. Therefore, redundant encoding of merge information can be avoided.

  Further, for example, according to the fourth embodiment, a new merge mode for merging the corresponding prediction unit in the lower layer and the prediction unit in the upper layer is introduced. Therefore, redundant encoding of motion vector information can be avoided for prediction units in the upper layer.

  In the present specification, an example in which information related to intra prediction and information related to inter prediction is multiplexed on the header of the encoded stream and transmitted from the encoding side to the decoding side has been mainly described. However, the method for transmitting such information is not limited to such an example. For example, these pieces of information may be transmitted or recorded as separate data associated with the encoded bitstream without being multiplexed into the encoded bitstream. Here, the term “associate” means that an image (which may be a part of an image such as a slice or a block) included in the bitstream and information corresponding to the image can be linked at the time of decoding. Means. That is, information may be transmitted on a transmission path different from that of the image (or bit stream). Information may be recorded on a recording medium (or another recording area of the same recording medium) different from the image (or bit stream). Furthermore, the information and the image (or bit stream) may be associated with each other in an arbitrary unit such as a plurality of frames, one frame, or a part of the frame.

  The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, but the technical scope of the present disclosure is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field of the present disclosure can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that it belongs to the technical scope of the present disclosure.

The following configurations also belong to the technical scope of the present disclosure.
(1)
A second in the second layer corresponding to a first prediction unit in the first layer of a scalable decoded image including a first layer and a second layer higher than the first layer. Setting information for setting a motion vector in the prediction unit, an information acquisition unit for acquiring the setting information related to the motion vector set in the first prediction unit;
A motion vector setting unit for setting a motion vector in the second prediction unit using the setting information acquired by the information acquisition unit;
An image processing apparatus comprising:
(2)
The setting information includes predictor information indicating a predictor used when predicting a motion vector of the first prediction unit,
The motion vector setting unit predicts a motion vector set in the second prediction unit using the predictor indicated by the predictor information.
The image processing apparatus according to (1).
(3)
The setting information includes predictor information indicating a predictor used when predicting a motion vector of the second prediction unit,
The predictor is selected from a plurality of predictor candidates including a predictor candidate based on a motion vector set in the first prediction unit.
The image processing apparatus according to (1).
(4)
The image processing apparatus according to (3), wherein a predictor candidate based on a motion vector set as the first prediction unit is assigned a minimum code number among the plurality of predictor candidates.
(5)
The information acquisition unit further acquires differential motion vector information indicating a difference between a motion vector set in the second prediction unit and a predicted motion vector;
The motion vector setting unit sets a motion vector generated by adding a difference indicated by the difference motion vector information to the predicted motion vector predicted using the predictor as the second prediction unit.
The image processing apparatus according to any one of (2) to (4).
(6)
The setting information includes merge information indicating whether a motion vector common to the first prediction unit and a prediction unit adjacent to the first prediction unit is set,
The motion vector setting unit sets a common motion vector for the second prediction unit and a prediction unit adjacent to the second prediction unit according to the merge information.
The image processing apparatus according to (1).
(7)
The setting information includes merge information indicating whether a motion vector common to the first prediction unit and the second prediction unit is set,
The motion vector setting unit, when the merge information indicates that a motion vector common to the first prediction unit and the second prediction unit is set, Setting a common motion vector as the second prediction unit;
The image processing apparatus according to (1).
(8)
The motion vector setting unit expands the motion vector set in the first prediction unit according to a spatial resolution ratio between the first layer and the second layer, and then The image processing apparatus according to any one of (3), 4 and 7, wherein a motion vector setting process is performed for each prediction unit.
(9)
The image processing device according to (8), wherein the motion vector setting unit rounds the expanded motion vector according to motion vector accuracy when the motion vector set in the first prediction unit is expanded.
(10)
The image processing apparatus according to any one of (1) to (7), wherein the first layer and the second layer are layers having different spatial resolutions.
(11)
The image processing device according to any one of (1) to (7), wherein the first layer and the second layer are layers having different noise ratios.
(12)
Any of (1) to (11), wherein the first prediction unit is a prediction unit in the first layer having a pixel corresponding to a pixel at a predetermined position in the second prediction unit. The image processing apparatus according to claim 1.
(13)
In any one of (1) to (11), the first prediction unit is a prediction unit having the largest overlap among prediction units in the first layer overlapping the second prediction unit. The image processing apparatus described.
(14)
A second in the second layer corresponding to a first prediction unit in the first layer of a scalable decoded image including a first layer and a second layer higher than the first layer. Setting information for setting a motion vector in the prediction unit, and acquiring the setting information related to the motion vector set in the first prediction unit;
Using the acquired setting information to set a motion vector in the second prediction unit;
An image processing method including:
(15)
A second in the second layer corresponding to a first prediction unit in the first layer of a scalable decoded image including a first layer and a second layer higher than the first layer. Setting information for setting a motion vector in the prediction unit, and an information generation unit that generates the setting information related to the motion vector set in the first prediction unit;
An encoding unit that encodes the setting information generated by the information generation unit;
An image processing apparatus comprising:
(16)
A second in the second layer corresponding to a first prediction unit in the first layer of a scalable decoded image including a first layer and a second layer higher than the first layer. Generating setting information related to the motion vector set in the first prediction unit, which is setting information for setting a motion vector in the prediction unit of
Encoding the generated setting information;
An image processing method including:

10 Image encoding device (image processing device)
146, 246, 346, 446 Information generation unit 16 Encoding unit 60 Image decoding device (image processing device)
191,291,391,491 Information acquisition unit 192,246,392,492 Motion vector setting unit

Claims (16)

A second in the second layer corresponding to a first prediction unit in the first layer of a scalable decoded image including a first layer and a second layer higher than the first layer. Setting information for setting a motion vector in the prediction unit, an information acquisition unit for acquiring the setting information related to the motion vector set in the first prediction unit;
A motion vector setting unit for setting a motion vector in the second prediction unit using the setting information acquired by the information acquisition unit;
An image processing apparatus comprising: The setting information includes predictor information indicating a predictor used when predicting a motion vector of the first prediction unit,
The motion vector setting unit predicts a motion vector set in the second prediction unit using the predictor indicated by the predictor information.
The image processing apparatus according to claim 1. The setting information includes predictor information indicating a predictor used when predicting a motion vector of the second prediction unit,
The predictor is selected from a plurality of predictor candidates including a predictor candidate based on a motion vector set in the first prediction unit.
The image processing apparatus according to claim 1.   The image processing apparatus according to claim 3, wherein a predictor candidate based on a motion vector set as the first prediction unit is assigned a minimum code number among the plurality of predictor candidates. The information acquisition unit further acquires differential motion vector information indicating a difference between a motion vector set in the second prediction unit and a predicted motion vector;
The motion vector setting unit sets a motion vector generated by adding a difference indicated by the difference motion vector information to the predicted motion vector predicted using the predictor as the second prediction unit.
The image processing apparatus according to claim 2. The setting information includes merge information indicating whether a motion vector common to the first prediction unit and a prediction unit adjacent to the first prediction unit is set,
The motion vector setting unit sets a common motion vector for the second prediction unit and a prediction unit adjacent to the second prediction unit according to the merge information.
The image processing apparatus according to claim 1. The setting information includes merge information indicating whether a motion vector common to the first prediction unit and the second prediction unit is set,
The motion vector setting unit, when the merge information indicates that a motion vector common to the first prediction unit and the second prediction unit is set, Setting a common motion vector as the second prediction unit;
The image processing apparatus according to claim 1.   The motion vector setting unit expands the motion vector set in the first prediction unit according to a spatial resolution ratio between the first layer and the second layer, and then The image processing apparatus according to claim 3, wherein a motion vector setting process is performed for each prediction unit.   The image processing apparatus according to claim 8, wherein the motion vector setting unit rounds the expanded motion vector according to motion vector accuracy when the motion vector set in the first prediction unit is expanded.   The image processing apparatus according to claim 1, wherein the first layer and the second layer are layers having different spatial resolutions.   The image processing apparatus according to claim 1, wherein the first layer and the second layer are layers having different noise ratios.   The image processing apparatus according to claim 1, wherein the first prediction unit is a prediction unit in the first layer having a pixel corresponding to a pixel at a predetermined position in the second prediction unit.   The image processing apparatus according to claim 1, wherein the first prediction unit is a prediction unit having the largest overlap among prediction units in the first layer that overlap with the second prediction unit. A second in the second layer corresponding to a first prediction unit in the first layer of a scalable decoded image including a first layer and a second layer higher than the first layer. Setting information for setting a motion vector in the prediction unit, and acquiring the setting information related to the motion vector set in the first prediction unit;
Using the acquired setting information to set a motion vector in the second prediction unit;
An image processing method including: A second in the second layer corresponding to a first prediction unit in the first layer of a scalable decoded image including a first layer and a second layer higher than the first layer. Setting information for setting a motion vector in the prediction unit, and an information generation unit that generates the setting information related to the motion vector set in the first prediction unit;
An encoding unit that encodes the setting information generated by the information generation unit;
An image processing apparatus comprising: A second in the second layer corresponding to a first prediction unit in the first layer of a scalable decoded image including a first layer and a second layer higher than the first layer. Generating setting information related to the motion vector set in the first prediction unit, which is setting information for setting a motion vector in the prediction unit of
Encoding the generated setting information;
An image processing method including:

Images